Vishesh 'ironeagle' Bangotra
a32eeaf2b4
- collect.py copies each repo's built site contents directly to a top-level <repo>/ dir (was libs|apis|wiki/<repo>/site nesting) - nginx uses `index index.html lib/index.html api/index.html` so /dagpipe/ -> lib/index.html, /auth-server/ -> api/index.html are served internally with no redirects - _index regenerates links against the flat layout (/dagpipe/lib/, /auth-server/api/, /mongo-ops/, /blog/...) - config.yml static entries point at vendored blog/ + media-manager/ - Dockerfile copies per-repo dirs flat into the nginx html root - removed stale libs/, apis/, wiki/, tutorials/ category trees
1 line
37 KiB
JSON
1 line
37 KiB
JSON
{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"lib/","title":"hexa","text":""},{"location":"lib/#hexa","title":"hexa","text":"<p>Hexa: Type-declared dependency injection for Python.</p>"},{"location":"lib/#hexa-classes","title":"Classes","text":""},{"location":"lib/#hexa.MethodSpec","title":"MethodSpec <code>dataclass</code>","text":"<pre><code>MethodSpec(\n name: str,\n args: list[str] = list(),\n kwargs: dict[str, str] = dict(),\n)\n</code></pre> <p>Specification of a single abstract method.</p>"},{"location":"lib/#hexa.PortNode","title":"PortNode <code>dataclass</code>","text":"<pre><code>PortNode(\n name: str,\n port_cls: str,\n kind: str = \"leaf\",\n attributes: dict[str, ConfigValue] = dict(),\n children: list[PortNode] = list(),\n methods: dict[str, MethodSpec] = dict(),\n)\n</code></pre> <p>A node in the hexa port tree.</p>"},{"location":"lib/#hexa.PortNode--parameters","title":"Parameters","text":"<p>name : str Slot name, e.g. <code>\"parser\"</code>, <code>\"amount_balance\"</code>. kind : str One of <code>\"leaf\"</code>, <code>\"port\"</code>, or <code>\"parent\"</code>. - <code>\"leaf\"</code>: no children (e.g. <code>NumberPort</code>) - <code>\"port\"</code>: has children (e.g. <code>TransactionParserPort</code>) - <code>\"parent\"</code>: the root of the tree (e.g. <code>ExtractionPipeline</code>) port_cls : str ABC class name that gives this port its shape, e.g. <code>\"TransactionParserPort\"</code>. Every port has a concrete typed shape. attributes : dict[str, ConfigValue] Annotated attributes: <code>name -> type_expr</code> for scalars, or a nested <code>dict</code> of sub-attributes (a struct) for grouped/structured values. children : list[PortNode] Nested port slots. methods : dict[str, MethodSpec] Abstract methods keyed by method name.</p>"},{"location":"lib/#hexa.PortNode-functions","title":"Functions","text":""},{"location":"lib/#hexa.PortNode.diff","title":"diff","text":"<pre><code>diff(other: PortNode, path: str = '') -> list[str]\n</code></pre> <p>Return a list of human-readable difference strings.</p> <p>An empty list means the trees are equal.</p>"},{"location":"lib/#hexa.PortNode.find","title":"find","text":"<pre><code>find(name: str) -> PortNode | None\n</code></pre> <p>Find a direct child by slot name.</p>"},{"location":"lib/#hexa.PortNode.walk","title":"walk","text":"<pre><code>walk() -> list[PortNode]\n</code></pre> <p>Return all nodes in depth-first order (self first).</p>"},{"location":"lib/#hexa-functions","title":"Functions","text":""},{"location":"lib/#hexa.build","title":"build","text":"<pre><code>build(\n cls: type[T],\n *,\n instances: dict[str, Any] | None = None,\n config: dict[str, Any] | None = None\n) -> T\n</code></pre> <p>Recursively instantiate the port tree from annotations.</p>"},{"location":"lib/#hexa.build--parameters","title":"Parameters","text":"<p>cls : type The root class to build (e.g. <code>AxisExtractionPipeline</code>). instances : dict, optional <code>{slot_name: object}</code> injected verbatim into every node whose annotations contain that name (runtime values \u2014 repos, handlers, clients \u2014 that must not be re-constructed). Applied to the whole tree by name. config : dict, optional <code>{field_name: value}</code> propagated by name to every node's annotated config fields (non-port attributes), replacing the default <code>None</code>.</p>"},{"location":"lib/#hexa.build--returns","title":"Returns","text":"<p>instance A fully wired instance of <code>cls</code> and all its nested ports.</p>"},{"location":"lib/check_matches/","title":"Check Matches","text":""},{"location":"lib/check_matches/#hexa.check_matches","title":"hexa.check_matches","text":"<p>Verify that two hexa representations agree.</p>"},{"location":"lib/check_matches/#hexa.check_matches-classes","title":"Classes","text":""},{"location":"lib/check_matches/#hexa.check_matches-functions","title":"Functions","text":""},{"location":"lib/check_matches/#hexa.check_matches.check_matches","title":"check_matches","text":"<pre><code>check_matches(\n path_a: str | Path, path_b: str | Path\n) -> bool | list[str]\n</code></pre> <p>Verify two representations agree.</p>"},{"location":"lib/check_matches/#hexa.check_matches.check_matches--parameters","title":"Parameters","text":"<p>path_a : str | Path Path to a .yaml or .py file. path_b : str | Path Path to a .yaml or .py file.</p>"},{"location":"lib/check_matches/#hexa.check_matches.check_matches--returns","title":"Returns","text":"<p>bool | list[str] True if they match exactly, otherwise a list of difference strings.</p>"},{"location":"lib/cli/","title":"Cli","text":""},{"location":"lib/cli/#hexa.cli","title":"hexa.cli","text":"<p>Command-line interface for hexa utilities.</p>"},{"location":"lib/cli/#hexa.cli-functions","title":"Functions","text":""},{"location":"lib/container/","title":"Container","text":""},{"location":"lib/container/#hexa.container","title":"hexa.container","text":"<p>Container to build a wired hexa pipeline from annotations.</p>"},{"location":"lib/container/#hexa.container-functions","title":"Functions","text":""},{"location":"lib/container/#hexa.container.build","title":"build","text":"<pre><code>build(\n cls: type[T],\n *,\n instances: dict[str, Any] | None = None,\n config: dict[str, Any] | None = None\n) -> T\n</code></pre> <p>Recursively instantiate the port tree from annotations.</p>"},{"location":"lib/container/#hexa.container.build--parameters","title":"Parameters","text":"<p>cls : type The root class to build (e.g. <code>AxisExtractionPipeline</code>). instances : dict, optional <code>{slot_name: object}</code> injected verbatim into every node whose annotations contain that name (runtime values \u2014 repos, handlers, clients \u2014 that must not be re-constructed). Applied to the whole tree by name. config : dict, optional <code>{field_name: value}</code> propagated by name to every node's annotated config fields (non-port attributes), replacing the default <code>None</code>.</p>"},{"location":"lib/container/#hexa.container.build--returns","title":"Returns","text":"<p>instance A fully wired instance of <code>cls</code> and all its nested ports.</p>"},{"location":"lib/design/","title":"hexa \u2014 Design (extraction pipeline, Axis/Icici)","text":"<p>Note: The extraction pipeline discussed below is a sample/example illustrating the hexa composition pattern.</p> <p>This document is the concrete counterpart to philosophy.md: where the files are, what the tree looks like for both banks, and how a future container instantiates it. Self-sufficient for a fresh agent.</p>"},{"location":"lib/design/#file-map-samplesextraction_pipeline","title":"File map (<code>samples/extraction_pipeline/</code>)","text":"File Layer Role <code>extraction_pipeline_abc.py</code> ABC contracts All port ABCs + <code>ExtractionPipeline</code> root. Declares attributes, the <code>ExtractionSource</code> dataclass, port slots, <code>@abstractmethod</code>s. <code>extraction_pipeline_impl.py</code> Generic <code>Impl*</code> Concrete default implementations of every port; <code>ImplExtractionPipeline</code> root; <code>clean_num</code> helper. <code>extraction_pipeline_ambiguity.py</code> Shared logic Ported <code>ResolutionContext</code>, <code>AmbiguityHandler</code>, <code>AmountBalanceNotFound</code>, <code>validate_signs</code> \u2014 used by <code>ImplAmountBalancePort</code>. <code>extraction_pipeline_banks.py</code> Composition root <code>AxisExtractionPipeline(ImplExtractionPipeline)</code> + <code>IciciExtractionPipeline</code> (each re-pins <code>txn_dicts</code>). <code>banks/axis/pdf.py</code> Bank specialization <code>AxisNumberPort</code>, <code>AxisDescPort</code>, <code>AxisAmountBalancePort</code>, <code>AxisTransactionParserPort</code>, <code>AxisTxnDictsPort</code>. <code>banks/icici/pdf.py</code> Bank specialization <code>Icici*</code> mirror of the above + Icici-only <code>adjust_balance</code>/<code>missing_number_candidates</code>. <code>extraction_pipeline.yaml</code> Contract spec Declarative spec of the port tree; interchangeable with <code>_abc.py</code> via the optional utilities."},{"location":"lib/design/#the-port-tree-abc-contract","title":"The port tree (ABC contract)","text":"<pre><code>ExtractionPipeline\n\u251c\u2500\u2500 source / pipeline / trust_fallback [attributes; source is a nested struct]\n\u251c\u2500\u2500 ingest \u2192 IngestPort (source \u2192 content)\n\u251c\u2500\u2500 raw_lines \u2192 RawLinesPort (content \u2192 raw_lines)\n\u251c\u2500\u2500 txn_blocks \u2192 TxnBlocksPort (raw_lines \u2192 txn_blocks)\n\u2502 \u251c\u2500\u2500 bucket_finder \u2192 BucketFinderPort\n\u2502 \u2514\u2500\u2500 splitter \u2192 SplitterPort\n\u251c\u2500\u2500 txn_dicts \u2192 TxnDictsPort (txn_blocks \u2192 txn_dicts)\n\u2502 \u2514\u2500\u2500 parser \u2192 TransactionParserPort\n\u2502 \u251c\u2500\u2500 number \u2192 NumberPort\n\u2502 \u251c\u2500\u2500 date \u2192 DatePort\n\u2502 \u251c\u2500\u2500 desc \u2192 DescPort\n\u2502 \u2514\u2500\u2500 amount_balance \u2192 AmountBalancePort\n\u2502 \u251c\u2500\u2500 number \u2192 NumberPort\n\u2502 \u2514\u2500\u2500 ambiguity \u2192 AmbiguityPort\n\u2514\u2500\u2500 raw_expense \u2192 RawExpensePort (txn_dicts \u2192 list[RawExpense], terminal)\n</code></pre>"},{"location":"lib/design/#generic-default-impl","title":"Generic default (<code>Impl*</code>)","text":"<p><code>ImplExtractionPipeline</code> pins every stage slot to a concrete <code>Impl*</code> port:</p> <pre><code>ImplExtractionPipeline\n\u251c\u2500\u2500 ingest: ImplIngestPort\n\u251c\u2500\u2500 raw_lines: ImplRawLinesPort\n\u251c\u2500\u2500 txn_blocks: ImplTxnBlocksPort\n\u2502 \u251c\u2500\u2500 bucket_finder: ImplBucketFinderPort\n\u2502 \u2514\u2500\u2500 splitter: ImplSplitterPort\n\u251c\u2500\u2500 txn_dicts: ImplTxnDictsPort\n\u2502 \u2514\u2500\u2500 parser: ImplTransactionParserPort\n\u2502 \u251c\u2500\u2500 number: ImplNumberPort\n\u2502 \u251c\u2500\u2500 date: ImplDatePort\n\u2502 \u251c\u2500\u2500 desc: ImplDescPort\n\u2502 \u2514\u2500\u2500 amount_balance: ImplAmountBalancePort\n\u2502 \u251c\u2500\u2500 number: ImplNumberPort\n\u2502 \u2514\u2500\u2500 ambiguity: ImplAmbiguityPort\n\u2514\u2500\u2500 raw_expense: ImplRawExpensePort\n</code></pre>"},{"location":"lib/design/#bank-variation-only-what-differs","title":"Bank variation (only what differs)","text":"<p>The root class re-pins only <code>txn_dicts</code>. Everything else inherits <code>Impl*</code>.</p>"},{"location":"lib/design/#axis-banksaxispdfpy","title":"Axis (<code>banks/axis/pdf.py</code>)","text":"<pre><code>class AxisExtractionPipeline(ImplExtractionPipeline):\n txn_dicts: AxisTxnDictsPort\n</code></pre> <pre><code>AxisTxnDictsPort(ImplTxnDictsPort)\n \u2514\u2500 parser: AxisTransactionParserPort(ImplTransactionParserPort)\n \u251c\u2500 number: AxisNumberPort # NUMBER_RE, all clean numbers\n \u251c\u2500 desc: AxisDescPort # STARTTERS = [upi/, imps/, neft-, neft/, ach/, ach-, 2a/]\n \u2514\u2500 amount_balance: AxisAmountBalancePort(ImplAmountBalancePort)\n \u2514\u2500 number: AxisNumberPort # re-pin back to axis number port\n</code></pre> <p>Notes: - <code>date</code> is not redeclared on <code>AxisTransactionParserPort</code> \u2014 it inherits <code>ImplDatePort</code> (correct already, adds no behavior). - <code>amount_balance.ambiguity</code> is not redeclared \u2014 inherits <code>ImplAmbiguityPort</code>.</p>"},{"location":"lib/design/#icici-banksicicipdfpy","title":"Icici (<code>banks/icici/pdf.py</code>)","text":"<pre><code>class IciciExtractionPipeline(ImplExtractionPipeline):\n txn_dicts: IciciTxnDictsPort\n</code></pre> <pre><code>IciciTxnDictsPort(ImplTxnDictsPort)\n \u2514\u2500 parser: IciciTransactionParserPort(ImplTransactionParserPort)\n \u251c\u2500 number: IciciNumberPort # NUMBER_RE, returns decimals[-2:] (last 2)\n \u251c\u2500 desc: IciciDescPort # STARTTERS + fix_ocr (UPl\u2192UPI, ..., ICIC]\u2192ICICI)\n \u2514\u2500 amount_balance: IciciAmountBalancePort(ImplAmountBalancePort)\n \u251c\u2500 number: IciciNumberPort # re-pin to icici number port\n \u251c\u2500 adjust_balance: # neg-markers [(-), (-}, {-}, {-) , ()] \u2192 -abs(balance)\n \u2514\u2500 missing_number_candidates: # Icici-only one-number hook\n</code></pre>"},{"location":"lib/design/#redundancy-rule-applied-in-both-banks","title":"Redundancy rule (applied in both banks)","text":"<p>Only redeclare a slot you are actually changing from <code>Impl*</code>. If a slot's behavior is already correct from the parent <code>Impl*</code>, omit it. This keeps the diff between a bank and <code>Impl*</code> minimal and unambiguous.</p> <p><code>date: ImplDatePort</code> was previously redeclared redundantly and has been removed \u2014 it added mental load on the extender for zero behavior change. (Fidelity audit: it is still concrete <code>ImplDatePort</code> in <code>ImplTransactionParserPort</code>, so behavior is unchanged.)</p>"},{"location":"lib/design/#container-buildcls-in-hexacontainerpy","title":"Container (<code>build(cls)</code> in <code>hexa/container.py</code>)","text":"<p>The annotations are metadata; a small reflection builder turns a root class into a wired instance:</p> <pre><code>build(AxisExtractionPipeline) \u2192 wired AxisExtractionPipeline\n</code></pre> <p>Algorithm (<code>build(cls)</code>):</p> <ol> <li>Walk <code>cls.__annotations__</code>, merged across the MRO so inherited slot annotations are visible (a bank subclass only adds/overrides a slot; the rest come from <code>Impl*</code>).</li> <li>For each port slot whose annotated type is a concrete <code>Impl*</code>/bank class (not an ABC, not a builtin), instantiate it.</li> <li>Recurse into that child \u2014 repeat until leaves (classes that declare no further port slots).</li> <li><code>setattr(parent, slot, child)</code> to wire each slot onto the parent instance.</li> <li>Apply config defaults from the class-level annotated attributes.</li> </ol> <p>Selection is fully static: the annotated type is the chosen implementation. The container holds no per-bank branch logic \u2014 <code>Axis</code> vs <code>Icici</code> is decided purely by which root class you call <code>build</code> on.</p>"},{"location":"lib/design/#control-flow-at-runtime-target","title":"Control flow at runtime (target)","text":"<p>A leaf-stage port exposes one entry method; stages chain data through the shared <code>run(source)</code> signature on the root:</p> <pre><code>run(source)\n ingest.materialize(source) \u2192 content\n raw_lines.extract(content) \u2192 raw_lines\n txn_blocks.extract(raw_lines) \u2192 {opening_balance: [blocks]}\n txn_dicts.extract(txn_blocks) \u2192 [txn dicts]\n raw_expense.build(txn_dicts) \u2192 [RawExpense] (terminal)\n</code></pre> <p>The stage ports map naturally onto dagpipe <code>Node</code>s (<code>ingest</code>, <code>raw_lines</code>, <code>txn_blocks</code>, <code>txn_dicts</code>, <code>raw_expense</code>), each node's <code>resolve</code> delegating to its wired port subtree, connected as a dagpipe <code>Graph</code> and run via <code>Engine</code>. That merge is a separate, future step.</p>"},{"location":"lib/design/#utilities-optional-hexa-agnostic","title":"Utilities (optional, hexa-agnostic)","text":"<p>The hexa utilities are implemented in <code>hexa/</code> (the package root) and they are agnostic of the extraction pipeline sample. It ships a set of optional code-gen / validation utilities so users can choose their own workflow \u2014 they are never required. A user is free to:</p> <ul> <li>hand-write an ABC file and never touch YAML, or</li> <li>start from YAML and generate the ABC file, or</li> <li>start from an ABC file and emit YAML for it, or</li> <li>use the ABC/YAML only as a spec (neither generated from the other).</li> </ul> <p>The utilities only ever translate between two interchangeable representations of the same contract \u2014 the <code>.yaml</code> spec and the <code>_abc.py</code> module. They never generate or manage the <code>Impl*</code>/bank files (those carry behavior and slot choice, which is a human decision).</p>"},{"location":"lib/design/#shared-model","title":"Shared model","text":"<p>All utilities serialize through one neutral model, a <code>PortNode</code> tree. It is NOT tied to the extraction sample \u2014 any port tree fits.</p> <pre><code>@dataclass\nclass PortNode:\n name: str # slot name, e.g. \"parser\", \"amount_balance\"\n kind: str # \"leaf\" | \"port\" | \"parent\"\n port_cls: str # ABC class name, e.g. \"TransactionParserPort\"\n attributes: dict[str, ConfigValue] # annotated attributes: name -> type expr,\n # or a nested struct dict for grouped values\n children: list[\"PortNode\"]\n methods: dict[str, MethodSpec] # abstract methods\n</code></pre>"},{"location":"lib/design/#1-parse_yamlpath-portnode","title":"1. <code>parse_yaml(path) -> PortNode</code>","text":"<p>Read a YAML spec into the <code>PortNode</code> tree (YAML \u2192 model).</p> <ul> <li>Parses completely (structural validation only, no type-checking).</li> <li>Supports the sample grammar (see <code>extraction_pipeline.yaml</code>): a root class whose inline attributes (scalars and nested structs), port slots, and reserved <code>methods:</code> recurse into child ports.</li> <li>Raises <code>ValueError</code> with file/line on structural errors.</li> </ul>"},{"location":"lib/design/#2-generate_abcpath_yaml-out_pathnone-str","title":"2. <code>generate_abc(path_yaml, out_path=None) -> str</code>","text":"<p>Generate <code>_abc.py</code> source from a YAML spec (YAML \u2192 ABC).</p> <ul> <li><code>parse_yaml(...)</code> \u2192 write one <code>class <PortCls>(ABC)</code> per <code>PortNode</code> with attribute annotations (nested structs become synthesized <code>@dataclass</code>es), port-slot annotations, and <code>@abstractmethod</code> stubs, matching the style of the hand-written <code>extraction_pipeline_abc.py</code>.</li> <li>Deterministic (YAML-order) output so it is idempotent / diff-friendly.</li> <li><code>out_path</code> optional; always returns the generated source.</li> </ul>"},{"location":"lib/design/#3-parse_abcmodule_or_path-portnode","title":"3. <code>parse_abc(module_or_path) -> PortNode</code>","text":"<p>Reflect over an ABC module into the <code>PortNode</code> tree (ABC \u2192 model).</p> <ul> <li>Reads <code>__annotations__</code> / <code>@abstractmethod</code> from generic ABC classes (no sample assumptions) to reconstruct the same <code>PortNode</code> tree.</li> </ul>"},{"location":"lib/design/#4-generate_yamlmodule_or_path-out_pathnone-str","title":"4. <code>generate_yaml(module_or_path, out_path=None) -> str</code>","text":"<p>Generate a <code>.yaml</code> spec from an ABC module (ABC \u2192 YAML).</p> <ul> <li><code>parse_abc(...)</code> \u2192 write the reverse of <code>generate_abc</code>.</li> <li>Lets a user author the ABC file first and emit YAML for it.</li> </ul>"},{"location":"lib/design/#5-check_matchespath_a-path_b-bool","title":"5. <code>check_matches(path_a, path_b) -> bool</code>","text":"<p>Verify two representations agree \u2014 YAML vs ABC, either direction.</p> <ul> <li>Builds both <code>PortNode</code> models (<code>parse_yaml</code> + <code>parse_abc</code>) and compares: classes exist and are <code>ABC</code>s, attributes match, port slots match, <code>@abstractmethod</code> signatures match (missing keyword-only params treated conservatively).</li> <li>Returns <code>True</code> only if the whole tree matches; otherwise <code>False</code> (or a diff).</li> </ul>"},{"location":"lib/design/#workflows-users-choice","title":"Workflows (user's choice)","text":"<pre><code># YAML-first\nedit spec.yaml\n \u2192 generate_abc(spec.yaml, abc.py) # regenerate ABCs\n \u2192 hand-write/update Impl* + banks\n \u2192 check_matches(spec.yaml, abc.py) == True # guard\n\n# ABC-first\nedit abc.py\n \u2192 generate_yaml(abc.py, spec.yaml) # emit YAML spec\n \u2192 check_matches(spec.yaml, abc.py) == True\n\n# hand-written both (no utilities at all)\nedit abc.py + spec.yaml independently # optional check_matches as a guard\n</code></pre> <p><code>Impl*</code>/bank files are always hand-extended on top and are never generated.</p>"},{"location":"lib/generate_abc/","title":"Generate Abc","text":""},{"location":"lib/generate_abc/#hexa.generate_abc","title":"hexa.generate_abc","text":"<p>Generate _abc.py source from a hexa PortNode tree.</p>"},{"location":"lib/generate_abc/#hexa.generate_abc-classes","title":"Classes","text":""},{"location":"lib/generate_abc/#hexa.generate_abc-functions","title":"Functions","text":""},{"location":"lib/generate_abc/#hexa.generate_abc.generate_abc","title":"generate_abc","text":"<pre><code>generate_abc(\n node: PortNode, out_path: str | Path | None = None\n) -> str\n</code></pre> <p>Generate ABC Python source from a :class:<code>PortNode</code> tree.</p>"},{"location":"lib/generate_abc/#hexa.generate_abc.generate_abc--parameters","title":"Parameters","text":"<p>node : PortNode The root of the parsed YAML (or generated) tree. out_path : str | Path | None Optional file path to write the generated source to.</p>"},{"location":"lib/generate_abc/#hexa.generate_abc.generate_abc--returns","title":"Returns","text":"<p>str The generated Python source code.</p>"},{"location":"lib/generate_yaml/","title":"Generate Yaml","text":""},{"location":"lib/generate_yaml/#hexa.generate_yaml","title":"hexa.generate_yaml","text":"<p>Generate YAML spec (nested mapping form) from a hexa PortNode tree.</p>"},{"location":"lib/generate_yaml/#hexa.generate_yaml-classes","title":"Classes","text":""},{"location":"lib/generate_yaml/#hexa.generate_yaml-functions","title":"Functions","text":""},{"location":"lib/generate_yaml/#hexa.generate_yaml.generate_yaml","title":"generate_yaml","text":"<pre><code>generate_yaml(\n node: PortNode, out_path: str | Path | None = None\n) -> str\n</code></pre> <p>Generate YAML source from a :class:<code>PortNode</code> tree.</p>"},{"location":"lib/generate_yaml/#hexa.generate_yaml.generate_yaml--parameters","title":"Parameters","text":"<p>node : PortNode The root of the tree. out_path : str | Path | None Optional file path to write the generated source to.</p>"},{"location":"lib/generate_yaml/#hexa.generate_yaml.generate_yaml--returns","title":"Returns","text":"<p>str The generated YAML source.</p>"},{"location":"lib/model/","title":"Model","text":""},{"location":"lib/model/#hexa.model","title":"hexa.model","text":"<p>Shared neutral model for hexa port trees.</p> <p>All utilities (<code>parse_yaml</code>, <code>generate_abc</code>, <code>parse_abc</code>, <code>generate_yaml</code>, <code>check_matches</code>) serialize through this single model. It is NOT tied to any sample \u2014 any port tree fits.</p>"},{"location":"lib/model/#hexa.model-classes","title":"Classes","text":""},{"location":"lib/model/#hexa.model.MethodSpec","title":"MethodSpec <code>dataclass</code>","text":"<pre><code>MethodSpec(\n name: str,\n args: list[str] = list(),\n kwargs: dict[str, str] = dict(),\n)\n</code></pre> <p>Specification of a single abstract method.</p>"},{"location":"lib/model/#hexa.model.PortNode","title":"PortNode <code>dataclass</code>","text":"<pre><code>PortNode(\n name: str,\n port_cls: str,\n kind: str = \"leaf\",\n attributes: dict[str, ConfigValue] = dict(),\n children: list[PortNode] = list(),\n methods: dict[str, MethodSpec] = dict(),\n)\n</code></pre> <p>A node in the hexa port tree.</p>"},{"location":"lib/model/#hexa.model.PortNode--parameters","title":"Parameters","text":"<p>name : str Slot name, e.g. <code>\"parser\"</code>, <code>\"amount_balance\"</code>. kind : str One of <code>\"leaf\"</code>, <code>\"port\"</code>, or <code>\"parent\"</code>. - <code>\"leaf\"</code>: no children (e.g. <code>NumberPort</code>) - <code>\"port\"</code>: has children (e.g. <code>TransactionParserPort</code>) - <code>\"parent\"</code>: the root of the tree (e.g. <code>ExtractionPipeline</code>) port_cls : str ABC class name that gives this port its shape, e.g. <code>\"TransactionParserPort\"</code>. Every port has a concrete typed shape. attributes : dict[str, ConfigValue] Annotated attributes: <code>name -> type_expr</code> for scalars, or a nested <code>dict</code> of sub-attributes (a struct) for grouped/structured values. children : list[PortNode] Nested port slots. methods : dict[str, MethodSpec] Abstract methods keyed by method name.</p>"},{"location":"lib/model/#hexa.model.PortNode-functions","title":"Functions","text":""},{"location":"lib/model/#hexa.model.PortNode.diff","title":"diff","text":"<pre><code>diff(other: PortNode, path: str = '') -> list[str]\n</code></pre> <p>Return a list of human-readable difference strings.</p> <p>An empty list means the trees are equal.</p>"},{"location":"lib/model/#hexa.model.PortNode.find","title":"find","text":"<pre><code>find(name: str) -> PortNode | None\n</code></pre> <p>Find a direct child by slot name.</p>"},{"location":"lib/model/#hexa.model.PortNode.walk","title":"walk","text":"<pre><code>walk() -> list[PortNode]\n</code></pre> <p>Return all nodes in depth-first order (self first).</p>"},{"location":"lib/parse_abc/","title":"Parse Abc","text":""},{"location":"lib/parse_abc/#hexa.parse_abc","title":"hexa.parse_abc","text":"<p>Parse an ABC Python module into a hexa PortNode tree.</p>"},{"location":"lib/parse_abc/#hexa.parse_abc-classes","title":"Classes","text":""},{"location":"lib/parse_abc/#hexa.parse_abc-functions","title":"Functions","text":""},{"location":"lib/parse_abc/#hexa.parse_abc.parse_abc","title":"parse_abc","text":"<pre><code>parse_abc(module_or_path: str | Path | Any) -> PortNode\n</code></pre> <p>Parse an ABC Python module into a :class:<code>PortNode</code> tree.</p> <p>ABC classes become port nodes. A nested-struct attribute is expressed as a <code>@dataclass</code> annotation on an ABC port (e.g. <code>source: ExtractionSource</code> where <code>ExtractionSource</code> is a <code>@dataclass</code>); the dataclass's fields are expanded into a nested <code>attributes</code> entry.</p>"},{"location":"lib/parse_abc/#hexa.parse_abc.parse_abc--parameters","title":"Parameters","text":"<p>module_or_path : str | Path | Any A file path to a <code>.py</code> file, a module import string, or a loaded module.</p>"},{"location":"lib/parse_abc/#hexa.parse_abc.parse_abc--returns","title":"Returns","text":"<p>PortNode The root node of the tree.</p>"},{"location":"lib/parse_yaml/","title":"Parse Yaml","text":""},{"location":"lib/parse_yaml/#hexa.parse_yaml","title":"hexa.parse_yaml","text":"<p>Parse a hexa YAML spec into a :class:<code>PortNode</code> tree.</p> <p>The YAML grammar uses a nested mapping format built from three member kinds:</p> <ul> <li>attribute: a <code>name: type</code> scalar (e.g. <code>source_type: str</code>)</li> <li>nested struct: a <code>name:</code> mapping of sub-attributes (e.g. <code>source:</code>)</li> <li>port: a <code>name:</code> mapping whose single key is a class whose value is a body, or a class key with a <code>...</code> body. A port carries a class shape.</li> </ul> <p>Example::</p> <pre><code>RootClassName:\n attr: type # scalar attribute\n source: # nested struct (all scalar sub-attributes)\n source_type: str\n bank: str\n methods: # reserved block of functions\n run:\n args: [a, b]\n ingest: # port (single non-scalar class key)\n IngestPort:\n source_type: str\n methods: {Method: ...}\n</code></pre> Disambiguation of a mapping value <p>A mapping with exactly one key whose value is a non-scalar body (a mapping, <code>...</code>, or a list) is a port. Any other mapping (multiple keys, or all scalar-typed values) is a nested struct of attributes.</p> <p>See <code>samples/minimal/sample.yaml</code> and <code>samples/extraction_pipeline/extraction_pipeline.yaml</code> for examples.</p>"},{"location":"lib/parse_yaml/#hexa.parse_yaml-classes","title":"Classes","text":""},{"location":"lib/parse_yaml/#hexa.parse_yaml-functions","title":"Functions","text":""},{"location":"lib/parse_yaml/#hexa.parse_yaml.parse_yaml","title":"parse_yaml","text":"<pre><code>parse_yaml(path: str | Path) -> PortNode\n</code></pre> <p>Parse a hexa YAML spec file into a :class:<code>PortNode</code> tree.</p>"},{"location":"lib/parse_yaml/#hexa.parse_yaml.parse_yaml--parameters","title":"Parameters","text":"<p>path : str | Path Path to the <code>.yaml</code> file.</p>"},{"location":"lib/parse_yaml/#hexa.parse_yaml.parse_yaml--returns","title":"Returns","text":"<p>PortNode The root node of the parsed tree.</p>"},{"location":"lib/parse_yaml/#hexa.parse_yaml.parse_yaml--raises","title":"Raises","text":"<p>ValueError On structural errors in the YAML.</p>"},{"location":"lib/philosophy/","title":"hexa \u2014 Philosophy","text":"<p>This document explains the why behind the type-declared dependency pattern used for the extraction pipeline sample \u2014 specifically the Axis/Icici bank variation. It is, together with design.md, self-sufficient: a fresh agent should be able to pick up the codebase from here and reason about (and extend) the pattern without needing the original session context.</p>"},{"location":"lib/philosophy/#one-idea","title":"One idea","text":"<p>A concrete class's annotated slot type is the dependency decision.</p> <p>Bank variation is expressed entirely as class annotations \u2014 never as imperative wiring, never as <code>__init__</code> parameters, never as a composition-root that assembles objects by hand. The tree of ports a pipeline needs is declared once, and a bank differs from the generic pipeline only by which concrete types are pinned onto which slots.</p>"},{"location":"lib/philosophy/#the-three-layers","title":"The three layers","text":"<p>The pattern is built from three fixed layers. Understanding which layer something belongs to is the whole mental model.</p>"},{"location":"lib/philosophy/#layer-1-abc-contracts-samplesextraction_pipelineextraction_pipeline_abcpy","title":"Layer 1 \u2014 ABC contracts (<code>samples/extraction_pipeline/extraction_pipeline_abc.py</code>)","text":"<p>Authoritative declaration of need (\"WHAT a stage requires\").</p> <ul> <li>Each port is an <code>ABC</code>.</li> <li>It declares attributes as class-level annotated defaults (e.g. <code>TransactionParserPort.min_numbers: int = 3</code>), optionally grouped as a nested struct via a <code>@dataclass</code> (e.g. <code>source: ExtractionSource</code>).</li> <li>It declares port slots as annotated attributes whose type is another ABC (e.g. <code>TransactionParserPort.number: NumberPort</code>, <code>amount_balance: AmountBalancePort</code>).</li> <li>It declares <code>@abstractmethod</code> bodies \u2014 the method signatures each concrete implementation must provide.</li> </ul> <p>The ABC never runs, never holds an instance, and never says which implementation to use. It only says what the shape is.</p>"},{"location":"lib/philosophy/#layer-2-generic-concurrent-implementations-samplesextraction_pipelineextraction_pipeline_implpy","title":"Layer 2 \u2014 generic concurrent implementations (<code>samples/extraction_pipeline/extraction_pipeline_impl.py</code>)","text":"<p>The common default behavior (\"Imple \u2014 the shared adaptor\").</p> <ul> <li><code>class ImplTransactionParserPort(TransactionParserPort)</code> subclasses the ABC, implements every <code>@abstractmethod</code>, and re-pins its port slots to concrete <code>Impl*</code> types: <code>number: ImplNumberPort</code>, <code>desc: ImplDescPort</code>, <code>amount_balance: ImplAmountBalancePort</code>.</li> <li><code>ImplAmountBalancePort</code> in turn pins <code>number: ImplNumberPort</code> and <code>ambiguity: ImplAmbiguityPort</code>.</li> <li>The root, <code>ImplExtractionPipeline</code>, pins its five stage slots to <code>Impl*</code> ports: <code>ingest</code>, <code>raw_lines</code>, <code>txn_blocks</code>, <code>txn_dicts</code>, <code>raw_expense</code>.</li> </ul> <p><code>Impl*</code> is the default that shared, bank-agnostic behavior lives in. Any slot a bank does not override falls back to <code>Impl*</code> at runtime.</p>"},{"location":"lib/philosophy/#layer-3-bank-specializations-samplesextraction_pipelinebanksaxisicicipdfpy-samplesextraction_pipelineextraction_pipeline_bankspy","title":"Layer 3 \u2014 bank specializations (<code>samples/extraction_pipeline/banks/{axis,icici}/pdf.py</code>, <code>samples/extraction_pipeline/extraction_pipeline_banks.py</code>)","text":"<p>The variation (\"what differs per bank\").</p> <ul> <li>Bank classes subclass the concrete <code>Impl*</code> classes, never the ABC \u2014 they extend, they never reinvent the contract.</li> <li>Each <code>Axis*</code>/<code>Icici*</code> class overrides only the members that differ from the generic <code>Impl*</code>; everything shared is inherited.</li> <li>A bank root is a thin annotated subclass of <code>ImplExtractionPipeline</code> that re-pins only the slots that differ:</li> </ul> <pre><code>class AxisExtractionPipeline(ImplExtractionPipeline):\n txn_dicts: AxisTxnDictsPort\n</code></pre>"},{"location":"lib/philosophy/#the-re-pinning-cascade","title":"The re-pinning cascade","text":"<p>Bank-specific wiring is hierarchical and cascading. Each level narrows exactly one slot, and its narrowed type drags the next level's narrowing along with it:</p> <pre><code>AxisExtractionPipeline\n \u2514\u2500 txn_dicts: AxisTxnDictsPort (only override on the root)\n \u2514\u2500 parser: AxisTransactionParserPort\n \u251c\u2500 number: AxisNumberPort (get_numbers \u2192 all clean nums)\n \u251c\u2500 desc: AxisDescPort (STARTTERS)\n \u2514\u2500 amount_balance: AxisAmountBalancePort\n \u2514\u2500 number: AxisNumberPort (re-pin: same bank's number port)\n</code></pre> <p>Siblings that are already correct in <code>Impl*</code> are inherited, not redeclared. E.g. the parser's <code>date</code> slot comes from <code>ImplTransactionParserPort</code>; the amount-balance port's <code>ambiguity</code> comes from <code>ImplAmountBalancePort</code>. Re-declaring them adds mental load on the extender with zero behavior change.</p> <p>Rule \u2014 only redeclare the slot you are actually changing. If a slot's behavior is already correct from <code>Impl*</code>, leave it out.</p>"},{"location":"lib/philosophy/#the-two-fidelity-guarantees-this-pattern-preserves","title":"The two fidelity guarantees this pattern preserves","text":"<ol> <li>Shared behavior stays shared. Slots like <code>ingest</code>, <code>raw_lines</code>, <code>txn_blocks</code>, <code>raw_expense</code>, parser <code>date</code>, and <code>ambiguity</code> are bank-agnostic. Leaving them at the <code>Impl*</code> default is deliberate, not an omission.</li> <li>Banks never reinvent. <code>Axis*</code>/<code>Icici*</code> only extend <code>Impl*</code>. The ABC contract in <code>samples/extraction_pipeline/extraction_pipeline_abc.py</code> is defined once and shared by every bank.</li> </ol>"},{"location":"lib/philosophy/#what-this-pattern-is-not","title":"What this pattern is NOT","text":"<ul> <li>No imperative assembly. No <code>build_axis_pipeline()</code> wiring objects by hand.</li> <li>No constructor injection. <code>Impl*</code> classes have no <code>__init__</code>; wiring is done by assigning concrete port instances onto annotated slots.</li> <li>YAML and ABCs are interchangeable, and both optional. The <code>.yaml</code> spec and the <code>_abc.py</code> module are two representations of the same contract. hexa is agnostic of the extraction pipeline sample and never imposes a workflow: a user can hand-write the ABC file, start from YAML (ABC generated), start from an ABC file (YAML emitted), or use both as a spec. The optional utilities in design.md \u00a7 Utilities translate between them and can verify they agree \u2014 but the bank <code>Impl*</code>/specialization files are always hand-written (their slots are a manual choice) and never generated.</li> </ul>"},{"location":"lib/philosophy/#runtime-instantiation","title":"Runtime instantiation","text":"<p>The annotations are the decision; <code>hexa.build(cls)</code> (see design.md \u00a7 Container) turns a root class into a wired instance by reading <code>__annotations__</code> + MRO, recursively instantiating each pinned concrete type, and <code>setattr</code>ing the child onto the parent slot. This document is scoped to the notation \u2014 the rest of the tree can be expressed purely as class annotations and filled at runtime by the container.</p>"}]} |