Vishesh 'ironeagle' Bangotra
7aadae1736
Enables dagpipe wiki on the docs hub alongside lib and mcp, removes stale nested lib pages, and picks up regenerated GSDFC docstrings in the MCP bundle.
1861 lines
121 KiB
JSON
1861 lines
121 KiB
JSON
{
|
||
"module": "dagpipe",
|
||
"content": {
|
||
"path": "dagpipe",
|
||
"docstring": "# Summary\n\nDirected acyclic graph execution framework for deterministic state propagation.\n\n`dagpipe` executes pipelines composed of nodes connected in a directed acyclic\ngraph (DAG). Each node receives an immutable `State` and optionally produces\nderived states for downstream nodes.\n\n# Installation\n\nInstall using pip:\n\n```bash\npip install dagpipe\n```\n\n---\n\n# Quick Start\n\n```python\nfrom dagpipe import State, Payload, Schema, Graph, Engine\nfrom dagpipe.node import Node\n\nclass HelloNode(Node):\n id = \"hello\"\n def resolve(self, state):\n yield self.fork(state, payload_update={\"msg\": \"hello\"})\n\n# Build and run\ngraph = Graph()\ngraph.add_root(HelloNode())\nengine = Engine(graph)\n\nclass MyState(State):\n schema = Schema({})\n\nresults = engine.run(MyState(payload=Payload({})))\n```\n\n---\n\n# Public API\n\nThis package re-exports the **core pipeline components**.\nConsumers should import from this namespace for standard usage.\n\n## Execution Core\n\n- `Engine`: Responsible for orchestrating node execution and state propagation.\n- `Graph`: Defines the execution topology and node relationships.\n- `Node`: Base class for defining execution logic and transformations.\n\n## State & Data\n\n- `State`: Represents an immutable execution snapshot at a point in time.\n- `Payload`: Immutable hierarchical container for execution data.\n- `Schema`: Defines and validates the allowed structure of payloads.\n- `SchemaError`: Raised when data violates the declared schema.\n\n## Declarative Pipelines\n\n- `Pipeline`: High-level wrapper for an engine, state type, and initial payload.\n- `load_pipeline`: Factory function to create a pipeline from YAML.\n\n---",
|
||
"objects": {
|
||
"State": {
|
||
"name": "State",
|
||
"kind": "class",
|
||
"path": "dagpipe.State",
|
||
"signature": "State(payload: Payload, confidence: float = 1.0, parent: State | None = None, depth: int = 0, history: tuple[str, ...] = tuple(), metadata: dict[str, Any] = dict())",
|
||
"docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.",
|
||
"members": {
|
||
"payload": {
|
||
"name": "payload",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.State.payload",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"schema": {
|
||
"name": "schema",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.State.schema",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"confidence": {
|
||
"name": "confidence",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.State.confidence",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"parent": {
|
||
"name": "parent",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.State.parent",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"depth": {
|
||
"name": "depth",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.State.depth",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"history": {
|
||
"name": "history",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.State.history",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"metadata": {
|
||
"name": "metadata",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.State.metadata",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.State.fork",
|
||
"signature": "fork(*, payload_update: Mapping[str, Any] | None = None, confidence_delta: float = 0.0, node_id: str | None = None, metadata_update: Mapping[str, Any] | None = None)",
|
||
"docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any] | None, optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str | None, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any] | None, optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history."
|
||
},
|
||
"lineage": {
|
||
"name": "lineage",
|
||
"kind": "function",
|
||
"path": "dagpipe.State.lineage",
|
||
"signature": "lineage()",
|
||
"docstring": "Return lineage from root to this State.\n\nReturns:\n tuple[State, ...]:\n Ordered execution lineage (root first)."
|
||
},
|
||
"get": {
|
||
"name": "get",
|
||
"kind": "function",
|
||
"path": "dagpipe.State.get",
|
||
"signature": "get(key: str, default: Any = None)",
|
||
"docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default."
|
||
},
|
||
"has": {
|
||
"name": "has",
|
||
"kind": "function",
|
||
"path": "dagpipe.State.has",
|
||
"signature": "has(key: str)",
|
||
"docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key."
|
||
}
|
||
}
|
||
},
|
||
"Payload": {
|
||
"name": "Payload",
|
||
"kind": "class",
|
||
"path": "dagpipe.Payload",
|
||
"signature": "Payload(_data: Mapping[str, Any])",
|
||
"docstring": "Immutable hierarchical container with dot-path access.\n\nAttributes:\n _data (Mapping[str, Any]):\n Immutable hierarchical data structure.\n\nNotes:\n **Responsibilities:**\n\n - Stores execution data used by `State`.\n - Supports efficient atomic updates without modifying existing instances.\n - `Payload` instances are fully thread-safe due to immutability.",
|
||
"members": {
|
||
"iter_paths": {
|
||
"name": "iter_paths",
|
||
"kind": "function",
|
||
"path": "dagpipe.Payload.iter_paths",
|
||
"signature": "iter_paths(data: Mapping[str, Any], prefix: str = '')",
|
||
"docstring": "Recursively yield dot-paths for all leaf nodes.\n\nArgs:\n data (Mapping[str, Any]):\n The mapping to iterate over.\n prefix (str, optional):\n Current path prefix.\n\nYields:\n str:\n Dot-path for each leaf node."
|
||
},
|
||
"get": {
|
||
"name": "get",
|
||
"kind": "function",
|
||
"path": "dagpipe.Payload.get",
|
||
"signature": "get(path: str, default: Any = None)",
|
||
"docstring": "Retrieve value using dot-path.\n\nArgs:\n path (str):\n Dot-separated path to the value.\n default (Any, optional):\n Default value if path doesn't exist.\n\nReturns:\n Any:\n The retrieved value or default."
|
||
},
|
||
"has": {
|
||
"name": "has",
|
||
"kind": "function",
|
||
"path": "dagpipe.Payload.has",
|
||
"signature": "has(path: str)",
|
||
"docstring": "Return True if path exists.\n\nArgs:\n path (str):\n Dot-separated path to check.\n\nReturns:\n bool:\n Existence of the path."
|
||
},
|
||
"update": {
|
||
"name": "update",
|
||
"kind": "function",
|
||
"path": "dagpipe.Payload.update",
|
||
"signature": "update(updates: Mapping[str, Any])",
|
||
"docstring": "Create a new `Payload` with dot-path updates applied.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path to value mapping.\n\nReturns:\n Payload:\n New immutable payload instance with updates.\n\nNotes:\n **Guarantees:**\n\n - Preserves existing data by copying only modified branches.\n - Returns a new immutable `Payload`."
|
||
},
|
||
"keys": {
|
||
"name": "keys",
|
||
"kind": "function",
|
||
"path": "dagpipe.Payload.keys",
|
||
"signature": "keys()",
|
||
"docstring": "Return top-level keys.\n\nReturns:\n Iterable[str]:\n Iterator over top-level keys."
|
||
},
|
||
"as_dict": {
|
||
"name": "as_dict",
|
||
"kind": "function",
|
||
"path": "dagpipe.Payload.as_dict",
|
||
"signature": "as_dict()",
|
||
"docstring": "Return underlying mapping.\n\nReturns:\n Mapping[str, Any]:\n Read-only view of the underlying data."
|
||
}
|
||
}
|
||
},
|
||
"Schema": {
|
||
"name": "Schema",
|
||
"kind": "class",
|
||
"path": "dagpipe.Schema",
|
||
"signature": "Schema(tree: Mapping[str, SchemaNode])",
|
||
"docstring": "Immutable hierarchical schema defining allowed payload structure.\n\nAttributes:\n tree (Mapping[str, SchemaNode]):\n Hierarchical schema definition.\n\nNotes:\n **Responsibilities:**\n\n - Validates `State` payloads and updates.\n - Reusable across all `State` instances.\n - Fully thread-safe due to immutability.",
|
||
"members": {
|
||
"tree": {
|
||
"name": "tree",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Schema.tree",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"validate_payload": {
|
||
"name": "validate_payload",
|
||
"kind": "function",
|
||
"path": "dagpipe.Schema.validate_payload",
|
||
"signature": "validate_payload(payload: Payload)",
|
||
"docstring": "Validate complete payload structure.\n\nArgs:\n payload (Payload):\n Payload to validate.\n\nRaises:\n SchemaError:\n If payload violates schema."
|
||
},
|
||
"validate_update": {
|
||
"name": "validate_update",
|
||
"kind": "function",
|
||
"path": "dagpipe.Schema.validate_update",
|
||
"signature": "validate_update(updates: Mapping[str, Any])",
|
||
"docstring": "Validate payload update paths.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path updates to validate.\n\nRaises:\n SchemaError:\n If any path is invalid according to the schema."
|
||
}
|
||
}
|
||
},
|
||
"SchemaError": {
|
||
"name": "SchemaError",
|
||
"kind": "class",
|
||
"path": "dagpipe.SchemaError",
|
||
"signature": null,
|
||
"docstring": "Raised when payload data violates the declared schema.\n\nIndicates invalid structure, invalid path, or invalid type."
|
||
},
|
||
"Node": {
|
||
"name": "Node",
|
||
"kind": "class",
|
||
"path": "dagpipe.Node",
|
||
"signature": null,
|
||
"docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.",
|
||
"members": {
|
||
"id": {
|
||
"name": "id",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Node.id",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"name": {
|
||
"name": "name",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Node.name",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"node_id_to_name": {
|
||
"name": "node_id_to_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.Node.node_id_to_name",
|
||
"signature": "node_id_to_name(node_id: str)",
|
||
"docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')."
|
||
},
|
||
"clean_id_and_name": {
|
||
"name": "clean_id_and_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.Node.clean_id_and_name",
|
||
"signature": "clean_id_and_name()",
|
||
"docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing."
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.Node.run",
|
||
"signature": "run(state: State)",
|
||
"docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object."
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.Node.fork",
|
||
"signature": "fork(state: State, *, payload_update: Any = None, confidence_delta: float = 0.0, metadata_update: Any = None)",
|
||
"docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Any, optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Any, optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history."
|
||
},
|
||
"resolve": {
|
||
"name": "resolve",
|
||
"kind": "function",
|
||
"path": "dagpipe.Node.resolve",
|
||
"signature": "resolve(state: State)",
|
||
"docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch."
|
||
},
|
||
"is_async": {
|
||
"name": "is_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.Node.is_async",
|
||
"signature": "is_async()",
|
||
"docstring": "Return whether this node executes asynchronously.\n\nReturns:\n bool:\n True if the node is an `AsyncNode` instance."
|
||
}
|
||
}
|
||
},
|
||
"AsyncNode": {
|
||
"name": "AsyncNode",
|
||
"kind": "class",
|
||
"path": "dagpipe.AsyncNode",
|
||
"signature": null,
|
||
"docstring": "Base class for nodes whose execution is asynchronous.\n\nSubclasses implement `resolve_async` (an async generator yielding derived\n`State` objects). The engine dispatches to `resolve_async` when running an\nasync traversal (see `Engine.run_async`).\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a no-op\nconsumer: calling `run` on an `AsyncNode` returns no states, signalling that\nan async engine is required.",
|
||
"members": {
|
||
"resolve": {
|
||
"name": "resolve",
|
||
"kind": "function",
|
||
"path": "dagpipe.AsyncNode.resolve",
|
||
"signature": "resolve(state: State)",
|
||
"docstring": "Execute no-op resolution in sync contexts.\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a\nno-op consumer: this returns no states, signalling that an async engine\nis required.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Empty tuple, since async execution is handled by\n `resolve_async`."
|
||
},
|
||
"resolve_async": {
|
||
"name": "resolve_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.AsyncNode.resolve_async",
|
||
"signature": "resolve_async(state: State)",
|
||
"docstring": "Execute node logic asynchronously.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n **Guarantees:**\n\n - Subclasses implement this.\n - Must not mutate the input state.\n - Should use `fork()` to create child states."
|
||
},
|
||
"run_async": {
|
||
"name": "run_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.AsyncNode.run_async",
|
||
"signature": "run_async(state: State)",
|
||
"docstring": "Execute this node asynchronously on a state, validating outputs.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve_async()` yields a non-`State` object."
|
||
}
|
||
}
|
||
},
|
||
"Graph": {
|
||
"name": "Graph",
|
||
"kind": "class",
|
||
"path": "dagpipe.Graph",
|
||
"signature": "Graph()",
|
||
"docstring": "Directed Acyclic Graph defining execution topology of `Node` objects.\n\nNotes:\n **Responsibilities:**\n\n - Stores node connectivity and validates that the topology remains acyclic.\n - Structure determines how `State` flows between nodes during execution.\n\n **Guarantees:**\n\n - Topology is acyclic. Node relationships remain consistent.\n - Thread-safe for concurrent reads after construction.",
|
||
"members": {
|
||
"add_edge": {
|
||
"name": "add_edge",
|
||
"kind": "function",
|
||
"path": "dagpipe.Graph.add_edge",
|
||
"signature": "add_edge(src: Node, dst: Node)",
|
||
"docstring": "Add a directed edge from `src` to `dst`.\n\nArgs:\n src (Node):\n Source node.\n\n dst (Node):\n Destination node.\n\nRaises:\n TypeError:\n If `src` or `dst` is not a `Node`.\n\n ValueError:\n If the edge would create a cycle or if `src` and `dst` are common.\n\nNotes:\n - Validates node types.\n - Prevents cycles.\n - Registers nodes if not present.\n - Updates parent and child mappings."
|
||
},
|
||
"add_root": {
|
||
"name": "add_root",
|
||
"kind": "function",
|
||
"path": "dagpipe.Graph.add_root",
|
||
"signature": "add_root(node: Node)",
|
||
"docstring": "Add a root node with no parents.\n\nArgs:\n node (Node):\n Node to add as a root.\n\nRaises:\n TypeError:\n If node is not a Node instance."
|
||
},
|
||
"children": {
|
||
"name": "children",
|
||
"kind": "function",
|
||
"path": "dagpipe.Graph.children",
|
||
"signature": "children(node: Node)",
|
||
"docstring": "Return child nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n tuple[Node, ...]:\n Outgoing neighbors."
|
||
},
|
||
"parents": {
|
||
"name": "parents",
|
||
"kind": "function",
|
||
"path": "dagpipe.Graph.parents",
|
||
"signature": "parents(node: Node)",
|
||
"docstring": "Return parent nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n tuple[Node, ...]:\n Incoming neighbors."
|
||
},
|
||
"roots": {
|
||
"name": "roots",
|
||
"kind": "function",
|
||
"path": "dagpipe.Graph.roots",
|
||
"signature": "roots()",
|
||
"docstring": "Return root nodes (nodes with no incoming edges).\n\nReturns:\n tuple[Node, ...]:\n Entry point nodes."
|
||
},
|
||
"nodes": {
|
||
"name": "nodes",
|
||
"kind": "function",
|
||
"path": "dagpipe.Graph.nodes",
|
||
"signature": "nodes()",
|
||
"docstring": "Return all nodes in the graph.\n\nReturns:\n tuple[Node, ...]:\n All registered nodes."
|
||
}
|
||
}
|
||
},
|
||
"Engine": {
|
||
"name": "Engine",
|
||
"kind": "class",
|
||
"path": "dagpipe.Engine",
|
||
"signature": "Engine(nodes_or_graph: Sequence[Node] | Graph, *, on_step: StepHook | None = None)",
|
||
"docstring": "Execution engine responsible for running pipeline logic.\n\nNotes:\n **Responsibilities:**\n\n - Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n - Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n - Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n - Supports step-wise / resumable execution and progress hooks.\n\n **Guarantees:**\n\n - Never mutates `State`, `Node`, or `Graph` instances.\n - `State` objects are never modified in place; each branch produces\n independent instances.\n - Execution order is deterministic and follows graph or pipeline topology.\n - Thread-safe for concurrent execution.",
|
||
"members": {
|
||
"MODE_LINEAR": {
|
||
"name": "MODE_LINEAR",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Engine.MODE_LINEAR",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"MODE_GRAPH": {
|
||
"name": "MODE_GRAPH",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Engine.MODE_GRAPH",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.Engine.run",
|
||
"signature": "run(root: State)",
|
||
"docstring": "Execute the pipeline starting from a root `State`.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nRaises:\n TypeError:\n If `root` is not a `State` instance.\n\n RuntimeError:\n If the engine execution mode is invalid.\n\nNotes:\n **Responsibilities:**\n\n - Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states."
|
||
},
|
||
"run_async": {
|
||
"name": "run_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.Engine.run_async",
|
||
"signature": "run_async(root: State)",
|
||
"docstring": "Execute the pipeline starting from `root`, dispatching sync vs async nodes.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nNotes:\n Each node is executed with `Node.run` when synchronous and\n `AsyncNode.run_async` when asynchronous. Linear and graph topologies\n are both supported."
|
||
},
|
||
"run_steps": {
|
||
"name": "run_steps",
|
||
"kind": "function",
|
||
"path": "dagpipe.Engine.run_steps",
|
||
"signature": "run_steps(root: State, *, resume_from: int | None = None, on_step: StepHook | None = None)",
|
||
"docstring": "Execute the pipeline step-by-step, yielding one `StepResult` per step.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int | None, optional):\n Skip steps at index < `resume_from` (for resume-after-partial).\n Steps are 0-indexed.\n\n on_step (StepHook | None, optional):\n Callback `(step, status, message)` invoked per step; falls back\n to the engine-level hook when unset.\n\nYields:\n StepResult:\n One per executed node/step, carrying the produced states.\n\nNotes:\n This is a synchronous, generator-based checkpoint interface compatible\n with the imperative resume-by-step behaviour of the legacy\n orchestrator. Use `run_steps_async` for async nodes."
|
||
},
|
||
"run_steps_async": {
|
||
"name": "run_steps_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.Engine.run_steps_async",
|
||
"signature": "run_steps_async(root: State, *, resume_from: int | None = None, on_step: AsyncStepHook | None = None)",
|
||
"docstring": "Async variant of `run_steps` supporting `AsyncNode` execution.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int | None, optional):\n Skip steps at index < `resume_from`.\n\n on_step (AsyncStepHook | None, optional):\n Async callback `(step, status, message)` invoked per step.\n\nYields:\n StepResult:\n One per executed node/step."
|
||
},
|
||
"nodes": {
|
||
"name": "nodes",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Engine.nodes",
|
||
"signature": null,
|
||
"docstring": "Return nodes managed by this engine.\n\nReturns:\n tuple[Node, ...]:\n Ordered sequence in linear mode or all nodes in graph mode."
|
||
}
|
||
}
|
||
},
|
||
"ProgressMessage": {
|
||
"name": "ProgressMessage",
|
||
"kind": "class",
|
||
"path": "dagpipe.ProgressMessage",
|
||
"signature": "ProgressMessage(*, lines: int | None = None, blocks: int | None = None, count: int | None = None, unit: str | None = None, raw_ocr_line: str | None = None, error: str | None = None, step: str = '', status: str = '')",
|
||
"docstring": "Lightweight progress payload emitted by engine step hooks.\n\nMirrors the imperative `ProgressMessage` used by the legacy orchestrator so\ncallers can surface counts/lines/errors without coupling the engine to pydantic.\n\nAttributes:\n lines (int | None):\n Optional count of processed lines.\n\n blocks (int | None):\n Optional count of processed blocks.\n\n count (int | None):\n Optional generic item count.\n\n unit (str | None):\n Optional unit for the count (e.g., 'pages').\n\n raw_ocr_line (str | None):\n Optional raw OCR line payload.\n\n error (str | None):\n Optional error description.\n\n step (str):\n Identifier of the step that emitted the message.\n\n status (str):\n Status label associated with the step.\n\nNotes:\n **Guarantees:**\n\n - Immutable after construction (attributes are never reassigned).\n - Independent of pydantic; safe to construct in the engine core.",
|
||
"members": {
|
||
"lines": {
|
||
"name": "lines",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.ProgressMessage.lines",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"blocks": {
|
||
"name": "blocks",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.ProgressMessage.blocks",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"count": {
|
||
"name": "count",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.ProgressMessage.count",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"unit": {
|
||
"name": "unit",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.ProgressMessage.unit",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"raw_ocr_line": {
|
||
"name": "raw_ocr_line",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.ProgressMessage.raw_ocr_line",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"error": {
|
||
"name": "error",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.ProgressMessage.error",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"step": {
|
||
"name": "step",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.ProgressMessage.step",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"status": {
|
||
"name": "status",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.ProgressMessage.status",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"as_dict": {
|
||
"name": "as_dict",
|
||
"kind": "function",
|
||
"path": "dagpipe.ProgressMessage.as_dict",
|
||
"signature": "as_dict()",
|
||
"docstring": "Return the message as a plain dictionary.\n\nReturns:\n dict[str, Any]:\n All attribute values keyed by their attribute name."
|
||
}
|
||
}
|
||
},
|
||
"StepResult": {
|
||
"name": "StepResult",
|
||
"kind": "class",
|
||
"path": "dagpipe.StepResult",
|
||
"signature": "StepResult(index: int, node_id: str, states: tuple[State, ...], completed: bool)",
|
||
"docstring": "A single checkpointed step within an async/resumable engine run.\n\nAttributes:\n index (int): Ordinal index of the step.\n node_id (str): Identifier of the node associated with this step.\n states (tuple[State, ...]): States produced by running this step.\n completed (bool): Whether this step succeeded (vs. paused/interrupted).",
|
||
"members": {
|
||
"index": {
|
||
"name": "index",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.StepResult.index",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"node_id": {
|
||
"name": "node_id",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.StepResult.node_id",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"states": {
|
||
"name": "states",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.StepResult.states",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"completed": {
|
||
"name": "completed",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.StepResult.completed",
|
||
"signature": null,
|
||
"docstring": null
|
||
}
|
||
}
|
||
},
|
||
"Pipeline": {
|
||
"name": "Pipeline",
|
||
"kind": "class",
|
||
"path": "dagpipe.Pipeline",
|
||
"signature": "Pipeline(engine: Engine, state_cls: type[State], initial_payload: Payload)",
|
||
"docstring": "Executable pipeline created from YAML configuration.\n\nAttributes:\n engine (Engine):\n Execution engine responsible for running the pipeline.\n\n state_cls (Type[State]):\n Dynamically created `State` subclass with configured schema.\n\n initial_payload (Payload):\n Default payload used when execution begins.\n\nNotes:\n **Responsibilities:**\n\n - Encapsulates engine, state type, and initial payload.\n - Provides a simplified interface for executing configured pipelines.\n - Safe for concurrent execution if underlying nodes are thread-safe.",
|
||
"members": {
|
||
"engine": {
|
||
"name": "engine",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Pipeline.engine",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"state_cls": {
|
||
"name": "state_cls",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Pipeline.state_cls",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"initial_payload": {
|
||
"name": "initial_payload",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.Pipeline.initial_payload",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.Pipeline.run",
|
||
"signature": "run(payload_override: Mapping[str, Any] | None = None)",
|
||
"docstring": "Execute the pipeline.\n\nArgs:\n payload_override (Mapping[str, Any] | None, optional):\n Payload values overriding initial payload.\n\nReturns:\n list[State]:\n Terminal execution states.\n\nNotes:\n **Responsibilities:**\n\n - Merges override payload with initial payload.\n - Creates root `State` and executes engine."
|
||
}
|
||
}
|
||
},
|
||
"load_pipeline": {
|
||
"name": "load_pipeline",
|
||
"kind": "function",
|
||
"path": "dagpipe.load_pipeline",
|
||
"signature": "load_pipeline(path: str)",
|
||
"docstring": "Load pipeline from YAML file.\n\nArgs:\n path (str):\n Path to YAML configuration file.\n\nReturns:\n Pipeline:\n Executable pipeline instance.\n\nNotes:\n **Responsibilities:**\n\n - Loads YAML configuration and builds schema.\n - Creates `State` subclass and loads `Node` instances.\n - Builds `Graph` topology and initializes `Engine`."
|
||
},
|
||
"engine": {
|
||
"name": "engine",
|
||
"kind": "module",
|
||
"path": "dagpipe.engine",
|
||
"signature": null,
|
||
"docstring": "# Summary\n\nExecution engine responsible for running pipelines and graphs.\n\nThe `Engine` executes `Node` objects and propagates immutable `State` instances\nthrough either a linear sequence or a directed acyclic graph (`Graph`).\nIt orchestrates execution order, branching, and state propagation.\n\n---\n\n# Guarantees\n\n- Deterministic execution and consistent state lineage.\n- Orchestrates execution without modifying `Graph`, `Node`, or `State` objects.",
|
||
"members": {
|
||
"Graph": {
|
||
"name": "Graph",
|
||
"kind": "class",
|
||
"path": "dagpipe.engine.Graph",
|
||
"signature": "Graph()",
|
||
"docstring": "Directed Acyclic Graph defining execution topology of `Node` objects.\n\nNotes:\n **Responsibilities:**\n\n - Stores node connectivity and validates that the topology remains acyclic.\n - Structure determines how `State` flows between nodes during execution.\n\n **Guarantees:**\n\n - Topology is acyclic. Node relationships remain consistent.\n - Thread-safe for concurrent reads after construction.",
|
||
"members": {
|
||
"add_edge": {
|
||
"name": "add_edge",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Graph.add_edge",
|
||
"signature": "add_edge(src: Node, dst: Node)",
|
||
"docstring": "Add a directed edge from `src` to `dst`.\n\nArgs:\n src (Node):\n Source node.\n\n dst (Node):\n Destination node.\n\nRaises:\n TypeError:\n If `src` or `dst` is not a `Node`.\n\n ValueError:\n If the edge would create a cycle or if `src` and `dst` are common.\n\nNotes:\n - Validates node types.\n - Prevents cycles.\n - Registers nodes if not present.\n - Updates parent and child mappings."
|
||
},
|
||
"add_root": {
|
||
"name": "add_root",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Graph.add_root",
|
||
"signature": "add_root(node: Node)",
|
||
"docstring": "Add a root node with no parents.\n\nArgs:\n node (Node):\n Node to add as a root.\n\nRaises:\n TypeError:\n If node is not a Node instance."
|
||
},
|
||
"children": {
|
||
"name": "children",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Graph.children",
|
||
"signature": "children(node: Node)",
|
||
"docstring": "Return child nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n tuple[Node, ...]:\n Outgoing neighbors."
|
||
},
|
||
"parents": {
|
||
"name": "parents",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Graph.parents",
|
||
"signature": "parents(node: Node)",
|
||
"docstring": "Return parent nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n tuple[Node, ...]:\n Incoming neighbors."
|
||
},
|
||
"roots": {
|
||
"name": "roots",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Graph.roots",
|
||
"signature": "roots()",
|
||
"docstring": "Return root nodes (nodes with no incoming edges).\n\nReturns:\n tuple[Node, ...]:\n Entry point nodes."
|
||
},
|
||
"nodes": {
|
||
"name": "nodes",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Graph.nodes",
|
||
"signature": "nodes()",
|
||
"docstring": "Return all nodes in the graph.\n\nReturns:\n tuple[Node, ...]:\n All registered nodes."
|
||
}
|
||
}
|
||
},
|
||
"AsyncNode": {
|
||
"name": "AsyncNode",
|
||
"kind": "class",
|
||
"path": "dagpipe.engine.AsyncNode",
|
||
"signature": null,
|
||
"docstring": "Base class for nodes whose execution is asynchronous.\n\nSubclasses implement `resolve_async` (an async generator yielding derived\n`State` objects). The engine dispatches to `resolve_async` when running an\nasync traversal (see `Engine.run_async`).\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a no-op\nconsumer: calling `run` on an `AsyncNode` returns no states, signalling that\nan async engine is required.",
|
||
"members": {
|
||
"resolve": {
|
||
"name": "resolve",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.AsyncNode.resolve",
|
||
"signature": "resolve(state: State)",
|
||
"docstring": "Execute no-op resolution in sync contexts.\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a\nno-op consumer: this returns no states, signalling that an async engine\nis required.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Empty tuple, since async execution is handled by\n `resolve_async`."
|
||
},
|
||
"resolve_async": {
|
||
"name": "resolve_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.AsyncNode.resolve_async",
|
||
"signature": "resolve_async(state: State)",
|
||
"docstring": "Execute node logic asynchronously.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n **Guarantees:**\n\n - Subclasses implement this.\n - Must not mutate the input state.\n - Should use `fork()` to create child states."
|
||
},
|
||
"run_async": {
|
||
"name": "run_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.AsyncNode.run_async",
|
||
"signature": "run_async(state: State)",
|
||
"docstring": "Execute this node asynchronously on a state, validating outputs.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve_async()` yields a non-`State` object."
|
||
}
|
||
}
|
||
},
|
||
"Node": {
|
||
"name": "Node",
|
||
"kind": "class",
|
||
"path": "dagpipe.engine.Node",
|
||
"signature": null,
|
||
"docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.",
|
||
"members": {
|
||
"id": {
|
||
"name": "id",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.Node.id",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"name": {
|
||
"name": "name",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.Node.name",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"node_id_to_name": {
|
||
"name": "node_id_to_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Node.node_id_to_name",
|
||
"signature": "node_id_to_name(node_id: str)",
|
||
"docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')."
|
||
},
|
||
"clean_id_and_name": {
|
||
"name": "clean_id_and_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Node.clean_id_and_name",
|
||
"signature": "clean_id_and_name()",
|
||
"docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing."
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Node.run",
|
||
"signature": "run(state: State)",
|
||
"docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object."
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Node.fork",
|
||
"signature": "fork(state: State, *, payload_update: Any = None, confidence_delta: float = 0.0, metadata_update: Any = None)",
|
||
"docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Any, optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Any, optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history."
|
||
},
|
||
"resolve": {
|
||
"name": "resolve",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Node.resolve",
|
||
"signature": "resolve(state: State)",
|
||
"docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch."
|
||
},
|
||
"is_async": {
|
||
"name": "is_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Node.is_async",
|
||
"signature": "is_async()",
|
||
"docstring": "Return whether this node executes asynchronously.\n\nReturns:\n bool:\n True if the node is an `AsyncNode` instance."
|
||
}
|
||
}
|
||
},
|
||
"State": {
|
||
"name": "State",
|
||
"kind": "class",
|
||
"path": "dagpipe.engine.State",
|
||
"signature": "State(payload: Payload, confidence: float = 1.0, parent: State | None = None, depth: int = 0, history: tuple[str, ...] = tuple(), metadata: dict[str, Any] = dict())",
|
||
"docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.",
|
||
"members": {
|
||
"payload": {
|
||
"name": "payload",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.State.payload",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"schema": {
|
||
"name": "schema",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.State.schema",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"confidence": {
|
||
"name": "confidence",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.State.confidence",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"parent": {
|
||
"name": "parent",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.State.parent",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"depth": {
|
||
"name": "depth",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.State.depth",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"history": {
|
||
"name": "history",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.State.history",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"metadata": {
|
||
"name": "metadata",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.State.metadata",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.State.fork",
|
||
"signature": "fork(*, payload_update: Mapping[str, Any] | None = None, confidence_delta: float = 0.0, node_id: str | None = None, metadata_update: Mapping[str, Any] | None = None)",
|
||
"docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any] | None, optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str | None, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any] | None, optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history."
|
||
},
|
||
"lineage": {
|
||
"name": "lineage",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.State.lineage",
|
||
"signature": "lineage()",
|
||
"docstring": "Return lineage from root to this State.\n\nReturns:\n tuple[State, ...]:\n Ordered execution lineage (root first)."
|
||
},
|
||
"get": {
|
||
"name": "get",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.State.get",
|
||
"signature": "get(key: str, default: Any = None)",
|
||
"docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default."
|
||
},
|
||
"has": {
|
||
"name": "has",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.State.has",
|
||
"signature": "has(key: str)",
|
||
"docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key."
|
||
}
|
||
}
|
||
},
|
||
"StepHook": {
|
||
"name": "StepHook",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.StepHook",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"AsyncStepHook": {
|
||
"name": "AsyncStepHook",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.AsyncStepHook",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"ProgressMessage": {
|
||
"name": "ProgressMessage",
|
||
"kind": "class",
|
||
"path": "dagpipe.engine.ProgressMessage",
|
||
"signature": "ProgressMessage(*, lines: int | None = None, blocks: int | None = None, count: int | None = None, unit: str | None = None, raw_ocr_line: str | None = None, error: str | None = None, step: str = '', status: str = '')",
|
||
"docstring": "Lightweight progress payload emitted by engine step hooks.\n\nMirrors the imperative `ProgressMessage` used by the legacy orchestrator so\ncallers can surface counts/lines/errors without coupling the engine to pydantic.\n\nAttributes:\n lines (int | None):\n Optional count of processed lines.\n\n blocks (int | None):\n Optional count of processed blocks.\n\n count (int | None):\n Optional generic item count.\n\n unit (str | None):\n Optional unit for the count (e.g., 'pages').\n\n raw_ocr_line (str | None):\n Optional raw OCR line payload.\n\n error (str | None):\n Optional error description.\n\n step (str):\n Identifier of the step that emitted the message.\n\n status (str):\n Status label associated with the step.\n\nNotes:\n **Guarantees:**\n\n - Immutable after construction (attributes are never reassigned).\n - Independent of pydantic; safe to construct in the engine core.",
|
||
"members": {
|
||
"lines": {
|
||
"name": "lines",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.ProgressMessage.lines",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"blocks": {
|
||
"name": "blocks",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.ProgressMessage.blocks",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"count": {
|
||
"name": "count",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.ProgressMessage.count",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"unit": {
|
||
"name": "unit",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.ProgressMessage.unit",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"raw_ocr_line": {
|
||
"name": "raw_ocr_line",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.ProgressMessage.raw_ocr_line",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"error": {
|
||
"name": "error",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.ProgressMessage.error",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"step": {
|
||
"name": "step",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.ProgressMessage.step",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"status": {
|
||
"name": "status",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.ProgressMessage.status",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"as_dict": {
|
||
"name": "as_dict",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.ProgressMessage.as_dict",
|
||
"signature": "as_dict() -> dict[str, Any]",
|
||
"docstring": "Return the message as a plain dictionary.\n\nReturns:\n dict[str, Any]:\n All attribute values keyed by their attribute name."
|
||
}
|
||
}
|
||
},
|
||
"StepResult": {
|
||
"name": "StepResult",
|
||
"kind": "class",
|
||
"path": "dagpipe.engine.StepResult",
|
||
"signature": "StepResult(index: int, node_id: str, states: tuple[State, ...], completed: bool)",
|
||
"docstring": "A single checkpointed step within an async/resumable engine run.\n\nAttributes:\n index (int): Ordinal index of the step.\n node_id (str): Identifier of the node associated with this step.\n states (tuple[State, ...]): States produced by running this step.\n completed (bool): Whether this step succeeded (vs. paused/interrupted).",
|
||
"members": {
|
||
"index": {
|
||
"name": "index",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.StepResult.index",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"node_id": {
|
||
"name": "node_id",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.StepResult.node_id",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"states": {
|
||
"name": "states",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.StepResult.states",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"completed": {
|
||
"name": "completed",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.StepResult.completed",
|
||
"signature": null,
|
||
"docstring": null
|
||
}
|
||
}
|
||
},
|
||
"Engine": {
|
||
"name": "Engine",
|
||
"kind": "class",
|
||
"path": "dagpipe.engine.Engine",
|
||
"signature": "Engine(nodes_or_graph: Sequence[Node] | Graph, *, on_step: StepHook | None = None)",
|
||
"docstring": "Execution engine responsible for running pipeline logic.\n\nNotes:\n **Responsibilities:**\n\n - Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n - Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n - Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n - Supports step-wise / resumable execution and progress hooks.\n\n **Guarantees:**\n\n - Never mutates `State`, `Node`, or `Graph` instances.\n - `State` objects are never modified in place; each branch produces\n independent instances.\n - Execution order is deterministic and follows graph or pipeline topology.\n - Thread-safe for concurrent execution.",
|
||
"members": {
|
||
"MODE_LINEAR": {
|
||
"name": "MODE_LINEAR",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.Engine.MODE_LINEAR",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"MODE_GRAPH": {
|
||
"name": "MODE_GRAPH",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.Engine.MODE_GRAPH",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Engine.run",
|
||
"signature": "run(root: State) -> list[State]",
|
||
"docstring": "Execute the pipeline starting from a root `State`.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nRaises:\n TypeError:\n If `root` is not a `State` instance.\n\n RuntimeError:\n If the engine execution mode is invalid.\n\nNotes:\n **Responsibilities:**\n\n - Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states."
|
||
},
|
||
"run_async": {
|
||
"name": "run_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Engine.run_async",
|
||
"signature": "run_async(root: State) -> list[State]",
|
||
"docstring": "Execute the pipeline starting from `root`, dispatching sync vs async nodes.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nNotes:\n Each node is executed with `Node.run` when synchronous and\n `AsyncNode.run_async` when asynchronous. Linear and graph topologies\n are both supported."
|
||
},
|
||
"run_steps": {
|
||
"name": "run_steps",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Engine.run_steps",
|
||
"signature": "run_steps(root: State, *, resume_from: int | None = None, on_step: StepHook | None = None) -> Iterator[StepResult]",
|
||
"docstring": "Execute the pipeline step-by-step, yielding one `StepResult` per step.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int | None, optional):\n Skip steps at index < `resume_from` (for resume-after-partial).\n Steps are 0-indexed.\n\n on_step (StepHook | None, optional):\n Callback `(step, status, message)` invoked per step; falls back\n to the engine-level hook when unset.\n\nYields:\n StepResult:\n One per executed node/step, carrying the produced states.\n\nNotes:\n This is a synchronous, generator-based checkpoint interface compatible\n with the imperative resume-by-step behaviour of the legacy\n orchestrator. Use `run_steps_async` for async nodes."
|
||
},
|
||
"run_steps_async": {
|
||
"name": "run_steps_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.engine.Engine.run_steps_async",
|
||
"signature": "run_steps_async(root: State, *, resume_from: int | None = None, on_step: AsyncStepHook | None = None) -> AsyncIterator[StepResult]",
|
||
"docstring": "Async variant of `run_steps` supporting `AsyncNode` execution.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int | None, optional):\n Skip steps at index < `resume_from`.\n\n on_step (AsyncStepHook | None, optional):\n Async callback `(step, status, message)` invoked per step.\n\nYields:\n StepResult:\n One per executed node/step."
|
||
},
|
||
"nodes": {
|
||
"name": "nodes",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.engine.Engine.nodes",
|
||
"signature": null,
|
||
"docstring": "Return nodes managed by this engine.\n\nReturns:\n tuple[Node, ...]:\n Ordered sequence in linear mode or all nodes in graph mode."
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"graph": {
|
||
"name": "graph",
|
||
"kind": "module",
|
||
"path": "dagpipe.graph",
|
||
"signature": null,
|
||
"docstring": "# Summary\n\nDefines DAG structure connecting nodes.\n\nA `Graph` describes execution topology only. It does not execute nodes or manage\n`State`. Execution is handled by an `Engine`.\n\n---\n\n# Responsibilities\n\n- Multiple roots, branching, and merging support.\n- Deterministic traversal based on topology.\n- Graph is mutable during construction but treated as immutable at runtime.",
|
||
"members": {
|
||
"Node": {
|
||
"name": "Node",
|
||
"kind": "class",
|
||
"path": "dagpipe.graph.Node",
|
||
"signature": null,
|
||
"docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.",
|
||
"members": {
|
||
"id": {
|
||
"name": "id",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.graph.Node.id",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"name": {
|
||
"name": "name",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.graph.Node.name",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"node_id_to_name": {
|
||
"name": "node_id_to_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Node.node_id_to_name",
|
||
"signature": "node_id_to_name(node_id: str)",
|
||
"docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')."
|
||
},
|
||
"clean_id_and_name": {
|
||
"name": "clean_id_and_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Node.clean_id_and_name",
|
||
"signature": "clean_id_and_name()",
|
||
"docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing."
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Node.run",
|
||
"signature": "run(state: State)",
|
||
"docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object."
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Node.fork",
|
||
"signature": "fork(state: State, *, payload_update: Any = None, confidence_delta: float = 0.0, metadata_update: Any = None)",
|
||
"docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Any, optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Any, optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history."
|
||
},
|
||
"resolve": {
|
||
"name": "resolve",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Node.resolve",
|
||
"signature": "resolve(state: State)",
|
||
"docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch."
|
||
},
|
||
"is_async": {
|
||
"name": "is_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Node.is_async",
|
||
"signature": "is_async()",
|
||
"docstring": "Return whether this node executes asynchronously.\n\nReturns:\n bool:\n True if the node is an `AsyncNode` instance."
|
||
}
|
||
}
|
||
},
|
||
"Graph": {
|
||
"name": "Graph",
|
||
"kind": "class",
|
||
"path": "dagpipe.graph.Graph",
|
||
"signature": "Graph()",
|
||
"docstring": "Directed Acyclic Graph defining execution topology of `Node` objects.\n\nNotes:\n **Responsibilities:**\n\n - Stores node connectivity and validates that the topology remains acyclic.\n - Structure determines how `State` flows between nodes during execution.\n\n **Guarantees:**\n\n - Topology is acyclic. Node relationships remain consistent.\n - Thread-safe for concurrent reads after construction.",
|
||
"members": {
|
||
"add_edge": {
|
||
"name": "add_edge",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Graph.add_edge",
|
||
"signature": "add_edge(src: Node, dst: Node) -> None",
|
||
"docstring": "Add a directed edge from `src` to `dst`.\n\nArgs:\n src (Node):\n Source node.\n\n dst (Node):\n Destination node.\n\nRaises:\n TypeError:\n If `src` or `dst` is not a `Node`.\n\n ValueError:\n If the edge would create a cycle or if `src` and `dst` are common.\n\nNotes:\n - Validates node types.\n - Prevents cycles.\n - Registers nodes if not present.\n - Updates parent and child mappings."
|
||
},
|
||
"add_root": {
|
||
"name": "add_root",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Graph.add_root",
|
||
"signature": "add_root(node: Node) -> None",
|
||
"docstring": "Add a root node with no parents.\n\nArgs:\n node (Node):\n Node to add as a root.\n\nRaises:\n TypeError:\n If node is not a Node instance."
|
||
},
|
||
"children": {
|
||
"name": "children",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Graph.children",
|
||
"signature": "children(node: Node) -> tuple[Node, ...]",
|
||
"docstring": "Return child nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n tuple[Node, ...]:\n Outgoing neighbors."
|
||
},
|
||
"parents": {
|
||
"name": "parents",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Graph.parents",
|
||
"signature": "parents(node: Node) -> tuple[Node, ...]",
|
||
"docstring": "Return parent nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n tuple[Node, ...]:\n Incoming neighbors."
|
||
},
|
||
"roots": {
|
||
"name": "roots",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Graph.roots",
|
||
"signature": "roots() -> tuple[Node, ...]",
|
||
"docstring": "Return root nodes (nodes with no incoming edges).\n\nReturns:\n tuple[Node, ...]:\n Entry point nodes."
|
||
},
|
||
"nodes": {
|
||
"name": "nodes",
|
||
"kind": "function",
|
||
"path": "dagpipe.graph.Graph.nodes",
|
||
"signature": "nodes() -> tuple[Node, ...]",
|
||
"docstring": "Return all nodes in the graph.\n\nReturns:\n tuple[Node, ...]:\n All registered nodes."
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"node": {
|
||
"name": "node",
|
||
"kind": "module",
|
||
"path": "dagpipe.node",
|
||
"signature": null,
|
||
"docstring": "# Summary\n\nDefines the `Node` abstraction used by `dagpipe`.\n\nA node represents a single unit of pipeline execution logic. It consumes one\n`State` and produces zero, one, or many new `State` objects.\n\nNodes are connected using a `Graph` and executed by an `Engine`.\n\n---\n\n# Design principles\n\n- **Pure:** Must not mutate input state.\n- **Deterministic:** Same input produces same output.\n- **Stateless:** Recommended to be stateless for reuse.\n- **Composable:** Nodes enable branching execution graphs.",
|
||
"members": {
|
||
"State": {
|
||
"name": "State",
|
||
"kind": "class",
|
||
"path": "dagpipe.node.State",
|
||
"signature": "State(payload: Payload, confidence: float = 1.0, parent: State | None = None, depth: int = 0, history: tuple[str, ...] = tuple(), metadata: dict[str, Any] = dict())",
|
||
"docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.",
|
||
"members": {
|
||
"payload": {
|
||
"name": "payload",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.State.payload",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"schema": {
|
||
"name": "schema",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.State.schema",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"confidence": {
|
||
"name": "confidence",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.State.confidence",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"parent": {
|
||
"name": "parent",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.State.parent",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"depth": {
|
||
"name": "depth",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.State.depth",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"history": {
|
||
"name": "history",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.State.history",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"metadata": {
|
||
"name": "metadata",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.State.metadata",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.State.fork",
|
||
"signature": "fork(*, payload_update: Mapping[str, Any] | None = None, confidence_delta: float = 0.0, node_id: str | None = None, metadata_update: Mapping[str, Any] | None = None)",
|
||
"docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any] | None, optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str | None, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any] | None, optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history."
|
||
},
|
||
"lineage": {
|
||
"name": "lineage",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.State.lineage",
|
||
"signature": "lineage()",
|
||
"docstring": "Return lineage from root to this State.\n\nReturns:\n tuple[State, ...]:\n Ordered execution lineage (root first)."
|
||
},
|
||
"get": {
|
||
"name": "get",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.State.get",
|
||
"signature": "get(key: str, default: Any = None)",
|
||
"docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default."
|
||
},
|
||
"has": {
|
||
"name": "has",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.State.has",
|
||
"signature": "has(key: str)",
|
||
"docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key."
|
||
}
|
||
}
|
||
},
|
||
"Node": {
|
||
"name": "Node",
|
||
"kind": "class",
|
||
"path": "dagpipe.node.Node",
|
||
"signature": null,
|
||
"docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.",
|
||
"members": {
|
||
"id": {
|
||
"name": "id",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.Node.id",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"name": {
|
||
"name": "name",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.node.Node.name",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"node_id_to_name": {
|
||
"name": "node_id_to_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.Node.node_id_to_name",
|
||
"signature": "node_id_to_name(node_id: str) -> str",
|
||
"docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')."
|
||
},
|
||
"clean_id_and_name": {
|
||
"name": "clean_id_and_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.Node.clean_id_and_name",
|
||
"signature": "clean_id_and_name() -> None",
|
||
"docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing."
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.Node.run",
|
||
"signature": "run(state: State) -> tuple[State, ...]",
|
||
"docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object."
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.Node.fork",
|
||
"signature": "fork(state: State, *, payload_update: Any = None, confidence_delta: float = 0.0, metadata_update: Any = None) -> State",
|
||
"docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Any, optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Any, optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history."
|
||
},
|
||
"resolve": {
|
||
"name": "resolve",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.Node.resolve",
|
||
"signature": "resolve(state: State) -> Iterable[State]",
|
||
"docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch."
|
||
},
|
||
"is_async": {
|
||
"name": "is_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.Node.is_async",
|
||
"signature": "is_async() -> bool",
|
||
"docstring": "Return whether this node executes asynchronously.\n\nReturns:\n bool:\n True if the node is an `AsyncNode` instance."
|
||
}
|
||
}
|
||
},
|
||
"AsyncNode": {
|
||
"name": "AsyncNode",
|
||
"kind": "class",
|
||
"path": "dagpipe.node.AsyncNode",
|
||
"signature": null,
|
||
"docstring": "Base class for nodes whose execution is asynchronous.\n\nSubclasses implement `resolve_async` (an async generator yielding derived\n`State` objects). The engine dispatches to `resolve_async` when running an\nasync traversal (see `Engine.run_async`).\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a no-op\nconsumer: calling `run` on an `AsyncNode` returns no states, signalling that\nan async engine is required.",
|
||
"members": {
|
||
"resolve": {
|
||
"name": "resolve",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.AsyncNode.resolve",
|
||
"signature": "resolve(state: State) -> Iterable[State]",
|
||
"docstring": "Execute no-op resolution in sync contexts.\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a\nno-op consumer: this returns no states, signalling that an async engine\nis required.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Empty tuple, since async execution is handled by\n `resolve_async`."
|
||
},
|
||
"resolve_async": {
|
||
"name": "resolve_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.AsyncNode.resolve_async",
|
||
"signature": "resolve_async(state: State) -> Iterable[State]",
|
||
"docstring": "Execute node logic asynchronously.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n **Guarantees:**\n\n - Subclasses implement this.\n - Must not mutate the input state.\n - Should use `fork()` to create child states."
|
||
},
|
||
"run_async": {
|
||
"name": "run_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.node.AsyncNode.run_async",
|
||
"signature": "run_async(state: State) -> tuple[State, ...]",
|
||
"docstring": "Execute this node asynchronously on a state, validating outputs.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve_async()` yields a non-`State` object."
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"state": {
|
||
"name": "state",
|
||
"kind": "module",
|
||
"path": "dagpipe.state",
|
||
"signature": null,
|
||
"docstring": "# Summary\n\nDefines the core `State` object used by `dagpipe`.\n\nThe `State` represents a single point in pipeline execution. It contains\narbitrary data and metadata and is designed to be immutable. Instead of\nmodifying an existing state, nodes create new child states via `fork()`.\n\n---\n\n# Design principles\n\n- **Immutability:** States must never be modified after creation.\n All transformations must create a new state via `fork()`.\n- **Cheap cloning:** Forking must be efficient since branching may create many states.\n- **Lineage tracking:** Each state maintains a reference to its parent and\n execution metadata for debugging and observability.\n- **Domain agnostic:** State contains generic key-value data and does not\n assume any schema.\n- **Engine-friendly:** State contains execution metadata such as depth and history.",
|
||
"members": {
|
||
"Payload": {
|
||
"name": "Payload",
|
||
"kind": "class",
|
||
"path": "dagpipe.state.Payload",
|
||
"signature": "Payload(_data: Mapping[str, Any])",
|
||
"docstring": "Immutable hierarchical container with dot-path access.\n\nAttributes:\n _data (Mapping[str, Any]):\n Immutable hierarchical data structure.\n\nNotes:\n **Responsibilities:**\n\n - Stores execution data used by `State`.\n - Supports efficient atomic updates without modifying existing instances.\n - `Payload` instances are fully thread-safe due to immutability.",
|
||
"members": {
|
||
"iter_paths": {
|
||
"name": "iter_paths",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.Payload.iter_paths",
|
||
"signature": "iter_paths(data: Mapping[str, Any], prefix: str = '') -> Iterable[str]",
|
||
"docstring": "Recursively yield dot-paths for all leaf nodes.\n\nArgs:\n data (Mapping[str, Any]):\n The mapping to iterate over.\n prefix (str, optional):\n Current path prefix.\n\nYields:\n str:\n Dot-path for each leaf node."
|
||
},
|
||
"get": {
|
||
"name": "get",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.Payload.get",
|
||
"signature": "get(path: str, default: Any = None) -> Any",
|
||
"docstring": "Retrieve value using dot-path.\n\nArgs:\n path (str):\n Dot-separated path to the value.\n default (Any, optional):\n Default value if path doesn't exist.\n\nReturns:\n Any:\n The retrieved value or default."
|
||
},
|
||
"has": {
|
||
"name": "has",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.Payload.has",
|
||
"signature": "has(path: str) -> bool",
|
||
"docstring": "Return True if path exists.\n\nArgs:\n path (str):\n Dot-separated path to check.\n\nReturns:\n bool:\n Existence of the path."
|
||
},
|
||
"update": {
|
||
"name": "update",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.Payload.update",
|
||
"signature": "update(updates: Mapping[str, Any]) -> Payload",
|
||
"docstring": "Create a new `Payload` with dot-path updates applied.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path to value mapping.\n\nReturns:\n Payload:\n New immutable payload instance with updates.\n\nNotes:\n **Guarantees:**\n\n - Preserves existing data by copying only modified branches.\n - Returns a new immutable `Payload`."
|
||
},
|
||
"keys": {
|
||
"name": "keys",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.Payload.keys",
|
||
"signature": "keys() -> Iterable[str]",
|
||
"docstring": "Return top-level keys.\n\nReturns:\n Iterable[str]:\n Iterator over top-level keys."
|
||
},
|
||
"as_dict": {
|
||
"name": "as_dict",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.Payload.as_dict",
|
||
"signature": "as_dict() -> Mapping[str, Any]",
|
||
"docstring": "Return underlying mapping.\n\nReturns:\n Mapping[str, Any]:\n Read-only view of the underlying data."
|
||
}
|
||
}
|
||
},
|
||
"SchemaNode": {
|
||
"name": "SchemaNode",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.SchemaNode",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"Schema": {
|
||
"name": "Schema",
|
||
"kind": "class",
|
||
"path": "dagpipe.state.Schema",
|
||
"signature": "Schema(tree: Mapping[str, SchemaNode])",
|
||
"docstring": "Immutable hierarchical schema defining allowed payload structure.\n\nAttributes:\n tree (Mapping[str, SchemaNode]):\n Hierarchical schema definition.\n\nNotes:\n **Responsibilities:**\n\n - Validates `State` payloads and updates.\n - Reusable across all `State` instances.\n - Fully thread-safe due to immutability.",
|
||
"members": {
|
||
"tree": {
|
||
"name": "tree",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.Schema.tree",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"validate_payload": {
|
||
"name": "validate_payload",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.Schema.validate_payload",
|
||
"signature": "validate_payload(payload: Payload) -> None",
|
||
"docstring": "Validate complete payload structure.\n\nArgs:\n payload (Payload):\n Payload to validate.\n\nRaises:\n SchemaError:\n If payload violates schema."
|
||
},
|
||
"validate_update": {
|
||
"name": "validate_update",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.Schema.validate_update",
|
||
"signature": "validate_update(updates: Mapping[str, Any]) -> None",
|
||
"docstring": "Validate payload update paths.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path updates to validate.\n\nRaises:\n SchemaError:\n If any path is invalid according to the schema."
|
||
}
|
||
}
|
||
},
|
||
"SchemaError": {
|
||
"name": "SchemaError",
|
||
"kind": "class",
|
||
"path": "dagpipe.state.SchemaError",
|
||
"signature": null,
|
||
"docstring": "Raised when payload data violates the declared schema.\n\nIndicates invalid structure, invalid path, or invalid type."
|
||
},
|
||
"State": {
|
||
"name": "State",
|
||
"kind": "class",
|
||
"path": "dagpipe.state.State",
|
||
"signature": "State(payload: Payload, confidence: float = 1.0, parent: State | None = None, depth: int = 0, history: tuple[str, ...] = tuple(), metadata: dict[str, Any] = dict())",
|
||
"docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.",
|
||
"members": {
|
||
"payload": {
|
||
"name": "payload",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.State.payload",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"schema": {
|
||
"name": "schema",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.State.schema",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"confidence": {
|
||
"name": "confidence",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.State.confidence",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"parent": {
|
||
"name": "parent",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.State.parent",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"depth": {
|
||
"name": "depth",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.State.depth",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"history": {
|
||
"name": "history",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.State.history",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"metadata": {
|
||
"name": "metadata",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.state.State.metadata",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.State.fork",
|
||
"signature": "fork(*, payload_update: Mapping[str, Any] | None = None, confidence_delta: float = 0.0, node_id: str | None = None, metadata_update: Mapping[str, Any] | None = None) -> State",
|
||
"docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any] | None, optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str | None, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any] | None, optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history."
|
||
},
|
||
"lineage": {
|
||
"name": "lineage",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.State.lineage",
|
||
"signature": "lineage() -> tuple[State, ...]",
|
||
"docstring": "Return lineage from root to this State.\n\nReturns:\n tuple[State, ...]:\n Ordered execution lineage (root first)."
|
||
},
|
||
"get": {
|
||
"name": "get",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.State.get",
|
||
"signature": "get(key: str, default: Any = None) -> Any",
|
||
"docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default."
|
||
},
|
||
"has": {
|
||
"name": "has",
|
||
"kind": "function",
|
||
"path": "dagpipe.state.State.has",
|
||
"signature": "has(key: str) -> bool",
|
||
"docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key."
|
||
}
|
||
}
|
||
}
|
||
}
|
||
},
|
||
"yaml_loader": {
|
||
"name": "yaml_loader",
|
||
"kind": "module",
|
||
"path": "dagpipe.yaml_loader",
|
||
"signature": null,
|
||
"docstring": "# Summary\n\nLoads dagpipe pipelines from YAML configuration.\n\nCreates fully configured pipeline objects from declarative YAML definitions,\nincluding `Schema`, `State` subclasses, `Node` instances, `Graph` topology,\nand initial payloads.",
|
||
"members": {
|
||
"Engine": {
|
||
"name": "Engine",
|
||
"kind": "class",
|
||
"path": "dagpipe.yaml_loader.Engine",
|
||
"signature": "Engine(nodes_or_graph: Sequence[Node] | Graph, *, on_step: StepHook | None = None)",
|
||
"docstring": "Execution engine responsible for running pipeline logic.\n\nNotes:\n **Responsibilities:**\n\n - Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n - Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n - Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n - Supports step-wise / resumable execution and progress hooks.\n\n **Guarantees:**\n\n - Never mutates `State`, `Node`, or `Graph` instances.\n - `State` objects are never modified in place; each branch produces\n independent instances.\n - Execution order is deterministic and follows graph or pipeline topology.\n - Thread-safe for concurrent execution.",
|
||
"members": {
|
||
"MODE_LINEAR": {
|
||
"name": "MODE_LINEAR",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Engine.MODE_LINEAR",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"MODE_GRAPH": {
|
||
"name": "MODE_GRAPH",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Engine.MODE_GRAPH",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Engine.run",
|
||
"signature": "run(root: State)",
|
||
"docstring": "Execute the pipeline starting from a root `State`.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nRaises:\n TypeError:\n If `root` is not a `State` instance.\n\n RuntimeError:\n If the engine execution mode is invalid.\n\nNotes:\n **Responsibilities:**\n\n - Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states."
|
||
},
|
||
"run_async": {
|
||
"name": "run_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Engine.run_async",
|
||
"signature": "run_async(root: State)",
|
||
"docstring": "Execute the pipeline starting from `root`, dispatching sync vs async nodes.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nNotes:\n Each node is executed with `Node.run` when synchronous and\n `AsyncNode.run_async` when asynchronous. Linear and graph topologies\n are both supported."
|
||
},
|
||
"run_steps": {
|
||
"name": "run_steps",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Engine.run_steps",
|
||
"signature": "run_steps(root: State, *, resume_from: int | None = None, on_step: StepHook | None = None)",
|
||
"docstring": "Execute the pipeline step-by-step, yielding one `StepResult` per step.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int | None, optional):\n Skip steps at index < `resume_from` (for resume-after-partial).\n Steps are 0-indexed.\n\n on_step (StepHook | None, optional):\n Callback `(step, status, message)` invoked per step; falls back\n to the engine-level hook when unset.\n\nYields:\n StepResult:\n One per executed node/step, carrying the produced states.\n\nNotes:\n This is a synchronous, generator-based checkpoint interface compatible\n with the imperative resume-by-step behaviour of the legacy\n orchestrator. Use `run_steps_async` for async nodes."
|
||
},
|
||
"run_steps_async": {
|
||
"name": "run_steps_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Engine.run_steps_async",
|
||
"signature": "run_steps_async(root: State, *, resume_from: int | None = None, on_step: AsyncStepHook | None = None)",
|
||
"docstring": "Async variant of `run_steps` supporting `AsyncNode` execution.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int | None, optional):\n Skip steps at index < `resume_from`.\n\n on_step (AsyncStepHook | None, optional):\n Async callback `(step, status, message)` invoked per step.\n\nYields:\n StepResult:\n One per executed node/step."
|
||
},
|
||
"nodes": {
|
||
"name": "nodes",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Engine.nodes",
|
||
"signature": null,
|
||
"docstring": "Return nodes managed by this engine.\n\nReturns:\n tuple[Node, ...]:\n Ordered sequence in linear mode or all nodes in graph mode."
|
||
}
|
||
}
|
||
},
|
||
"Graph": {
|
||
"name": "Graph",
|
||
"kind": "class",
|
||
"path": "dagpipe.yaml_loader.Graph",
|
||
"signature": "Graph()",
|
||
"docstring": "Directed Acyclic Graph defining execution topology of `Node` objects.\n\nNotes:\n **Responsibilities:**\n\n - Stores node connectivity and validates that the topology remains acyclic.\n - Structure determines how `State` flows between nodes during execution.\n\n **Guarantees:**\n\n - Topology is acyclic. Node relationships remain consistent.\n - Thread-safe for concurrent reads after construction.",
|
||
"members": {
|
||
"add_edge": {
|
||
"name": "add_edge",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Graph.add_edge",
|
||
"signature": "add_edge(src: Node, dst: Node)",
|
||
"docstring": "Add a directed edge from `src` to `dst`.\n\nArgs:\n src (Node):\n Source node.\n\n dst (Node):\n Destination node.\n\nRaises:\n TypeError:\n If `src` or `dst` is not a `Node`.\n\n ValueError:\n If the edge would create a cycle or if `src` and `dst` are common.\n\nNotes:\n - Validates node types.\n - Prevents cycles.\n - Registers nodes if not present.\n - Updates parent and child mappings."
|
||
},
|
||
"add_root": {
|
||
"name": "add_root",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Graph.add_root",
|
||
"signature": "add_root(node: Node)",
|
||
"docstring": "Add a root node with no parents.\n\nArgs:\n node (Node):\n Node to add as a root.\n\nRaises:\n TypeError:\n If node is not a Node instance."
|
||
},
|
||
"children": {
|
||
"name": "children",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Graph.children",
|
||
"signature": "children(node: Node)",
|
||
"docstring": "Return child nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n tuple[Node, ...]:\n Outgoing neighbors."
|
||
},
|
||
"parents": {
|
||
"name": "parents",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Graph.parents",
|
||
"signature": "parents(node: Node)",
|
||
"docstring": "Return parent nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n tuple[Node, ...]:\n Incoming neighbors."
|
||
},
|
||
"roots": {
|
||
"name": "roots",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Graph.roots",
|
||
"signature": "roots()",
|
||
"docstring": "Return root nodes (nodes with no incoming edges).\n\nReturns:\n tuple[Node, ...]:\n Entry point nodes."
|
||
},
|
||
"nodes": {
|
||
"name": "nodes",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Graph.nodes",
|
||
"signature": "nodes()",
|
||
"docstring": "Return all nodes in the graph.\n\nReturns:\n tuple[Node, ...]:\n All registered nodes."
|
||
}
|
||
}
|
||
},
|
||
"Node": {
|
||
"name": "Node",
|
||
"kind": "class",
|
||
"path": "dagpipe.yaml_loader.Node",
|
||
"signature": null,
|
||
"docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.",
|
||
"members": {
|
||
"id": {
|
||
"name": "id",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Node.id",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"name": {
|
||
"name": "name",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Node.name",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"node_id_to_name": {
|
||
"name": "node_id_to_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Node.node_id_to_name",
|
||
"signature": "node_id_to_name(node_id: str)",
|
||
"docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')."
|
||
},
|
||
"clean_id_and_name": {
|
||
"name": "clean_id_and_name",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Node.clean_id_and_name",
|
||
"signature": "clean_id_and_name()",
|
||
"docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing."
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Node.run",
|
||
"signature": "run(state: State)",
|
||
"docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object."
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Node.fork",
|
||
"signature": "fork(state: State, *, payload_update: Any = None, confidence_delta: float = 0.0, metadata_update: Any = None)",
|
||
"docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Any, optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Any, optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history."
|
||
},
|
||
"resolve": {
|
||
"name": "resolve",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Node.resolve",
|
||
"signature": "resolve(state: State)",
|
||
"docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch."
|
||
},
|
||
"is_async": {
|
||
"name": "is_async",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Node.is_async",
|
||
"signature": "is_async()",
|
||
"docstring": "Return whether this node executes asynchronously.\n\nReturns:\n bool:\n True if the node is an `AsyncNode` instance."
|
||
}
|
||
}
|
||
},
|
||
"Payload": {
|
||
"name": "Payload",
|
||
"kind": "class",
|
||
"path": "dagpipe.yaml_loader.Payload",
|
||
"signature": "Payload(_data: Mapping[str, Any])",
|
||
"docstring": "Immutable hierarchical container with dot-path access.\n\nAttributes:\n _data (Mapping[str, Any]):\n Immutable hierarchical data structure.\n\nNotes:\n **Responsibilities:**\n\n - Stores execution data used by `State`.\n - Supports efficient atomic updates without modifying existing instances.\n - `Payload` instances are fully thread-safe due to immutability.",
|
||
"members": {
|
||
"iter_paths": {
|
||
"name": "iter_paths",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Payload.iter_paths",
|
||
"signature": "iter_paths(data: Mapping[str, Any], prefix: str = '')",
|
||
"docstring": "Recursively yield dot-paths for all leaf nodes.\n\nArgs:\n data (Mapping[str, Any]):\n The mapping to iterate over.\n prefix (str, optional):\n Current path prefix.\n\nYields:\n str:\n Dot-path for each leaf node."
|
||
},
|
||
"get": {
|
||
"name": "get",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Payload.get",
|
||
"signature": "get(path: str, default: Any = None)",
|
||
"docstring": "Retrieve value using dot-path.\n\nArgs:\n path (str):\n Dot-separated path to the value.\n default (Any, optional):\n Default value if path doesn't exist.\n\nReturns:\n Any:\n The retrieved value or default."
|
||
},
|
||
"has": {
|
||
"name": "has",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Payload.has",
|
||
"signature": "has(path: str)",
|
||
"docstring": "Return True if path exists.\n\nArgs:\n path (str):\n Dot-separated path to check.\n\nReturns:\n bool:\n Existence of the path."
|
||
},
|
||
"update": {
|
||
"name": "update",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Payload.update",
|
||
"signature": "update(updates: Mapping[str, Any])",
|
||
"docstring": "Create a new `Payload` with dot-path updates applied.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path to value mapping.\n\nReturns:\n Payload:\n New immutable payload instance with updates.\n\nNotes:\n **Guarantees:**\n\n - Preserves existing data by copying only modified branches.\n - Returns a new immutable `Payload`."
|
||
},
|
||
"keys": {
|
||
"name": "keys",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Payload.keys",
|
||
"signature": "keys()",
|
||
"docstring": "Return top-level keys.\n\nReturns:\n Iterable[str]:\n Iterator over top-level keys."
|
||
},
|
||
"as_dict": {
|
||
"name": "as_dict",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Payload.as_dict",
|
||
"signature": "as_dict()",
|
||
"docstring": "Return underlying mapping.\n\nReturns:\n Mapping[str, Any]:\n Read-only view of the underlying data."
|
||
}
|
||
}
|
||
},
|
||
"Schema": {
|
||
"name": "Schema",
|
||
"kind": "class",
|
||
"path": "dagpipe.yaml_loader.Schema",
|
||
"signature": "Schema(tree: Mapping[str, SchemaNode])",
|
||
"docstring": "Immutable hierarchical schema defining allowed payload structure.\n\nAttributes:\n tree (Mapping[str, SchemaNode]):\n Hierarchical schema definition.\n\nNotes:\n **Responsibilities:**\n\n - Validates `State` payloads and updates.\n - Reusable across all `State` instances.\n - Fully thread-safe due to immutability.",
|
||
"members": {
|
||
"tree": {
|
||
"name": "tree",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Schema.tree",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"validate_payload": {
|
||
"name": "validate_payload",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Schema.validate_payload",
|
||
"signature": "validate_payload(payload: Payload)",
|
||
"docstring": "Validate complete payload structure.\n\nArgs:\n payload (Payload):\n Payload to validate.\n\nRaises:\n SchemaError:\n If payload violates schema."
|
||
},
|
||
"validate_update": {
|
||
"name": "validate_update",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Schema.validate_update",
|
||
"signature": "validate_update(updates: Mapping[str, Any])",
|
||
"docstring": "Validate payload update paths.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path updates to validate.\n\nRaises:\n SchemaError:\n If any path is invalid according to the schema."
|
||
}
|
||
}
|
||
},
|
||
"State": {
|
||
"name": "State",
|
||
"kind": "class",
|
||
"path": "dagpipe.yaml_loader.State",
|
||
"signature": "State(payload: Payload, confidence: float = 1.0, parent: State | None = None, depth: int = 0, history: tuple[str, ...] = tuple(), metadata: dict[str, Any] = dict())",
|
||
"docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.",
|
||
"members": {
|
||
"payload": {
|
||
"name": "payload",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.State.payload",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"schema": {
|
||
"name": "schema",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.State.schema",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"confidence": {
|
||
"name": "confidence",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.State.confidence",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"parent": {
|
||
"name": "parent",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.State.parent",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"depth": {
|
||
"name": "depth",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.State.depth",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"history": {
|
||
"name": "history",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.State.history",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"metadata": {
|
||
"name": "metadata",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.State.metadata",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"fork": {
|
||
"name": "fork",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.State.fork",
|
||
"signature": "fork(*, payload_update: Mapping[str, Any] | None = None, confidence_delta: float = 0.0, node_id: str | None = None, metadata_update: Mapping[str, Any] | None = None)",
|
||
"docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any] | None, optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str | None, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any] | None, optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history."
|
||
},
|
||
"lineage": {
|
||
"name": "lineage",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.State.lineage",
|
||
"signature": "lineage()",
|
||
"docstring": "Return lineage from root to this State.\n\nReturns:\n tuple[State, ...]:\n Ordered execution lineage (root first)."
|
||
},
|
||
"get": {
|
||
"name": "get",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.State.get",
|
||
"signature": "get(key: str, default: Any = None)",
|
||
"docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default."
|
||
},
|
||
"has": {
|
||
"name": "has",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.State.has",
|
||
"signature": "has(key: str)",
|
||
"docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key."
|
||
}
|
||
}
|
||
},
|
||
"Pipeline": {
|
||
"name": "Pipeline",
|
||
"kind": "class",
|
||
"path": "dagpipe.yaml_loader.Pipeline",
|
||
"signature": "Pipeline(engine: Engine, state_cls: type[State], initial_payload: Payload)",
|
||
"docstring": "Executable pipeline created from YAML configuration.\n\nAttributes:\n engine (Engine):\n Execution engine responsible for running the pipeline.\n\n state_cls (Type[State]):\n Dynamically created `State` subclass with configured schema.\n\n initial_payload (Payload):\n Default payload used when execution begins.\n\nNotes:\n **Responsibilities:**\n\n - Encapsulates engine, state type, and initial payload.\n - Provides a simplified interface for executing configured pipelines.\n - Safe for concurrent execution if underlying nodes are thread-safe.",
|
||
"members": {
|
||
"engine": {
|
||
"name": "engine",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Pipeline.engine",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"state_cls": {
|
||
"name": "state_cls",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Pipeline.state_cls",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"initial_payload": {
|
||
"name": "initial_payload",
|
||
"kind": "attribute",
|
||
"path": "dagpipe.yaml_loader.Pipeline.initial_payload",
|
||
"signature": null,
|
||
"docstring": null
|
||
},
|
||
"run": {
|
||
"name": "run",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.Pipeline.run",
|
||
"signature": "run(payload_override: Mapping[str, Any] | None = None) -> list[State]",
|
||
"docstring": "Execute the pipeline.\n\nArgs:\n payload_override (Mapping[str, Any] | None, optional):\n Payload values overriding initial payload.\n\nReturns:\n list[State]:\n Terminal execution states.\n\nNotes:\n **Responsibilities:**\n\n - Merges override payload with initial payload.\n - Creates root `State` and executes engine."
|
||
}
|
||
}
|
||
},
|
||
"load_pipeline": {
|
||
"name": "load_pipeline",
|
||
"kind": "function",
|
||
"path": "dagpipe.yaml_loader.load_pipeline",
|
||
"signature": "load_pipeline(path: str) -> Pipeline",
|
||
"docstring": "Load pipeline from YAML file.\n\nArgs:\n path (str):\n Path to YAML configuration file.\n\nReturns:\n Pipeline:\n Executable pipeline instance.\n\nNotes:\n **Responsibilities:**\n\n - Loads YAML configuration and builds schema.\n - Creates `State` subclass and loads `Node` instances.\n - Builds `Graph` topology and initializes `Engine`."
|
||
}
|
||
}
|
||
}
|
||
}
|
||
}
|
||
} |