{ "module": "dagpipe", "content": { "path": "dagpipe", "docstring": "# Summary\n\nDirected acyclic graph execution framework for deterministic state propagation.\n\n`dagpipe` executes pipelines composed of nodes connected in a directed acyclic\ngraph (DAG). Each node receives an immutable `State` and optionally produces\nderived states for downstream nodes.\n\n# Installation\n\nInstall using pip:\n\n```bash\npip install dagpipe\n```\n\n---\n\n# Quick Start\n\n```python\nfrom dagpipe import State, Payload, Schema, Graph, Engine\nfrom dagpipe.node import Node\n\nclass HelloNode(Node):\n id = \"hello\"\n def resolve(self, state):\n yield self.fork(state, payload_update={\"msg\": \"hello\"})\n\n# Build and run\ngraph = Graph()\ngraph.add_root(HelloNode())\nengine = Engine(graph)\n\nclass MyState(State):\n schema = Schema({})\n\nresults = engine.run(MyState(payload=Payload({})))\n```\n\n---\n\n# Public API\n\nThis package re-exports the **core pipeline components**.\nConsumers should import from this namespace for standard usage.\n\n## Execution Core\n\n- `Engine`: Responsible for orchestrating node execution and state propagation.\n- `Graph`: Defines the execution topology and node relationships.\n- `Node`: Base class for defining execution logic and transformations.\n\n## State & Data\n\n- `State`: Represents an immutable execution snapshot at a point in time.\n- `Payload`: Immutable hierarchical container for execution data.\n- `Schema`: Defines and validates the allowed structure of payloads.\n- `SchemaError`: Raised when data violates the declared schema.\n\n## Declarative Pipelines\n\n- `Pipeline`: High-level wrapper for an engine, state type, and initial payload.\n- `load_pipeline`: Factory function to create a pipeline from YAML.\n\n---", "objects": { "State": { "name": "State", "kind": "class", "path": "dagpipe.State", "signature": "", "docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.", "members": { "payload": { "name": "payload", "kind": "attribute", "path": "dagpipe.State.payload", "signature": "", "docstring": null }, "schema": { "name": "schema", "kind": "attribute", "path": "dagpipe.State.schema", "signature": "", "docstring": null }, "confidence": { "name": "confidence", "kind": "attribute", "path": "dagpipe.State.confidence", "signature": "", "docstring": null }, "parent": { "name": "parent", "kind": "attribute", "path": "dagpipe.State.parent", "signature": "", "docstring": null }, "depth": { "name": "depth", "kind": "attribute", "path": "dagpipe.State.depth", "signature": "", "docstring": null }, "history": { "name": "history", "kind": "attribute", "path": "dagpipe.State.history", "signature": "", "docstring": null }, "metadata": { "name": "metadata", "kind": "attribute", "path": "dagpipe.State.metadata", "signature": "", "docstring": null }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.State.fork", "signature": "", "docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any], optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any], optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history." }, "lineage": { "name": "lineage", "kind": "function", "path": "dagpipe.State.lineage", "signature": "", "docstring": "Return lineage from root to this State.\n\nReturns:\n Tuple[State, ...]:\n Ordered execution lineage (root first)." }, "get": { "name": "get", "kind": "function", "path": "dagpipe.State.get", "signature": "", "docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default." }, "has": { "name": "has", "kind": "function", "path": "dagpipe.State.has", "signature": "", "docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key." } } }, "Payload": { "name": "Payload", "kind": "class", "path": "dagpipe.Payload", "signature": "", "docstring": "Immutable hierarchical container with dot-path access.\n\nAttributes:\n _data (Mapping[str, Any]):\n Immutable hierarchical data structure.\n\nNotes:\n **Responsibilities:**\n\n - Stores execution data used by `State`.\n - Supports efficient atomic updates without modifying existing instances.\n - `Payload` instances are fully thread-safe due to immutability.", "members": { "iter_paths": { "name": "iter_paths", "kind": "function", "path": "dagpipe.Payload.iter_paths", "signature": "", "docstring": "Recursively yield dot-paths for all leaf nodes.\n\nArgs:\n data (Mapping[str, Any]):\n The mapping to iterate over.\n prefix (str, optional):\n Current path prefix.\n\nReturns:\n Iterable[str]:\n Generator yielding dot-paths." }, "get": { "name": "get", "kind": "function", "path": "dagpipe.Payload.get", "signature": "", "docstring": "Retrieve value using dot-path.\n\nArgs:\n path (str):\n Dot-separated path to the value.\n default (Any, optional):\n Default value if path doesn't exist.\n\nReturns:\n Any:\n The retrieved value or default." }, "has": { "name": "has", "kind": "function", "path": "dagpipe.Payload.has", "signature": "", "docstring": "Return True if path exists.\n\nArgs:\n path (str):\n Dot-separated path to check.\n\nReturns:\n bool:\n Existence of the path." }, "update": { "name": "update", "kind": "function", "path": "dagpipe.Payload.update", "signature": "", "docstring": "Create a new `Payload` with dot-path updates applied.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path to value mapping.\n\nReturns:\n Payload:\n New immutable payload instance with updates.\n\nNotes:\n **Guarantees:**\n\n - Preserves existing data by copying only modified branches.\n - Returns a new immutable `Payload`." }, "keys": { "name": "keys", "kind": "function", "path": "dagpipe.Payload.keys", "signature": "", "docstring": "Return top-level keys.\n\nReturns:\n Iterable[str]:\n Iterator over top-level keys." }, "as_dict": { "name": "as_dict", "kind": "function", "path": "dagpipe.Payload.as_dict", "signature": "", "docstring": "Return underlying mapping.\n\nReturns:\n Mapping[str, Any]:\n Read-only view of the underlying data." } } }, "Schema": { "name": "Schema", "kind": "class", "path": "dagpipe.Schema", "signature": "", "docstring": "Immutable hierarchical schema defining allowed payload structure.\n\nAttributes:\n tree (Mapping[str, SchemaNode]):\n Hierarchical schema definition.\n\nNotes:\n **Responsibilities:**\n\n - Validates `State` payloads and updates.\n - Reusable across all `State` instances.\n - Fully thread-safe due to immutability.", "members": { "tree": { "name": "tree", "kind": "attribute", "path": "dagpipe.Schema.tree", "signature": "", "docstring": null }, "validate_payload": { "name": "validate_payload", "kind": "function", "path": "dagpipe.Schema.validate_payload", "signature": "", "docstring": "Validate complete payload structure.\n\nArgs:\n payload (Payload):\n Payload to validate.\n\nRaises:\n SchemaError:\n If payload violates schema." }, "validate_update": { "name": "validate_update", "kind": "function", "path": "dagpipe.Schema.validate_update", "signature": "", "docstring": "Validate payload update paths.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path updates to validate.\n\nRaises:\n SchemaError:\n If any path is invalid according to the schema." } } }, "SchemaError": { "name": "SchemaError", "kind": "class", "path": "dagpipe.SchemaError", "signature": "", "docstring": "Raised when payload data violates the declared schema.\n\nIndicates invalid structure, invalid path, or invalid type.\n---" }, "Node": { "name": "Node", "kind": "class", "path": "dagpipe.Node", "signature": "", "docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.", "members": { "id": { "name": "id", "kind": "attribute", "path": "dagpipe.Node.id", "signature": "", "docstring": null }, "name": { "name": "name", "kind": "attribute", "path": "dagpipe.Node.name", "signature": "", "docstring": null }, "node_id_to_name": { "name": "node_id_to_name", "kind": "function", "path": "dagpipe.Node.node_id_to_name", "signature": "", "docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')." }, "clean_id_and_name": { "name": "clean_id_and_name", "kind": "function", "path": "dagpipe.Node.clean_id_and_name", "signature": "", "docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing." }, "run": { "name": "run", "kind": "function", "path": "dagpipe.Node.run", "signature": "", "docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object." }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.Node.fork", "signature": "", "docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Mapping[str, Any], optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Mapping[str, Any], optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history." }, "resolve": { "name": "resolve", "kind": "function", "path": "dagpipe.Node.resolve", "signature": "", "docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nYields:\n State:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch." }, "is_async": { "name": "is_async", "kind": "function", "path": "dagpipe.Node.is_async", "signature": "", "docstring": "Return whether this node executes asynchronously." } } }, "AsyncNode": { "name": "AsyncNode", "kind": "class", "path": "dagpipe.AsyncNode", "signature": "", "docstring": "Base class for nodes whose execution is asynchronous.\n\nSubclasses implement `resolve_async` (an async generator yielding derived\n`State` objects). The engine dispatches to `resolve_async` when running an\nasync traversal (see `Engine.run_async`).\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a no-op\nconsumer: calling `run` on an `AsyncNode` returns no states, signalling that\nan async engine is required.", "members": { "resolve": { "name": "resolve", "kind": "function", "path": "dagpipe.AsyncNode.resolve", "signature": "", "docstring": null }, "resolve_async": { "name": "resolve_async", "kind": "function", "path": "dagpipe.AsyncNode.resolve_async", "signature": "", "docstring": "Execute node logic asynchronously.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n Subclasses implement this. Must not mutate the input state.\n Should use `fork()` to create child states." }, "run_async": { "name": "run_async", "kind": "function", "path": "dagpipe.AsyncNode.run_async", "signature": "", "docstring": "Execute this node asynchronously on a state, validating outputs." } } }, "Graph": { "name": "Graph", "kind": "class", "path": "dagpipe.Graph", "signature": "", "docstring": "Directed Acyclic Graph defining execution topology of `Node` objects.\n\nNotes:\n **Responsibilities:**\n\n - Stores node connectivity and validates that the topology remains acyclic.\n - Structure determines how `State` flows between nodes during execution.\n\n **Guarantees:**\n\n - Topology is acyclic. Node relationships remain consistent.\n - Thread-safe for concurrent reads after construction.", "members": { "add_edge": { "name": "add_edge", "kind": "function", "path": "dagpipe.Graph.add_edge", "signature": "", "docstring": "Add a directed edge from `src` to `dst`.\n\nArgs:\n src (Node):\n Source node.\n\n dst (Node):\n Destination node.\n\nRaises:\n TypeError:\n If `src` or `dst` is not a `Node`.\n\n ValueError:\n If the edge would create a cycle or if `src` and `dst` are common.\n\nNotes:\n - Validates node types.\n - Prevents cycles.\n - Registers nodes if not present.\n - Updates parent and child mappings." }, "add_root": { "name": "add_root", "kind": "function", "path": "dagpipe.Graph.add_root", "signature": "", "docstring": "Add a root node with no parents.\n\nArgs:\n node (Node):\n Node to add as a root.\n\nRaises:\n TypeError:\n If node is not a Node instance." }, "children": { "name": "children", "kind": "function", "path": "dagpipe.Graph.children", "signature": "", "docstring": "Return child nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n Tuple[Node, ...]:\n Outgoing neighbors." }, "parents": { "name": "parents", "kind": "function", "path": "dagpipe.Graph.parents", "signature": "", "docstring": "Return parent nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n Tuple[Node, ...]:\n Incoming neighbors." }, "roots": { "name": "roots", "kind": "function", "path": "dagpipe.Graph.roots", "signature": "", "docstring": "Return root nodes (nodes with no incoming edges).\n\nReturns:\n Tuple[Node, ...]:\n Entry point nodes." }, "nodes": { "name": "nodes", "kind": "function", "path": "dagpipe.Graph.nodes", "signature": "", "docstring": "Return all nodes in the graph.\n\nReturns:\n Tuple[Node, ...]:\n All registered nodes." } } }, "Engine": { "name": "Engine", "kind": "class", "path": "dagpipe.Engine", "signature": "", "docstring": "Execution engine responsible for running pipeline logic.\n\nNotes:\n **Responsibilities:**\n\n - Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n - Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n - Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n - Supports step-wise / resumable execution and progress hooks.\n\n **Guarantees:**\n\n - Never mutates `State`, `Node`, or `Graph` instances.\n - `State` objects are never modified in place; each branch produces\n independent instances.\n - Execution order is deterministic and follows graph or pipeline topology.\n - Thread-safe for concurrent execution.", "members": { "MODE_LINEAR": { "name": "MODE_LINEAR", "kind": "attribute", "path": "dagpipe.Engine.MODE_LINEAR", "signature": "", "docstring": null }, "MODE_GRAPH": { "name": "MODE_GRAPH", "kind": "attribute", "path": "dagpipe.Engine.MODE_GRAPH", "signature": "", "docstring": null }, "run": { "name": "run", "kind": "function", "path": "dagpipe.Engine.run", "signature": "", "docstring": "Execute the pipeline starting from a root `State`.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nRaises:\n TypeError:\n If `root` is not a `State` instance.\n\n RuntimeError:\n If the engine execution mode is invalid.\n\nNotes:\n **Responsibilities:**\n\n - Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states." }, "run_async": { "name": "run_async", "kind": "function", "path": "dagpipe.Engine.run_async", "signature": "", "docstring": "Execute the pipeline starting from `root`, dispatching sync vs async nodes.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nNotes:\n Each node is executed with `Node.run` when synchronous and\n `AsyncNode.run_async` when asynchronous. Linear and graph topologies\n are both supported." }, "run_steps": { "name": "run_steps", "kind": "function", "path": "dagpipe.Engine.run_steps", "signature": "", "docstring": "Execute the pipeline step-by-step, yielding one `StepResult` per step.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int, optional):\n Skip steps at index < `resume_from` (for resume-after-partial).\n Steps are 0-indexed.\n\n on_step (StepHook, optional):\n Callback `(step, status, message)` invoked per step; falls back\n to the engine-level hook when unset.\n\nYields:\n StepResult:\n One per executed node/step, carrying the produced states.\n\nNotes:\n This is a synchronous, generator-based checkpoint interface compatible\n with the imperative resume-by-step behaviour of the legacy\n orchestrator. Use `run_steps_async` for async nodes." }, "run_steps_async": { "name": "run_steps_async", "kind": "function", "path": "dagpipe.Engine.run_steps_async", "signature": "", "docstring": "Async variant of `run_steps` supporting `AsyncNode` execution.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int, optional):\n Skip steps at index < `resume_from`.\n\n on_step (AsyncStepHook, optional):\n Async callback `(step, status, message)` invoked per step.\n\nYields:\n StepResult:\n One per executed node/step." }, "nodes": { "name": "nodes", "kind": "attribute", "path": "dagpipe.Engine.nodes", "signature": "", "docstring": "Return nodes managed by this engine.\n\nReturns:\n tuple[Node, ...]:\n Ordered sequence in linear mode or all nodes in graph mode." } } }, "ProgressMessage": { "name": "ProgressMessage", "kind": "class", "path": "dagpipe.ProgressMessage", "signature": "", "docstring": "Lightweight progress payload emitted by engine step hooks.\n\nMirrors the imperative `ProgressMessage` used by the legacy orchestrator so\ncallers can surface counts/lines/errors without coupling the engine to pydantic.", "members": { "lines": { "name": "lines", "kind": "attribute", "path": "dagpipe.ProgressMessage.lines", "signature": "", "docstring": null }, "blocks": { "name": "blocks", "kind": "attribute", "path": "dagpipe.ProgressMessage.blocks", "signature": "", "docstring": null }, "count": { "name": "count", "kind": "attribute", "path": "dagpipe.ProgressMessage.count", "signature": "", "docstring": null }, "unit": { "name": "unit", "kind": "attribute", "path": "dagpipe.ProgressMessage.unit", "signature": "", "docstring": null }, "raw_ocr_line": { "name": "raw_ocr_line", "kind": "attribute", "path": "dagpipe.ProgressMessage.raw_ocr_line", "signature": "", "docstring": null }, "error": { "name": "error", "kind": "attribute", "path": "dagpipe.ProgressMessage.error", "signature": "", "docstring": null }, "step": { "name": "step", "kind": "attribute", "path": "dagpipe.ProgressMessage.step", "signature": "", "docstring": null }, "status": { "name": "status", "kind": "attribute", "path": "dagpipe.ProgressMessage.status", "signature": "", "docstring": null }, "as_dict": { "name": "as_dict", "kind": "function", "path": "dagpipe.ProgressMessage.as_dict", "signature": "", "docstring": null } } }, "StepResult": { "name": "StepResult", "kind": "class", "path": "dagpipe.StepResult", "signature": "", "docstring": "A single checkpointed step within an async/resumable engine run.\n\nAttributes:\n index (int): Ordinal index of the step.\n node_id (str): Identifier of the node associated with this step.\n states (Tuple[State, ...]): States produced by running this step.\n completed (bool): Whether this step succeeded (vs. paused/interrupted).", "members": { "index": { "name": "index", "kind": "attribute", "path": "dagpipe.StepResult.index", "signature": "", "docstring": null }, "node_id": { "name": "node_id", "kind": "attribute", "path": "dagpipe.StepResult.node_id", "signature": "", "docstring": null }, "states": { "name": "states", "kind": "attribute", "path": "dagpipe.StepResult.states", "signature": "", "docstring": null }, "completed": { "name": "completed", "kind": "attribute", "path": "dagpipe.StepResult.completed", "signature": "", "docstring": null } } }, "Pipeline": { "name": "Pipeline", "kind": "class", "path": "dagpipe.Pipeline", "signature": "", "docstring": "Executable pipeline created from YAML configuration.\n\nAttributes:\n engine (Engine):\n Execution engine responsible for running the pipeline.\n\n state_cls (Type[State]):\n Dynamically created `State` subclass with configured schema.\n\n initial_payload (Payload):\n Default payload used when execution begins.\n\nNotes:\n **Responsibilities:**\n\n - Encapsulates engine, state type, and initial payload.\n - Provides a simplified interface for executing configured pipelines.\n - Safe for concurrent execution if underlying nodes are thread-safe.", "members": { "engine": { "name": "engine", "kind": "attribute", "path": "dagpipe.Pipeline.engine", "signature": "", "docstring": null }, "state_cls": { "name": "state_cls", "kind": "attribute", "path": "dagpipe.Pipeline.state_cls", "signature": "", "docstring": null }, "initial_payload": { "name": "initial_payload", "kind": "attribute", "path": "dagpipe.Pipeline.initial_payload", "signature": "", "docstring": null }, "run": { "name": "run", "kind": "function", "path": "dagpipe.Pipeline.run", "signature": "", "docstring": "Execute the pipeline.\n\nArgs:\n payload_override (Mapping[str, Any], optional):\n Payload values overriding initial payload.\n\nReturns:\n list[State]:\n Terminal execution states.\n\nNotes:\n **Responsibilities:**\n\n - Merges override payload with initial payload.\n - Creates root `State` and executes engine." } } }, "load_pipeline": { "name": "load_pipeline", "kind": "function", "path": "dagpipe.load_pipeline", "signature": "", "docstring": "Load pipeline from YAML file.\n\nArgs:\n path (str):\n Path to YAML configuration file.\n\nReturns:\n Pipeline:\n Executable pipeline instance.\n\nNotes:\n **Responsibilities:**\n\n - Loads YAML configuration and builds schema.\n - Creates `State` subclass and loads `Node` instances.\n - Builds `Graph` topology and initializes `Engine`." }, "engine": { "name": "engine", "kind": "module", "path": "dagpipe.engine", "signature": null, "docstring": "# Summary\n\nExecution engine responsible for running pipelines and graphs.\n\nThe `Engine` executes `Node` objects and propagates immutable `State` instances\nthrough either a linear sequence or a directed acyclic graph (`Graph`).\nIt orchestrates execution order, branching, and state propagation.\n\n---\n\n# Guarantees\n\n- Deterministic execution and consistent state lineage.\n- Orchestrates execution without modifying `Graph`, `Node`, or `State` objects.", "members": { "deque": { "name": "deque", "kind": "alias", "path": "dagpipe.engine.deque", "signature": "", "docstring": null }, "AsyncIterator": { "name": "AsyncIterator", "kind": "alias", "path": "dagpipe.engine.AsyncIterator", "signature": "", "docstring": null }, "Awaitable": { "name": "Awaitable", "kind": "alias", "path": "dagpipe.engine.Awaitable", "signature": "", "docstring": null }, "Callable": { "name": "Callable", "kind": "alias", "path": "dagpipe.engine.Callable", "signature": "", "docstring": null }, "Iterator": { "name": "Iterator", "kind": "alias", "path": "dagpipe.engine.Iterator", "signature": "", "docstring": null }, "Sequence": { "name": "Sequence", "kind": "alias", "path": "dagpipe.engine.Sequence", "signature": "", "docstring": null }, "Any": { "name": "Any", "kind": "alias", "path": "dagpipe.engine.Any", "signature": "", "docstring": null }, "Optional": { "name": "Optional", "kind": "alias", "path": "dagpipe.engine.Optional", "signature": "", "docstring": null }, "Graph": { "name": "Graph", "kind": "class", "path": "dagpipe.engine.Graph", "signature": "", "docstring": "Directed Acyclic Graph defining execution topology of `Node` objects.\n\nNotes:\n **Responsibilities:**\n\n - Stores node connectivity and validates that the topology remains acyclic.\n - Structure determines how `State` flows between nodes during execution.\n\n **Guarantees:**\n\n - Topology is acyclic. Node relationships remain consistent.\n - Thread-safe for concurrent reads after construction.", "members": { "add_edge": { "name": "add_edge", "kind": "function", "path": "dagpipe.engine.Graph.add_edge", "signature": "", "docstring": "Add a directed edge from `src` to `dst`.\n\nArgs:\n src (Node):\n Source node.\n\n dst (Node):\n Destination node.\n\nRaises:\n TypeError:\n If `src` or `dst` is not a `Node`.\n\n ValueError:\n If the edge would create a cycle or if `src` and `dst` are common.\n\nNotes:\n - Validates node types.\n - Prevents cycles.\n - Registers nodes if not present.\n - Updates parent and child mappings." }, "add_root": { "name": "add_root", "kind": "function", "path": "dagpipe.engine.Graph.add_root", "signature": "", "docstring": "Add a root node with no parents.\n\nArgs:\n node (Node):\n Node to add as a root.\n\nRaises:\n TypeError:\n If node is not a Node instance." }, "children": { "name": "children", "kind": "function", "path": "dagpipe.engine.Graph.children", "signature": "", "docstring": "Return child nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n Tuple[Node, ...]:\n Outgoing neighbors." }, "parents": { "name": "parents", "kind": "function", "path": "dagpipe.engine.Graph.parents", "signature": "", "docstring": "Return parent nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n Tuple[Node, ...]:\n Incoming neighbors." }, "roots": { "name": "roots", "kind": "function", "path": "dagpipe.engine.Graph.roots", "signature": "", "docstring": "Return root nodes (nodes with no incoming edges).\n\nReturns:\n Tuple[Node, ...]:\n Entry point nodes." }, "nodes": { "name": "nodes", "kind": "function", "path": "dagpipe.engine.Graph.nodes", "signature": "", "docstring": "Return all nodes in the graph.\n\nReturns:\n Tuple[Node, ...]:\n All registered nodes." } } }, "AsyncNode": { "name": "AsyncNode", "kind": "class", "path": "dagpipe.engine.AsyncNode", "signature": "", "docstring": "Base class for nodes whose execution is asynchronous.\n\nSubclasses implement `resolve_async` (an async generator yielding derived\n`State` objects). The engine dispatches to `resolve_async` when running an\nasync traversal (see `Engine.run_async`).\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a no-op\nconsumer: calling `run` on an `AsyncNode` returns no states, signalling that\nan async engine is required.", "members": { "resolve": { "name": "resolve", "kind": "function", "path": "dagpipe.engine.AsyncNode.resolve", "signature": "", "docstring": null }, "resolve_async": { "name": "resolve_async", "kind": "function", "path": "dagpipe.engine.AsyncNode.resolve_async", "signature": "", "docstring": "Execute node logic asynchronously.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n Subclasses implement this. Must not mutate the input state.\n Should use `fork()` to create child states." }, "run_async": { "name": "run_async", "kind": "function", "path": "dagpipe.engine.AsyncNode.run_async", "signature": "", "docstring": "Execute this node asynchronously on a state, validating outputs." } } }, "Node": { "name": "Node", "kind": "class", "path": "dagpipe.engine.Node", "signature": "", "docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.", "members": { "id": { "name": "id", "kind": "attribute", "path": "dagpipe.engine.Node.id", "signature": "", "docstring": null }, "name": { "name": "name", "kind": "attribute", "path": "dagpipe.engine.Node.name", "signature": "", "docstring": null }, "node_id_to_name": { "name": "node_id_to_name", "kind": "function", "path": "dagpipe.engine.Node.node_id_to_name", "signature": "", "docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')." }, "clean_id_and_name": { "name": "clean_id_and_name", "kind": "function", "path": "dagpipe.engine.Node.clean_id_and_name", "signature": "", "docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing." }, "run": { "name": "run", "kind": "function", "path": "dagpipe.engine.Node.run", "signature": "", "docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object." }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.engine.Node.fork", "signature": "", "docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Mapping[str, Any], optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Mapping[str, Any], optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history." }, "resolve": { "name": "resolve", "kind": "function", "path": "dagpipe.engine.Node.resolve", "signature": "", "docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nYields:\n State:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch." }, "is_async": { "name": "is_async", "kind": "function", "path": "dagpipe.engine.Node.is_async", "signature": "", "docstring": "Return whether this node executes asynchronously." } } }, "State": { "name": "State", "kind": "class", "path": "dagpipe.engine.State", "signature": "", "docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.", "members": { "payload": { "name": "payload", "kind": "attribute", "path": "dagpipe.engine.State.payload", "signature": "", "docstring": null }, "schema": { "name": "schema", "kind": "attribute", "path": "dagpipe.engine.State.schema", "signature": "", "docstring": null }, "confidence": { "name": "confidence", "kind": "attribute", "path": "dagpipe.engine.State.confidence", "signature": "", "docstring": null }, "parent": { "name": "parent", "kind": "attribute", "path": "dagpipe.engine.State.parent", "signature": "", "docstring": null }, "depth": { "name": "depth", "kind": "attribute", "path": "dagpipe.engine.State.depth", "signature": "", "docstring": null }, "history": { "name": "history", "kind": "attribute", "path": "dagpipe.engine.State.history", "signature": "", "docstring": null }, "metadata": { "name": "metadata", "kind": "attribute", "path": "dagpipe.engine.State.metadata", "signature": "", "docstring": null }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.engine.State.fork", "signature": "", "docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any], optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any], optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history." }, "lineage": { "name": "lineage", "kind": "function", "path": "dagpipe.engine.State.lineage", "signature": "", "docstring": "Return lineage from root to this State.\n\nReturns:\n Tuple[State, ...]:\n Ordered execution lineage (root first)." }, "get": { "name": "get", "kind": "function", "path": "dagpipe.engine.State.get", "signature": "", "docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default." }, "has": { "name": "has", "kind": "function", "path": "dagpipe.engine.State.has", "signature": "", "docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key." } } }, "StepHook": { "name": "StepHook", "kind": "attribute", "path": "dagpipe.engine.StepHook", "signature": null, "docstring": null }, "AsyncStepHook": { "name": "AsyncStepHook", "kind": "attribute", "path": "dagpipe.engine.AsyncStepHook", "signature": null, "docstring": null }, "ProgressMessage": { "name": "ProgressMessage", "kind": "class", "path": "dagpipe.engine.ProgressMessage", "signature": "", "docstring": "Lightweight progress payload emitted by engine step hooks.\n\nMirrors the imperative `ProgressMessage` used by the legacy orchestrator so\ncallers can surface counts/lines/errors without coupling the engine to pydantic.", "members": { "lines": { "name": "lines", "kind": "attribute", "path": "dagpipe.engine.ProgressMessage.lines", "signature": null, "docstring": null }, "blocks": { "name": "blocks", "kind": "attribute", "path": "dagpipe.engine.ProgressMessage.blocks", "signature": null, "docstring": null }, "count": { "name": "count", "kind": "attribute", "path": "dagpipe.engine.ProgressMessage.count", "signature": null, "docstring": null }, "unit": { "name": "unit", "kind": "attribute", "path": "dagpipe.engine.ProgressMessage.unit", "signature": null, "docstring": null }, "raw_ocr_line": { "name": "raw_ocr_line", "kind": "attribute", "path": "dagpipe.engine.ProgressMessage.raw_ocr_line", "signature": null, "docstring": null }, "error": { "name": "error", "kind": "attribute", "path": "dagpipe.engine.ProgressMessage.error", "signature": null, "docstring": null }, "step": { "name": "step", "kind": "attribute", "path": "dagpipe.engine.ProgressMessage.step", "signature": null, "docstring": null }, "status": { "name": "status", "kind": "attribute", "path": "dagpipe.engine.ProgressMessage.status", "signature": null, "docstring": null }, "as_dict": { "name": "as_dict", "kind": "function", "path": "dagpipe.engine.ProgressMessage.as_dict", "signature": "", "docstring": null } } }, "StepResult": { "name": "StepResult", "kind": "class", "path": "dagpipe.engine.StepResult", "signature": "", "docstring": "A single checkpointed step within an async/resumable engine run.\n\nAttributes:\n index (int): Ordinal index of the step.\n node_id (str): Identifier of the node associated with this step.\n states (Tuple[State, ...]): States produced by running this step.\n completed (bool): Whether this step succeeded (vs. paused/interrupted).", "members": { "index": { "name": "index", "kind": "attribute", "path": "dagpipe.engine.StepResult.index", "signature": null, "docstring": null }, "node_id": { "name": "node_id", "kind": "attribute", "path": "dagpipe.engine.StepResult.node_id", "signature": null, "docstring": null }, "states": { "name": "states", "kind": "attribute", "path": "dagpipe.engine.StepResult.states", "signature": null, "docstring": null }, "completed": { "name": "completed", "kind": "attribute", "path": "dagpipe.engine.StepResult.completed", "signature": null, "docstring": null } } }, "Engine": { "name": "Engine", "kind": "class", "path": "dagpipe.engine.Engine", "signature": "", "docstring": "Execution engine responsible for running pipeline logic.\n\nNotes:\n **Responsibilities:**\n\n - Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n - Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n - Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n - Supports step-wise / resumable execution and progress hooks.\n\n **Guarantees:**\n\n - Never mutates `State`, `Node`, or `Graph` instances.\n - `State` objects are never modified in place; each branch produces\n independent instances.\n - Execution order is deterministic and follows graph or pipeline topology.\n - Thread-safe for concurrent execution.", "members": { "MODE_LINEAR": { "name": "MODE_LINEAR", "kind": "attribute", "path": "dagpipe.engine.Engine.MODE_LINEAR", "signature": null, "docstring": null }, "MODE_GRAPH": { "name": "MODE_GRAPH", "kind": "attribute", "path": "dagpipe.engine.Engine.MODE_GRAPH", "signature": null, "docstring": null }, "run": { "name": "run", "kind": "function", "path": "dagpipe.engine.Engine.run", "signature": "", "docstring": "Execute the pipeline starting from a root `State`.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nRaises:\n TypeError:\n If `root` is not a `State` instance.\n\n RuntimeError:\n If the engine execution mode is invalid.\n\nNotes:\n **Responsibilities:**\n\n - Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states." }, "run_async": { "name": "run_async", "kind": "function", "path": "dagpipe.engine.Engine.run_async", "signature": "", "docstring": "Execute the pipeline starting from `root`, dispatching sync vs async nodes.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nNotes:\n Each node is executed with `Node.run` when synchronous and\n `AsyncNode.run_async` when asynchronous. Linear and graph topologies\n are both supported." }, "run_steps": { "name": "run_steps", "kind": "function", "path": "dagpipe.engine.Engine.run_steps", "signature": "", "docstring": "Execute the pipeline step-by-step, yielding one `StepResult` per step.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int, optional):\n Skip steps at index < `resume_from` (for resume-after-partial).\n Steps are 0-indexed.\n\n on_step (StepHook, optional):\n Callback `(step, status, message)` invoked per step; falls back\n to the engine-level hook when unset.\n\nYields:\n StepResult:\n One per executed node/step, carrying the produced states.\n\nNotes:\n This is a synchronous, generator-based checkpoint interface compatible\n with the imperative resume-by-step behaviour of the legacy\n orchestrator. Use `run_steps_async` for async nodes." }, "run_steps_async": { "name": "run_steps_async", "kind": "function", "path": "dagpipe.engine.Engine.run_steps_async", "signature": "", "docstring": "Async variant of `run_steps` supporting `AsyncNode` execution.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int, optional):\n Skip steps at index < `resume_from`.\n\n on_step (AsyncStepHook, optional):\n Async callback `(step, status, message)` invoked per step.\n\nYields:\n StepResult:\n One per executed node/step." }, "nodes": { "name": "nodes", "kind": "attribute", "path": "dagpipe.engine.Engine.nodes", "signature": null, "docstring": "Return nodes managed by this engine.\n\nReturns:\n tuple[Node, ...]:\n Ordered sequence in linear mode or all nodes in graph mode." } } }, "Incomplete": { "name": "Incomplete", "kind": "alias", "path": "dagpipe.engine.Incomplete", "signature": "", "docstring": null } } }, "graph": { "name": "graph", "kind": "module", "path": "dagpipe.graph", "signature": null, "docstring": "# Summary\n\nDefines DAG structure connecting nodes.\n\nA `Graph` describes execution topology only. It does not execute nodes or manage\n`State`. Execution is handled by an `Engine`.\n\n---\n\n# Responsibilities\n\n- Multiple roots, branching, and merging support.\n- Deterministic traversal based on topology.\n- Graph is mutable during construction but treated as immutable at runtime.", "members": { "defaultdict": { "name": "defaultdict", "kind": "alias", "path": "dagpipe.graph.defaultdict", "signature": "", "docstring": null }, "Node": { "name": "Node", "kind": "class", "path": "dagpipe.graph.Node", "signature": "", "docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.", "members": { "id": { "name": "id", "kind": "attribute", "path": "dagpipe.graph.Node.id", "signature": "", "docstring": null }, "name": { "name": "name", "kind": "attribute", "path": "dagpipe.graph.Node.name", "signature": "", "docstring": null }, "node_id_to_name": { "name": "node_id_to_name", "kind": "function", "path": "dagpipe.graph.Node.node_id_to_name", "signature": "", "docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')." }, "clean_id_and_name": { "name": "clean_id_and_name", "kind": "function", "path": "dagpipe.graph.Node.clean_id_and_name", "signature": "", "docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing." }, "run": { "name": "run", "kind": "function", "path": "dagpipe.graph.Node.run", "signature": "", "docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object." }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.graph.Node.fork", "signature": "", "docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Mapping[str, Any], optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Mapping[str, Any], optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history." }, "resolve": { "name": "resolve", "kind": "function", "path": "dagpipe.graph.Node.resolve", "signature": "", "docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nYields:\n State:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch." }, "is_async": { "name": "is_async", "kind": "function", "path": "dagpipe.graph.Node.is_async", "signature": "", "docstring": "Return whether this node executes asynchronously." } } }, "Graph": { "name": "Graph", "kind": "class", "path": "dagpipe.graph.Graph", "signature": "", "docstring": "Directed Acyclic Graph defining execution topology of `Node` objects.\n\nNotes:\n **Responsibilities:**\n\n - Stores node connectivity and validates that the topology remains acyclic.\n - Structure determines how `State` flows between nodes during execution.\n\n **Guarantees:**\n\n - Topology is acyclic. Node relationships remain consistent.\n - Thread-safe for concurrent reads after construction.", "members": { "add_edge": { "name": "add_edge", "kind": "function", "path": "dagpipe.graph.Graph.add_edge", "signature": "", "docstring": "Add a directed edge from `src` to `dst`.\n\nArgs:\n src (Node):\n Source node.\n\n dst (Node):\n Destination node.\n\nRaises:\n TypeError:\n If `src` or `dst` is not a `Node`.\n\n ValueError:\n If the edge would create a cycle or if `src` and `dst` are common.\n\nNotes:\n - Validates node types.\n - Prevents cycles.\n - Registers nodes if not present.\n - Updates parent and child mappings." }, "add_root": { "name": "add_root", "kind": "function", "path": "dagpipe.graph.Graph.add_root", "signature": "", "docstring": "Add a root node with no parents.\n\nArgs:\n node (Node):\n Node to add as a root.\n\nRaises:\n TypeError:\n If node is not a Node instance." }, "children": { "name": "children", "kind": "function", "path": "dagpipe.graph.Graph.children", "signature": "", "docstring": "Return child nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n Tuple[Node, ...]:\n Outgoing neighbors." }, "parents": { "name": "parents", "kind": "function", "path": "dagpipe.graph.Graph.parents", "signature": "", "docstring": "Return parent nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n Tuple[Node, ...]:\n Incoming neighbors." }, "roots": { "name": "roots", "kind": "function", "path": "dagpipe.graph.Graph.roots", "signature": "", "docstring": "Return root nodes (nodes with no incoming edges).\n\nReturns:\n Tuple[Node, ...]:\n Entry point nodes." }, "nodes": { "name": "nodes", "kind": "function", "path": "dagpipe.graph.Graph.nodes", "signature": "", "docstring": "Return all nodes in the graph.\n\nReturns:\n Tuple[Node, ...]:\n All registered nodes." } } } } }, "node": { "name": "node", "kind": "module", "path": "dagpipe.node", "signature": null, "docstring": "# Summary\n\nDefines the `Node` abstraction used by `dagpipe`.\n\nA node represents a single unit of pipeline execution logic. It consumes one\n`State` and produces zero, one, or many new `State` objects.\n\nNodes are connected using a `Graph` and executed by an `Engine`.\n\n---\n\n# Design principles\n\n- **Pure:** Must not mutate input state.\n- **Deterministic:** Same input produces same output.\n- **Stateless:** Recommended to be stateless for reuse.\n- **Composable:** Nodes enable branching execution graphs.", "members": { "inspect": { "name": "inspect", "kind": "alias", "path": "dagpipe.node.inspect", "signature": "", "docstring": null }, "re": { "name": "re", "kind": "alias", "path": "dagpipe.node.re", "signature": "", "docstring": null }, "ABC": { "name": "ABC", "kind": "alias", "path": "dagpipe.node.ABC", "signature": "", "docstring": null }, "abstractmethod": { "name": "abstractmethod", "kind": "alias", "path": "dagpipe.node.abstractmethod", "signature": "", "docstring": null }, "Iterable": { "name": "Iterable", "kind": "alias", "path": "dagpipe.node.Iterable", "signature": "", "docstring": null }, "Iterator": { "name": "Iterator", "kind": "alias", "path": "dagpipe.node.Iterator", "signature": "", "docstring": null }, "Any": { "name": "Any", "kind": "alias", "path": "dagpipe.node.Any", "signature": "", "docstring": null }, "cast": { "name": "cast", "kind": "alias", "path": "dagpipe.node.cast", "signature": "", "docstring": null }, "State": { "name": "State", "kind": "class", "path": "dagpipe.node.State", "signature": "", "docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.", "members": { "payload": { "name": "payload", "kind": "attribute", "path": "dagpipe.node.State.payload", "signature": "", "docstring": null }, "schema": { "name": "schema", "kind": "attribute", "path": "dagpipe.node.State.schema", "signature": "", "docstring": null }, "confidence": { "name": "confidence", "kind": "attribute", "path": "dagpipe.node.State.confidence", "signature": "", "docstring": null }, "parent": { "name": "parent", "kind": "attribute", "path": "dagpipe.node.State.parent", "signature": "", "docstring": null }, "depth": { "name": "depth", "kind": "attribute", "path": "dagpipe.node.State.depth", "signature": "", "docstring": null }, "history": { "name": "history", "kind": "attribute", "path": "dagpipe.node.State.history", "signature": "", "docstring": null }, "metadata": { "name": "metadata", "kind": "attribute", "path": "dagpipe.node.State.metadata", "signature": "", "docstring": null }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.node.State.fork", "signature": "", "docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any], optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any], optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history." }, "lineage": { "name": "lineage", "kind": "function", "path": "dagpipe.node.State.lineage", "signature": "", "docstring": "Return lineage from root to this State.\n\nReturns:\n Tuple[State, ...]:\n Ordered execution lineage (root first)." }, "get": { "name": "get", "kind": "function", "path": "dagpipe.node.State.get", "signature": "", "docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default." }, "has": { "name": "has", "kind": "function", "path": "dagpipe.node.State.has", "signature": "", "docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key." } } }, "Node": { "name": "Node", "kind": "class", "path": "dagpipe.node.Node", "signature": "", "docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.", "members": { "id": { "name": "id", "kind": "attribute", "path": "dagpipe.node.Node.id", "signature": null, "docstring": null }, "name": { "name": "name", "kind": "attribute", "path": "dagpipe.node.Node.name", "signature": null, "docstring": null }, "node_id_to_name": { "name": "node_id_to_name", "kind": "function", "path": "dagpipe.node.Node.node_id_to_name", "signature": "", "docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')." }, "clean_id_and_name": { "name": "clean_id_and_name", "kind": "function", "path": "dagpipe.node.Node.clean_id_and_name", "signature": "", "docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing." }, "run": { "name": "run", "kind": "function", "path": "dagpipe.node.Node.run", "signature": "", "docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object." }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.node.Node.fork", "signature": "", "docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Mapping[str, Any], optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Mapping[str, Any], optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history." }, "resolve": { "name": "resolve", "kind": "function", "path": "dagpipe.node.Node.resolve", "signature": "", "docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nYields:\n State:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch." }, "is_async": { "name": "is_async", "kind": "function", "path": "dagpipe.node.Node.is_async", "signature": "", "docstring": "Return whether this node executes asynchronously." } } }, "AsyncNode": { "name": "AsyncNode", "kind": "class", "path": "dagpipe.node.AsyncNode", "signature": "", "docstring": "Base class for nodes whose execution is asynchronous.\n\nSubclasses implement `resolve_async` (an async generator yielding derived\n`State` objects). The engine dispatches to `resolve_async` when running an\nasync traversal (see `Engine.run_async`).\n\nSync-only engines (and the base `Node.run`) treat an `AsyncNode` as a no-op\nconsumer: calling `run` on an `AsyncNode` returns no states, signalling that\nan async engine is required.", "members": { "resolve": { "name": "resolve", "kind": "function", "path": "dagpipe.node.AsyncNode.resolve", "signature": "", "docstring": null }, "resolve_async": { "name": "resolve_async", "kind": "function", "path": "dagpipe.node.AsyncNode.resolve_async", "signature": "", "docstring": "Execute node logic asynchronously.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n Iterable[State]:\n Derived execution state(s).\n\nNotes:\n Subclasses implement this. Must not mutate the input state.\n Should use `fork()` to create child states." }, "run_async": { "name": "run_async", "kind": "function", "path": "dagpipe.node.AsyncNode.run_async", "signature": "", "docstring": "Execute this node asynchronously on a state, validating outputs." } } }, "abc": { "name": "abc", "kind": "alias", "path": "dagpipe.node.abc", "signature": "", "docstring": null } } }, "state": { "name": "state", "kind": "module", "path": "dagpipe.state", "signature": null, "docstring": "# Summary\n\nDefines the core `State` object used by `dagpipe`.\n\nThe `State` represents a single point in pipeline execution. It contains\narbitrary data and metadata and is designed to be immutable. Instead of\nmodifying an existing state, nodes create new child states via `fork()`.\n\n---\n\n# Design principles\n\n- **Immutability:** States must never be modified after creation.\n All transformations must create a new state via `fork()`.\n- **Cheap cloning:** Forking must be efficient since branching may create many states.\n- **Lineage tracking:** Each state maintains a reference to its parent and\n execution metadata for debugging and observability.\n- **Domain agnostic:** State contains generic key-value data and does not\n assume any schema.\n- **Engine-friendly:** State contains execution metadata such as depth and history.", "members": { "Iterable": { "name": "Iterable", "kind": "alias", "path": "dagpipe.state.Iterable", "signature": "", "docstring": null }, "Mapping": { "name": "Mapping", "kind": "alias", "path": "dagpipe.state.Mapping", "signature": "", "docstring": null }, "dataclass": { "name": "dataclass", "kind": "alias", "path": "dagpipe.state.dataclass", "signature": "", "docstring": null }, "field": { "name": "field", "kind": "alias", "path": "dagpipe.state.field", "signature": "", "docstring": null }, "UnionType": { "name": "UnionType", "kind": "alias", "path": "dagpipe.state.UnionType", "signature": "", "docstring": null }, "Any": { "name": "Any", "kind": "alias", "path": "dagpipe.state.Any", "signature": "", "docstring": null }, "ClassVar": { "name": "ClassVar", "kind": "alias", "path": "dagpipe.state.ClassVar", "signature": "", "docstring": null }, "Optional": { "name": "Optional", "kind": "alias", "path": "dagpipe.state.Optional", "signature": "", "docstring": null }, "Union": { "name": "Union", "kind": "alias", "path": "dagpipe.state.Union", "signature": "", "docstring": null }, "get_args": { "name": "get_args", "kind": "alias", "path": "dagpipe.state.get_args", "signature": "", "docstring": null }, "Payload": { "name": "Payload", "kind": "class", "path": "dagpipe.state.Payload", "signature": "", "docstring": "Immutable hierarchical container with dot-path access.\n\nAttributes:\n _data (Mapping[str, Any]):\n Immutable hierarchical data structure.\n\nNotes:\n **Responsibilities:**\n\n - Stores execution data used by `State`.\n - Supports efficient atomic updates without modifying existing instances.\n - `Payload` instances are fully thread-safe due to immutability.", "members": { "iter_paths": { "name": "iter_paths", "kind": "function", "path": "dagpipe.state.Payload.iter_paths", "signature": "", "docstring": "Recursively yield dot-paths for all leaf nodes.\n\nArgs:\n data (Mapping[str, Any]):\n The mapping to iterate over.\n prefix (str, optional):\n Current path prefix.\n\nReturns:\n Iterable[str]:\n Generator yielding dot-paths." }, "get": { "name": "get", "kind": "function", "path": "dagpipe.state.Payload.get", "signature": "", "docstring": "Retrieve value using dot-path.\n\nArgs:\n path (str):\n Dot-separated path to the value.\n default (Any, optional):\n Default value if path doesn't exist.\n\nReturns:\n Any:\n The retrieved value or default." }, "has": { "name": "has", "kind": "function", "path": "dagpipe.state.Payload.has", "signature": "", "docstring": "Return True if path exists.\n\nArgs:\n path (str):\n Dot-separated path to check.\n\nReturns:\n bool:\n Existence of the path." }, "update": { "name": "update", "kind": "function", "path": "dagpipe.state.Payload.update", "signature": "", "docstring": "Create a new `Payload` with dot-path updates applied.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path to value mapping.\n\nReturns:\n Payload:\n New immutable payload instance with updates.\n\nNotes:\n **Guarantees:**\n\n - Preserves existing data by copying only modified branches.\n - Returns a new immutable `Payload`." }, "keys": { "name": "keys", "kind": "function", "path": "dagpipe.state.Payload.keys", "signature": "", "docstring": "Return top-level keys.\n\nReturns:\n Iterable[str]:\n Iterator over top-level keys." }, "as_dict": { "name": "as_dict", "kind": "function", "path": "dagpipe.state.Payload.as_dict", "signature": "", "docstring": "Return underlying mapping.\n\nReturns:\n Mapping[str, Any]:\n Read-only view of the underlying data." } } }, "SchemaNode": { "name": "SchemaNode", "kind": "attribute", "path": "dagpipe.state.SchemaNode", "signature": null, "docstring": null }, "Schema": { "name": "Schema", "kind": "class", "path": "dagpipe.state.Schema", "signature": "", "docstring": "Immutable hierarchical schema defining allowed payload structure.\n\nAttributes:\n tree (Mapping[str, SchemaNode]):\n Hierarchical schema definition.\n\nNotes:\n **Responsibilities:**\n\n - Validates `State` payloads and updates.\n - Reusable across all `State` instances.\n - Fully thread-safe due to immutability.", "members": { "tree": { "name": "tree", "kind": "attribute", "path": "dagpipe.state.Schema.tree", "signature": null, "docstring": null }, "validate_payload": { "name": "validate_payload", "kind": "function", "path": "dagpipe.state.Schema.validate_payload", "signature": "", "docstring": "Validate complete payload structure.\n\nArgs:\n payload (Payload):\n Payload to validate.\n\nRaises:\n SchemaError:\n If payload violates schema." }, "validate_update": { "name": "validate_update", "kind": "function", "path": "dagpipe.state.Schema.validate_update", "signature": "", "docstring": "Validate payload update paths.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path updates to validate.\n\nRaises:\n SchemaError:\n If any path is invalid according to the schema." } } }, "SchemaError": { "name": "SchemaError", "kind": "class", "path": "dagpipe.state.SchemaError", "signature": "", "docstring": "Raised when payload data violates the declared schema.\n\nIndicates invalid structure, invalid path, or invalid type.\n---" }, "State": { "name": "State", "kind": "class", "path": "dagpipe.state.State", "signature": "", "docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.", "members": { "payload": { "name": "payload", "kind": "attribute", "path": "dagpipe.state.State.payload", "signature": null, "docstring": null }, "schema": { "name": "schema", "kind": "attribute", "path": "dagpipe.state.State.schema", "signature": null, "docstring": null }, "confidence": { "name": "confidence", "kind": "attribute", "path": "dagpipe.state.State.confidence", "signature": null, "docstring": null }, "parent": { "name": "parent", "kind": "attribute", "path": "dagpipe.state.State.parent", "signature": null, "docstring": null }, "depth": { "name": "depth", "kind": "attribute", "path": "dagpipe.state.State.depth", "signature": null, "docstring": null }, "history": { "name": "history", "kind": "attribute", "path": "dagpipe.state.State.history", "signature": null, "docstring": null }, "metadata": { "name": "metadata", "kind": "attribute", "path": "dagpipe.state.State.metadata", "signature": null, "docstring": null }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.state.State.fork", "signature": "", "docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any], optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any], optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history." }, "lineage": { "name": "lineage", "kind": "function", "path": "dagpipe.state.State.lineage", "signature": "", "docstring": "Return lineage from root to this State.\n\nReturns:\n Tuple[State, ...]:\n Ordered execution lineage (root first)." }, "get": { "name": "get", "kind": "function", "path": "dagpipe.state.State.get", "signature": "", "docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default." }, "has": { "name": "has", "kind": "function", "path": "dagpipe.state.State.has", "signature": "", "docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key." } } }, "Incomplete": { "name": "Incomplete", "kind": "alias", "path": "dagpipe.state.Incomplete", "signature": "", "docstring": null } } }, "yaml_loader": { "name": "yaml_loader", "kind": "module", "path": "dagpipe.yaml_loader", "signature": null, "docstring": "# Summary\n\nLoads dagpipe pipelines from YAML configuration.\n\nCreates fully configured pipeline objects from declarative YAML definitions,\nincluding `Schema`, `State` subclasses, `Node` instances, `Graph` topology,\nand initial payloads.", "members": { "importlib": { "name": "importlib", "kind": "alias", "path": "dagpipe.yaml_loader.importlib", "signature": "", "docstring": null }, "types": { "name": "types", "kind": "alias", "path": "dagpipe.yaml_loader.types", "signature": "", "docstring": null }, "Mapping": { "name": "Mapping", "kind": "alias", "path": "dagpipe.yaml_loader.Mapping", "signature": "", "docstring": null }, "dataclass": { "name": "dataclass", "kind": "alias", "path": "dagpipe.yaml_loader.dataclass", "signature": "", "docstring": null }, "Any": { "name": "Any", "kind": "alias", "path": "dagpipe.yaml_loader.Any", "signature": "", "docstring": null }, "cast": { "name": "cast", "kind": "alias", "path": "dagpipe.yaml_loader.cast", "signature": "", "docstring": null }, "yaml": { "name": "yaml", "kind": "alias", "path": "dagpipe.yaml_loader.yaml", "signature": "", "docstring": null }, "Engine": { "name": "Engine", "kind": "class", "path": "dagpipe.yaml_loader.Engine", "signature": "", "docstring": "Execution engine responsible for running pipeline logic.\n\nNotes:\n **Responsibilities:**\n\n - Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n - Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n - Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n - Supports step-wise / resumable execution and progress hooks.\n\n **Guarantees:**\n\n - Never mutates `State`, `Node`, or `Graph` instances.\n - `State` objects are never modified in place; each branch produces\n independent instances.\n - Execution order is deterministic and follows graph or pipeline topology.\n - Thread-safe for concurrent execution.", "members": { "MODE_LINEAR": { "name": "MODE_LINEAR", "kind": "attribute", "path": "dagpipe.yaml_loader.Engine.MODE_LINEAR", "signature": "", "docstring": null }, "MODE_GRAPH": { "name": "MODE_GRAPH", "kind": "attribute", "path": "dagpipe.yaml_loader.Engine.MODE_GRAPH", "signature": "", "docstring": null }, "run": { "name": "run", "kind": "function", "path": "dagpipe.yaml_loader.Engine.run", "signature": "", "docstring": "Execute the pipeline starting from a root `State`.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nRaises:\n TypeError:\n If `root` is not a `State` instance.\n\n RuntimeError:\n If the engine execution mode is invalid.\n\nNotes:\n **Responsibilities:**\n\n - Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states." }, "run_async": { "name": "run_async", "kind": "function", "path": "dagpipe.yaml_loader.Engine.run_async", "signature": "", "docstring": "Execute the pipeline starting from `root`, dispatching sync vs async nodes.\n\nArgs:\n root (State):\n Initial execution state.\n\nReturns:\n list[State]:\n Terminal execution states produced by the pipeline.\n\nNotes:\n Each node is executed with `Node.run` when synchronous and\n `AsyncNode.run_async` when asynchronous. Linear and graph topologies\n are both supported." }, "run_steps": { "name": "run_steps", "kind": "function", "path": "dagpipe.yaml_loader.Engine.run_steps", "signature": "", "docstring": "Execute the pipeline step-by-step, yielding one `StepResult` per step.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int, optional):\n Skip steps at index < `resume_from` (for resume-after-partial).\n Steps are 0-indexed.\n\n on_step (StepHook, optional):\n Callback `(step, status, message)` invoked per step; falls back\n to the engine-level hook when unset.\n\nYields:\n StepResult:\n One per executed node/step, carrying the produced states.\n\nNotes:\n This is a synchronous, generator-based checkpoint interface compatible\n with the imperative resume-by-step behaviour of the legacy\n orchestrator. Use `run_steps_async` for async nodes." }, "run_steps_async": { "name": "run_steps_async", "kind": "function", "path": "dagpipe.yaml_loader.Engine.run_steps_async", "signature": "", "docstring": "Async variant of `run_steps` supporting `AsyncNode` execution.\n\nArgs:\n root (State):\n Initial execution state.\n\n resume_from (int, optional):\n Skip steps at index < `resume_from`.\n\n on_step (AsyncStepHook, optional):\n Async callback `(step, status, message)` invoked per step.\n\nYields:\n StepResult:\n One per executed node/step." }, "nodes": { "name": "nodes", "kind": "attribute", "path": "dagpipe.yaml_loader.Engine.nodes", "signature": "", "docstring": "Return nodes managed by this engine.\n\nReturns:\n tuple[Node, ...]:\n Ordered sequence in linear mode or all nodes in graph mode." } } }, "Graph": { "name": "Graph", "kind": "class", "path": "dagpipe.yaml_loader.Graph", "signature": "", "docstring": "Directed Acyclic Graph defining execution topology of `Node` objects.\n\nNotes:\n **Responsibilities:**\n\n - Stores node connectivity and validates that the topology remains acyclic.\n - Structure determines how `State` flows between nodes during execution.\n\n **Guarantees:**\n\n - Topology is acyclic. Node relationships remain consistent.\n - Thread-safe for concurrent reads after construction.", "members": { "add_edge": { "name": "add_edge", "kind": "function", "path": "dagpipe.yaml_loader.Graph.add_edge", "signature": "", "docstring": "Add a directed edge from `src` to `dst`.\n\nArgs:\n src (Node):\n Source node.\n\n dst (Node):\n Destination node.\n\nRaises:\n TypeError:\n If `src` or `dst` is not a `Node`.\n\n ValueError:\n If the edge would create a cycle or if `src` and `dst` are common.\n\nNotes:\n - Validates node types.\n - Prevents cycles.\n - Registers nodes if not present.\n - Updates parent and child mappings." }, "add_root": { "name": "add_root", "kind": "function", "path": "dagpipe.yaml_loader.Graph.add_root", "signature": "", "docstring": "Add a root node with no parents.\n\nArgs:\n node (Node):\n Node to add as a root.\n\nRaises:\n TypeError:\n If node is not a Node instance." }, "children": { "name": "children", "kind": "function", "path": "dagpipe.yaml_loader.Graph.children", "signature": "", "docstring": "Return child nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n Tuple[Node, ...]:\n Outgoing neighbors." }, "parents": { "name": "parents", "kind": "function", "path": "dagpipe.yaml_loader.Graph.parents", "signature": "", "docstring": "Return parent nodes of a node.\n\nArgs:\n node (Node):\n Node to query.\n\nReturns:\n Tuple[Node, ...]:\n Incoming neighbors." }, "roots": { "name": "roots", "kind": "function", "path": "dagpipe.yaml_loader.Graph.roots", "signature": "", "docstring": "Return root nodes (nodes with no incoming edges).\n\nReturns:\n Tuple[Node, ...]:\n Entry point nodes." }, "nodes": { "name": "nodes", "kind": "function", "path": "dagpipe.yaml_loader.Graph.nodes", "signature": "", "docstring": "Return all nodes in the graph.\n\nReturns:\n Tuple[Node, ...]:\n All registered nodes." } } }, "Node": { "name": "Node", "kind": "class", "path": "dagpipe.yaml_loader.Node", "signature": "", "docstring": "Base class for all dagpipe execution nodes.\n\nAttributes:\n id (str):\n Unique identifier of the node (snake_case dotted format).\n\n name (str):\n Human-readable display name.\n\nNotes:\n **Responsibilities:**\n\n - Represents a deterministic unit of execution in the pipeline graph.\n - Consumes one `State` and produces zero, one, or many derived states.\n - Defines execution logic and enables branching, filtering, and transformation.\n\n **Guarantees:**\n\n - Nodes must never mutate the input `State`.\n - Instances are singletons per subclass and reused across executions.", "members": { "id": { "name": "id", "kind": "attribute", "path": "dagpipe.yaml_loader.Node.id", "signature": "", "docstring": null }, "name": { "name": "name", "kind": "attribute", "path": "dagpipe.yaml_loader.Node.name", "signature": "", "docstring": null }, "node_id_to_name": { "name": "node_id_to_name", "kind": "function", "path": "dagpipe.yaml_loader.Node.node_id_to_name", "signature": "", "docstring": "Convert a dotted snake_case node ID into a human-readable name.\n\nArgs:\n node_id (str):\n Unique node identifier (e.g., 'entity.resolve.numeric_merchant').\n\nReturns:\n str:\n Human-readable display name (e.g., 'Entity › Resolve › Numeric Merchant')." }, "clean_id_and_name": { "name": "clean_id_and_name", "kind": "function", "path": "dagpipe.yaml_loader.Node.clean_id_and_name", "signature": "", "docstring": "Normalize and validate node ID and display name.\n\nRaises:\n TypeError:\n If ID is not a string.\n ValueError:\n If ID format is invalid.\n\nNotes:\n **Guarantees:**\n\n - Generates ID from module and class name if missing.\n - Validates ID format.\n - Generates human-readable name if missing." }, "run": { "name": "run", "kind": "function", "path": "dagpipe.yaml_loader.Node.run", "signature": "", "docstring": "Execute this node on a `State`.\n\nArgs:\n state (State):\n Input execution state.\n\nReturns:\n tuple[State, ...]:\n Derived execution states.\n\nRaises:\n TypeError:\n If `resolve()` yields a non-`State` object." }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.yaml_loader.Node.fork", "signature": "", "docstring": "Create a child `State` attributed to this node.\n\nArgs:\n state (State):\n Parent execution state.\n\n payload_update (Mapping[str, Any], optional):\n Dot-path payload updates.\n\n confidence_delta (float, optional):\n Confidence adjustment.\n\n metadata_update (Mapping[str, Any], optional):\n Metadata updates.\n\nReturns:\n State:\n New child execution state.\n\nNotes:\n **Responsibilities:**\n\n - Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history." }, "resolve": { "name": "resolve", "kind": "function", "path": "dagpipe.yaml_loader.Node.resolve", "signature": "", "docstring": "Execute node logic.\n\nArgs:\n state (State):\n Input execution state.\n\nYields:\n State:\n Derived execution state(s).\n\nNotes:\n **Responsibilities:**\n\n - Subclasses implement specific resolution behavior.\n - Must not mutate input state.\n - Should use `fork()` to create child states.\n - May yield zero states to terminate a branch." }, "is_async": { "name": "is_async", "kind": "function", "path": "dagpipe.yaml_loader.Node.is_async", "signature": "", "docstring": "Return whether this node executes asynchronously." } } }, "Payload": { "name": "Payload", "kind": "class", "path": "dagpipe.yaml_loader.Payload", "signature": "", "docstring": "Immutable hierarchical container with dot-path access.\n\nAttributes:\n _data (Mapping[str, Any]):\n Immutable hierarchical data structure.\n\nNotes:\n **Responsibilities:**\n\n - Stores execution data used by `State`.\n - Supports efficient atomic updates without modifying existing instances.\n - `Payload` instances are fully thread-safe due to immutability.", "members": { "iter_paths": { "name": "iter_paths", "kind": "function", "path": "dagpipe.yaml_loader.Payload.iter_paths", "signature": "", "docstring": "Recursively yield dot-paths for all leaf nodes.\n\nArgs:\n data (Mapping[str, Any]):\n The mapping to iterate over.\n prefix (str, optional):\n Current path prefix.\n\nReturns:\n Iterable[str]:\n Generator yielding dot-paths." }, "get": { "name": "get", "kind": "function", "path": "dagpipe.yaml_loader.Payload.get", "signature": "", "docstring": "Retrieve value using dot-path.\n\nArgs:\n path (str):\n Dot-separated path to the value.\n default (Any, optional):\n Default value if path doesn't exist.\n\nReturns:\n Any:\n The retrieved value or default." }, "has": { "name": "has", "kind": "function", "path": "dagpipe.yaml_loader.Payload.has", "signature": "", "docstring": "Return True if path exists.\n\nArgs:\n path (str):\n Dot-separated path to check.\n\nReturns:\n bool:\n Existence of the path." }, "update": { "name": "update", "kind": "function", "path": "dagpipe.yaml_loader.Payload.update", "signature": "", "docstring": "Create a new `Payload` with dot-path updates applied.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path to value mapping.\n\nReturns:\n Payload:\n New immutable payload instance with updates.\n\nNotes:\n **Guarantees:**\n\n - Preserves existing data by copying only modified branches.\n - Returns a new immutable `Payload`." }, "keys": { "name": "keys", "kind": "function", "path": "dagpipe.yaml_loader.Payload.keys", "signature": "", "docstring": "Return top-level keys.\n\nReturns:\n Iterable[str]:\n Iterator over top-level keys." }, "as_dict": { "name": "as_dict", "kind": "function", "path": "dagpipe.yaml_loader.Payload.as_dict", "signature": "", "docstring": "Return underlying mapping.\n\nReturns:\n Mapping[str, Any]:\n Read-only view of the underlying data." } } }, "Schema": { "name": "Schema", "kind": "class", "path": "dagpipe.yaml_loader.Schema", "signature": "", "docstring": "Immutable hierarchical schema defining allowed payload structure.\n\nAttributes:\n tree (Mapping[str, SchemaNode]):\n Hierarchical schema definition.\n\nNotes:\n **Responsibilities:**\n\n - Validates `State` payloads and updates.\n - Reusable across all `State` instances.\n - Fully thread-safe due to immutability.", "members": { "tree": { "name": "tree", "kind": "attribute", "path": "dagpipe.yaml_loader.Schema.tree", "signature": "", "docstring": null }, "validate_payload": { "name": "validate_payload", "kind": "function", "path": "dagpipe.yaml_loader.Schema.validate_payload", "signature": "", "docstring": "Validate complete payload structure.\n\nArgs:\n payload (Payload):\n Payload to validate.\n\nRaises:\n SchemaError:\n If payload violates schema." }, "validate_update": { "name": "validate_update", "kind": "function", "path": "dagpipe.yaml_loader.Schema.validate_update", "signature": "", "docstring": "Validate payload update paths.\n\nArgs:\n updates (Mapping[str, Any]):\n Dot-path updates to validate.\n\nRaises:\n SchemaError:\n If any path is invalid according to the schema." } } }, "State": { "name": "State", "kind": "class", "path": "dagpipe.yaml_loader.State", "signature": "", "docstring": "Immutable execution state propagated through dagpipe pipeline.\n\nAttributes:\n payload (Payload):\n Execution data container.\n\n schema (ClassVar[Schema]):\n Payload validation schema.\n\n confidence (float):\n Execution confidence score.\n\n parent (Optional[State]):\n Parent state reference.\n\n depth (int):\n Execution depth.\n\n history (Tuple[str, ...]):\n Ordered node execution lineage.\n\n metadata (Dict[str, Any]):\n Execution metadata.\n\nNotes:\n **Responsibilities:**\n\n - Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n - Fundamental unit of execution in `dagpipe`.\n - Fully thread-safe due to immutability.", "members": { "payload": { "name": "payload", "kind": "attribute", "path": "dagpipe.yaml_loader.State.payload", "signature": "", "docstring": null }, "schema": { "name": "schema", "kind": "attribute", "path": "dagpipe.yaml_loader.State.schema", "signature": "", "docstring": null }, "confidence": { "name": "confidence", "kind": "attribute", "path": "dagpipe.yaml_loader.State.confidence", "signature": "", "docstring": null }, "parent": { "name": "parent", "kind": "attribute", "path": "dagpipe.yaml_loader.State.parent", "signature": "", "docstring": null }, "depth": { "name": "depth", "kind": "attribute", "path": "dagpipe.yaml_loader.State.depth", "signature": "", "docstring": null }, "history": { "name": "history", "kind": "attribute", "path": "dagpipe.yaml_loader.State.history", "signature": "", "docstring": null }, "metadata": { "name": "metadata", "kind": "attribute", "path": "dagpipe.yaml_loader.State.metadata", "signature": "", "docstring": null }, "fork": { "name": "fork", "kind": "function", "path": "dagpipe.yaml_loader.State.fork", "signature": "", "docstring": "Create a new child `State` derived from this state.\n\nArgs:\n payload_update (Mapping[str, Any], optional):\n Dot-path updates applied to the payload.\n\n confidence_delta (float, optional):\n Adjustment applied to current confidence.\n\n node_id (str, optional):\n Identifier of the node creating this state.\n\n metadata_update (Mapping[str, Any], optional):\n Updates merged into state metadata.\n\nReturns:\n State:\n A new immutable `State` instance.\n\nNotes:\n **Guarantees:**\n\n - This is the only supported mechanism for modifying execution data.\n - Validates payload updates, preserves lineage, increments depth,\n and appends to history." }, "lineage": { "name": "lineage", "kind": "function", "path": "dagpipe.yaml_loader.State.lineage", "signature": "", "docstring": "Return lineage from root to this State.\n\nReturns:\n Tuple[State, ...]:\n Ordered execution lineage (root first)." }, "get": { "name": "get", "kind": "function", "path": "dagpipe.yaml_loader.State.get", "signature": "", "docstring": "Retrieve payload value.\n\nArgs:\n key (str):\n Dot-path key.\n default (Any, optional):\n Fallback value.\n\nReturns:\n Any:\n Stored value or default." }, "has": { "name": "has", "kind": "function", "path": "dagpipe.yaml_loader.State.has", "signature": "", "docstring": "Check whether payload contains key.\n\nArgs:\n key (str):\n Dot-path key.\n\nReturns:\n bool:\n Existence of the key." } } }, "Pipeline": { "name": "Pipeline", "kind": "class", "path": "dagpipe.yaml_loader.Pipeline", "signature": "", "docstring": "Executable pipeline created from YAML configuration.\n\nAttributes:\n engine (Engine):\n Execution engine responsible for running the pipeline.\n\n state_cls (Type[State]):\n Dynamically created `State` subclass with configured schema.\n\n initial_payload (Payload):\n Default payload used when execution begins.\n\nNotes:\n **Responsibilities:**\n\n - Encapsulates engine, state type, and initial payload.\n - Provides a simplified interface for executing configured pipelines.\n - Safe for concurrent execution if underlying nodes are thread-safe.", "members": { "engine": { "name": "engine", "kind": "attribute", "path": "dagpipe.yaml_loader.Pipeline.engine", "signature": null, "docstring": null }, "state_cls": { "name": "state_cls", "kind": "attribute", "path": "dagpipe.yaml_loader.Pipeline.state_cls", "signature": null, "docstring": null }, "initial_payload": { "name": "initial_payload", "kind": "attribute", "path": "dagpipe.yaml_loader.Pipeline.initial_payload", "signature": null, "docstring": null }, "run": { "name": "run", "kind": "function", "path": "dagpipe.yaml_loader.Pipeline.run", "signature": "", "docstring": "Execute the pipeline.\n\nArgs:\n payload_override (Mapping[str, Any], optional):\n Payload values overriding initial payload.\n\nReturns:\n list[State]:\n Terminal execution states.\n\nNotes:\n **Responsibilities:**\n\n - Merges override payload with initial payload.\n - Creates root `State` and executes engine." } } }, "load_pipeline": { "name": "load_pipeline", "kind": "function", "path": "dagpipe.yaml_loader.load_pipeline", "signature": "", "docstring": "Load pipeline from YAML file.\n\nArgs:\n path (str):\n Path to YAML configuration file.\n\nReturns:\n Pipeline:\n Executable pipeline instance.\n\nNotes:\n **Responsibilities:**\n\n - Loads YAML configuration and builds schema.\n - Creates `State` subclass and loads `Node` instances.\n - Builds `Graph` topology and initializes `Engine`." } } } } } }