{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"dagpipe","text":""},{"location":"#dagpipe","title":"dagpipe","text":""},{"location":"#dagpipe--summary","title":"Summary","text":"
Directed acyclic graph execution framework for deterministic state propagation.
dagpipe executes pipelines composed of nodes connected in a directed acyclic graph (DAG). Each node receives an immutable State and optionally produces derived states for downstream nodes.
Install using pip:
pip install dagpipe\n"},{"location":"#dagpipe--quick-start","title":"Quick Start","text":"from dagpipe import State, Payload, Schema, Graph, Engine\nfrom dagpipe.node import Node\n\nclass HelloNode(Node):\n id = \"hello\"\n def resolve(self, state):\n yield self.fork(state, payload_update={\"msg\": \"hello\"})\n\n# Build and run\ngraph = Graph()\ngraph.add_root(HelloNode())\nengine = Engine(graph)\n\nclass MyState(State):\n schema = Schema({})\n\nresults = engine.run(MyState(payload=Payload({})))\n"},{"location":"#dagpipe--public-api","title":"Public API","text":"This package re-exports the core pipeline components. Consumers should import from this namespace for standard usage.
"},{"location":"#dagpipe--execution-core","title":"Execution Core","text":"Engine: Responsible for orchestrating node execution and state propagation.Graph: Defines the execution topology and node relationships.Node: Base class for defining execution logic and transformations.State: Represents an immutable execution snapshot at a point in time.Payload: Immutable hierarchical container for execution data.Schema: Defines and validates the allowed structure of payloads.SchemaError: Raised when data violates the declared schema.Pipeline: High-level wrapper for an engine, state type, and initial payload.load_pipeline: Factory function to create a pipeline from YAML. Bases: Node
Base class for nodes whose execution is asynchronous.
Subclasses implement resolve_async (an async generator yielding derived State objects). The engine dispatches to resolve_async when running an async traversal (see Engine.run_async).
Sync-only engines (and the base Node.run) treat an AsyncNode as a no-op consumer: calling run on an AsyncNode returns no states, signalling that an async engine is required.
__hash__() -> int\n Return stable hash based on node ID.
Returns:
Name Type Descriptionint int Hash of the node ID, allowing nodes to be used as dict keys.
"},{"location":"#dagpipe.AsyncNode.__new__","title":"__new__","text":"__new__(*args: Any, **kwargs: Any) -> AsyncNode\n Create or reuse an async node instance.
Parameters:
Name Type Description Default*args Any Positional constructor arguments forwarded to __init__.
() **kwargs Any Keyword constructor arguments forwarded to __init__.
{} Returns:
Name Type DescriptionAsyncNode AsyncNode A fresh instance for subclasses declaring a parameterized __init__, or the shared singleton for stateless subclasses.
__repr__() -> str\n Return computation identity based on node ID.
Returns:
Name Type Descriptionstr str String of the form <Node {id}>.
__str__() -> str\n Return user-facing display name.
Returns:
Name Type Descriptionstr str String of the form <Node {name}>.
classmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"#dagpipe.AsyncNode.fork","title":"fork","text":"fork(\n state: State,\n *,\n payload_update: Any = None,\n confidence_delta: float = 0.0,\n metadata_update: Any = None\n) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Any Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Any Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history.\n"},{"location":"#dagpipe.AsyncNode.is_async","title":"is_async","text":"is_async() -> bool\n Return whether this node executes asynchronously.
Returns:
Name Type Descriptionbool bool True if the node is an AsyncNode instance.
staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"#dagpipe.AsyncNode.resolve","title":"resolve","text":"resolve(state: State) -> Iterable[State]\n Execute no-op resolution in sync contexts.
Sync-only engines (and the base Node.run) treat an AsyncNode as a no-op consumer: this returns no states, signalling that an async engine is required.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Empty tuple, since async execution is handled by resolve_async.
async","text":"resolve_async(state: State) -> Iterable[State]\n Execute node logic asynchronously.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Derived execution state(s).
NotesGuarantees:
- Subclasses implement this.\n- Must not mutate the input state.\n- Should use `fork()` to create child states.\n"},{"location":"#dagpipe.AsyncNode.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
async","text":"run_async(state: State) -> tuple[State, ...]\n Execute this node asynchronously on a state, validating outputs.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve_async() yields a non-State object.
Engine(\n nodes_or_graph: Sequence[Node] | Graph,\n *,\n on_step: StepHook | None = None\n)\n Execution engine responsible for running pipeline logic.
NotesResponsibilities:
- Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n- Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n- Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n- Supports step-wise / resumable execution and progress hooks.\n Guarantees:
- Never mutates `State`, `Node`, or `Graph` instances.\n- `State` objects are never modified in place; each branch produces\n independent instances.\n- Execution order is deterministic and follows graph or pipeline topology.\n- Thread-safe for concurrent execution.\n Create an engine from a node sequence or a graph.
Parameters:
Name Type Description Defaultnodes_or_graph Sequence[Node] | Graph Either an ordered sequence of Node instances (linear mode) or a Graph defining the execution topology (graph mode).
on_step StepHook | None Default per-step callback (step, status, message) used when a step run does not supply its own hook.
None Raises:
Type DescriptionTypeError If a sequence element is not a Node, or if nodes_or_graph is neither a Sequence[Node] nor a Graph.
property","text":"nodes: tuple[Node, ...]\n Return nodes managed by this engine.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Ordered sequence in linear mode or all nodes in graph mode.
"},{"location":"#dagpipe.Engine-functions","title":"Functions","text":""},{"location":"#dagpipe.Engine.__repr__","title":"__repr__","text":"__repr__() -> str\n Return the canonical string representation of the object.
Returns:
Name Type Descriptionstr str Representation that uniquely identifies the object and its configuration.
"},{"location":"#dagpipe.Engine.run","title":"run","text":"run(root: State) -> list[State]\n Execute the pipeline starting from a root State.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type Descriptionlist[State] list[State]: Terminal execution states produced by the pipeline.
Raises:
Type DescriptionTypeError If root is not a State instance.
RuntimeError If the engine execution mode is invalid.
NotesResponsibilities:
- Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states.\n"},{"location":"#dagpipe.Engine.run_async","title":"run_async async","text":"run_async(root: State) -> list[State]\n Execute the pipeline starting from root, dispatching sync vs async nodes.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type Descriptionlist[State] list[State]: Terminal execution states produced by the pipeline.
NotesEach node is executed with Node.run when synchronous and AsyncNode.run_async when asynchronous. Linear and graph topologies are both supported.
run_steps(\n root: State,\n *,\n resume_from: int | None = None,\n on_step: StepHook | None = None\n) -> Iterator[StepResult]\n Execute the pipeline step-by-step, yielding one StepResult per step.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredresume_from int | None Skip steps at index < resume_from (for resume-after-partial). Steps are 0-indexed.
None on_step StepHook | None Callback (step, status, message) invoked per step; falls back to the engine-level hook when unset.
None Yields:
Name Type DescriptionStepResult StepResult One per executed node/step, carrying the produced states.
NotesThis is a synchronous, generator-based checkpoint interface compatible with the imperative resume-by-step behaviour of the legacy orchestrator. Use run_steps_async for async nodes.
async","text":"run_steps_async(\n root: State,\n *,\n resume_from: int | None = None,\n on_step: AsyncStepHook | None = None\n) -> AsyncIterator[StepResult]\n Async variant of run_steps supporting AsyncNode execution.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredresume_from int | None Skip steps at index < resume_from.
None on_step AsyncStepHook | None Async callback (step, status, message) invoked per step.
None Yields:
Name Type DescriptionStepResult AsyncIterator[StepResult] One per executed node/step.
"},{"location":"#dagpipe.Graph","title":"Graph","text":"Graph()\n Directed Acyclic Graph defining execution topology of Node objects.
Responsibilities:
- Stores node connectivity and validates that the topology remains acyclic.\n- Structure determines how `State` flows between nodes during execution.\n Guarantees:
- Topology is acyclic. Node relationships remain consistent.\n- Thread-safe for concurrent reads after construction.\n Create an empty Graph.
Initializes node registry and edge mappings.
"},{"location":"#dagpipe.Graph-functions","title":"Functions","text":""},{"location":"#dagpipe.Graph.__repr__","title":"__repr__","text":"__repr__() -> str\n Return a compact graph description.
Returns:
Name Type Descriptionstr str A string describing the graph as Graph(nodes=N, edges=M) where N and M describe the current registry size.
add_edge(src: Node, dst: Node) -> None\n Add a directed edge from src to dst.
Parameters:
Name Type Description Defaultsrc Node Source node.
requireddst Node Destination node.
requiredRaises:
Type DescriptionTypeError If src or dst is not a Node.
ValueError If the edge would create a cycle or if src and dst are common.
add_root(node: Node) -> None\n Add a root node with no parents.
Parameters:
Name Type Description Defaultnode Node Node to add as a root.
requiredRaises:
Type DescriptionTypeError If node is not a Node instance.
"},{"location":"#dagpipe.Graph.children","title":"children","text":"children(node: Node) -> tuple[Node, ...]\n Return child nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Outgoing neighbors.
"},{"location":"#dagpipe.Graph.nodes","title":"nodes","text":"nodes() -> tuple[Node, ...]\n Return all nodes in the graph.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: All registered nodes.
"},{"location":"#dagpipe.Graph.parents","title":"parents","text":"parents(node: Node) -> tuple[Node, ...]\n Return parent nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Incoming neighbors.
"},{"location":"#dagpipe.Graph.roots","title":"roots","text":"roots() -> tuple[Node, ...]\n Return root nodes (nodes with no incoming edges).
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Entry point nodes.
"},{"location":"#dagpipe.Node","title":"Node","text":" Bases: ABC
Base class for all dagpipe execution nodes.
Attributes:
Name Type Descriptionid str Unique identifier of the node (snake_case dotted format).
name str Human-readable display name.
NotesResponsibilities:
- Represents a deterministic unit of execution in the pipeline graph.\n- Consumes one `State` and produces zero, one, or many derived states.\n- Defines execution logic and enables branching, filtering, and transformation.\n Guarantees:
- Nodes must never mutate the input `State`.\n- Instances are singletons per subclass and reused across executions.\n"},{"location":"#dagpipe.Node-functions","title":"Functions","text":""},{"location":"#dagpipe.Node.__hash__","title":"__hash__","text":"__hash__() -> int\n Return stable hash based on node ID.
Returns:
Name Type Descriptionint int Hash of the node ID, allowing nodes to be used as dict keys.
"},{"location":"#dagpipe.Node.__new__","title":"__new__","text":"__new__(*args: Any, **kwargs: Any) -> Node\n Create or reuse a node instance.
Parameters:
Name Type Description Default*args Any Positional constructor arguments forwarded to __init__.
() **kwargs Any Keyword constructor arguments forwarded to __init__.
{} Returns:
Name Type DescriptionNode Node A fresh instance for subclasses declaring a parameterized __init__, or the shared singleton for stateless subclasses.
Guarantees:
- Stateless subclasses (no parameterized `__init__`) share one\n singleton instance per class \u2014 matching the original dagpipe\n behaviour underpinning `set_registry`-style configuration.\n- Subclasses that declare an `__init__` requiring instance-state\n arguments get a fresh instance per construction so pipeline\n builders can inject per-run dependencies.\n"},{"location":"#dagpipe.Node.__repr__","title":"__repr__","text":"__repr__() -> str\n Return computation identity based on node ID.
Returns:
Name Type Descriptionstr str String of the form <Node {id}>.
__str__() -> str\n Return user-facing display name.
Returns:
Name Type Descriptionstr str String of the form <Node {name}>.
classmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"#dagpipe.Node.fork","title":"fork","text":"fork(\n state: State,\n *,\n payload_update: Any = None,\n confidence_delta: float = 0.0,\n metadata_update: Any = None\n) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Any Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Any Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history.\n"},{"location":"#dagpipe.Node.is_async","title":"is_async","text":"is_async() -> bool\n Return whether this node executes asynchronously.
Returns:
Name Type Descriptionbool bool True if the node is an AsyncNode instance.
staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"#dagpipe.Node.resolve","title":"resolveabstractmethod","text":"resolve(state: State) -> Iterable[State]\n Execute node logic.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Derived execution state(s).
NotesResponsibilities:
- Subclasses implement specific resolution behavior.\n- Must not mutate input state.\n- Should use `fork()` to create child states.\n- May yield zero states to terminate a branch.\n"},{"location":"#dagpipe.Node.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
dataclass","text":"Payload(_data: Mapping[str, Any])\n Immutable hierarchical container with dot-path access.
Attributes:
Name Type Description_data Mapping[str, Any] Immutable hierarchical data structure.
NotesResponsibilities:
- Stores execution data used by `State`.\n- Supports efficient atomic updates without modifying existing instances.\n- `Payload` instances are fully thread-safe due to immutability.\n"},{"location":"#dagpipe.Payload-functions","title":"Functions","text":""},{"location":"#dagpipe.Payload.__repr__","title":"__repr__","text":"__repr__() -> str\n Return a concise payload description.
Returns:
Name Type Descriptionstr str String of the form Payload(keys=[...]) listing top-level keys.
as_dict() -> Mapping[str, Any]\n Return underlying mapping.
Returns:
Type DescriptionMapping[str, Any] Mapping[str, Any]: Read-only view of the underlying data.
"},{"location":"#dagpipe.Payload.get","title":"get","text":"get(path: str, default: Any = None) -> Any\n Retrieve value using dot-path.
Parameters:
Name Type Description Defaultpath str Dot-separated path to the value.
requireddefault Any Default value if path doesn't exist.
None Returns:
Name Type DescriptionAny Any The retrieved value or default.
"},{"location":"#dagpipe.Payload.has","title":"has","text":"has(path: str) -> bool\n Return True if path exists.
Parameters:
Name Type Description Defaultpath str Dot-separated path to check.
requiredReturns:
Name Type Descriptionbool bool Existence of the path.
"},{"location":"#dagpipe.Payload.iter_paths","title":"iter_pathsclassmethod","text":"iter_paths(\n data: Mapping[str, Any], prefix: str = \"\"\n) -> Iterable[str]\n Recursively yield dot-paths for all leaf nodes.
Parameters:
Name Type Description Defaultdata Mapping[str, Any] The mapping to iterate over.
requiredprefix str Current path prefix.
'' Yields:
Name Type Descriptionstr Iterable[str] Dot-path for each leaf node.
"},{"location":"#dagpipe.Payload.keys","title":"keys","text":"keys() -> Iterable[str]\n Return top-level keys.
Returns:
Type DescriptionIterable[str] Iterable[str]: Iterator over top-level keys.
"},{"location":"#dagpipe.Payload.update","title":"update","text":"update(updates: Mapping[str, Any]) -> Payload\n Create a new Payload with dot-path updates applied.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path to value mapping.
requiredReturns:
Name Type DescriptionPayload Payload New immutable payload instance with updates.
NotesGuarantees:
- Preserves existing data by copying only modified branches.\n- Returns a new immutable `Payload`.\n"},{"location":"#dagpipe.Pipeline","title":"Pipeline dataclass","text":"Pipeline(\n engine: Engine,\n state_cls: type[State],\n initial_payload: Payload,\n)\n Executable pipeline created from YAML configuration.
Attributes:
Name Type Descriptionengine Engine Execution engine responsible for running the pipeline.
state_cls Type[State] Dynamically created State subclass with configured schema.
initial_payload Payload Default payload used when execution begins.
NotesResponsibilities:
- Encapsulates engine, state type, and initial payload.\n- Provides a simplified interface for executing configured pipelines.\n- Safe for concurrent execution if underlying nodes are thread-safe.\n"},{"location":"#dagpipe.Pipeline-functions","title":"Functions","text":""},{"location":"#dagpipe.Pipeline.run","title":"run","text":"run(\n payload_override: Mapping[str, Any] | None = None,\n) -> list[State]\n Execute the pipeline.
Parameters:
Name Type Description Defaultpayload_override Mapping[str, Any] | None Payload values overriding initial payload.
None Returns:
Type Descriptionlist[State] list[State]: Terminal execution states.
NotesResponsibilities:
- Merges override payload with initial payload.\n- Creates root `State` and executes engine.\n"},{"location":"#dagpipe.ProgressMessage","title":"ProgressMessage","text":"ProgressMessage(\n *,\n lines: int | None = None,\n blocks: int | None = None,\n count: int | None = None,\n unit: str | None = None,\n raw_ocr_line: str | None = None,\n error: str | None = None,\n step: str = \"\",\n status: str = \"\"\n)\n Lightweight progress payload emitted by engine step hooks.
Mirrors the imperative ProgressMessage used by the legacy orchestrator so callers can surface counts/lines/errors without coupling the engine to pydantic.
Attributes:
Name Type Descriptionlines int | None Optional count of processed lines.
blocks int | None Optional count of processed blocks.
count int | None Optional generic item count.
unit str | None Optional unit for the count (e.g., 'pages').
raw_ocr_line str | None Optional raw OCR line payload.
error str | None Optional error description.
step str Identifier of the step that emitted the message.
status str Status label associated with the step.
NotesGuarantees:
- Immutable after construction (attributes are never reassigned).\n- Independent of pydantic; safe to construct in the engine core.\n Create a progress message.
Parameters:
Name Type Description Defaultlines int | None Optional count of processed lines.
None blocks int | None Optional count of processed blocks.
None count int | None Optional generic item count.
None unit str | None Optional unit for the count (e.g., 'pages').
None raw_ocr_line str | None Optional raw OCR line payload.
None error str | None Optional error description.
None step str Identifier of the step that emitted the message.
'' status str Status label associated with the step.
''"},{"location":"#dagpipe.ProgressMessage-functions","title":"Functions","text":""},{"location":"#dagpipe.ProgressMessage.as_dict","title":"as_dict","text":"as_dict() -> dict[str, Any]\n Return the message as a plain dictionary.
Returns:
Type Descriptiondict[str, Any] dict[str, Any]: All attribute values keyed by their attribute name.
"},{"location":"#dagpipe.Schema","title":"Schemadataclass","text":"Schema(tree: Mapping[str, SchemaNode])\n Immutable hierarchical schema defining allowed payload structure.
Attributes:
Name Type Descriptiontree Mapping[str, SchemaNode] Hierarchical schema definition.
NotesResponsibilities:
- Validates `State` payloads and updates.\n- Reusable across all `State` instances.\n- Fully thread-safe due to immutability.\n"},{"location":"#dagpipe.Schema-functions","title":"Functions","text":""},{"location":"#dagpipe.Schema.validate_payload","title":"validate_payload","text":"validate_payload(payload: Payload) -> None\n Validate complete payload structure.
Parameters:
Name Type Description Defaultpayload Payload Payload to validate.
requiredRaises:
Type DescriptionSchemaError If payload violates schema.
"},{"location":"#dagpipe.Schema.validate_update","title":"validate_update","text":"validate_update(updates: Mapping[str, Any]) -> None\n Validate payload update paths.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path updates to validate.
requiredRaises:
Type DescriptionSchemaError If any path is invalid according to the schema.
"},{"location":"#dagpipe.SchemaError","title":"SchemaError","text":" Bases: Exception
Raised when payload data violates the declared schema.
Indicates invalid structure, invalid path, or invalid type.
"},{"location":"#dagpipe.State","title":"Statedataclass","text":"State(\n payload: Payload,\n confidence: float = 1.0,\n parent: State | None = None,\n depth: int = 0,\n history: tuple[str, ...] = tuple(),\n metadata: dict[str, Any] = dict(),\n)\n Immutable execution state propagated through dagpipe pipeline.
Attributes:
Name Type Descriptionpayload Payload Execution data container.
schema ClassVar[Schema] Payload validation schema.
confidence float Execution confidence score.
parent Optional[State] Parent state reference.
depth int Execution depth.
history Tuple[str, ...] Ordered node execution lineage.
metadata Dict[str, Any] Execution metadata.
NotesResponsibilities:
- Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n- Fundamental unit of execution in `dagpipe`.\n- Fully thread-safe due to immutability.\n"},{"location":"#dagpipe.State-functions","title":"Functions","text":""},{"location":"#dagpipe.State.__post_init__","title":"__post_init__","text":"__post_init__() -> None\n Validate the payload against the declared schema.
Raises:
Type DescriptionSchemaError If the payload violates the schema declared on the subclass.
"},{"location":"#dagpipe.State.__repr__","title":"__repr__","text":"__repr__() -> str\n Concise debug representation.
Avoids printing full data for large states.
"},{"location":"#dagpipe.State.fork","title":"fork","text":"fork(\n *,\n payload_update: Mapping[str, Any] | None = None,\n confidence_delta: float = 0.0,\n node_id: str | None = None,\n metadata_update: Mapping[str, Any] | None = None\n) -> State\n Create a new child State derived from this state.
Parameters:
Name Type Description Defaultpayload_update Mapping[str, Any] | None Dot-path updates applied to the payload.
None confidence_delta float Adjustment applied to current confidence.
0.0 node_id str | None Identifier of the node creating this state.
None metadata_update Mapping[str, Any] | None Updates merged into state metadata.
None Returns:
Name Type DescriptionState State A new immutable State instance.
Guarantees:
- This is the only supported mechanism for modifying execution data.\n- Validates payload updates, preserves lineage, increments depth,\n and appends to history.\n"},{"location":"#dagpipe.State.get","title":"get","text":"get(key: str, default: Any = None) -> Any\n Retrieve payload value.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requireddefault Any Fallback value.
None Returns:
Name Type DescriptionAny Any Stored value or default.
"},{"location":"#dagpipe.State.has","title":"has","text":"has(key: str) -> bool\n Check whether payload contains key.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requiredReturns:
Name Type Descriptionbool bool Existence of the key.
"},{"location":"#dagpipe.State.lineage","title":"lineage","text":"lineage() -> tuple[State, ...]\n Return lineage from root to this State.
Returns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Ordered execution lineage (root first).
"},{"location":"#dagpipe.StepResult","title":"StepResult","text":"StepResult(\n index: int,\n node_id: str,\n states: tuple[State, ...],\n completed: bool,\n)\n A single checkpointed step within an async/resumable engine run.
Attributes:
Name Type Descriptionindex int Ordinal index of the step.
node_id str Identifier of the node associated with this step.
states tuple[State, ...] States produced by running this step.
completed bool Whether this step succeeded (vs. paused/interrupted).
Initialise StepResult.
Parameters:
Name Type Description Defaultindex int Ordinal index of the step.
requirednode_id str Identifier of the node associated with this step.
requiredstates tuple[State, ...] States produced by running this step.
requiredcompleted bool Whether this step succeeded (vs. paused/interrupted).
required"},{"location":"#dagpipe.StepResult-functions","title":"Functions","text":""},{"location":"#dagpipe-functions","title":"Functions","text":""},{"location":"#dagpipe.load_pipeline","title":"load_pipeline","text":"load_pipeline(path: str) -> Pipeline\n Load pipeline from YAML file.
Parameters:
Name Type Description Defaultpath str Path to YAML configuration file.
requiredReturns:
Name Type DescriptionPipeline Pipeline Executable pipeline instance.
NotesResponsibilities:
- Loads YAML configuration and builds schema.\n- Creates `State` subclass and loads `Node` instances.\n- Builds `Graph` topology and initializes `Engine`.\n"},{"location":"engine/","title":"Engine","text":""},{"location":"engine/#dagpipe.engine","title":"dagpipe.engine","text":""},{"location":"engine/#dagpipe.engine--summary","title":"Summary","text":"Execution engine responsible for running pipelines and graphs.
The Engine executes Node objects and propagates immutable State instances through either a linear sequence or a directed acyclic graph (Graph). It orchestrates execution order, branching, and state propagation.
Graph, Node, or State objects.Engine(\n nodes_or_graph: Sequence[Node] | Graph,\n *,\n on_step: StepHook | None = None\n)\n Execution engine responsible for running pipeline logic.
NotesResponsibilities:
- Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n- Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n- Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n- Supports step-wise / resumable execution and progress hooks.\n Guarantees:
- Never mutates `State`, `Node`, or `Graph` instances.\n- `State` objects are never modified in place; each branch produces\n independent instances.\n- Execution order is deterministic and follows graph or pipeline topology.\n- Thread-safe for concurrent execution.\n Create an engine from a node sequence or a graph.
Parameters:
Name Type Description Defaultnodes_or_graph Sequence[Node] | Graph Either an ordered sequence of Node instances (linear mode) or a Graph defining the execution topology (graph mode).
on_step StepHook | None Default per-step callback (step, status, message) used when a step run does not supply its own hook.
None Raises:
Type DescriptionTypeError If a sequence element is not a Node, or if nodes_or_graph is neither a Sequence[Node] nor a Graph.
property","text":"nodes: tuple[Node, ...]\n Return nodes managed by this engine.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Ordered sequence in linear mode or all nodes in graph mode.
"},{"location":"engine/#dagpipe.engine.Engine-functions","title":"Functions","text":""},{"location":"engine/#dagpipe.engine.Engine.__repr__","title":"__repr__","text":"__repr__() -> str\n Return the canonical string representation of the object.
Returns:
Name Type Descriptionstr str Representation that uniquely identifies the object and its configuration.
"},{"location":"engine/#dagpipe.engine.Engine.run","title":"run","text":"run(root: State) -> list[State]\n Execute the pipeline starting from a root State.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type Descriptionlist[State] list[State]: Terminal execution states produced by the pipeline.
Raises:
Type DescriptionTypeError If root is not a State instance.
RuntimeError If the engine execution mode is invalid.
NotesResponsibilities:
- Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states.\n"},{"location":"engine/#dagpipe.engine.Engine.run_async","title":"run_async async","text":"run_async(root: State) -> list[State]\n Execute the pipeline starting from root, dispatching sync vs async nodes.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type Descriptionlist[State] list[State]: Terminal execution states produced by the pipeline.
NotesEach node is executed with Node.run when synchronous and AsyncNode.run_async when asynchronous. Linear and graph topologies are both supported.
run_steps(\n root: State,\n *,\n resume_from: int | None = None,\n on_step: StepHook | None = None\n) -> Iterator[StepResult]\n Execute the pipeline step-by-step, yielding one StepResult per step.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredresume_from int | None Skip steps at index < resume_from (for resume-after-partial). Steps are 0-indexed.
None on_step StepHook | None Callback (step, status, message) invoked per step; falls back to the engine-level hook when unset.
None Yields:
Name Type DescriptionStepResult StepResult One per executed node/step, carrying the produced states.
NotesThis is a synchronous, generator-based checkpoint interface compatible with the imperative resume-by-step behaviour of the legacy orchestrator. Use run_steps_async for async nodes.
async","text":"run_steps_async(\n root: State,\n *,\n resume_from: int | None = None,\n on_step: AsyncStepHook | None = None\n) -> AsyncIterator[StepResult]\n Async variant of run_steps supporting AsyncNode execution.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredresume_from int | None Skip steps at index < resume_from.
None on_step AsyncStepHook | None Async callback (step, status, message) invoked per step.
None Yields:
Name Type DescriptionStepResult AsyncIterator[StepResult] One per executed node/step.
"},{"location":"engine/#dagpipe.engine.ProgressMessage","title":"ProgressMessage","text":"ProgressMessage(\n *,\n lines: int | None = None,\n blocks: int | None = None,\n count: int | None = None,\n unit: str | None = None,\n raw_ocr_line: str | None = None,\n error: str | None = None,\n step: str = \"\",\n status: str = \"\"\n)\n Lightweight progress payload emitted by engine step hooks.
Mirrors the imperative ProgressMessage used by the legacy orchestrator so callers can surface counts/lines/errors without coupling the engine to pydantic.
Attributes:
Name Type Descriptionlines int | None Optional count of processed lines.
blocks int | None Optional count of processed blocks.
count int | None Optional generic item count.
unit str | None Optional unit for the count (e.g., 'pages').
raw_ocr_line str | None Optional raw OCR line payload.
error str | None Optional error description.
step str Identifier of the step that emitted the message.
status str Status label associated with the step.
NotesGuarantees:
- Immutable after construction (attributes are never reassigned).\n- Independent of pydantic; safe to construct in the engine core.\n Create a progress message.
Parameters:
Name Type Description Defaultlines int | None Optional count of processed lines.
None blocks int | None Optional count of processed blocks.
None count int | None Optional generic item count.
None unit str | None Optional unit for the count (e.g., 'pages').
None raw_ocr_line str | None Optional raw OCR line payload.
None error str | None Optional error description.
None step str Identifier of the step that emitted the message.
'' status str Status label associated with the step.
''"},{"location":"engine/#dagpipe.engine.ProgressMessage-functions","title":"Functions","text":""},{"location":"engine/#dagpipe.engine.ProgressMessage.as_dict","title":"as_dict","text":"as_dict() -> dict[str, Any]\n Return the message as a plain dictionary.
Returns:
Type Descriptiondict[str, Any] dict[str, Any]: All attribute values keyed by their attribute name.
"},{"location":"engine/#dagpipe.engine.StepResult","title":"StepResult","text":"StepResult(\n index: int,\n node_id: str,\n states: tuple[State, ...],\n completed: bool,\n)\n A single checkpointed step within an async/resumable engine run.
Attributes:
Name Type Descriptionindex int Ordinal index of the step.
node_id str Identifier of the node associated with this step.
states tuple[State, ...] States produced by running this step.
completed bool Whether this step succeeded (vs. paused/interrupted).
Initialise StepResult.
Parameters:
Name Type Description Defaultindex int Ordinal index of the step.
requirednode_id str Identifier of the node associated with this step.
requiredstates tuple[State, ...] States produced by running this step.
requiredcompleted bool Whether this step succeeded (vs. paused/interrupted).
required"},{"location":"engine/#dagpipe.engine.StepResult-functions","title":"Functions","text":""},{"location":"graph/","title":"Graph","text":""},{"location":"graph/#dagpipe.graph","title":"dagpipe.graph","text":""},{"location":"graph/#dagpipe.graph--summary","title":"Summary","text":"Defines DAG structure connecting nodes.
A Graph describes execution topology only. It does not execute nodes or manage State. Execution is handled by an Engine.
Graph()\n Directed Acyclic Graph defining execution topology of Node objects.
Responsibilities:
- Stores node connectivity and validates that the topology remains acyclic.\n- Structure determines how `State` flows between nodes during execution.\n Guarantees:
- Topology is acyclic. Node relationships remain consistent.\n- Thread-safe for concurrent reads after construction.\n Create an empty Graph.
Initializes node registry and edge mappings.
"},{"location":"graph/#dagpipe.graph.Graph-functions","title":"Functions","text":""},{"location":"graph/#dagpipe.graph.Graph.__repr__","title":"__repr__","text":"__repr__() -> str\n Return a compact graph description.
Returns:
Name Type Descriptionstr str A string describing the graph as Graph(nodes=N, edges=M) where N and M describe the current registry size.
add_edge(src: Node, dst: Node) -> None\n Add a directed edge from src to dst.
Parameters:
Name Type Description Defaultsrc Node Source node.
requireddst Node Destination node.
requiredRaises:
Type DescriptionTypeError If src or dst is not a Node.
ValueError If the edge would create a cycle or if src and dst are common.
add_root(node: Node) -> None\n Add a root node with no parents.
Parameters:
Name Type Description Defaultnode Node Node to add as a root.
requiredRaises:
Type DescriptionTypeError If node is not a Node instance.
"},{"location":"graph/#dagpipe.graph.Graph.children","title":"children","text":"children(node: Node) -> tuple[Node, ...]\n Return child nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Outgoing neighbors.
"},{"location":"graph/#dagpipe.graph.Graph.nodes","title":"nodes","text":"nodes() -> tuple[Node, ...]\n Return all nodes in the graph.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: All registered nodes.
"},{"location":"graph/#dagpipe.graph.Graph.parents","title":"parents","text":"parents(node: Node) -> tuple[Node, ...]\n Return parent nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Incoming neighbors.
"},{"location":"graph/#dagpipe.graph.Graph.roots","title":"roots","text":"roots() -> tuple[Node, ...]\n Return root nodes (nodes with no incoming edges).
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Entry point nodes.
"},{"location":"node/","title":"Node","text":""},{"location":"node/#dagpipe.node","title":"dagpipe.node","text":""},{"location":"node/#dagpipe.node--summary","title":"Summary","text":"Defines the Node abstraction used by dagpipe.
A node represents a single unit of pipeline execution logic. It consumes one State and produces zero, one, or many new State objects.
Nodes are connected using a Graph and executed by an Engine.
Bases: Node
Base class for nodes whose execution is asynchronous.
Subclasses implement resolve_async (an async generator yielding derived State objects). The engine dispatches to resolve_async when running an async traversal (see Engine.run_async).
Sync-only engines (and the base Node.run) treat an AsyncNode as a no-op consumer: calling run on an AsyncNode returns no states, signalling that an async engine is required.
__hash__() -> int\n Return stable hash based on node ID.
Returns:
Name Type Descriptionint int Hash of the node ID, allowing nodes to be used as dict keys.
"},{"location":"node/#dagpipe.node.AsyncNode.__new__","title":"__new__","text":"__new__(*args: Any, **kwargs: Any) -> AsyncNode\n Create or reuse an async node instance.
Parameters:
Name Type Description Default*args Any Positional constructor arguments forwarded to __init__.
() **kwargs Any Keyword constructor arguments forwarded to __init__.
{} Returns:
Name Type DescriptionAsyncNode AsyncNode A fresh instance for subclasses declaring a parameterized __init__, or the shared singleton for stateless subclasses.
__repr__() -> str\n Return computation identity based on node ID.
Returns:
Name Type Descriptionstr str String of the form <Node {id}>.
__str__() -> str\n Return user-facing display name.
Returns:
Name Type Descriptionstr str String of the form <Node {name}>.
classmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"node/#dagpipe.node.AsyncNode.fork","title":"fork","text":"fork(\n state: State,\n *,\n payload_update: Any = None,\n confidence_delta: float = 0.0,\n metadata_update: Any = None\n) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Any Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Any Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history.\n"},{"location":"node/#dagpipe.node.AsyncNode.is_async","title":"is_async","text":"is_async() -> bool\n Return whether this node executes asynchronously.
Returns:
Name Type Descriptionbool bool True if the node is an AsyncNode instance.
staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"node/#dagpipe.node.AsyncNode.resolve","title":"resolve","text":"resolve(state: State) -> Iterable[State]\n Execute no-op resolution in sync contexts.
Sync-only engines (and the base Node.run) treat an AsyncNode as a no-op consumer: this returns no states, signalling that an async engine is required.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Empty tuple, since async execution is handled by resolve_async.
async","text":"resolve_async(state: State) -> Iterable[State]\n Execute node logic asynchronously.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Derived execution state(s).
NotesGuarantees:
- Subclasses implement this.\n- Must not mutate the input state.\n- Should use `fork()` to create child states.\n"},{"location":"node/#dagpipe.node.AsyncNode.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
async","text":"run_async(state: State) -> tuple[State, ...]\n Execute this node asynchronously on a state, validating outputs.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve_async() yields a non-State object.
Bases: ABC
Base class for all dagpipe execution nodes.
Attributes:
Name Type Descriptionid str Unique identifier of the node (snake_case dotted format).
name str Human-readable display name.
NotesResponsibilities:
- Represents a deterministic unit of execution in the pipeline graph.\n- Consumes one `State` and produces zero, one, or many derived states.\n- Defines execution logic and enables branching, filtering, and transformation.\n Guarantees:
- Nodes must never mutate the input `State`.\n- Instances are singletons per subclass and reused across executions.\n"},{"location":"node/#dagpipe.node.Node-functions","title":"Functions","text":""},{"location":"node/#dagpipe.node.Node.__hash__","title":"__hash__","text":"__hash__() -> int\n Return stable hash based on node ID.
Returns:
Name Type Descriptionint int Hash of the node ID, allowing nodes to be used as dict keys.
"},{"location":"node/#dagpipe.node.Node.__new__","title":"__new__","text":"__new__(*args: Any, **kwargs: Any) -> Node\n Create or reuse a node instance.
Parameters:
Name Type Description Default*args Any Positional constructor arguments forwarded to __init__.
() **kwargs Any Keyword constructor arguments forwarded to __init__.
{} Returns:
Name Type DescriptionNode Node A fresh instance for subclasses declaring a parameterized __init__, or the shared singleton for stateless subclasses.
Guarantees:
- Stateless subclasses (no parameterized `__init__`) share one\n singleton instance per class \u2014 matching the original dagpipe\n behaviour underpinning `set_registry`-style configuration.\n- Subclasses that declare an `__init__` requiring instance-state\n arguments get a fresh instance per construction so pipeline\n builders can inject per-run dependencies.\n"},{"location":"node/#dagpipe.node.Node.__repr__","title":"__repr__","text":"__repr__() -> str\n Return computation identity based on node ID.
Returns:
Name Type Descriptionstr str String of the form <Node {id}>.
__str__() -> str\n Return user-facing display name.
Returns:
Name Type Descriptionstr str String of the form <Node {name}>.
classmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"node/#dagpipe.node.Node.fork","title":"fork","text":"fork(\n state: State,\n *,\n payload_update: Any = None,\n confidence_delta: float = 0.0,\n metadata_update: Any = None\n) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Any Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Any Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history.\n"},{"location":"node/#dagpipe.node.Node.is_async","title":"is_async","text":"is_async() -> bool\n Return whether this node executes asynchronously.
Returns:
Name Type Descriptionbool bool True if the node is an AsyncNode instance.
staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"node/#dagpipe.node.Node.resolve","title":"resolveabstractmethod","text":"resolve(state: State) -> Iterable[State]\n Execute node logic.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Derived execution state(s).
NotesResponsibilities:
- Subclasses implement specific resolution behavior.\n- Must not mutate input state.\n- Should use `fork()` to create child states.\n- May yield zero states to terminate a branch.\n"},{"location":"node/#dagpipe.node.Node.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
Defines the core State object used by dagpipe.
The State represents a single point in pipeline execution. It contains arbitrary data and metadata and is designed to be immutable. Instead of modifying an existing state, nodes create new child states via fork().
fork().dataclass","text":"Payload(_data: Mapping[str, Any])\n Immutable hierarchical container with dot-path access.
Attributes:
Name Type Description_data Mapping[str, Any] Immutable hierarchical data structure.
NotesResponsibilities:
- Stores execution data used by `State`.\n- Supports efficient atomic updates without modifying existing instances.\n- `Payload` instances are fully thread-safe due to immutability.\n"},{"location":"state/#dagpipe.state.Payload-functions","title":"Functions","text":""},{"location":"state/#dagpipe.state.Payload.__repr__","title":"__repr__","text":"__repr__() -> str\n Return a concise payload description.
Returns:
Name Type Descriptionstr str String of the form Payload(keys=[...]) listing top-level keys.
as_dict() -> Mapping[str, Any]\n Return underlying mapping.
Returns:
Type DescriptionMapping[str, Any] Mapping[str, Any]: Read-only view of the underlying data.
"},{"location":"state/#dagpipe.state.Payload.get","title":"get","text":"get(path: str, default: Any = None) -> Any\n Retrieve value using dot-path.
Parameters:
Name Type Description Defaultpath str Dot-separated path to the value.
requireddefault Any Default value if path doesn't exist.
None Returns:
Name Type DescriptionAny Any The retrieved value or default.
"},{"location":"state/#dagpipe.state.Payload.has","title":"has","text":"has(path: str) -> bool\n Return True if path exists.
Parameters:
Name Type Description Defaultpath str Dot-separated path to check.
requiredReturns:
Name Type Descriptionbool bool Existence of the path.
"},{"location":"state/#dagpipe.state.Payload.iter_paths","title":"iter_pathsclassmethod","text":"iter_paths(\n data: Mapping[str, Any], prefix: str = \"\"\n) -> Iterable[str]\n Recursively yield dot-paths for all leaf nodes.
Parameters:
Name Type Description Defaultdata Mapping[str, Any] The mapping to iterate over.
requiredprefix str Current path prefix.
'' Yields:
Name Type Descriptionstr Iterable[str] Dot-path for each leaf node.
"},{"location":"state/#dagpipe.state.Payload.keys","title":"keys","text":"keys() -> Iterable[str]\n Return top-level keys.
Returns:
Type DescriptionIterable[str] Iterable[str]: Iterator over top-level keys.
"},{"location":"state/#dagpipe.state.Payload.update","title":"update","text":"update(updates: Mapping[str, Any]) -> Payload\n Create a new Payload with dot-path updates applied.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path to value mapping.
requiredReturns:
Name Type DescriptionPayload Payload New immutable payload instance with updates.
NotesGuarantees:
- Preserves existing data by copying only modified branches.\n- Returns a new immutable `Payload`.\n"},{"location":"state/#dagpipe.state.Schema","title":"Schema dataclass","text":"Schema(tree: Mapping[str, SchemaNode])\n Immutable hierarchical schema defining allowed payload structure.
Attributes:
Name Type Descriptiontree Mapping[str, SchemaNode] Hierarchical schema definition.
NotesResponsibilities:
- Validates `State` payloads and updates.\n- Reusable across all `State` instances.\n- Fully thread-safe due to immutability.\n"},{"location":"state/#dagpipe.state.Schema-functions","title":"Functions","text":""},{"location":"state/#dagpipe.state.Schema.validate_payload","title":"validate_payload","text":"validate_payload(payload: Payload) -> None\n Validate complete payload structure.
Parameters:
Name Type Description Defaultpayload Payload Payload to validate.
requiredRaises:
Type DescriptionSchemaError If payload violates schema.
"},{"location":"state/#dagpipe.state.Schema.validate_update","title":"validate_update","text":"validate_update(updates: Mapping[str, Any]) -> None\n Validate payload update paths.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path updates to validate.
requiredRaises:
Type DescriptionSchemaError If any path is invalid according to the schema.
"},{"location":"state/#dagpipe.state.SchemaError","title":"SchemaError","text":" Bases: Exception
Raised when payload data violates the declared schema.
Indicates invalid structure, invalid path, or invalid type.
"},{"location":"state/#dagpipe.state.State","title":"Statedataclass","text":"State(\n payload: Payload,\n confidence: float = 1.0,\n parent: State | None = None,\n depth: int = 0,\n history: tuple[str, ...] = tuple(),\n metadata: dict[str, Any] = dict(),\n)\n Immutable execution state propagated through dagpipe pipeline.
Attributes:
Name Type Descriptionpayload Payload Execution data container.
schema ClassVar[Schema] Payload validation schema.
confidence float Execution confidence score.
parent Optional[State] Parent state reference.
depth int Execution depth.
history Tuple[str, ...] Ordered node execution lineage.
metadata Dict[str, Any] Execution metadata.
NotesResponsibilities:
- Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n- Fundamental unit of execution in `dagpipe`.\n- Fully thread-safe due to immutability.\n"},{"location":"state/#dagpipe.state.State-functions","title":"Functions","text":""},{"location":"state/#dagpipe.state.State.__post_init__","title":"__post_init__","text":"__post_init__() -> None\n Validate the payload against the declared schema.
Raises:
Type DescriptionSchemaError If the payload violates the schema declared on the subclass.
"},{"location":"state/#dagpipe.state.State.__repr__","title":"__repr__","text":"__repr__() -> str\n Concise debug representation.
Avoids printing full data for large states.
"},{"location":"state/#dagpipe.state.State.fork","title":"fork","text":"fork(\n *,\n payload_update: Mapping[str, Any] | None = None,\n confidence_delta: float = 0.0,\n node_id: str | None = None,\n metadata_update: Mapping[str, Any] | None = None\n) -> State\n Create a new child State derived from this state.
Parameters:
Name Type Description Defaultpayload_update Mapping[str, Any] | None Dot-path updates applied to the payload.
None confidence_delta float Adjustment applied to current confidence.
0.0 node_id str | None Identifier of the node creating this state.
None metadata_update Mapping[str, Any] | None Updates merged into state metadata.
None Returns:
Name Type DescriptionState State A new immutable State instance.
Guarantees:
- This is the only supported mechanism for modifying execution data.\n- Validates payload updates, preserves lineage, increments depth,\n and appends to history.\n"},{"location":"state/#dagpipe.state.State.get","title":"get","text":"get(key: str, default: Any = None) -> Any\n Retrieve payload value.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requireddefault Any Fallback value.
None Returns:
Name Type DescriptionAny Any Stored value or default.
"},{"location":"state/#dagpipe.state.State.has","title":"has","text":"has(key: str) -> bool\n Check whether payload contains key.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requiredReturns:
Name Type Descriptionbool bool Existence of the key.
"},{"location":"state/#dagpipe.state.State.lineage","title":"lineage","text":"lineage() -> tuple[State, ...]\n Return lineage from root to this State.
Returns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Ordered execution lineage (root first).
"},{"location":"yaml_loader/","title":"Yaml Loader","text":""},{"location":"yaml_loader/#dagpipe.yaml_loader","title":"dagpipe.yaml_loader","text":""},{"location":"yaml_loader/#dagpipe.yaml_loader--summary","title":"Summary","text":"Loads dagpipe pipelines from YAML configuration.
Creates fully configured pipeline objects from declarative YAML definitions, including Schema, State subclasses, Node instances, Graph topology, and initial payloads.
dataclass","text":"Pipeline(\n engine: Engine,\n state_cls: type[State],\n initial_payload: Payload,\n)\n Executable pipeline created from YAML configuration.
Attributes:
Name Type Descriptionengine Engine Execution engine responsible for running the pipeline.
state_cls Type[State] Dynamically created State subclass with configured schema.
initial_payload Payload Default payload used when execution begins.
NotesResponsibilities:
- Encapsulates engine, state type, and initial payload.\n- Provides a simplified interface for executing configured pipelines.\n- Safe for concurrent execution if underlying nodes are thread-safe.\n"},{"location":"yaml_loader/#dagpipe.yaml_loader.Pipeline-functions","title":"Functions","text":""},{"location":"yaml_loader/#dagpipe.yaml_loader.Pipeline.run","title":"run","text":"run(\n payload_override: Mapping[str, Any] | None = None,\n) -> list[State]\n Execute the pipeline.
Parameters:
Name Type Description Defaultpayload_override Mapping[str, Any] | None Payload values overriding initial payload.
None Returns:
Type Descriptionlist[State] list[State]: Terminal execution states.
NotesResponsibilities:
- Merges override payload with initial payload.\n- Creates root `State` and executes engine.\n"},{"location":"yaml_loader/#dagpipe.yaml_loader-functions","title":"Functions","text":""},{"location":"yaml_loader/#dagpipe.yaml_loader.load_pipeline","title":"load_pipeline","text":"load_pipeline(path: str) -> Pipeline\n Load pipeline from YAML file.
Parameters:
Name Type Description Defaultpath str Path to YAML configuration file.
requiredReturns:
Name Type DescriptionPipeline Pipeline Executable pipeline instance.
NotesResponsibilities:
- Loads YAML configuration and builds schema.\n- Creates `State` subclass and loads `Node` instances.\n- Builds `Graph` topology and initializes `Engine`.\n"},{"location":"dagpipe/","title":"Dagpipe","text":"Directed acyclic graph execution framework for deterministic state propagation.
dagpipe executes pipelines composed of nodes connected in a directed acyclic graph (DAG). Each node receives an immutable State and optionally produces derived states for downstream nodes.
Install using pip:
pip install dagpipe\n"},{"location":"dagpipe/#dagpipe--quick-start","title":"Quick Start","text":"from dagpipe import State, Payload, Schema, Graph, Engine\nfrom dagpipe.node import Node\n\nclass HelloNode(Node):\n id = \"hello\"\n def resolve(self, state):\n yield self.fork(state, payload_update={\"msg\": \"hello\"})\n\n# Build and run\ngraph = Graph()\ngraph.add_root(HelloNode())\nengine = Engine(graph)\n\nclass MyState(State):\n schema = Schema({})\n\nresults = engine.run(MyState(payload=Payload({})))\n"},{"location":"dagpipe/#dagpipe--public-api","title":"Public API","text":"This package re-exports the core pipeline components. Consumers should import from this namespace for standard usage.
"},{"location":"dagpipe/#dagpipe--execution-core","title":"Execution Core","text":"Engine: Responsible for orchestrating node execution and state propagation.Graph: Defines the execution topology and node relationships.Node: Base class for defining execution logic and transformations.State: Represents an immutable execution snapshot at a point in time.Payload: Immutable hierarchical container for execution data.Schema: Defines and validates the allowed structure of payloads.SchemaError: Raised when data violates the declared schema.Pipeline: High-level wrapper for an engine, state type, and initial payload.load_pipeline: Factory function to create a pipeline from YAML. Bases: Node
Base class for nodes whose execution is asynchronous.
Subclasses implement resolve_async (an async generator yielding derived State objects). The engine dispatches to resolve_async when running an async traversal (see Engine.run_async).
Sync-only engines (and the base Node.run) treat an AsyncNode as a no-op consumer: calling run on an AsyncNode returns no states, signalling that an async engine is required.
__hash__() -> int\n Return stable hash based on node ID.
Returns:
Name Type Descriptionint int Hash of the node ID, allowing nodes to be used as dict keys.
"},{"location":"dagpipe/#dagpipe.AsyncNode.__new__","title":"__new__","text":"__new__(*args: Any, **kwargs: Any) -> AsyncNode\n Create or reuse an async node instance.
Parameters:
Name Type Description Default*args Any Positional constructor arguments forwarded to __init__.
() **kwargs Any Keyword constructor arguments forwarded to __init__.
{} Returns:
Name Type DescriptionAsyncNode AsyncNode A fresh instance for subclasses declaring a parameterized __init__, or the shared singleton for stateless subclasses.
__repr__() -> str\n Return computation identity based on node ID.
Returns:
Name Type Descriptionstr str String of the form <Node {id}>.
__str__() -> str\n Return user-facing display name.
Returns:
Name Type Descriptionstr str String of the form <Node {name}>.
classmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"dagpipe/#dagpipe.AsyncNode.fork","title":"fork","text":"fork(\n state: State,\n *,\n payload_update: Any = None,\n confidence_delta: float = 0.0,\n metadata_update: Any = None\n) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Any Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Any Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history.\n"},{"location":"dagpipe/#dagpipe.AsyncNode.is_async","title":"is_async","text":"is_async() -> bool\n Return whether this node executes asynchronously.
Returns:
Name Type Descriptionbool bool True if the node is an AsyncNode instance.
staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"dagpipe/#dagpipe.AsyncNode.resolve","title":"resolve","text":"resolve(state: State) -> Iterable[State]\n Execute no-op resolution in sync contexts.
Sync-only engines (and the base Node.run) treat an AsyncNode as a no-op consumer: this returns no states, signalling that an async engine is required.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Empty tuple, since async execution is handled by resolve_async.
async","text":"resolve_async(state: State) -> Iterable[State]\n Execute node logic asynchronously.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Derived execution state(s).
NotesGuarantees:
- Subclasses implement this.\n- Must not mutate the input state.\n- Should use `fork()` to create child states.\n"},{"location":"dagpipe/#dagpipe.AsyncNode.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
async","text":"run_async(state: State) -> tuple[State, ...]\n Execute this node asynchronously on a state, validating outputs.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve_async() yields a non-State object.
Engine(\n nodes_or_graph: Sequence[Node] | Graph,\n *,\n on_step: StepHook | None = None\n)\n Execution engine responsible for running pipeline logic.
NotesResponsibilities:
- Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n- Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n- Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n- Supports step-wise / resumable execution and progress hooks.\n Guarantees:
- Never mutates `State`, `Node`, or `Graph` instances.\n- `State` objects are never modified in place; each branch produces\n independent instances.\n- Execution order is deterministic and follows graph or pipeline topology.\n- Thread-safe for concurrent execution.\n Create an engine from a node sequence or a graph.
Parameters:
Name Type Description Defaultnodes_or_graph Sequence[Node] | Graph Either an ordered sequence of Node instances (linear mode) or a Graph defining the execution topology (graph mode).
on_step StepHook | None Default per-step callback (step, status, message) used when a step run does not supply its own hook.
None Raises:
Type DescriptionTypeError If a sequence element is not a Node, or if nodes_or_graph is neither a Sequence[Node] nor a Graph.
property","text":"nodes: tuple[Node, ...]\n Return nodes managed by this engine.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Ordered sequence in linear mode or all nodes in graph mode.
"},{"location":"dagpipe/#dagpipe.Engine-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.Engine.__repr__","title":"__repr__","text":"__repr__() -> str\n Return the canonical string representation of the object.
Returns:
Name Type Descriptionstr str Representation that uniquely identifies the object and its configuration.
"},{"location":"dagpipe/#dagpipe.Engine.run","title":"run","text":"run(root: State) -> list[State]\n Execute the pipeline starting from a root State.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type Descriptionlist[State] list[State]: Terminal execution states produced by the pipeline.
Raises:
Type DescriptionTypeError If root is not a State instance.
RuntimeError If the engine execution mode is invalid.
NotesResponsibilities:
- Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states.\n"},{"location":"dagpipe/#dagpipe.Engine.run_async","title":"run_async async","text":"run_async(root: State) -> list[State]\n Execute the pipeline starting from root, dispatching sync vs async nodes.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type Descriptionlist[State] list[State]: Terminal execution states produced by the pipeline.
NotesEach node is executed with Node.run when synchronous and AsyncNode.run_async when asynchronous. Linear and graph topologies are both supported.
run_steps(\n root: State,\n *,\n resume_from: int | None = None,\n on_step: StepHook | None = None\n) -> Iterator[StepResult]\n Execute the pipeline step-by-step, yielding one StepResult per step.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredresume_from int | None Skip steps at index < resume_from (for resume-after-partial). Steps are 0-indexed.
None on_step StepHook | None Callback (step, status, message) invoked per step; falls back to the engine-level hook when unset.
None Yields:
Name Type DescriptionStepResult StepResult One per executed node/step, carrying the produced states.
NotesThis is a synchronous, generator-based checkpoint interface compatible with the imperative resume-by-step behaviour of the legacy orchestrator. Use run_steps_async for async nodes.
async","text":"run_steps_async(\n root: State,\n *,\n resume_from: int | None = None,\n on_step: AsyncStepHook | None = None\n) -> AsyncIterator[StepResult]\n Async variant of run_steps supporting AsyncNode execution.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredresume_from int | None Skip steps at index < resume_from.
None on_step AsyncStepHook | None Async callback (step, status, message) invoked per step.
None Yields:
Name Type DescriptionStepResult AsyncIterator[StepResult] One per executed node/step.
"},{"location":"dagpipe/#dagpipe.Graph","title":"Graph","text":"Graph()\n Directed Acyclic Graph defining execution topology of Node objects.
Responsibilities:
- Stores node connectivity and validates that the topology remains acyclic.\n- Structure determines how `State` flows between nodes during execution.\n Guarantees:
- Topology is acyclic. Node relationships remain consistent.\n- Thread-safe for concurrent reads after construction.\n Create an empty Graph.
Initializes node registry and edge mappings.
"},{"location":"dagpipe/#dagpipe.Graph-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.Graph.__repr__","title":"__repr__","text":"__repr__() -> str\n Return a compact graph description.
Returns:
Name Type Descriptionstr str A string describing the graph as Graph(nodes=N, edges=M) where N and M describe the current registry size.
add_edge(src: Node, dst: Node) -> None\n Add a directed edge from src to dst.
Parameters:
Name Type Description Defaultsrc Node Source node.
requireddst Node Destination node.
requiredRaises:
Type DescriptionTypeError If src or dst is not a Node.
ValueError If the edge would create a cycle or if src and dst are common.
add_root(node: Node) -> None\n Add a root node with no parents.
Parameters:
Name Type Description Defaultnode Node Node to add as a root.
requiredRaises:
Type DescriptionTypeError If node is not a Node instance.
"},{"location":"dagpipe/#dagpipe.Graph.children","title":"children","text":"children(node: Node) -> tuple[Node, ...]\n Return child nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Outgoing neighbors.
"},{"location":"dagpipe/#dagpipe.Graph.nodes","title":"nodes","text":"nodes() -> tuple[Node, ...]\n Return all nodes in the graph.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: All registered nodes.
"},{"location":"dagpipe/#dagpipe.Graph.parents","title":"parents","text":"parents(node: Node) -> tuple[Node, ...]\n Return parent nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Incoming neighbors.
"},{"location":"dagpipe/#dagpipe.Graph.roots","title":"roots","text":"roots() -> tuple[Node, ...]\n Return root nodes (nodes with no incoming edges).
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Entry point nodes.
"},{"location":"dagpipe/#dagpipe.Node","title":"Node","text":" Bases: ABC
Base class for all dagpipe execution nodes.
Attributes:
Name Type Descriptionid str Unique identifier of the node (snake_case dotted format).
name str Human-readable display name.
NotesResponsibilities:
- Represents a deterministic unit of execution in the pipeline graph.\n- Consumes one `State` and produces zero, one, or many derived states.\n- Defines execution logic and enables branching, filtering, and transformation.\n Guarantees:
- Nodes must never mutate the input `State`.\n- Instances are singletons per subclass and reused across executions.\n"},{"location":"dagpipe/#dagpipe.Node-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.Node.__hash__","title":"__hash__","text":"__hash__() -> int\n Return stable hash based on node ID.
Returns:
Name Type Descriptionint int Hash of the node ID, allowing nodes to be used as dict keys.
"},{"location":"dagpipe/#dagpipe.Node.__new__","title":"__new__","text":"__new__(*args: Any, **kwargs: Any) -> Node\n Create or reuse a node instance.
Parameters:
Name Type Description Default*args Any Positional constructor arguments forwarded to __init__.
() **kwargs Any Keyword constructor arguments forwarded to __init__.
{} Returns:
Name Type DescriptionNode Node A fresh instance for subclasses declaring a parameterized __init__, or the shared singleton for stateless subclasses.
Guarantees:
- Stateless subclasses (no parameterized `__init__`) share one\n singleton instance per class \u2014 matching the original dagpipe\n behaviour underpinning `set_registry`-style configuration.\n- Subclasses that declare an `__init__` requiring instance-state\n arguments get a fresh instance per construction so pipeline\n builders can inject per-run dependencies.\n"},{"location":"dagpipe/#dagpipe.Node.__repr__","title":"__repr__","text":"__repr__() -> str\n Return computation identity based on node ID.
Returns:
Name Type Descriptionstr str String of the form <Node {id}>.
__str__() -> str\n Return user-facing display name.
Returns:
Name Type Descriptionstr str String of the form <Node {name}>.
classmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"dagpipe/#dagpipe.Node.fork","title":"fork","text":"fork(\n state: State,\n *,\n payload_update: Any = None,\n confidence_delta: float = 0.0,\n metadata_update: Any = None\n) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Any Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Any Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history.\n"},{"location":"dagpipe/#dagpipe.Node.is_async","title":"is_async","text":"is_async() -> bool\n Return whether this node executes asynchronously.
Returns:
Name Type Descriptionbool bool True if the node is an AsyncNode instance.
staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"dagpipe/#dagpipe.Node.resolve","title":"resolveabstractmethod","text":"resolve(state: State) -> Iterable[State]\n Execute node logic.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Derived execution state(s).
NotesResponsibilities:
- Subclasses implement specific resolution behavior.\n- Must not mutate input state.\n- Should use `fork()` to create child states.\n- May yield zero states to terminate a branch.\n"},{"location":"dagpipe/#dagpipe.Node.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
dataclass","text":"Payload(_data: Mapping[str, Any])\n Immutable hierarchical container with dot-path access.
Attributes:
Name Type Description_data Mapping[str, Any] Immutable hierarchical data structure.
NotesResponsibilities:
- Stores execution data used by `State`.\n- Supports efficient atomic updates without modifying existing instances.\n- `Payload` instances are fully thread-safe due to immutability.\n"},{"location":"dagpipe/#dagpipe.Payload-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.Payload.__repr__","title":"__repr__","text":"__repr__() -> str\n Return a concise payload description.
Returns:
Name Type Descriptionstr str String of the form Payload(keys=[...]) listing top-level keys.
as_dict() -> Mapping[str, Any]\n Return underlying mapping.
Returns:
Type DescriptionMapping[str, Any] Mapping[str, Any]: Read-only view of the underlying data.
"},{"location":"dagpipe/#dagpipe.Payload.get","title":"get","text":"get(path: str, default: Any = None) -> Any\n Retrieve value using dot-path.
Parameters:
Name Type Description Defaultpath str Dot-separated path to the value.
requireddefault Any Default value if path doesn't exist.
None Returns:
Name Type DescriptionAny Any The retrieved value or default.
"},{"location":"dagpipe/#dagpipe.Payload.has","title":"has","text":"has(path: str) -> bool\n Return True if path exists.
Parameters:
Name Type Description Defaultpath str Dot-separated path to check.
requiredReturns:
Name Type Descriptionbool bool Existence of the path.
"},{"location":"dagpipe/#dagpipe.Payload.iter_paths","title":"iter_pathsclassmethod","text":"iter_paths(\n data: Mapping[str, Any], prefix: str = \"\"\n) -> Iterable[str]\n Recursively yield dot-paths for all leaf nodes.
Parameters:
Name Type Description Defaultdata Mapping[str, Any] The mapping to iterate over.
requiredprefix str Current path prefix.
'' Yields:
Name Type Descriptionstr Iterable[str] Dot-path for each leaf node.
"},{"location":"dagpipe/#dagpipe.Payload.keys","title":"keys","text":"keys() -> Iterable[str]\n Return top-level keys.
Returns:
Type DescriptionIterable[str] Iterable[str]: Iterator over top-level keys.
"},{"location":"dagpipe/#dagpipe.Payload.update","title":"update","text":"update(updates: Mapping[str, Any]) -> Payload\n Create a new Payload with dot-path updates applied.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path to value mapping.
requiredReturns:
Name Type DescriptionPayload Payload New immutable payload instance with updates.
NotesGuarantees:
- Preserves existing data by copying only modified branches.\n- Returns a new immutable `Payload`.\n"},{"location":"dagpipe/#dagpipe.Pipeline","title":"Pipeline dataclass","text":"Pipeline(\n engine: Engine,\n state_cls: type[State],\n initial_payload: Payload,\n)\n Executable pipeline created from YAML configuration.
Attributes:
Name Type Descriptionengine Engine Execution engine responsible for running the pipeline.
state_cls Type[State] Dynamically created State subclass with configured schema.
initial_payload Payload Default payload used when execution begins.
NotesResponsibilities:
- Encapsulates engine, state type, and initial payload.\n- Provides a simplified interface for executing configured pipelines.\n- Safe for concurrent execution if underlying nodes are thread-safe.\n"},{"location":"dagpipe/#dagpipe.Pipeline-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.Pipeline.run","title":"run","text":"run(\n payload_override: Mapping[str, Any] | None = None,\n) -> list[State]\n Execute the pipeline.
Parameters:
Name Type Description Defaultpayload_override Mapping[str, Any] | None Payload values overriding initial payload.
None Returns:
Type Descriptionlist[State] list[State]: Terminal execution states.
NotesResponsibilities:
- Merges override payload with initial payload.\n- Creates root `State` and executes engine.\n"},{"location":"dagpipe/#dagpipe.ProgressMessage","title":"ProgressMessage","text":"ProgressMessage(\n *,\n lines: int | None = None,\n blocks: int | None = None,\n count: int | None = None,\n unit: str | None = None,\n raw_ocr_line: str | None = None,\n error: str | None = None,\n step: str = \"\",\n status: str = \"\"\n)\n Lightweight progress payload emitted by engine step hooks.
Mirrors the imperative ProgressMessage used by the legacy orchestrator so callers can surface counts/lines/errors without coupling the engine to pydantic.
Attributes:
Name Type Descriptionlines int | None Optional count of processed lines.
blocks int | None Optional count of processed blocks.
count int | None Optional generic item count.
unit str | None Optional unit for the count (e.g., 'pages').
raw_ocr_line str | None Optional raw OCR line payload.
error str | None Optional error description.
step str Identifier of the step that emitted the message.
status str Status label associated with the step.
NotesGuarantees:
- Immutable after construction (attributes are never reassigned).\n- Independent of pydantic; safe to construct in the engine core.\n Create a progress message.
Parameters:
Name Type Description Defaultlines int | None Optional count of processed lines.
None blocks int | None Optional count of processed blocks.
None count int | None Optional generic item count.
None unit str | None Optional unit for the count (e.g., 'pages').
None raw_ocr_line str | None Optional raw OCR line payload.
None error str | None Optional error description.
None step str Identifier of the step that emitted the message.
'' status str Status label associated with the step.
''"},{"location":"dagpipe/#dagpipe.ProgressMessage-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.ProgressMessage.as_dict","title":"as_dict","text":"as_dict() -> dict[str, Any]\n Return the message as a plain dictionary.
Returns:
Type Descriptiondict[str, Any] dict[str, Any]: All attribute values keyed by their attribute name.
"},{"location":"dagpipe/#dagpipe.Schema","title":"Schemadataclass","text":"Schema(tree: Mapping[str, SchemaNode])\n Immutable hierarchical schema defining allowed payload structure.
Attributes:
Name Type Descriptiontree Mapping[str, SchemaNode] Hierarchical schema definition.
NotesResponsibilities:
- Validates `State` payloads and updates.\n- Reusable across all `State` instances.\n- Fully thread-safe due to immutability.\n"},{"location":"dagpipe/#dagpipe.Schema-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.Schema.validate_payload","title":"validate_payload","text":"validate_payload(payload: Payload) -> None\n Validate complete payload structure.
Parameters:
Name Type Description Defaultpayload Payload Payload to validate.
requiredRaises:
Type DescriptionSchemaError If payload violates schema.
"},{"location":"dagpipe/#dagpipe.Schema.validate_update","title":"validate_update","text":"validate_update(updates: Mapping[str, Any]) -> None\n Validate payload update paths.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path updates to validate.
requiredRaises:
Type DescriptionSchemaError If any path is invalid according to the schema.
"},{"location":"dagpipe/#dagpipe.SchemaError","title":"SchemaError","text":" Bases: Exception
Raised when payload data violates the declared schema.
Indicates invalid structure, invalid path, or invalid type.
"},{"location":"dagpipe/#dagpipe.State","title":"Statedataclass","text":"State(\n payload: Payload,\n confidence: float = 1.0,\n parent: State | None = None,\n depth: int = 0,\n history: tuple[str, ...] = tuple(),\n metadata: dict[str, Any] = dict(),\n)\n Immutable execution state propagated through dagpipe pipeline.
Attributes:
Name Type Descriptionpayload Payload Execution data container.
schema ClassVar[Schema] Payload validation schema.
confidence float Execution confidence score.
parent Optional[State] Parent state reference.
depth int Execution depth.
history Tuple[str, ...] Ordered node execution lineage.
metadata Dict[str, Any] Execution metadata.
NotesResponsibilities:
- Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n- Fundamental unit of execution in `dagpipe`.\n- Fully thread-safe due to immutability.\n"},{"location":"dagpipe/#dagpipe.State-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.State.__post_init__","title":"__post_init__","text":"__post_init__() -> None\n Validate the payload against the declared schema.
Raises:
Type DescriptionSchemaError If the payload violates the schema declared on the subclass.
"},{"location":"dagpipe/#dagpipe.State.__repr__","title":"__repr__","text":"__repr__() -> str\n Concise debug representation.
Avoids printing full data for large states.
"},{"location":"dagpipe/#dagpipe.State.fork","title":"fork","text":"fork(\n *,\n payload_update: Mapping[str, Any] | None = None,\n confidence_delta: float = 0.0,\n node_id: str | None = None,\n metadata_update: Mapping[str, Any] | None = None\n) -> State\n Create a new child State derived from this state.
Parameters:
Name Type Description Defaultpayload_update Mapping[str, Any] | None Dot-path updates applied to the payload.
None confidence_delta float Adjustment applied to current confidence.
0.0 node_id str | None Identifier of the node creating this state.
None metadata_update Mapping[str, Any] | None Updates merged into state metadata.
None Returns:
Name Type DescriptionState State A new immutable State instance.
Guarantees:
- This is the only supported mechanism for modifying execution data.\n- Validates payload updates, preserves lineage, increments depth,\n and appends to history.\n"},{"location":"dagpipe/#dagpipe.State.get","title":"get","text":"get(key: str, default: Any = None) -> Any\n Retrieve payload value.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requireddefault Any Fallback value.
None Returns:
Name Type DescriptionAny Any Stored value or default.
"},{"location":"dagpipe/#dagpipe.State.has","title":"has","text":"has(key: str) -> bool\n Check whether payload contains key.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requiredReturns:
Name Type Descriptionbool bool Existence of the key.
"},{"location":"dagpipe/#dagpipe.State.lineage","title":"lineage","text":"lineage() -> tuple[State, ...]\n Return lineage from root to this State.
Returns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Ordered execution lineage (root first).
"},{"location":"dagpipe/#dagpipe.StepResult","title":"StepResult","text":"StepResult(\n index: int,\n node_id: str,\n states: tuple[State, ...],\n completed: bool,\n)\n A single checkpointed step within an async/resumable engine run.
Attributes:
Name Type Descriptionindex int Ordinal index of the step.
node_id str Identifier of the node associated with this step.
states tuple[State, ...] States produced by running this step.
completed bool Whether this step succeeded (vs. paused/interrupted).
Initialise StepResult.
Parameters:
Name Type Description Defaultindex int Ordinal index of the step.
requirednode_id str Identifier of the node associated with this step.
requiredstates tuple[State, ...] States produced by running this step.
requiredcompleted bool Whether this step succeeded (vs. paused/interrupted).
required"},{"location":"dagpipe/#dagpipe.StepResult-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe-functions","title":"Functions","text":""},{"location":"dagpipe/#dagpipe.load_pipeline","title":"load_pipeline","text":"load_pipeline(path: str) -> Pipeline\n Load pipeline from YAML file.
Parameters:
Name Type Description Defaultpath str Path to YAML configuration file.
requiredReturns:
Name Type DescriptionPipeline Pipeline Executable pipeline instance.
NotesResponsibilities:
- Loads YAML configuration and builds schema.\n- Creates `State` subclass and loads `Node` instances.\n- Builds `Graph` topology and initializes `Engine`.\n"},{"location":"dagpipe/engine/","title":"Engine","text":""},{"location":"dagpipe/engine/#dagpipe.engine","title":"dagpipe.engine","text":""},{"location":"dagpipe/engine/#dagpipe.engine--summary","title":"Summary","text":"Execution engine responsible for running pipelines and graphs.
The Engine executes Node objects and propagates immutable State instances through either a linear sequence or a directed acyclic graph (Graph). It orchestrates execution order, branching, and state propagation.
Graph, Node, or State objects.Engine(\n nodes_or_graph: Sequence[Node] | Graph,\n *,\n on_step: StepHook | None = None\n)\n Execution engine responsible for running pipeline logic.
NotesResponsibilities:
- Accepts either a linear sequence of `Node` objects or a `Graph`\n defining execution topology.\n- Propagates immutable `State` objects through `Node` objects and\n collects terminal states.\n- Supports synchronous (`run`) and asynchronous (`run_async`)\n execution, dispatching per-node.\n- Supports step-wise / resumable execution and progress hooks.\n Guarantees:
- Never mutates `State`, `Node`, or `Graph` instances.\n- `State` objects are never modified in place; each branch produces\n independent instances.\n- Execution order is deterministic and follows graph or pipeline topology.\n- Thread-safe for concurrent execution.\n Create an engine from a node sequence or a graph.
Parameters:
Name Type Description Defaultnodes_or_graph Sequence[Node] | Graph Either an ordered sequence of Node instances (linear mode) or a Graph defining the execution topology (graph mode).
on_step StepHook | None Default per-step callback (step, status, message) used when a step run does not supply its own hook.
None Raises:
Type DescriptionTypeError If a sequence element is not a Node, or if nodes_or_graph is neither a Sequence[Node] nor a Graph.
property","text":"nodes: tuple[Node, ...]\n Return nodes managed by this engine.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Ordered sequence in linear mode or all nodes in graph mode.
"},{"location":"dagpipe/engine/#dagpipe.engine.Engine-functions","title":"Functions","text":""},{"location":"dagpipe/engine/#dagpipe.engine.Engine.__repr__","title":"__repr__","text":"__repr__() -> str\n Return the canonical string representation of the object.
Returns:
Name Type Descriptionstr str Representation that uniquely identifies the object and its configuration.
"},{"location":"dagpipe/engine/#dagpipe.engine.Engine.run","title":"run","text":"run(root: State) -> list[State]\n Execute the pipeline starting from a root State.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type Descriptionlist[State] list[State]: Terminal execution states produced by the pipeline.
Raises:
Type DescriptionTypeError If root is not a State instance.
RuntimeError If the engine execution mode is invalid.
NotesResponsibilities:
- Selects execution mode, propagates state through nodes, creates\n new instances for branches, and collects terminal states.\n"},{"location":"dagpipe/engine/#dagpipe.engine.Engine.run_async","title":"run_async async","text":"run_async(root: State) -> list[State]\n Execute the pipeline starting from root, dispatching sync vs async nodes.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type Descriptionlist[State] list[State]: Terminal execution states produced by the pipeline.
NotesEach node is executed with Node.run when synchronous and AsyncNode.run_async when asynchronous. Linear and graph topologies are both supported.
run_steps(\n root: State,\n *,\n resume_from: int | None = None,\n on_step: StepHook | None = None\n) -> Iterator[StepResult]\n Execute the pipeline step-by-step, yielding one StepResult per step.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredresume_from int | None Skip steps at index < resume_from (for resume-after-partial). Steps are 0-indexed.
None on_step StepHook | None Callback (step, status, message) invoked per step; falls back to the engine-level hook when unset.
None Yields:
Name Type DescriptionStepResult StepResult One per executed node/step, carrying the produced states.
NotesThis is a synchronous, generator-based checkpoint interface compatible with the imperative resume-by-step behaviour of the legacy orchestrator. Use run_steps_async for async nodes.
async","text":"run_steps_async(\n root: State,\n *,\n resume_from: int | None = None,\n on_step: AsyncStepHook | None = None\n) -> AsyncIterator[StepResult]\n Async variant of run_steps supporting AsyncNode execution.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredresume_from int | None Skip steps at index < resume_from.
None on_step AsyncStepHook | None Async callback (step, status, message) invoked per step.
None Yields:
Name Type DescriptionStepResult AsyncIterator[StepResult] One per executed node/step.
"},{"location":"dagpipe/engine/#dagpipe.engine.ProgressMessage","title":"ProgressMessage","text":"ProgressMessage(\n *,\n lines: int | None = None,\n blocks: int | None = None,\n count: int | None = None,\n unit: str | None = None,\n raw_ocr_line: str | None = None,\n error: str | None = None,\n step: str = \"\",\n status: str = \"\"\n)\n Lightweight progress payload emitted by engine step hooks.
Mirrors the imperative ProgressMessage used by the legacy orchestrator so callers can surface counts/lines/errors without coupling the engine to pydantic.
Attributes:
Name Type Descriptionlines int | None Optional count of processed lines.
blocks int | None Optional count of processed blocks.
count int | None Optional generic item count.
unit str | None Optional unit for the count (e.g., 'pages').
raw_ocr_line str | None Optional raw OCR line payload.
error str | None Optional error description.
step str Identifier of the step that emitted the message.
status str Status label associated with the step.
NotesGuarantees:
- Immutable after construction (attributes are never reassigned).\n- Independent of pydantic; safe to construct in the engine core.\n Create a progress message.
Parameters:
Name Type Description Defaultlines int | None Optional count of processed lines.
None blocks int | None Optional count of processed blocks.
None count int | None Optional generic item count.
None unit str | None Optional unit for the count (e.g., 'pages').
None raw_ocr_line str | None Optional raw OCR line payload.
None error str | None Optional error description.
None step str Identifier of the step that emitted the message.
'' status str Status label associated with the step.
''"},{"location":"dagpipe/engine/#dagpipe.engine.ProgressMessage-functions","title":"Functions","text":""},{"location":"dagpipe/engine/#dagpipe.engine.ProgressMessage.as_dict","title":"as_dict","text":"as_dict() -> dict[str, Any]\n Return the message as a plain dictionary.
Returns:
Type Descriptiondict[str, Any] dict[str, Any]: All attribute values keyed by their attribute name.
"},{"location":"dagpipe/engine/#dagpipe.engine.StepResult","title":"StepResult","text":"StepResult(\n index: int,\n node_id: str,\n states: tuple[State, ...],\n completed: bool,\n)\n A single checkpointed step within an async/resumable engine run.
Attributes:
Name Type Descriptionindex int Ordinal index of the step.
node_id str Identifier of the node associated with this step.
states tuple[State, ...] States produced by running this step.
completed bool Whether this step succeeded (vs. paused/interrupted).
Initialise StepResult.
Parameters:
Name Type Description Defaultindex int Ordinal index of the step.
requirednode_id str Identifier of the node associated with this step.
requiredstates tuple[State, ...] States produced by running this step.
requiredcompleted bool Whether this step succeeded (vs. paused/interrupted).
required"},{"location":"dagpipe/engine/#dagpipe.engine.StepResult-functions","title":"Functions","text":""},{"location":"dagpipe/graph/","title":"Graph","text":""},{"location":"dagpipe/graph/#dagpipe.graph","title":"dagpipe.graph","text":""},{"location":"dagpipe/graph/#dagpipe.graph--summary","title":"Summary","text":"Defines DAG structure connecting nodes.
A Graph describes execution topology only. It does not execute nodes or manage State. Execution is handled by an Engine.
Graph()\n Directed Acyclic Graph defining execution topology of Node objects.
Responsibilities:
- Stores node connectivity and validates that the topology remains acyclic.\n- Structure determines how `State` flows between nodes during execution.\n Guarantees:
- Topology is acyclic. Node relationships remain consistent.\n- Thread-safe for concurrent reads after construction.\n Create an empty Graph.
Initializes node registry and edge mappings.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph-functions","title":"Functions","text":""},{"location":"dagpipe/graph/#dagpipe.graph.Graph.__repr__","title":"__repr__","text":"__repr__() -> str\n Return a compact graph description.
Returns:
Name Type Descriptionstr str A string describing the graph as Graph(nodes=N, edges=M) where N and M describe the current registry size.
add_edge(src: Node, dst: Node) -> None\n Add a directed edge from src to dst.
Parameters:
Name Type Description Defaultsrc Node Source node.
requireddst Node Destination node.
requiredRaises:
Type DescriptionTypeError If src or dst is not a Node.
ValueError If the edge would create a cycle or if src and dst are common.
add_root(node: Node) -> None\n Add a root node with no parents.
Parameters:
Name Type Description Defaultnode Node Node to add as a root.
requiredRaises:
Type DescriptionTypeError If node is not a Node instance.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.children","title":"children","text":"children(node: Node) -> tuple[Node, ...]\n Return child nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Outgoing neighbors.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.nodes","title":"nodes","text":"nodes() -> tuple[Node, ...]\n Return all nodes in the graph.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: All registered nodes.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.parents","title":"parents","text":"parents(node: Node) -> tuple[Node, ...]\n Return parent nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Incoming neighbors.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.roots","title":"roots","text":"roots() -> tuple[Node, ...]\n Return root nodes (nodes with no incoming edges).
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Entry point nodes.
"},{"location":"dagpipe/node/","title":"Node","text":""},{"location":"dagpipe/node/#dagpipe.node","title":"dagpipe.node","text":""},{"location":"dagpipe/node/#dagpipe.node--summary","title":"Summary","text":"Defines the Node abstraction used by dagpipe.
A node represents a single unit of pipeline execution logic. It consumes one State and produces zero, one, or many new State objects.
Nodes are connected using a Graph and executed by an Engine.
Bases: Node
Base class for nodes whose execution is asynchronous.
Subclasses implement resolve_async (an async generator yielding derived State objects). The engine dispatches to resolve_async when running an async traversal (see Engine.run_async).
Sync-only engines (and the base Node.run) treat an AsyncNode as a no-op consumer: calling run on an AsyncNode returns no states, signalling that an async engine is required.
__hash__() -> int\n Return stable hash based on node ID.
Returns:
Name Type Descriptionint int Hash of the node ID, allowing nodes to be used as dict keys.
"},{"location":"dagpipe/node/#dagpipe.node.AsyncNode.__new__","title":"__new__","text":"__new__(*args: Any, **kwargs: Any) -> AsyncNode\n Create or reuse an async node instance.
Parameters:
Name Type Description Default*args Any Positional constructor arguments forwarded to __init__.
() **kwargs Any Keyword constructor arguments forwarded to __init__.
{} Returns:
Name Type DescriptionAsyncNode AsyncNode A fresh instance for subclasses declaring a parameterized __init__, or the shared singleton for stateless subclasses.
__repr__() -> str\n Return computation identity based on node ID.
Returns:
Name Type Descriptionstr str String of the form <Node {id}>.
__str__() -> str\n Return user-facing display name.
Returns:
Name Type Descriptionstr str String of the form <Node {name}>.
classmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"dagpipe/node/#dagpipe.node.AsyncNode.fork","title":"fork","text":"fork(\n state: State,\n *,\n payload_update: Any = None,\n confidence_delta: float = 0.0,\n metadata_update: Any = None\n) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Any Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Any Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history.\n"},{"location":"dagpipe/node/#dagpipe.node.AsyncNode.is_async","title":"is_async","text":"is_async() -> bool\n Return whether this node executes asynchronously.
Returns:
Name Type Descriptionbool bool True if the node is an AsyncNode instance.
staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"dagpipe/node/#dagpipe.node.AsyncNode.resolve","title":"resolve","text":"resolve(state: State) -> Iterable[State]\n Execute no-op resolution in sync contexts.
Sync-only engines (and the base Node.run) treat an AsyncNode as a no-op consumer: this returns no states, signalling that an async engine is required.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Empty tuple, since async execution is handled by resolve_async.
async","text":"resolve_async(state: State) -> Iterable[State]\n Execute node logic asynchronously.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Derived execution state(s).
NotesGuarantees:
- Subclasses implement this.\n- Must not mutate the input state.\n- Should use `fork()` to create child states.\n"},{"location":"dagpipe/node/#dagpipe.node.AsyncNode.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
async","text":"run_async(state: State) -> tuple[State, ...]\n Execute this node asynchronously on a state, validating outputs.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve_async() yields a non-State object.
Bases: ABC
Base class for all dagpipe execution nodes.
Attributes:
Name Type Descriptionid str Unique identifier of the node (snake_case dotted format).
name str Human-readable display name.
NotesResponsibilities:
- Represents a deterministic unit of execution in the pipeline graph.\n- Consumes one `State` and produces zero, one, or many derived states.\n- Defines execution logic and enables branching, filtering, and transformation.\n Guarantees:
- Nodes must never mutate the input `State`.\n- Instances are singletons per subclass and reused across executions.\n"},{"location":"dagpipe/node/#dagpipe.node.Node-functions","title":"Functions","text":""},{"location":"dagpipe/node/#dagpipe.node.Node.__hash__","title":"__hash__","text":"__hash__() -> int\n Return stable hash based on node ID.
Returns:
Name Type Descriptionint int Hash of the node ID, allowing nodes to be used as dict keys.
"},{"location":"dagpipe/node/#dagpipe.node.Node.__new__","title":"__new__","text":"__new__(*args: Any, **kwargs: Any) -> Node\n Create or reuse a node instance.
Parameters:
Name Type Description Default*args Any Positional constructor arguments forwarded to __init__.
() **kwargs Any Keyword constructor arguments forwarded to __init__.
{} Returns:
Name Type DescriptionNode Node A fresh instance for subclasses declaring a parameterized __init__, or the shared singleton for stateless subclasses.
Guarantees:
- Stateless subclasses (no parameterized `__init__`) share one\n singleton instance per class \u2014 matching the original dagpipe\n behaviour underpinning `set_registry`-style configuration.\n- Subclasses that declare an `__init__` requiring instance-state\n arguments get a fresh instance per construction so pipeline\n builders can inject per-run dependencies.\n"},{"location":"dagpipe/node/#dagpipe.node.Node.__repr__","title":"__repr__","text":"__repr__() -> str\n Return computation identity based on node ID.
Returns:
Name Type Descriptionstr str String of the form <Node {id}>.
__str__() -> str\n Return user-facing display name.
Returns:
Name Type Descriptionstr str String of the form <Node {name}>.
classmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"dagpipe/node/#dagpipe.node.Node.fork","title":"fork","text":"fork(\n state: State,\n *,\n payload_update: Any = None,\n confidence_delta: float = 0.0,\n metadata_update: Any = None\n) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Any Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Any Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around `State.fork()` that automatically\n records this node's ID in state history.\n"},{"location":"dagpipe/node/#dagpipe.node.Node.is_async","title":"is_async","text":"is_async() -> bool\n Return whether this node executes asynchronously.
Returns:
Name Type Descriptionbool bool True if the node is an AsyncNode instance.
staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"dagpipe/node/#dagpipe.node.Node.resolve","title":"resolveabstractmethod","text":"resolve(state: State) -> Iterable[State]\n Execute node logic.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type DescriptionIterable[State] Iterable[State]: Derived execution state(s).
NotesResponsibilities:
- Subclasses implement specific resolution behavior.\n- Must not mutate input state.\n- Should use `fork()` to create child states.\n- May yield zero states to terminate a branch.\n"},{"location":"dagpipe/node/#dagpipe.node.Node.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
Defines the core State object used by dagpipe.
The State represents a single point in pipeline execution. It contains arbitrary data and metadata and is designed to be immutable. Instead of modifying an existing state, nodes create new child states via fork().
fork().dataclass","text":"Payload(_data: Mapping[str, Any])\n Immutable hierarchical container with dot-path access.
Attributes:
Name Type Description_data Mapping[str, Any] Immutable hierarchical data structure.
NotesResponsibilities:
- Stores execution data used by `State`.\n- Supports efficient atomic updates without modifying existing instances.\n- `Payload` instances are fully thread-safe due to immutability.\n"},{"location":"dagpipe/state/#dagpipe.state.Payload-functions","title":"Functions","text":""},{"location":"dagpipe/state/#dagpipe.state.Payload.__repr__","title":"__repr__","text":"__repr__() -> str\n Return a concise payload description.
Returns:
Name Type Descriptionstr str String of the form Payload(keys=[...]) listing top-level keys.
as_dict() -> Mapping[str, Any]\n Return underlying mapping.
Returns:
Type DescriptionMapping[str, Any] Mapping[str, Any]: Read-only view of the underlying data.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.get","title":"get","text":"get(path: str, default: Any = None) -> Any\n Retrieve value using dot-path.
Parameters:
Name Type Description Defaultpath str Dot-separated path to the value.
requireddefault Any Default value if path doesn't exist.
None Returns:
Name Type DescriptionAny Any The retrieved value or default.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.has","title":"has","text":"has(path: str) -> bool\n Return True if path exists.
Parameters:
Name Type Description Defaultpath str Dot-separated path to check.
requiredReturns:
Name Type Descriptionbool bool Existence of the path.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.iter_paths","title":"iter_pathsclassmethod","text":"iter_paths(\n data: Mapping[str, Any], prefix: str = \"\"\n) -> Iterable[str]\n Recursively yield dot-paths for all leaf nodes.
Parameters:
Name Type Description Defaultdata Mapping[str, Any] The mapping to iterate over.
requiredprefix str Current path prefix.
'' Yields:
Name Type Descriptionstr Iterable[str] Dot-path for each leaf node.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.keys","title":"keys","text":"keys() -> Iterable[str]\n Return top-level keys.
Returns:
Type DescriptionIterable[str] Iterable[str]: Iterator over top-level keys.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.update","title":"update","text":"update(updates: Mapping[str, Any]) -> Payload\n Create a new Payload with dot-path updates applied.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path to value mapping.
requiredReturns:
Name Type DescriptionPayload Payload New immutable payload instance with updates.
NotesGuarantees:
- Preserves existing data by copying only modified branches.\n- Returns a new immutable `Payload`.\n"},{"location":"dagpipe/state/#dagpipe.state.Schema","title":"Schema dataclass","text":"Schema(tree: Mapping[str, SchemaNode])\n Immutable hierarchical schema defining allowed payload structure.
Attributes:
Name Type Descriptiontree Mapping[str, SchemaNode] Hierarchical schema definition.
NotesResponsibilities:
- Validates `State` payloads and updates.\n- Reusable across all `State` instances.\n- Fully thread-safe due to immutability.\n"},{"location":"dagpipe/state/#dagpipe.state.Schema-functions","title":"Functions","text":""},{"location":"dagpipe/state/#dagpipe.state.Schema.validate_payload","title":"validate_payload","text":"validate_payload(payload: Payload) -> None\n Validate complete payload structure.
Parameters:
Name Type Description Defaultpayload Payload Payload to validate.
requiredRaises:
Type DescriptionSchemaError If payload violates schema.
"},{"location":"dagpipe/state/#dagpipe.state.Schema.validate_update","title":"validate_update","text":"validate_update(updates: Mapping[str, Any]) -> None\n Validate payload update paths.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path updates to validate.
requiredRaises:
Type DescriptionSchemaError If any path is invalid according to the schema.
"},{"location":"dagpipe/state/#dagpipe.state.SchemaError","title":"SchemaError","text":" Bases: Exception
Raised when payload data violates the declared schema.
Indicates invalid structure, invalid path, or invalid type.
"},{"location":"dagpipe/state/#dagpipe.state.State","title":"Statedataclass","text":"State(\n payload: Payload,\n confidence: float = 1.0,\n parent: State | None = None,\n depth: int = 0,\n history: tuple[str, ...] = tuple(),\n metadata: dict[str, Any] = dict(),\n)\n Immutable execution state propagated through dagpipe pipeline.
Attributes:
Name Type Descriptionpayload Payload Execution data container.
schema ClassVar[Schema] Payload validation schema.
confidence float Execution confidence score.
parent Optional[State] Parent state reference.
depth int Execution depth.
history Tuple[str, ...] Ordered node execution lineage.
metadata Dict[str, Any] Execution metadata.
NotesResponsibilities:
- Represents a complete execution snapshot at a specific point in\n pipeline traversal.\n- Fundamental unit of execution in `dagpipe`.\n- Fully thread-safe due to immutability.\n"},{"location":"dagpipe/state/#dagpipe.state.State-functions","title":"Functions","text":""},{"location":"dagpipe/state/#dagpipe.state.State.__post_init__","title":"__post_init__","text":"__post_init__() -> None\n Validate the payload against the declared schema.
Raises:
Type DescriptionSchemaError If the payload violates the schema declared on the subclass.
"},{"location":"dagpipe/state/#dagpipe.state.State.__repr__","title":"__repr__","text":"__repr__() -> str\n Concise debug representation.
Avoids printing full data for large states.
"},{"location":"dagpipe/state/#dagpipe.state.State.fork","title":"fork","text":"fork(\n *,\n payload_update: Mapping[str, Any] | None = None,\n confidence_delta: float = 0.0,\n node_id: str | None = None,\n metadata_update: Mapping[str, Any] | None = None\n) -> State\n Create a new child State derived from this state.
Parameters:
Name Type Description Defaultpayload_update Mapping[str, Any] | None Dot-path updates applied to the payload.
None confidence_delta float Adjustment applied to current confidence.
0.0 node_id str | None Identifier of the node creating this state.
None metadata_update Mapping[str, Any] | None Updates merged into state metadata.
None Returns:
Name Type DescriptionState State A new immutable State instance.
Guarantees:
- This is the only supported mechanism for modifying execution data.\n- Validates payload updates, preserves lineage, increments depth,\n and appends to history.\n"},{"location":"dagpipe/state/#dagpipe.state.State.get","title":"get","text":"get(key: str, default: Any = None) -> Any\n Retrieve payload value.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requireddefault Any Fallback value.
None Returns:
Name Type DescriptionAny Any Stored value or default.
"},{"location":"dagpipe/state/#dagpipe.state.State.has","title":"has","text":"has(key: str) -> bool\n Check whether payload contains key.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requiredReturns:
Name Type Descriptionbool bool Existence of the key.
"},{"location":"dagpipe/state/#dagpipe.state.State.lineage","title":"lineage","text":"lineage() -> tuple[State, ...]\n Return lineage from root to this State.
Returns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Ordered execution lineage (root first).
"},{"location":"dagpipe/yaml_loader/","title":"Yaml Loader","text":""},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader","title":"dagpipe.yaml_loader","text":""},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader--summary","title":"Summary","text":"Loads dagpipe pipelines from YAML configuration.
Creates fully configured pipeline objects from declarative YAML definitions, including Schema, State subclasses, Node instances, Graph topology, and initial payloads.
dataclass","text":"Pipeline(\n engine: Engine,\n state_cls: type[State],\n initial_payload: Payload,\n)\n Executable pipeline created from YAML configuration.
Attributes:
Name Type Descriptionengine Engine Execution engine responsible for running the pipeline.
state_cls Type[State] Dynamically created State subclass with configured schema.
initial_payload Payload Default payload used when execution begins.
NotesResponsibilities:
- Encapsulates engine, state type, and initial payload.\n- Provides a simplified interface for executing configured pipelines.\n- Safe for concurrent execution if underlying nodes are thread-safe.\n"},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader.Pipeline-functions","title":"Functions","text":""},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader.Pipeline.run","title":"run","text":"run(\n payload_override: Mapping[str, Any] | None = None,\n) -> list[State]\n Execute the pipeline.
Parameters:
Name Type Description Defaultpayload_override Mapping[str, Any] | None Payload values overriding initial payload.
None Returns:
Type Descriptionlist[State] list[State]: Terminal execution states.
NotesResponsibilities:
- Merges override payload with initial payload.\n- Creates root `State` and executes engine.\n"},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader-functions","title":"Functions","text":""},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader.load_pipeline","title":"load_pipeline","text":"load_pipeline(path: str) -> Pipeline\n Load pipeline from YAML file.
Parameters:
Name Type Description Defaultpath str Path to YAML configuration file.
requiredReturns:
Name Type DescriptionPipeline Pipeline Executable pipeline instance.
NotesResponsibilities:
- Loads YAML configuration and builds schema.\n- Creates `State` subclass and loads `Node` instances.\n- Builds `Graph` topology and initializes `Engine`.\n"}]}