{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"dagpipe","text":"
Directed acyclic graph execution framework for deterministic state propagation.
"},{"location":"#dagpipe--summary","title":"Summary","text":"dagpipe executes pipelines composed of nodes connected in a directed acyclic graph (DAG). Each node receives an immutable state and optionally produces derived states for downstream nodes.
Install using pip:
pip install dagpipe\n"},{"location":"#dagpipe--quick-start","title":"Quick Start","text":"from dagpipe import State, Payload, Schema, Graph, Engine\nfrom dagpipe.node import Node\n\nclass HelloNode(Node):\n id = \"hello\"\n def resolve(self, state):\n yield self.fork(state, payload_update={\"msg\": \"hello\"})\n\n# Build and run\ngraph = Graph()\ngraph.add_root(HelloNode())\nengine = Engine(graph)\n\nclass MyState(State):\n schema = Schema({})\n\nresults = engine.run(MyState(payload=Payload({})))\n"},{"location":"#dagpipe--public-api","title":"Public API","text":"This package re-exports the core pipeline components. Consumers should import from this namespace for standard usage.
Execution Core: - Engine - Graph - Node
State & Data: - State / MyState (custom base) - Payload - Schema / SchemaError
Declarative Pipelines: - Pipeline - load_pipeline
"},{"location":"#dagpipe-classes","title":"Classes","text":""},{"location":"#dagpipe-functions","title":"Functions","text":""},{"location":"engine/","title":"Engine","text":""},{"location":"engine/#dagpipe.engine","title":"dagpipe.engine","text":"Execution engine responsible for running pipelines and graphs.
"},{"location":"engine/#dagpipe.engine--summary","title":"Summary","text":"The Engine executes Node objects and propagates immutable State instances through either a linear sequence or a directed acyclic graph (Graph). It orchestrates execution order, branching, and state propagation.
NotesGuarantees:
- Deterministic execution and consistent state lineage.\n- Orchestrates execution without modifying Graph, Node, or State objects.\n"},{"location":"engine/#dagpipe.engine-classes","title":"Classes","text":""},{"location":"engine/#dagpipe.engine.Engine","title":"Engine","text":"Engine(nodes_or_graph: Union[Sequence[Node], Graph])\n Execution engine responsible for running pipeline logic.
NotesResponsibilities:
- Accepts either a linear sequence of Nodes or a Graph defining execution topology.\n- Propagates immutable State objects through Nodes and collects terminal States.\n Guarantees:
- Never mutates State, Node, or Graph instances.\n- State objects are never modified in place; each branch produces independent instances.\n- Execution order is deterministic and follows graph or pipeline topology.\n- Thread-safe for concurrent execution.\n Initialize the execution engine.
Parameters:
Name Type Description Defaultnodes_or_graph Sequence[Node] | Graph Pipeline definition. May be a linear sequence or a DAG.
requiredRaises:
Type DescriptionTypeError If input is not a Sequence[Node] or Graph, or contains invalid node types.
NotesResponsibilities:
- Detects execution mode (linear or DAG) and validates node types and structure.\n"},{"location":"engine/#dagpipe.engine.Engine-attributes","title":"Attributes","text":""},{"location":"engine/#dagpipe.engine.Engine.nodes","title":"nodes property","text":"nodes: tuple[Node, ...]\n Return nodes managed by this engine.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Ordered sequence in linear mode or all nodes in graph mode.
"},{"location":"engine/#dagpipe.engine.Engine-functions","title":"Functions","text":""},{"location":"engine/#dagpipe.engine.Engine.__repr__","title":"__repr__","text":"__repr__() -> str\n Return the canonical string representation of the object.
Returns:
Name Type Descriptionstr str Representation that uniquely identifies the object and its configuration.
"},{"location":"engine/#dagpipe.engine.Engine.run","title":"run","text":"run(root: State) -> List[State]\n Execute the pipeline starting from a root State.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type DescriptionList[State] list[State]: Terminal execution states produced by the pipeline.
Raises:
Type DescriptionTypeError If root is not a State instance.
RuntimeError If the engine execution mode is invalid.
NotesResponsibilities:
- Selects execution mode, propagates state through nodes, creates new instances for branches, and collects terminal states.\n"},{"location":"graph/","title":"Graph","text":""},{"location":"graph/#dagpipe.graph","title":"dagpipe.graph","text":"Defines DAG structure connecting Nodes.
"},{"location":"graph/#dagpipe.graph--summary","title":"Summary","text":"Graph describes execution topology only. It does not execute nodes or manage State. Execution is handled by Engine.
NotesResponsibilities:
- Multiple roots, branching, and merging support.\n- Deterministic traversal based on topology.\n- Graph is mutable during construction but treated as immutable at runtime.\n"},{"location":"graph/#dagpipe.graph-classes","title":"Classes","text":""},{"location":"graph/#dagpipe.graph.Graph","title":"Graph","text":"Graph()\n Directed Acyclic Graph defining execution topology of Nodes.
NotesResponsibilities:
- Stores node connectivity and validates that the topology remains acyclic.\n- Structure determines how State flows between nodes during execution.\n Guarantees:
- Topology is acyclic. Node relationships remain consistent.\n- Thread-safe for concurrent reads after construction.\n Create an empty Graph.
"},{"location":"graph/#dagpipe.graph.Graph--initializes-node-registry-and-edge-mappings","title":"Initializes node registry and edge mappings.","text":""},{"location":"graph/#dagpipe.graph.Graph-functions","title":"Functions","text":""},{"location":"graph/#dagpipe.graph.Graph.__repr__","title":"__repr__","text":"__repr__() -> str\n Return debug representation.
"},{"location":"graph/#dagpipe.graph.Graph.__repr__--returns","title":"Returns","text":"str
"},{"location":"graph/#dagpipe.graph.Graph.add_edge","title":"add_edge","text":"add_edge(src: Node, dst: Node) -> None\n"},{"location":"graph/#dagpipe.graph.Graph.add_edge--add-directed-edge-from-src-to-dst","title":"Add directed edge from src to dst.","text":""},{"location":"graph/#dagpipe.graph.Graph.add_edge--parameters","title":"Parameters","text":"src : Node required: True source node
Noderequired: True destination node
"},{"location":"graph/#dagpipe.graph.Graph.add_edge--raises","title":"Raises","text":"TypeError if src or dst is not a Node
ValueError if edge would create a cycle
ValueError if src and dst are the same node
"},{"location":"graph/#dagpipe.graph.Graph.add_edge--behavior","title":"Behavior","text":"add_root(node: Node) -> None\n Add a root node with no parents.
Parameters:
Name Type Description Defaultnode Node Node to add as a root.
requiredRaises:
Type DescriptionTypeError If node is not a Node instance.
"},{"location":"graph/#dagpipe.graph.Graph.children","title":"children","text":"children(node: Node) -> Tuple[Node, ...]\n Return child nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type DescriptionTuple[Node, ...] Tuple[Node, ...]: Outgoing neighbors.
"},{"location":"graph/#dagpipe.graph.Graph.nodes","title":"nodes","text":"nodes() -> Tuple[Node, ...]\n Return all nodes in the graph.
Returns:
Type DescriptionTuple[Node, ...] Tuple[Node, ...]: All registered nodes.
"},{"location":"graph/#dagpipe.graph.Graph.parents","title":"parents","text":"parents(node: Node) -> Tuple[Node, ...]\n Return parent nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type DescriptionTuple[Node, ...] Tuple[Node, ...]: Incoming neighbors.
"},{"location":"graph/#dagpipe.graph.Graph.roots","title":"roots","text":"roots() -> Tuple[Node, ...]\n Return root nodes (nodes with no incoming edges).
Returns:
Type DescriptionTuple[Node, ...] Tuple[Node, ...]: Entry point nodes.
"},{"location":"node/","title":"Node","text":""},{"location":"node/#dagpipe.node","title":"dagpipe.node","text":"Defines the Node abstraction used by dagpipe.
"},{"location":"node/#dagpipe.node--summary","title":"Summary","text":"A Node represents a single unit of pipeline execution logic. It consumes one State and produces zero, one, or many new States.
Nodes are connected using Graph and executed by Engine.
NotesDesign Principles:
- **Pure:** Must not mutate input state.\n- **Deterministic:** Same input produces same output.\n- **Stateless:** Recommended to be stateless for reuse.\n- **Composable:** Nodes enable branching execution graphs.\n"},{"location":"node/#dagpipe.node-classes","title":"Classes","text":""},{"location":"node/#dagpipe.node.Node","title":"Node","text":" Bases: ABC
Base class for all dagpipe execution nodes.
Attributes:
Name Type Descriptionid str Unique identifier of the node (snake_case dotted format).
name str Human-readable display name.
NotesResponsibilities:
- Represents a deterministic unit of execution in the pipeline graph.\n- Consumes one State and produces zero, one, or many derived States.\n- Defines execution logic and enables branching, filtering, and transformation.\n Guarantees:
- Nodes must never mutate the input State. Instances are singletons per subclass and reused across executions.\n"},{"location":"node/#dagpipe.node.Node-functions","title":"Functions","text":""},{"location":"node/#dagpipe.node.Node.__hash__","title":"__hash__","text":"__hash__() -> int\n Return stable hash based on node ID.
"},{"location":"node/#dagpipe.node.Node.__hash__--returns","title":"Returns","text":"int
"},{"location":"node/#dagpipe.node.Node.__new__","title":"__new__","text":"__new__()\n Create or reuse singleton instance of the Node subclass.
Returns:
Name Type DescriptionNode Singleton instance of the subclass.
"},{"location":"node/#dagpipe.node.Node.__repr__","title":"__repr__","text":"__repr__() -> str\n Return debug representation.
"},{"location":"node/#dagpipe.node.Node.__repr__--returns","title":"Returns","text":"str
"},{"location":"node/#dagpipe.node.Node.__str__","title":"__str__","text":"__str__() -> str\n Return display representation.
"},{"location":"node/#dagpipe.node.Node.__str__--returns","title":"Returns","text":"str
"},{"location":"node/#dagpipe.node.Node.clean_id_and_name","title":"clean_id_and_nameclassmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"node/#dagpipe.node.Node.fork","title":"fork","text":"fork(state: State, *, payload_update=None, confidence_delta=0.0, metadata_update=None) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Mapping[str, Any] Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Mapping[str, Any] Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around State.fork() that automatically records this node's ID in state history.\n"},{"location":"node/#dagpipe.node.Node.node_id_to_name","title":"node_id_to_name staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"node/#dagpipe.node.Node.resolve","title":"resolveabstractmethod","text":"resolve(state: State) -> Iterable[State]\n Execute node logic.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredYields:
Name Type DescriptionState Iterable[State] Derived execution state(s).
NotesResponsibilities:
- Subclasses implement specific resolution behavior. Must not mutate input state. Should use fork() to create child states. May yield zero states to terminate a branch.\n"},{"location":"node/#dagpipe.node.Node.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
"},{"location":"state/","title":"State","text":""},{"location":"state/#dagpipe.state","title":"dagpipe.state","text":"Defines the core State object used by dagpipe.
"},{"location":"state/#dagpipe.state--summary","title":"Summary","text":"The State represents a single point in pipeline execution. It contains arbitrary data and metadata and is designed to be immutable. Instead of modifying an existing state, nodes create new child states via fork().
NotesKey design principles:
- **Immutability:** States must never be modified after creation. All transformations must create a new state via fork().\n- **Cheap cloning:** Forking must be efficient since branching may create many states.\n- **Lineage tracking:** Each state maintains a reference to its parent and execution metadata for debugging and observability.\n- **Domain agnostic:** State contains generic key\u2011value data and does not assume any schema.\n- **Engine\u2011friendly:** State contains execution metadata such as depth and history.\n"},{"location":"state/#dagpipe.state-classes","title":"Classes","text":""},{"location":"state/#dagpipe.state.Payload","title":"Payload dataclass","text":"Payload(_data: Mapping[str, Any])\n Immutable hierarchical container with dot-path access.
Attributes:
Name Type Description_data Mapping[str, Any] Immutable hierarchical data structure.
NotesResponsibilities:
- Stores execution data used by State. Supports efficient atomic updates without modifying existing instances.\n- Payload instances are fully thread-safe due to immutability.\n"},{"location":"state/#dagpipe.state.Payload-functions","title":"Functions","text":""},{"location":"state/#dagpipe.state.Payload.as_dict","title":"as_dict","text":"as_dict() -> Mapping[str, Any]\n Return underlying mapping.
Returns:
Type DescriptionMapping[str, Any] Mapping[str, Any]: Read-only view of the underlying data.
"},{"location":"state/#dagpipe.state.Payload.get","title":"get","text":"get(path: str, default: Any = None) -> Any\n Retrieve value using dot-path.
Parameters:
Name Type Description Defaultpath str Dot-separated path to the value.
requireddefault Any Default value if path doesn't exist.
None Returns:
Name Type DescriptionAny Any The retrieved value or default.
"},{"location":"state/#dagpipe.state.Payload.has","title":"has","text":"has(path: str) -> bool\n Return True if path exists.
Parameters:
Name Type Description Defaultpath str Dot-separated path to check.
requiredReturns:
Name Type Descriptionbool bool Existence of the path.
"},{"location":"state/#dagpipe.state.Payload.iter_paths","title":"iter_pathsclassmethod","text":"iter_paths(data: Mapping[str, Any], prefix: str = '') -> Iterable[str]\n Recursively yield dot-paths for all leaf nodes.
Parameters:
Name Type Description Defaultdata Mapping[str, Any] The mapping to iterate over.
requiredprefix str Current path prefix.
'' Returns:
Type DescriptionIterable[str] Iterable[str]: Generator yielding dot-paths.
"},{"location":"state/#dagpipe.state.Payload.keys","title":"keys","text":"keys() -> Iterable[str]\n Return top-level keys.
Returns:
Type DescriptionIterable[str] Iterable[str]: Iterator over top-level keys.
"},{"location":"state/#dagpipe.state.Payload.update","title":"update","text":"update(updates: Mapping[str, Any]) -> Payload\n Create a new Payload with dot-path updates applied.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path to value mapping.
requiredReturns:
Name Type DescriptionPayload Payload New immutable payload instance with updates.
NotesGuarantees:
- Preserves existing data by copying only modified branches. Returns a new immutable payload.\n"},{"location":"state/#dagpipe.state.Schema","title":"Schema dataclass","text":"Schema(tree: Mapping[str, SchemaNode])\n Immutable hierarchical schema defining allowed payload structure.
Attributes:
Name Type Descriptiontree Mapping[str, SchemaNode] Hierarchical schema definition.
NotesResponsibilities:
- Validates State payloads and updates. Reuseable across all State instances.\n- Fully thread-safe due to immutability.\n"},{"location":"state/#dagpipe.state.Schema-functions","title":"Functions","text":""},{"location":"state/#dagpipe.state.Schema.validate_payload","title":"validate_payload","text":"validate_payload(payload: Payload) -> None\n Validate complete payload structure.
Parameters:
Name Type Description Defaultpayload Payload Payload to validate.
requiredRaises:
Type DescriptionSchemaError If payload violates schema.
"},{"location":"state/#dagpipe.state.Schema.validate_update","title":"validate_update","text":"validate_update(updates: Mapping[str, Any]) -> None\n Validate payload update paths.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path updates to validate.
requiredRaises:
Type DescriptionSchemaError If any path is invalid according to the schema.
"},{"location":"state/#dagpipe.state.SchemaError","title":"SchemaError","text":" Bases: Exception
Raised when payload data violates the declared schema.
"},{"location":"state/#dagpipe.state.SchemaError--indicates-invalid-structure-invalid-path-or-invalid-type","title":"Indicates invalid structure, invalid path, or invalid type.","text":""},{"location":"state/#dagpipe.state.State","title":"Statedataclass","text":"State(payload: Payload, confidence: float = 1.0, parent: Optional[State] = None, depth: int = 0, history: Tuple[str, ...] = tuple(), metadata: Dict[str, Any] = dict())\n Immutable execution state propagated through dagpipe pipeline.
Attributes:
Name Type Descriptionpayload Payload Execution data container.
schema ClassVar[Schema] Payload validation schema.
confidence float Execution confidence score.
parent Optional[State] Parent state reference.
depth int Execution depth.
history Tuple[str, ...] Ordered node execution lineage.
metadata Dict[str, Any] Execution metadata.
NotesResponsibilities:
- Represents a complete execution snapshot at a specific point in pipeline traversal. Fundamental unit of execution in dagpipe.\n- Fully thread-safe due to immutability.\n"},{"location":"state/#dagpipe.state.State-functions","title":"Functions","text":""},{"location":"state/#dagpipe.state.State.__repr__","title":"__repr__","text":"__repr__() -> str\n Concise debug representation.
Avoids printing full data for large states.
"},{"location":"state/#dagpipe.state.State.fork","title":"fork","text":"fork(*, payload_update: Optional[Mapping[str, Any]] = None, confidence_delta: float = 0.0, node_id: Optional[str] = None, metadata_update: Optional[Mapping[str, Any]] = None) -> State\n Create a new child State derived from this State.
Parameters:
Name Type Description Defaultpayload_update Mapping[str, Any] Dot-path updates applied to the payload.
None confidence_delta float Adjustment applied to current confidence.
0.0 node_id str Identifier of the node creating this state.
None metadata_update Mapping[str, Any] Updates merged into state metadata.
None Returns:
Name Type DescriptionState State A new immutable State instance.
NotesGuarantees:
- This is the only supported mechanism for modifying execution data.\n- Validates payload updates, preserves lineage, increments depth, and appends to history.\n"},{"location":"state/#dagpipe.state.State.get","title":"get","text":"get(key: str, default: Any = None) -> Any\n Retrieve payload value.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requireddefault Any Fallback value.
None Returns:
Name Type DescriptionAny Any Stored value or default.
"},{"location":"state/#dagpipe.state.State.has","title":"has","text":"has(key: str) -> bool\n Check whether payload contains key.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requiredReturns:
Name Type Descriptionbool bool Existence of the key.
"},{"location":"state/#dagpipe.state.State.lineage","title":"lineage","text":"lineage() -> Tuple[State, ...]\n Return lineage from root to this State.
Returns:
Type DescriptionTuple[State, ...] Tuple[State, ...]: Ordered execution lineage (root first).
"},{"location":"yaml_loader/","title":"Yaml Loader","text":""},{"location":"yaml_loader/#dagpipe.yaml_loader","title":"dagpipe.yaml_loader","text":"Loads dagpipe pipelines from YAML configuration.
"},{"location":"yaml_loader/#dagpipe.yaml_loader--summary","title":"Summary","text":"Creates fully configured pipeline objects from declarative YAML definitions, including Schema, State subclasses, Node instances, Graph topology, and initial Payloads.
"},{"location":"yaml_loader/#dagpipe.yaml_loader-classes","title":"Classes","text":""},{"location":"yaml_loader/#dagpipe.yaml_loader.Pipeline","title":"Pipelinedataclass","text":"Pipeline(engine: Engine, state_cls: Type[State], initial_payload: Payload)\n Executable pipeline created from YAML configuration.
Attributes:
Name Type Descriptionengine Engine Execution engine responsible for running the pipeline.
state_cls Type[State] Dynamically created State subclass with configured schema.
initial_payload Payload Default payload used when execution begins.
NotesResponsibilities:
- Encapsulates engine, state type, and initial payload. Provides a simplified interface for executing configured pipelines.\n- Safe for concurrent execution if underlying Nodes are thread-safe.\n"},{"location":"yaml_loader/#dagpipe.yaml_loader.Pipeline-functions","title":"Functions","text":""},{"location":"yaml_loader/#dagpipe.yaml_loader.Pipeline.run","title":"run","text":"run(payload_override: Optional[Mapping[str, Any]] = None)\n Execute pipeline.
Parameters:
Name Type Description Defaultpayload_override Mapping[str, Any] Payload values overriding initial payload.
None Returns:
Type Descriptionlist[State]: Terminal execution states.
NotesResponsibilities:
- Merges override payload with initial payload, creates root State, and executes engine.\n"},{"location":"yaml_loader/#dagpipe.yaml_loader-functions","title":"Functions","text":""},{"location":"yaml_loader/#dagpipe.yaml_loader.load_pipeline","title":"load_pipeline","text":"load_pipeline(path: str) -> Pipeline\n Load pipeline from YAML file.
Parameters:
Name Type Description Defaultpath str Path to YAML configuration file.
requiredReturns:
Name Type DescriptionPipeline Pipeline Executable pipeline instance.
NotesResponsibilities:
- Loads YAML configuration, builds schema, creates State subclass, loads Node instances, builds Graph topology, and initializes Engine.\n"},{"location":"dagpipe/","title":"Dagpipe","text":"Directed acyclic graph execution framework for deterministic state propagation.
"},{"location":"dagpipe/#dagpipe--summary","title":"Summary","text":"dagpipe executes pipelines composed of nodes connected in a directed acyclic graph (DAG). Each node receives an immutable state and optionally produces derived states for downstream nodes.
Install using pip:
pip install dagpipe\n"},{"location":"dagpipe/#dagpipe--quick-start","title":"Quick Start","text":"from dagpipe import State, Payload, Schema, Graph, Engine\nfrom dagpipe.node import Node\n\nclass HelloNode(Node):\n id = \"hello\"\n def resolve(self, state):\n yield self.fork(state, payload_update={\"msg\": \"hello\"})\n\n# Build and run\ngraph = Graph()\ngraph.add_root(HelloNode())\nengine = Engine(graph)\n\nclass MyState(State):\n schema = Schema({})\n\nresults = engine.run(MyState(payload=Payload({})))\n"},{"location":"dagpipe/#dagpipe--public-api","title":"Public API","text":"This package re-exports the core pipeline components. Consumers should import from this namespace for standard usage.
Execution Core: - Engine - Graph - Node
State & Data: - State / MyState (custom base) - Payload - Schema / SchemaError
Declarative Pipelines: - Pipeline - load_pipeline
"},{"location":"dagpipe/#dagpipe-classes","title":"Classes","text":""},{"location":"dagpipe/#dagpipe-functions","title":"Functions","text":""},{"location":"dagpipe/engine/","title":"Engine","text":""},{"location":"dagpipe/engine/#dagpipe.engine","title":"dagpipe.engine","text":"Execution engine responsible for running pipelines and graphs.
"},{"location":"dagpipe/engine/#dagpipe.engine--summary","title":"Summary","text":"The Engine executes Node objects and propagates immutable State instances through either a linear sequence or a directed acyclic graph (Graph). It orchestrates execution order, branching, and state propagation.
NotesGuarantees:
- Deterministic execution and consistent state lineage.\n- Orchestrates execution without modifying Graph, Node, or State objects.\n"},{"location":"dagpipe/engine/#dagpipe.engine-classes","title":"Classes","text":""},{"location":"dagpipe/engine/#dagpipe.engine.Engine","title":"Engine","text":"Engine(nodes_or_graph: Union[Sequence[Node], Graph])\n Execution engine responsible for running pipeline logic.
NotesResponsibilities:
- Accepts either a linear sequence of Nodes or a Graph defining execution topology.\n- Propagates immutable State objects through Nodes and collects terminal States.\n Guarantees:
- Never mutates State, Node, or Graph instances.\n- State objects are never modified in place; each branch produces independent instances.\n- Execution order is deterministic and follows graph or pipeline topology.\n- Thread-safe for concurrent execution.\n Initialize the execution engine.
Parameters:
Name Type Description Defaultnodes_or_graph Sequence[Node] | Graph Pipeline definition. May be a linear sequence or a DAG.
requiredRaises:
Type DescriptionTypeError If input is not a Sequence[Node] or Graph, or contains invalid node types.
NotesResponsibilities:
- Detects execution mode (linear or DAG) and validates node types and structure.\n"},{"location":"dagpipe/engine/#dagpipe.engine.Engine-attributes","title":"Attributes","text":""},{"location":"dagpipe/engine/#dagpipe.engine.Engine.nodes","title":"nodes property","text":"nodes: tuple[Node, ...]\n Return nodes managed by this engine.
Returns:
Type Descriptiontuple[Node, ...] tuple[Node, ...]: Ordered sequence in linear mode or all nodes in graph mode.
"},{"location":"dagpipe/engine/#dagpipe.engine.Engine-functions","title":"Functions","text":""},{"location":"dagpipe/engine/#dagpipe.engine.Engine.__repr__","title":"__repr__","text":"__repr__() -> str\n Return the canonical string representation of the object.
Returns:
Name Type Descriptionstr str Representation that uniquely identifies the object and its configuration.
"},{"location":"dagpipe/engine/#dagpipe.engine.Engine.run","title":"run","text":"run(root: State) -> List[State]\n Execute the pipeline starting from a root State.
Parameters:
Name Type Description Defaultroot State Initial execution state.
requiredReturns:
Type DescriptionList[State] list[State]: Terminal execution states produced by the pipeline.
Raises:
Type DescriptionTypeError If root is not a State instance.
RuntimeError If the engine execution mode is invalid.
NotesResponsibilities:
- Selects execution mode, propagates state through nodes, creates new instances for branches, and collects terminal states.\n"},{"location":"dagpipe/graph/","title":"Graph","text":""},{"location":"dagpipe/graph/#dagpipe.graph","title":"dagpipe.graph","text":"Defines DAG structure connecting Nodes.
"},{"location":"dagpipe/graph/#dagpipe.graph--summary","title":"Summary","text":"Graph describes execution topology only. It does not execute nodes or manage State. Execution is handled by Engine.
NotesResponsibilities:
- Multiple roots, branching, and merging support.\n- Deterministic traversal based on topology.\n- Graph is mutable during construction but treated as immutable at runtime.\n"},{"location":"dagpipe/graph/#dagpipe.graph-classes","title":"Classes","text":""},{"location":"dagpipe/graph/#dagpipe.graph.Graph","title":"Graph","text":"Graph()\n Directed Acyclic Graph defining execution topology of Nodes.
NotesResponsibilities:
- Stores node connectivity and validates that the topology remains acyclic.\n- Structure determines how State flows between nodes during execution.\n Guarantees:
- Topology is acyclic. Node relationships remain consistent.\n- Thread-safe for concurrent reads after construction.\n Create an empty Graph.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph--initializes-node-registry-and-edge-mappings","title":"Initializes node registry and edge mappings.","text":""},{"location":"dagpipe/graph/#dagpipe.graph.Graph-functions","title":"Functions","text":""},{"location":"dagpipe/graph/#dagpipe.graph.Graph.__repr__","title":"__repr__","text":"__repr__() -> str\n Return debug representation.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.__repr__--returns","title":"Returns","text":"str
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.add_edge","title":"add_edge","text":"add_edge(src: Node, dst: Node) -> None\n"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.add_edge--add-directed-edge-from-src-to-dst","title":"Add directed edge from src to dst.","text":""},{"location":"dagpipe/graph/#dagpipe.graph.Graph.add_edge--parameters","title":"Parameters","text":"src : Node required: True source node
Noderequired: True destination node
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.add_edge--raises","title":"Raises","text":"TypeError if src or dst is not a Node
ValueError if edge would create a cycle
ValueError if src and dst are the same node
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.add_edge--behavior","title":"Behavior","text":"add_root(node: Node) -> None\n Add a root node with no parents.
Parameters:
Name Type Description Defaultnode Node Node to add as a root.
requiredRaises:
Type DescriptionTypeError If node is not a Node instance.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.children","title":"children","text":"children(node: Node) -> Tuple[Node, ...]\n Return child nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type DescriptionTuple[Node, ...] Tuple[Node, ...]: Outgoing neighbors.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.nodes","title":"nodes","text":"nodes() -> Tuple[Node, ...]\n Return all nodes in the graph.
Returns:
Type DescriptionTuple[Node, ...] Tuple[Node, ...]: All registered nodes.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.parents","title":"parents","text":"parents(node: Node) -> Tuple[Node, ...]\n Return parent nodes of a node.
Parameters:
Name Type Description Defaultnode Node Node to query.
requiredReturns:
Type DescriptionTuple[Node, ...] Tuple[Node, ...]: Incoming neighbors.
"},{"location":"dagpipe/graph/#dagpipe.graph.Graph.roots","title":"roots","text":"roots() -> Tuple[Node, ...]\n Return root nodes (nodes with no incoming edges).
Returns:
Type DescriptionTuple[Node, ...] Tuple[Node, ...]: Entry point nodes.
"},{"location":"dagpipe/node/","title":"Node","text":""},{"location":"dagpipe/node/#dagpipe.node","title":"dagpipe.node","text":"Defines the Node abstraction used by dagpipe.
"},{"location":"dagpipe/node/#dagpipe.node--summary","title":"Summary","text":"A Node represents a single unit of pipeline execution logic. It consumes one State and produces zero, one, or many new States.
Nodes are connected using Graph and executed by Engine.
NotesDesign Principles:
- **Pure:** Must not mutate input state.\n- **Deterministic:** Same input produces same output.\n- **Stateless:** Recommended to be stateless for reuse.\n- **Composable:** Nodes enable branching execution graphs.\n"},{"location":"dagpipe/node/#dagpipe.node-classes","title":"Classes","text":""},{"location":"dagpipe/node/#dagpipe.node.Node","title":"Node","text":" Bases: ABC
Base class for all dagpipe execution nodes.
Attributes:
Name Type Descriptionid str Unique identifier of the node (snake_case dotted format).
name str Human-readable display name.
NotesResponsibilities:
- Represents a deterministic unit of execution in the pipeline graph.\n- Consumes one State and produces zero, one, or many derived States.\n- Defines execution logic and enables branching, filtering, and transformation.\n Guarantees:
- Nodes must never mutate the input State. Instances are singletons per subclass and reused across executions.\n"},{"location":"dagpipe/node/#dagpipe.node.Node-functions","title":"Functions","text":""},{"location":"dagpipe/node/#dagpipe.node.Node.__hash__","title":"__hash__","text":"__hash__() -> int\n Return stable hash based on node ID.
"},{"location":"dagpipe/node/#dagpipe.node.Node.__hash__--returns","title":"Returns","text":"int
"},{"location":"dagpipe/node/#dagpipe.node.Node.__new__","title":"__new__","text":"__new__()\n Create or reuse singleton instance of the Node subclass.
Returns:
Name Type DescriptionNode Singleton instance of the subclass.
"},{"location":"dagpipe/node/#dagpipe.node.Node.__repr__","title":"__repr__","text":"__repr__() -> str\n Return debug representation.
"},{"location":"dagpipe/node/#dagpipe.node.Node.__repr__--returns","title":"Returns","text":"str
"},{"location":"dagpipe/node/#dagpipe.node.Node.__str__","title":"__str__","text":"__str__() -> str\n Return display representation.
"},{"location":"dagpipe/node/#dagpipe.node.Node.__str__--returns","title":"Returns","text":"str
"},{"location":"dagpipe/node/#dagpipe.node.Node.clean_id_and_name","title":"clean_id_and_nameclassmethod","text":"clean_id_and_name() -> None\n Normalize and validate node ID and display name.
Raises:
Type DescriptionTypeError If ID is not a string.
ValueError If ID format is invalid.
NotesGuarantees:
- Generates ID from module and class name if missing.\n- Validates ID format.\n- Generates human-readable name if missing.\n"},{"location":"dagpipe/node/#dagpipe.node.Node.fork","title":"fork","text":"fork(state: State, *, payload_update=None, confidence_delta=0.0, metadata_update=None) -> State\n Create a child State attributed to this node.
Parameters:
Name Type Description Defaultstate State Parent execution state.
requiredpayload_update Mapping[str, Any] Dot-path payload updates.
None confidence_delta float Confidence adjustment.
0.0 metadata_update Mapping[str, Any] Metadata updates.
None Returns:
Name Type DescriptionState State New child execution state.
NotesResponsibilities:
- Convenience wrapper around State.fork() that automatically records this node's ID in state history.\n"},{"location":"dagpipe/node/#dagpipe.node.Node.node_id_to_name","title":"node_id_to_name staticmethod","text":"node_id_to_name(node_id: str) -> str\n Convert a dotted snake_case node ID into a human-readable name.
Parameters:
Name Type Description Defaultnode_id str Unique node identifier (e.g., 'entity.resolve.numeric_merchant').
requiredReturns:
Name Type Descriptionstr str Human-readable display name (e.g., 'Entity \u203a Resolve \u203a Numeric Merchant').
"},{"location":"dagpipe/node/#dagpipe.node.Node.resolve","title":"resolveabstractmethod","text":"resolve(state: State) -> Iterable[State]\n Execute node logic.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredYields:
Name Type DescriptionState Iterable[State] Derived execution state(s).
NotesResponsibilities:
- Subclasses implement specific resolution behavior. Must not mutate input state. Should use fork() to create child states. May yield zero states to terminate a branch.\n"},{"location":"dagpipe/node/#dagpipe.node.Node.run","title":"run","text":"run(state: State) -> tuple[State, ...]\n Execute this node on a State.
Parameters:
Name Type Description Defaultstate State Input execution state.
requiredReturns:
Type Descriptiontuple[State, ...] tuple[State, ...]: Derived execution states.
Raises:
Type DescriptionTypeError If resolve() yields a non-State object.
"},{"location":"dagpipe/state/","title":"State","text":""},{"location":"dagpipe/state/#dagpipe.state","title":"dagpipe.state","text":"Defines the core State object used by dagpipe.
"},{"location":"dagpipe/state/#dagpipe.state--summary","title":"Summary","text":"The State represents a single point in pipeline execution. It contains arbitrary data and metadata and is designed to be immutable. Instead of modifying an existing state, nodes create new child states via fork().
NotesKey design principles:
- **Immutability:** States must never be modified after creation. All transformations must create a new state via fork().\n- **Cheap cloning:** Forking must be efficient since branching may create many states.\n- **Lineage tracking:** Each state maintains a reference to its parent and execution metadata for debugging and observability.\n- **Domain agnostic:** State contains generic key\u2011value data and does not assume any schema.\n- **Engine\u2011friendly:** State contains execution metadata such as depth and history.\n"},{"location":"dagpipe/state/#dagpipe.state-classes","title":"Classes","text":""},{"location":"dagpipe/state/#dagpipe.state.Payload","title":"Payload dataclass","text":"Payload(_data: Mapping[str, Any])\n Immutable hierarchical container with dot-path access.
Attributes:
Name Type Description_data Mapping[str, Any] Immutable hierarchical data structure.
NotesResponsibilities:
- Stores execution data used by State. Supports efficient atomic updates without modifying existing instances.\n- Payload instances are fully thread-safe due to immutability.\n"},{"location":"dagpipe/state/#dagpipe.state.Payload-functions","title":"Functions","text":""},{"location":"dagpipe/state/#dagpipe.state.Payload.as_dict","title":"as_dict","text":"as_dict() -> Mapping[str, Any]\n Return underlying mapping.
Returns:
Type DescriptionMapping[str, Any] Mapping[str, Any]: Read-only view of the underlying data.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.get","title":"get","text":"get(path: str, default: Any = None) -> Any\n Retrieve value using dot-path.
Parameters:
Name Type Description Defaultpath str Dot-separated path to the value.
requireddefault Any Default value if path doesn't exist.
None Returns:
Name Type DescriptionAny Any The retrieved value or default.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.has","title":"has","text":"has(path: str) -> bool\n Return True if path exists.
Parameters:
Name Type Description Defaultpath str Dot-separated path to check.
requiredReturns:
Name Type Descriptionbool bool Existence of the path.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.iter_paths","title":"iter_pathsclassmethod","text":"iter_paths(data: Mapping[str, Any], prefix: str = '') -> Iterable[str]\n Recursively yield dot-paths for all leaf nodes.
Parameters:
Name Type Description Defaultdata Mapping[str, Any] The mapping to iterate over.
requiredprefix str Current path prefix.
'' Returns:
Type DescriptionIterable[str] Iterable[str]: Generator yielding dot-paths.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.keys","title":"keys","text":"keys() -> Iterable[str]\n Return top-level keys.
Returns:
Type DescriptionIterable[str] Iterable[str]: Iterator over top-level keys.
"},{"location":"dagpipe/state/#dagpipe.state.Payload.update","title":"update","text":"update(updates: Mapping[str, Any]) -> Payload\n Create a new Payload with dot-path updates applied.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path to value mapping.
requiredReturns:
Name Type DescriptionPayload Payload New immutable payload instance with updates.
NotesGuarantees:
- Preserves existing data by copying only modified branches. Returns a new immutable payload.\n"},{"location":"dagpipe/state/#dagpipe.state.Schema","title":"Schema dataclass","text":"Schema(tree: Mapping[str, SchemaNode])\n Immutable hierarchical schema defining allowed payload structure.
Attributes:
Name Type Descriptiontree Mapping[str, SchemaNode] Hierarchical schema definition.
NotesResponsibilities:
- Validates State payloads and updates. Reuseable across all State instances.\n- Fully thread-safe due to immutability.\n"},{"location":"dagpipe/state/#dagpipe.state.Schema-functions","title":"Functions","text":""},{"location":"dagpipe/state/#dagpipe.state.Schema.validate_payload","title":"validate_payload","text":"validate_payload(payload: Payload) -> None\n Validate complete payload structure.
Parameters:
Name Type Description Defaultpayload Payload Payload to validate.
requiredRaises:
Type DescriptionSchemaError If payload violates schema.
"},{"location":"dagpipe/state/#dagpipe.state.Schema.validate_update","title":"validate_update","text":"validate_update(updates: Mapping[str, Any]) -> None\n Validate payload update paths.
Parameters:
Name Type Description Defaultupdates Mapping[str, Any] Dot-path updates to validate.
requiredRaises:
Type DescriptionSchemaError If any path is invalid according to the schema.
"},{"location":"dagpipe/state/#dagpipe.state.SchemaError","title":"SchemaError","text":" Bases: Exception
Raised when payload data violates the declared schema.
"},{"location":"dagpipe/state/#dagpipe.state.SchemaError--indicates-invalid-structure-invalid-path-or-invalid-type","title":"Indicates invalid structure, invalid path, or invalid type.","text":""},{"location":"dagpipe/state/#dagpipe.state.State","title":"Statedataclass","text":"State(payload: Payload, confidence: float = 1.0, parent: Optional[State] = None, depth: int = 0, history: Tuple[str, ...] = tuple(), metadata: Dict[str, Any] = dict())\n Immutable execution state propagated through dagpipe pipeline.
Attributes:
Name Type Descriptionpayload Payload Execution data container.
schema ClassVar[Schema] Payload validation schema.
confidence float Execution confidence score.
parent Optional[State] Parent state reference.
depth int Execution depth.
history Tuple[str, ...] Ordered node execution lineage.
metadata Dict[str, Any] Execution metadata.
NotesResponsibilities:
- Represents a complete execution snapshot at a specific point in pipeline traversal. Fundamental unit of execution in dagpipe.\n- Fully thread-safe due to immutability.\n"},{"location":"dagpipe/state/#dagpipe.state.State-functions","title":"Functions","text":""},{"location":"dagpipe/state/#dagpipe.state.State.__repr__","title":"__repr__","text":"__repr__() -> str\n Concise debug representation.
Avoids printing full data for large states.
"},{"location":"dagpipe/state/#dagpipe.state.State.fork","title":"fork","text":"fork(*, payload_update: Optional[Mapping[str, Any]] = None, confidence_delta: float = 0.0, node_id: Optional[str] = None, metadata_update: Optional[Mapping[str, Any]] = None) -> State\n Create a new child State derived from this State.
Parameters:
Name Type Description Defaultpayload_update Mapping[str, Any] Dot-path updates applied to the payload.
None confidence_delta float Adjustment applied to current confidence.
0.0 node_id str Identifier of the node creating this state.
None metadata_update Mapping[str, Any] Updates merged into state metadata.
None Returns:
Name Type DescriptionState State A new immutable State instance.
NotesGuarantees:
- This is the only supported mechanism for modifying execution data.\n- Validates payload updates, preserves lineage, increments depth, and appends to history.\n"},{"location":"dagpipe/state/#dagpipe.state.State.get","title":"get","text":"get(key: str, default: Any = None) -> Any\n Retrieve payload value.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requireddefault Any Fallback value.
None Returns:
Name Type DescriptionAny Any Stored value or default.
"},{"location":"dagpipe/state/#dagpipe.state.State.has","title":"has","text":"has(key: str) -> bool\n Check whether payload contains key.
Parameters:
Name Type Description Defaultkey str Dot-path key.
requiredReturns:
Name Type Descriptionbool bool Existence of the key.
"},{"location":"dagpipe/state/#dagpipe.state.State.lineage","title":"lineage","text":"lineage() -> Tuple[State, ...]\n Return lineage from root to this State.
Returns:
Type DescriptionTuple[State, ...] Tuple[State, ...]: Ordered execution lineage (root first).
"},{"location":"dagpipe/yaml_loader/","title":"Yaml Loader","text":""},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader","title":"dagpipe.yaml_loader","text":"Loads dagpipe pipelines from YAML configuration.
"},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader--summary","title":"Summary","text":"Creates fully configured pipeline objects from declarative YAML definitions, including Schema, State subclasses, Node instances, Graph topology, and initial Payloads.
"},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader-classes","title":"Classes","text":""},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader.Pipeline","title":"Pipelinedataclass","text":"Pipeline(engine: Engine, state_cls: Type[State], initial_payload: Payload)\n Executable pipeline created from YAML configuration.
Attributes:
Name Type Descriptionengine Engine Execution engine responsible for running the pipeline.
state_cls Type[State] Dynamically created State subclass with configured schema.
initial_payload Payload Default payload used when execution begins.
NotesResponsibilities:
- Encapsulates engine, state type, and initial payload. Provides a simplified interface for executing configured pipelines.\n- Safe for concurrent execution if underlying Nodes are thread-safe.\n"},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader.Pipeline-functions","title":"Functions","text":""},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader.Pipeline.run","title":"run","text":"run(payload_override: Optional[Mapping[str, Any]] = None)\n Execute pipeline.
Parameters:
Name Type Description Defaultpayload_override Mapping[str, Any] Payload values overriding initial payload.
None Returns:
Type Descriptionlist[State]: Terminal execution states.
NotesResponsibilities:
- Merges override payload with initial payload, creates root State, and executes engine.\n"},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader-functions","title":"Functions","text":""},{"location":"dagpipe/yaml_loader/#dagpipe.yaml_loader.load_pipeline","title":"load_pipeline","text":"load_pipeline(path: str) -> Pipeline\n Load pipeline from YAML file.
Parameters:
Name Type Description Defaultpath str Path to YAML configuration file.
requiredReturns:
Name Type DescriptionPipeline Pipeline Executable pipeline instance.
NotesResponsibilities:
- Loads YAML configuration, builds schema, creates State subclass, loads Node instances, builds Graph topology, and initializes Engine.\n"}]}