{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Standards Reference","text":"

The reference half of the Standards description: the enforceable rules, patterns, and the internal library catalog that agents consult while coding.

Read the Wiki for the philosophy and rationale; these pages are the reference that mirrors the standards MCP bundle.

"},{"location":"02_layering_rules/","title":"Layering Rules","text":""},{"location":"02_layering_rules/#summary","title":"Summary","text":"

The dependency direction is fixed. Violating it is a design defect.

"},{"location":"02_layering_rules/#allowed-imports","title":"Allowed imports","text":"
interfaces/*      ->  domain/*            (services, ports, models)\ninterfaces/*      ->  infra/adapters/*    (only read-only value objects)\ndomain/*          ->  domain/*            (same layer)\ndomain/*          ->  pipelines/*         (call a pipeline via a port)\ninfra/adapters/*  ->  domain/ports/*      (implement the interface)\nmain.py/init.py   ->  everything          (composition root only)\n
"},{"location":"02_layering_rules/#forbidden-imports","title":"Forbidden imports","text":"
domain/*          ->  interfaces/*        (never)\ndomain/*          ->  fastapi/starlette   (never)\ninfra/adapters/*  ->  interfaces/*        (never)\ninterfaces/*      ->  infra/db/*          (state lives in resources, not imports)\n
"},{"location":"02_layering_rules/#why","title":"Why","text":""},{"location":"02_layering_rules/#the-portadapter-contract","title":"The port/adapter contract","text":"

A port is an abstract class or protocol declared in domain/<x>/ports/. An adapter implements it in infra/adapters/. The composition root binds the two.

domain/<resource>/ports/db.py       -> repository port + document model\ndomain/<resource>/ports/service.py  -> service + factory\ninfra/db/repos/<vendor>.py          -> repository adapter factory\ninfra/adapters/service.py           -> generic CRUD service base\n
"},{"location":"02_layering_rules/#enforcement","title":"Enforcement","text":""},{"location":"03_composition_root/","title":"Composition Root","text":""},{"location":"03_composition_root/#summary","title":"Summary","text":"

All wiring happens in exactly one module. It builds a flat resource registry and exposes it to the HTTP layer and to scripts.

"},{"location":"03_composition_root/#the-resource-registry","title":"The resource registry","text":"

init.py returns a plain dict[str, Any]. Keys follow two conventions:

Plus named infrastructure adaptors:

The API stores the dict on app.state.resources during lifespan startup. Handlers read from it; nothing imports concrete adapters.

"},{"location":"03_composition_root/#shape","title":"Shape","text":"
_resources: dict = {}\n_initialized = False\n\nasync def init(settings=None) -> dict:\n    global _initialized\n    if _initialized:\n        return _resources\n\n    await setup_database(uri=settings.mongo_uri, db_name=settings.mongo_db)\n\n    # 1. leaf repositories\n    user_repo = create_user_repo()\n\n    # 2. dependent repositories (dependencies passed by keyword)\n    wallet_repo = create_wallet_repo(user_repo=user_repo)\n\n    # 3. services (build via the DI library, injecting the repo)\n    _resources[\"user_repo\"]   = user_repo\n    _resources[\"wallet_repo\"] = wallet_repo\n    _resources[\"user_svc\"]    = get_user_service(user_repo)\n    _resources[\"wallet_svc\"]  = get_wallet_service(wallet_repo)\n\n    # 4. infrastructure adaptors (optional)\n    _resources[\"http_cache_store\"] = HttpCacheStore(...)\n    _resources[\"progress_bus\"]     = ProgressEventBus()\n\n    _initialized = True\n    return _resources\n\nasync def shutdown() -> None:\n    global _initialized\n    await close_stores(_resources)\n    await shutdown_database()\n    _resources.clear()\n    _initialized = False\n
"},{"location":"03_composition_root/#typed-accessors","title":"Typed accessors","text":"

Expose a get_svc(name) helper with one @overload per key so scripts get typing without importing concrete classes:

@overload\ndef get_svc(name: Literal[\"user_svc\"]) -> UserService: ...\n@overload\ndef get_svc(name: Literal[\"wallet_svc\"]) -> WalletService: ...\n\ndef get_svc(name: str):\n    return _resources[name]\n

Also expose get_db() (raw database handle) and pure helpers such as compute_fingerprint(payload) (canonical hash used for deduplication).

"},{"location":"03_composition_root/#rules","title":"Rules","text":""},{"location":"04_spec_first_routing/","title":"Spec-First Routing","text":""},{"location":"04_spec_first_routing/#summary","title":"Summary","text":"

The OpenAPI document is the single source of truth for the HTTP contract. Handlers are bound to it by name; there is no decorator router.

"},{"location":"04_spec_first_routing/#the-contract","title":"The contract","text":""},{"location":"04_spec_first_routing/#app-bootstrap","title":"App bootstrap","text":"
# main.py\nfrom openapi_first.app import OpenAPIFirstApp\n\napp = OpenAPIFirstApp(\n    openapi_path=\"openapi.yaml\",\n    routes_module=interfaces.openapi,   # module whose namespace holds handlers\n    title=\"<App> API\",\n    version=\"0.1.0\",\n    lifespan=lifespan,\n)\n

lifespan calls the composition root:

@asynccontextmanager\nasync def lifespan(app):\n    app.state.resources = await init()\n    yield\n    await shutdown()\n
"},{"location":"04_spec_first_routing/#handler-registry","title":"Handler registry","text":"

interfaces/openapi.py is a flat re-export of every handler, so the framework can resolve operationId across the whole app:

from interfaces.item.routes import *\nfrom interfaces.wallet.routes import *\nfrom interfaces.progress.sse import *\n
"},{"location":"04_spec_first_routing/#handler-shape","title":"Handler shape","text":"

Handlers are thin and synchronous with respect to business logic:

async def create_item(request: Request, body: dict):\n    svc = request.app.state.resources[\"item_svc\"]\n    return await svc.create(Item.model_validate(body))\n
"},{"location":"04_spec_first_routing/#conventions","title":"Conventions","text":""},{"location":"04_spec_first_routing/#codegen","title":"Codegen","text":"

The openapi_first library can scaffold models and route stubs from the spec, and ships app templates (CRUD app, model app, service). Prefer codegen over hand-writing boilerplate.

"},{"location":"05_crud_template/","title":"CRUD Template","text":""},{"location":"05_crud_template/#summary","title":"Summary","text":"

Every resource follows the same four-file slice. Repetition is intentional: it makes the codebase predictable and generatable.

"},{"location":"05_crud_template/#the-slice","title":"The slice","text":"
domain/<resource>/\n    models.py               # pure Pydantic model (no DB fields)\n    ports/\n        db.py               # document model + repository factory\n        service.py          # service class + factory\n        converters.py       # (optional) transforms\n
"},{"location":"05_crud_template/#modelspy","title":"models.py","text":"

A framework-free Pydantic model describing the business shape.

class Item(BaseModel):\n    name: str\n    quantity: int = 0\n
"},{"location":"05_crud_template/#portsdbpy","title":"ports/db.py","text":"

The persistence model extends the business model and adds database concerns. The repository factory selects the right repository flavour.

class ItemDocument(Item, BaseDocument):   # BaseDocument adds id/created_at/...\n    pass\n\ndef create_item_repo(*, owner_repo=None) -> ItemRepository:\n    return repo_factory(\n        collection=\"items\",\n        document=ItemDocument,\n        populate_rules=[PopulateRule(field=\"owner\", collection=\"users\")],\n        cache=None,               # or a CachedBaseRepository config\n    )\n
"},{"location":"05_crud_template/#portsservicepy","title":"ports/service.py","text":"

The service is built through the DI library, injecting the repository. The generic CRUD base provides create/read/update/patch/delete/search/....

class ItemService(CrudsServiceAdaptor[Item, ItemDocument]):\n    model = Item\n    schema = ItemDocument\n\ndef get_item_service(repo) -> ItemService:\n    return build(ItemService, instances={\"repo\": repo})\n
"},{"location":"05_crud_template/#naming-conventions","title":"Naming conventions","text":"Concept Convention collection plural snake case (items) document <Resource>Document repository factory create_<resource>_repo service <Resource>Service service factory get_<resource>_service resource key <resource>_svc operationId list_items, create_item, ..."},{"location":"05_crud_template/#optional-files","title":"Optional files","text":""},{"location":"06_pipeline_scaffold/","title":"Pipeline Scaffold","text":""},{"location":"06_pipeline_scaffold/#summary","title":"Summary","text":"

Data-driven processing is expressed as a composition graph. The graph is built by a DI library; execution is driven by a DAG engine. Stages are pure and individually testable.

"},{"location":"06_pipeline_scaffold/#when-to-use-a-pipeline","title":"When to use a pipeline","text":"

Use a pipeline when a resource is produced by a multi-step transformation of an input (a document, a file, a message) rather than by a simple CRUD write.

"},{"location":"06_pipeline_scaffold/#structure-of-a-family","title":"Structure of a family","text":"

A family is one input shape (e.g. a file format or a message kind). Each family has the same skeleton:

pipelines/<family>/\n    abc.py            # port contracts (stage interfaces + root interface)\n    impl.py           # concrete stages + root implementation\n    pipeline.yaml     # graph spec: nodes, edges, roots, schema\n    resources/\n        pipeline.yaml # runtime spec consumed by the loader\n        patterns/*.yaml  # rule fragments (pattern registry)\n    banks/<variant>.py   # optional variants that re-pin a few slots\n
"},{"location":"06_pipeline_scaffold/#shared-scaffolding","title":"Shared scaffolding","text":"
pipelines/shared/\n    ports.py     # RegistryPort, FinalizePort, etc.\n    loader.py    # build(root_cls) -> wires the graph + engine\n    registry.py  # resolve_root(...) -> root class for a (kind, variant)\n
"},{"location":"06_pipeline_scaffold/#the-loader","title":"The loader","text":"
def build(root_cls, *, instances=None, config=None):\n    # merge resources/patterns/*.yaml into a pattern registry\n    # bind registry/config into each graph node by id\n    # build the schema (state) class and the engine\n    root = root_cls()\n    root._engine = Engine(graph, state_cls, payload_cls)\n    return root\n
"},{"location":"06_pipeline_scaffold/#the-registry","title":"The registry","text":"

A single lookup replaces hand-rolled if/else selection:

_FAMILIES = {\n    \"csv\":  {\"acme\": {\"heuristic\": AcmeCsvPipeline}},\n    \"json\": {\"globex\": {\"heuristic\": GlobexJsonPipeline}},\n}\n\ndef resolve_root(kind, variant, mode=\"heuristic\"):\n    return _FAMILIES[kind][variant][mode]\n

Adding a new variant means adding one class and one registry entry.

"},{"location":"06_pipeline_scaffold/#driving-a-pipeline","title":"Driving a pipeline","text":"

Stages are driven explicitly so intermediate state can be persisted:

pipeline = build(resolve_root(kind, variant))\ncontent = pipeline.ingest.materialize(raw_bytes)\nrecords = pipeline.reader.read(content)\nitems   = pipeline.normalize.build(records)\n
"},{"location":"06_pipeline_scaffold/#testing","title":"Testing","text":"

Because stages are pure, test each stage in isolation and test the assembled root against fixture inputs whose expected outputs are stored as golden files. A parity test replays fixtures headlessly.

"},{"location":"07_worker_and_progress/","title":"Worker & Progress","text":""},{"location":"07_worker_and_progress/#summary","title":"Summary","text":"

Long-running work runs in a background worker. Jobs are durable documents that move through an explicit status machine. Progress is streamed to clients.

"},{"location":"07_worker_and_progress/#the-status-machine","title":"The status machine","text":"
pending -> processing -> completed\n                     \\-> failed    (retried up to N times)\n                     \\-> paused    (waiting for human input)\n
"},{"location":"07_worker_and_progress/#the-poll-loop","title":"The poll loop","text":"
async def poll(resources):\n    backoff = MIN_POLL\n    while True:\n        jobs = await job_svc.search(filter={\"status\": \"pending\"}, limit=100)\n        backoff = MIN_POLL if jobs else min(backoff * 2, MAX_POLL)\n        for job in jobs:\n            if (job.retry_count or 0) >= MAX_RETRIES:\n                continue\n            await job_svc.update_status(job.id, \"processing\")\n            try:\n                await process(job, resources)\n            except PauseForResolution:\n                await job_svc.update_status(job.id, \"paused\")\n            except Exception as e:\n                await job_svc.update_status(\n                    job.id, \"failed\", error=str(e),\n                    retry_count=(job.retry_count or 0) + 1,\n                )\n        await asyncio.sleep(backoff)\n

Tunables live in environment variables: WORKER_POLL_MIN, WORKER_POLL_MAX, WORKER_MAX_RETRIES.

"},{"location":"07_worker_and_progress/#progress-events","title":"Progress events","text":"

A worker publishes structured progress events to a bus (a port). The HTTP edge exposes a streaming endpoint that replays events for a given job id.

class ProgressEvent(BaseModel):\n    job_id: str\n    step: str\n    status: str            # started | progress | completed | skipped | paused\n    message: dict | None = None\n

The bus is an in-memory adapter by default; because it is a port, it can be swapped for a durable broker. Emit from the orchestrator at each stage:

await self.bus.publish(ProgressEvent(job_id=jid, step=\"normalize\",\n                                     status=\"completed\", message={\"count\": n}))\n
"},{"location":"07_worker_and_progress/#streaming-endpoint","title":"Streaming endpoint","text":"
async def stream_job_events(request, id):\n    bus = request.app.state.resources.get(\"progress_bus\")\n    if bus is None:\n        return {\"events\": []}\n\n    async def gen():\n        sent = 0\n        while True:\n            events = bus.get_events(id)\n            for e in events[sent:]:\n                yield f\"data: {e.model_dump_json(exclude_none=True)}\\n\\n\"\n                sent += 1\n            await asyncio.sleep(0.5)\n\n    return StreamingResponse(gen(), media_type=\"text/event-stream\")\n

Known limitation. The default in-memory bus is per-process. If the worker and the API run as separate processes, events published by the worker are not visible to the API. For production, back the bus with a shared broker. The port boundary makes this a one-adapter change.

"},{"location":"08_human_in_loop/","title":"Human-in-the-Loop","text":""},{"location":"08_human_in_loop/#summary","title":"Summary","text":"

Some inputs are ambiguous and must be resolved by a human before processing can continue. The blueprint models this as a durable pause: an exception is raised, a question is recorded, the job pauses, and later resumes.

"},{"location":"08_human_in_loop/#the-mechanism","title":"The mechanism","text":"
  1. During processing, a stage raises a domain exception carrying the context needed to ask the question:
raise PauseForResolution(\n    job_id=jid,\n    reference=\"<stable key for this decision>\",\n    options=[...],\n    context={...},\n)\n
  1. The orchestrator catches it, records a question document, and re-raises so the worker marks the job paused.
  2. A client lists pending questions and answers one:
GET  /jobs/{id}/questions            -> list pending questions\nPOST /questions/{qid}/resolve  {choice}  -> record the choice\n
  1. When the last pending question is resolved, the resolve endpoint flips the parent job back to pending.
  2. The worker picks it up again, rebuilds a resolution map from previously answered questions, and seeds it into the pipeline so the same decision is not asked twice.
"},{"location":"08_human_in_loop/#the-question-document","title":"The question document","text":"
class Question(BaseDocument):\n    job_id: str\n    reference: str            # stable key: stage|field|value\n    status: str               # pending | resolved\n    options: list[dict]\n    context: dict\n    chosen: dict | None = None\n    resolved_at: str | None = None\n
"},{"location":"08_human_in_loop/#resume-map","title":"Resume map","text":"

The worker turns answered questions into a lookup keyed by reference, then injects it into the pipeline so that stage can skip re-raising:

resolved = {q.reference: q.chosen for q in await svc.search(\n    filter={\"job_id\": jid, \"status\": \"resolved\"})}\n\npipeline = build(root, instances={\"pause\": handler}, config={\"resolved\": resolved})\n
"},{"location":"08_human_in_loop/#resolve-endpoint","title":"Resolve endpoint","text":"
async def resolve_question(request, id, body):\n    svc = request.app.state.resources[\"question_svc\"]\n    q = await svc.read(id)\n    await svc.update(id, {\"chosen\": body[\"chosen\"], \"status\": \"resolved\",\n                          \"resolved_at\": now()})\n\n    pending = await svc.search(filter={\"job_id\": q.job_id, \"status\": \"pending\"},\n                               limit=1)\n    if not pending:\n        jobs = request.app.state.resources[\"job_svc\"]\n        job = await jobs.read(q.job_id)\n        if job and job.status == \"paused\":\n            await jobs.update_status(q.job_id, \"pending\")\n    return q\n
"},{"location":"08_human_in_loop/#cli-parity","title":"CLI parity","text":"

The same flow is available through scripts/CLI: list, get, create, questions, resolve, retry.

"},{"location":"09_infra_adaptors/","title":"Infra Adapters","text":""},{"location":"09_infra_adaptors/#summary","title":"Summary","text":"

All external systems are reached through adapters in infra/. Prefer an existing internal library over writing a new adapter.

"},{"location":"09_infra_adaptors/#database-mongodb","title":"Database (MongoDB)","text":" 
repo_factory(collection, document, engine=None,\n             populate_rules=None, cache=None)\n
PopulateRule(field=\"owner\", collection=\"users\",\n             nested_rules=[], max_depth=2, projection=None)\n

Library: the Mongo operations library (see registry). Do not write raw Motor calls in the domain.

"},{"location":"09_infra_adaptors/#http-response-cache","title":"HTTP response cache","text":"

A generic GET cache sits behind a decorator.

@cache_response(ttl=60)\nasync def list_items(request): ...\n
"},{"location":"09_infra_adaptors/#event-progress-bus","title":"Event / progress bus","text":"

A port with an in-memory default adapter (publish, get_events). Swap for a durable broker in production (see Worker & Progress).

"},{"location":"09_infra_adaptors/#external-clients-ingestion","title":"External clients (ingestion)","text":"

External fetchers (files, mail, HTTP) are adapters that wrap ingestion libraries. Construct them in the composition root and inject into orchestrators.

Libraries: the ingestion library (formats) and the mail library (email).

"},{"location":"09_infra_adaptors/#llm-clients","title":"LLM clients","text":"

When a pipeline calls a model provider, route through a pool adaptor that handles provider selection, retries, and client lifecycle:

pool = setup_async_pool(provider=settings.llm_provider)\nresult = await pool.complete(model=\"...\", messages=[...])\n

Providers are selected by LLM_PROVIDER and configured by environment.

"},{"location":"09_infra_adaptors/#key-value-store","title":"Key-value store","text":"

For report/snapshot storage, provide a pluggable store port with a Redis adapter and an in-memory fallback.

"},{"location":"09_infra_adaptors/#rule","title":"Rule","text":"

If a capability already exists as a library (DB, DI, ingestion, mail, auth, docs), use it. Writing a private copy of a library is a defect.

"},{"location":"12_genericity_matrix/","title":"Genericity Matrix","text":""},{"location":"12_genericity_matrix/#summary","title":"Summary","text":"

This page states precisely what is reusable as-is, what is a template to adapt, and what is app-specific and must be authored. Use it to scope a new build.

"},{"location":"12_genericity_matrix/#reusable-as-is-do-not-rewrite","title":"Reusable as-is (do not rewrite)","text":"Area Artifact Layering & dependency rule the package layout and import rules Composition root the init.py shape, idempotency, get_svc overloading Web edge OpenAPIFirstApp bootstrap, handler-name = operationId, thin handlers CRUD slice the four-file pattern (model / ports-db / ports-service / converters) Generic CRUD service the base service adaptor Repository factory base / populating / cached selection Worker the poll loop, backoff, retry, status machine Progress bus the bus port + in-memory adapter + streaming endpoint Human-in-the-loop the durable pause + question document + resolve endpoint + resume map Infra adaptors HTTP cache decorator, KV store port, LLM pool scaffold Testing integration harness with fakes, fixture-parity pattern Docs GSDFC + generated stubs + MCP bundle pipeline"},{"location":"12_genericity_matrix/#template-to-adapt-copy-then-change-names","title":"Template to adapt (copy, then change names)","text":"Artifact Change pyproject.toml project name, dependencies Dockerfile / docker-compose.yaml service name, ports, services .env.example connection strings, provider keys openapi.yaml every path/schema/operationId scripts/ seed and operator tasks"},{"location":"12_genericity_matrix/#app-specific-must-be-authored","title":"App-specific (must be authored)","text":"Artifact Why Resource models the business shape Repository populate/cache rules the relationships Service business logic domain behaviour Pipeline families (if any) the transformation graph Pattern/rule resources data-driven policy Fixtures & golden outputs test data"},{"location":"12_genericity_matrix/#the-two-axes","title":"The two axes","text":"

Everything app-specific reduces to two axes:

  1. Resource model \u2014 the set of resources, their fields, relationships, and the OpenAPI contract.
  2. Pipeline family \u2014 only if inputs require multi-step transformation.

If you can specify those two axes, the rest of the app follows mechanically.

"},{"location":"13_library_registry/","title":"Library Registry","text":""},{"location":"13_library_registry/#summary","title":"Summary","text":"

The Aetos ecosystem provides internal libraries for the shared concerns of every application. Use them. Writing a private copy of any capability below is a defect.

Each library also exposes its own documentation as an MCP server. An AI agent can read a library's full API by connecting to its MCP and reading doc://index, doc://nav, and doc://modules/{module}.

"},{"location":"13_library_registry/#catalog","title":"Catalog","text":"Library Import Purpose Use instead of MCP DAG Pipe dagpipe Deterministic state-transformation DAG engine hand-rolled orchestration/topological ordering :8006 Hexa hexa Declarative DI / composition-root tooling manual wiring, hand-written factories (see note) Mongo Ops mongo_ops Async MongoDB repositories, cache, populate, transactions raw Motor CRUD and connection lifecycle :8007 Omniread omniread Format-agnostic fetch + parse (HTML, PDF, CSV, XLSX) custom scrapers/parsers :8003 OpenAPI First openapi_first Spec-first FastAPI bootstrap, client, codegen decorator routers and hand-written clients :8004 Mail Intake mail_intake Provider-agnostic read-only email ingestion Gmail OAuth + MIME parsing :8002 Py JWT jwtlib JWT auth primitives (register/login/introspect) password hashing + token code :8005 Doc Forge docforge Docstring compiler \u2192 MkDocs + MCP docs custom doc generators :8001 Auth Server (service) Central identity service (register/login/me/introspect) building auth endpoints yourself \u2014"},{"location":"13_library_registry/#how-to-choose","title":"How to choose","text":"You need to\u2026 Use persist and query documents Mongo Ops compose dependencies Hexa run a multi-step transformation DAG Pipe read/parse an arbitrary input format Omniread expose a contract-first HTTP API OpenAPI First ingest email Mail Intake authenticate users / verify tokens Py JWT (+ Auth Server) generate documentation Doc Forge"},{"location":"13_library_registry/#discovery","title":"Discovery","text":"

A single standards MCP exposes find_library(capability), which returns the matching library, its import name, its \"use instead of\" guidance, and its MCP endpoint. Query it before writing any infrastructure.

"},{"location":"13_library_registry/#notes","title":"Notes","text":""}]}