Vishesh 'ironeagle' Bangotra
06ff5bce6e
- move templates into src/templates; renumber wiki pages; add app-components page - embed agentic build prompt as a copyable markdown block; add mermaid diagrams - build.py writes <name>/mcp; config static bundle -> standards/mcp - registry reads libraries.json as bytes (committed file was UTF-16LE); normalize to UTF-8 - Dockerfile: drop stale blog copy; per-repo <name>/mcp copies - drop nested standards/mcp/standards duplicate and empty root mcp/
1 line
20 KiB
JSON
1 line
20 KiB
JSON
{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"\ud83e\udde9 Standards \u2014 the Aetos App Blueprint","text":"<p>One reusable architecture for every backend application in the Aetos ecosystem: a spec-first HTTP edge, a pure domain core, a pluggable infrastructure layer, and a single composition root.</p> <p>Fill in two axes only and a new app is done: your resource model, and (if needed) your pipeline family. The blueprint itself never names a business domain \u2014 expenses, health metrics, inventory, and analytics are all the same shape underneath.</p> <p>Doc model: this wiki is written for humans \u2014 how-to guides, the build recipe, and quality gates. The enforceable rules (layering, composition root, CRUD template, pipeline scaffold) and the library registry live in the reference pages, mirrored by the machine-readable <code>standards</code> MCP bundle. Query <code>find_library</code> before writing any infrastructure.</p>"},{"location":"#key-features","title":"\ud83d\ude80 Key Features","text":"<ul> <li>\ud83e\udded Spec-first routing \u2014 the OpenAPI document, not decorator routers, is the source of truth (operationId lookup)</li> <li>\ud83c\udfd7\ufe0f One-way layering \u2014 <code>interfaces \u2192 domain \u2192 infra</code>, enforced</li> <li>\ud83d\udd0c Single composition root \u2014 every dependency is wired in exactly one place</li> <li>\ud83e\udde9 Generic, not app-specific \u2014 reusable rules versus two authoring axes</li> <li>\ud83d\udcda Library-backed \u2014 DB, DI, pipelines, ingestion, auth, and docs come from internal libraries, never hand-written</li> <li>\ud83d\udea6 Explicit state machines \u2014 durable job statuses, progress, and human-in-the-loop pauses</li> </ul>"},{"location":"#documentation-structure","title":"\ud83d\udcc1 Documentation Structure","text":"Section Description Overview Philosophy, the one-way dependency rule, architecture App Components The component model and how the pieces wire together Build Recipe The ordered, agentic checklist for a new app Testing & Quality Test layout and acceptance gates <p>Reference (separate site, mirrors the MCP bundle): the enforceable rules \u2014 Layering, Composition Root, Spec-First Routing, CRUD Template, Pipeline Scaffold, Worker & Progress, Human-in-the-Loop, Infra Adapters, Genericity Matrix, Library Registry.</p>"},{"location":"#related-resources","title":"\ud83d\udd17 Related Resources","text":"<ul> <li>MCP bundle \u2014 <code>standards</code> server on <code>:8009</code>: read <code>doc://index</code>, <code>doc://nav</code>, <code>doc://modules/{module}</code>; call <code>find_library(capability)</code></li> <li>Library registry \u2014 the catalog of internal libraries behind every app (<code>libraries.json</code>, served as <code>registry://libraries</code>)</li> <li>Template \u2014 the runnable skeleton at <code>standards/src/templates/app/</code></li> <li>Copy the repo's <code>AGENTS.md</code>-style build prompt from Build Recipe</li> </ul> <p>\u00a9 Aetoskia Internal \u2014 <code>standards</code></p>"},{"location":"01_overview/","title":"Overview","text":"<p>The blueprint is a hexagonal (ports & adapters) application with a spec-first web edge and a single composition root. It favors explicit wiring over implicit magic and existing internal libraries over hand-written infrastructure. Start here to build the mental model; the enforceable details live in the reference pages.</p>"},{"location":"01_overview/#architecture","title":"\ud83c\udfd7\ufe0f Architecture","text":"<pre><code>flowchart LR\n subgraph EDGE[\"interfaces/ \u2014 HTTP edge\"]\n OAS[\"openapi.yaml<br/>operationId contract\"]\n RT[\"<resource>/routes.py<br/>thin handlers\"]\n end\n\n subgraph CORE[\"domain/ \u2014 pure business core\"]\n MD[\"models\"]\n PT[\"ports (abstract)\"]\n SV[\"services\"]\n end\n\n subgraph INFRA[\"infra/ \u2014 adapters\"]\n RP[\"mongo repos (mongo-ops)\"]\n AD[\"cache \u00b7 stores \u00b7 llm \u00b7 mail\"]\n DB[(\"MongoDB\")]\n end\n\n CR[\"init.py<br/>composition root\"]\n\n OAS --> RT\n RT --> SV\n SV --> PT\n SV --> RP\n PT -. depends on .- RP\n CR --- RT\n CR --- SV\n CR --- RP\n RP --> DB\n AD --- CR</code></pre> <p>The rule holds at every edge: code in an outer ring may depend on a ring closer to the core, never the reverse. <code>domain/</code> never imports a web framework or a database driver.</p> <p>Legend \u2014 solid arrows are compile-time calls; dotted arrows are the port/adapter contract, resolved by the composition root at startup.</p>"},{"location":"01_overview/#the-one-way-dependency-rule","title":"\ud83d\udd04 The one-way dependency rule","text":"Ring May import Never imports <code>interfaces/</code> <code>domain</code>, framework glue (glue only) business logic, <code>infra</code> directly <code>domain/</code> nothing but its own ports web frameworks, drivers, <code>infra</code> <code>infra/</code> <code>domain</code> ports, drivers, libraries \u2014 <code>init.py</code> everything (the wiring point) \u2014"},{"location":"01_overview/#core-principles","title":"\ud83e\udded Core principles","text":"<ol> <li>One-way dependencies. <code>interfaces -> domain -> infra</code> is allowed; the reverse is forbidden.</li> <li>Ports in the domain, adapters in infra. A service depends on an abstract port; the composition root injects a concrete adapter.</li> <li>One composition root. Nothing constructs its own dependencies. All wiring lives in one module (<code>init.py</code>) exposed as a flat resource registry.</li> <li>Spec-first. The HTTP contract is declared in <code>openapi.yaml</code>; route handlers are looked up by <code>operationId</code>. There are no decorator routers.</li> <li>Reuse the libraries. Orchestration, DI, Mongo repositories, ingestion, auth, and docs come from internal libraries \u2014 see the Library Registry.</li> <li>Explicit state machines. Long-running work moves through explicit, durable statuses and can pause for human input.</li> </ol>"},{"location":"01_overview/#runtime-processes","title":"\u2699\ufe0f Runtime processes","text":"<p>A standard app may run as up to three processes that share the same composition root:</p> Process Entry point Responsibility API <code>main.py</code> serve HTTP; start/stop resources via lifespan Worker <code>worker.py</code> poll durable jobs, run pipelines, publish progress Scripts <code>python -m scripts.<name></code> one-shot operator tasks"},{"location":"01_overview/#read-next","title":"\u27a1\ufe0f Read Next","text":"<ul> <li>02 \u2013 App Components \u00b7 03 \u2013 Build Recipe</li> </ul>"},{"location":"02_components/","title":"App Components","text":"<p>What ships in every standard app, and how the pieces relate. This is the blueprint's component model \u2014 <code>templates/app/</code> is a runnable instance of it.</p>"},{"location":"02_components/#component-model","title":"\ud83d\uddc2\ufe0f Component model","text":"<pre><code>flowchart LR\n subgraph EDGE[\"interfaces/ \u2014 HTTP edge\"]\n OAS[\"openapi.yaml\"]\n RU[\"resource routes\"]\n JU[\"job routes\"]\n end\n\n subgraph CORE[\"domain/ \u2014 business core\"]\n RI[\"<resource> \u00b7 models \u00b7 ports \u00b7 services\"]\n JI[\"<job/worker> \u00b7 models \u00b7 ports \u00b7 services\"]\n PI[\"pipelines/<family>\"]\n end\n\n subgraph INFRA[\"infra/ \u2014 adapters\"]\n DB[(\"MongoDB repos\")]\n CA[\"redis cache\"]\n ST[\"stores \u00b7 progress bus\"]\n LL[\"llm clients\"]\n end\n\n W[\"worker.py\"]\n CR[\"init.py \u2014 composition root\"]\n\n OAS --> RU\n OAS --> JU\n RU --> RI\n JU --> JI\n JI --> PI\n PI --> DB\n PI --> LL\n RI --> CA\n RI --> DB\n JI --> ST\n W --> JI\n W --> PI\n\n CR === RU\n CR === JU\n CR === RI\n CR === JI\n CR === PI\n CR === W\n CR === DB\n CR === CA\n CR === ST</code></pre> <p>Legend \u2014 solid arrows are direct calls; thick links are wires created by the composition root. Every component is injected, never self-constructed.</p>"},{"location":"02_components/#component-roles","title":"\ud83d\udce6 Component roles","text":""},{"location":"02_components/#1-interfaces-the-edge","title":"1. <code>interfaces/</code> \u2014 the edge","text":"<p>Thin handlers bound to <code>openapi.yaml</code> <code>operationId</code>s. They parse the request, call a service, and shape the response. No business rules, no I/O.</p> <ul> <li><code>interfaces/openapi.py</code> re-exports every route module so the OpenAPI-first app can resolve handlers.</li> <li>Routes may mount the <code>cache</code> decorator on read endpoints.</li> </ul>"},{"location":"02_components/#2-domain-the-core","title":"2. <code>domain/</code> \u2014 the core","text":"<p>Pure models, ports (abstract contracts), and services (use cases). No framework imports, no drivers.</p> <ul> <li><code><aggregate>/models.py</code> \u2014 dataclasses, enums, document shapes.</li> <li><code><aggregate>/ports/db.py</code> \u2014 repo contract; <code><aggregate>/ports/service.py</code> \u2014 service contract.</li> <li>Services compose ports and enforce the business rules; the composition root injects the implementations.</li> </ul>"},{"location":"02_components/#3-infra-the-adapters","title":"3. <code>infra/</code> \u2014 the adapters","text":"<p>Concrete implementations of the domain ports plus cross-cutting infrastructure:</p> <ul> <li><code>db/</code> \u2014 mongo-ops repositories and the connection lifecycle.</li> <li><code>adapters/</code> \u2014 cache store, progress bus, storages, fetchers (library-backed).</li> <li>Anything else the domain needs, implemented against its ports.</li> </ul>"},{"location":"02_components/#4-pipelines-data-driven-families-optional","title":"4. <code>pipelines/</code> \u2014 data-driven families (optional)","text":"<p>Extraction and enrichment families, each a deterministic step chain (<code>abc.py</code> / <code>impl.py</code> / <code>pipeline.yaml</code> / <code>resources/</code>). Roots are registered in the pattern registry and driven either by a worker job or a CLI.</p>"},{"location":"02_components/#5-workerpy-the-job-worker-optional","title":"5. <code>worker.py</code> \u2014 the job worker (optional)","text":"<p>A separate process that polls durable job collections, runs pipeline roots, and publishes progress to the progress bus. It shares the same composition root as the API.</p>"},{"location":"02_components/#6-initpy-the-composition-root","title":"6. <code>init.py</code> \u2014 the composition root","text":"<p>The only place anything is constructed: leaf repositories \u2192 dependent repositories \u2192 services \u2192 adapters. Exposes <code>get_svc</code> overloads and starts / stops resources on the app lifespan.</p>"},{"location":"02_components/#how-the-request-flows","title":"\ud83d\udd17 How the request flows","text":"<pre><code>sequenceDiagram\n participant C as Client\n participant R as route (interfaces)\n participant S as service (domain)\n participant P as repo (infra)\n participant DB as MongoDB\n\n C->>R: POST /items (operationId)\n R->>S: svc.create(payload)\n S->>P: repo.insert(doc)\n P->>DB: insert\n DB-->>P: ack\n P-->>S: doc\n S-->>R: result\n R-->>C: 200 JSON</code></pre>"},{"location":"02_components/#related","title":"Related","text":"<ul> <li>01 \u2013 Overview \u00b7 03 \u2013 Build Recipe \u00b7 04 \u2013 Testing & Quality</li> </ul>"},{"location":"03_build_recipe/","title":"Build Recipe","text":"<p>The ordered, deterministic checklist for generating a new application of the standard shape. Every step is fixed; only steps 2 and 3 need app-specific authoring. Give an AI agent the copyable prompt below (or read it from the <code>standards</code> MCP bundle as <code>doc://modules/build_recipe</code>) and it can produce a complete app in one pass.</p>"},{"location":"03_build_recipe/#quick-start","title":"\ud83d\ude80 Quick start","text":"<pre><code># Copy the runnable skeleton (from the standards docs checkout)\ncp -r <docs>/standards/src/templates/app apps/<your-app>\ncd apps/<your-app>\n</code></pre> <p>Then follow the numbered steps in the prompt.</p>"},{"location":"03_build_recipe/#agent-prompt-copyable","title":"\ud83d\udccb Agent prompt (copyable)","text":"<pre><code># Build a new application from the Aetos Standards blueprint\n\nYou are coding a complete new backend application conforming to the Aetos\nstandard app blueprint. Execute every step in order. Only steps 2 and 3 use the\napp-specific details provided by the user; everything else is deterministic.\n\nBefore writing any code, connect to the `standards` MCP server and read:\n- `doc://nav` (the module list)\n- `doc://modules/layering_rules` (the dependency rules)\n- `doc://modules/genericity_matrix` (what is generic vs app-specific)\nThen call `find_library` with capabilities like `db`, `di`, `routing`,\n`ingestion`, `auth`, `docs`, `pipeline`, and note the libraries that already\nexist. Never reimplement one.\n\n1. **Scaffold** \u2014 copy `src/templates/app/` into `apps/<your-app>/`. Keep the\n package tree: `domain/`, `infra/`, `interfaces/`, `pipelines/`, `tests/`,\n `scripts/`, plus `pyproject.toml`, `openapi.yaml`, `config.py`, `init.py`,\n `main.py`, `worker.py`, `Dockerfile`, `docker-compose.yaml`, `.env.example`.\n\n2. **Author the spec (app-specific)** \u2014 write `openapi.yaml`: resources,\n schemas, `operationId`s, security, and extension metadata. This is the\n contract and drives all routing.\n\n3. **Write each CRUD slice (app-specific)** \u2014 per resource create the\n four-file slice: `models.py`, `ports/db.py`, `ports/service.py`, and\n optional `converters.py`. Follow the CRUD template.\n\n4. **Wire the composition root** \u2014 in `init.py`, build leaf repositories, then\n dependent repositories, then services; add infra adapters (db, cache,\n stores, progress bus, fetchers). Expose `get_svc` overloads.\n\n5. **Add routes** \u2014 create `interfaces/<resource>/routes.py` thin handlers\n bound to `operationId`s and re-export them in `interfaces/openapi.py`. Add\n the cache decorator to read endpoints.\n\n6. **Bootstrap app and worker** \u2014 confirm `main.py` runs the OpenAPI-first app\n with its lifespan. If the app has long-running jobs, adapt `worker.py` to\n poll the job collection, run the pipeline, and publish progress.\n\n7. **Add pipeline families (only if data-driven)** \u2014 create\n `pipelines/<family>/` with `abc.py`, `impl.py`, `pipeline.yaml`,\n `resources/`, and register the root. Add fixtures and a parity test.\n\n8. **Add infrastructure only where needed** \u2014 prefer a library-backed adapter.\n Never hand-write what an internal library already provides.\n\n9. **Tests** \u2014 integration tests with fakes at the HTTP edge; unit tests for\n domain and pipeline logic; a config-consistency test for rule-driven\n pipelines.\n\n10. **Scripts and seeds** \u2014 operator scripts under `scripts/` using\n `init()` / `get_svc()` / `shutdown()`.\n\n11. **Docs** \u2014 author GSDFC docstrings, generate `.pyi` stubs, mark the package\n `py.typed`, and build the docs.\n\n**Definition of done:**\n- `python -c \"import main\"` succeeds.\n- `pytest` passes.\n- The app serves every operation in the spec; the worker drains its queue.\n- No domain module imports a web framework.\n- No infrastructure was reimplemented that a library already provides.\n</code></pre>"},{"location":"03_build_recipe/#where-the-steps-land","title":"\ud83e\udded Where the steps land","text":"<p>The steps map onto the layers as follows:</p> Step Package / file Depends on 1 \u00b7 Scaffold whole <code>templates/app/</code> \u2014 2 \u00b7 Spec <code>openapi.yaml</code> \u2014 3 \u00b7 CRUD slices <code>domain/<resource>/</code> spec, CRUD template 4 \u00b7 Wiring <code>init.py</code> every slice 5 \u00b7 Routes <code>interfaces/<resource>/</code> services, openapi route registry 6 \u00b7 Bootstrap <code>main.py</code>, <code>worker.py</code> composition root 7 \u00b7 Pipelines <code>pipelines/<family>/</code> library pipeline engine 8 \u00b7 Infra <code>infra/</code> domain ports 9 \u00b7 Tests <code>tests/</code> app + fixtures 10 \u00b7 Scripts <code>scripts/</code> composition root 11 \u00b7 Docs docstrings + docs build code, Doc Forge"},{"location":"03_build_recipe/#definition-of-done","title":"\u26a0\ufe0f Definition of done","text":"<ul> <li><code>python -c \"import main\"</code> succeeds.</li> <li><code>pytest</code> passes.</li> <li>The app serves the spec's operations; the worker drains its queue.</li> <li>No domain module imports a web framework.</li> <li>No infrastructure was reimplemented that a library already provides.</li> </ul>"},{"location":"03_build_recipe/#related","title":"Related","text":"<ul> <li>01 \u2013 Overview \u00b7 02 \u2013 App Components \u00b7 04 \u2013 Testing & Quality</li> </ul>"},{"location":"04_development/01_testing/","title":"Testing & Quality Gates","text":"<p>Two suites run under pytest with asyncio auto mode: integration (the HTTP edge against fakes) and unit (pure domain and pipeline logic). Finish a blueprint app only when every gate below passes.</p>"},{"location":"04_development/01_testing/#layout","title":"\ud83d\uddc2\ufe0f Layout","text":"<pre><code>tests/\n integration/ # TestClient(app) + mocked resources\n unit/ # models, converters, pipeline stages/roots\n conftest.py # fixtures\n</code></pre>"},{"location":"04_development/01_testing/#integration-tests","title":"\ud83d\ude80 Integration tests","text":"<ul> <li>Build the app with <code>TestClient</code>.</li> <li>Replace <code>app.state.resources</code> with fake repositories/services so no database is required.</li> <li>Patch the auth introspection call (<code>httpx.AsyncClient.post</code>) to return a canned identity.</li> <li>Assert status codes and bodies for list/get/create/update/delete, plus <code>400</code>, <code>404</code>, <code>204</code>, and the streaming endpoint's fallback.</li> </ul> <pre><code>client = TestClient(app)\napp.state.resources = {\"item_svc\": FakeItemService(), ...}\n\ndef test_create_item():\n r = client.post(\"/items\", json={\"name\": \"x\"})\n assert r.status_code == 200\n</code></pre>"},{"location":"04_development/01_testing/#unit-tests","title":"\ud83e\uddea Unit tests","text":"<ul> <li>Domain models and converters: pure, no I/O.</li> <li>Pipeline stages: feed a fixture state, assert the transformed state.</li> <li>Assembled pipeline roots: fixture parity \u2014 replay a stored fixture input and compare against a golden output file.</li> </ul>"},{"location":"04_development/01_testing/#config-consistency-tests","title":"\ud83d\udd10 Config-consistency tests","text":"<p>For rule-driven pipelines, add a test that cross-references rules against the declared pattern registry: every flag emitted must be defined; every defined flag must be reachable. This catches drift without running the pipeline.</p>"},{"location":"04_development/01_testing/#quality-gates","title":"\u2705 Quality gates","text":"<p>A definition of done for any standard app:</p> <ul> <li><code>python -c \"import main\"</code> succeeds.</li> <li><code>pytest</code> passes.</li> <li>The app serves the spec's operations; the worker drains its queue.</li> <li>No domain module imports a web framework.</li> <li>No infrastructure was reimplemented that a library already provides.</li> </ul>"},{"location":"04_development/01_testing/#conventions","title":"\ud83d\udca1 Conventions","text":"<ul> <li><code>asyncio_mode = \"auto\"</code>; no manual event loops.</li> <li>Fixtures are checked into the repo (inputs + golden outputs).</li> <li>Tests skip (not fail) when a required fixture is absent.</li> <li>Do not mock the domain; mock only the infrastructure edge.</li> </ul>"},{"location":"04_development/01_testing/#related","title":"Related","text":"<ul> <li>02 \u2013 App Components \u00b7 03 \u2013 Build Recipe \u00b7 Overview</li> </ul>"}]} |