Vishesh 'ironeagle' Bangotra
5a1204fbcb
All checks were successful
continuous-integration/drone/push Build is passing
- drone npm template gains backfill-npm-from-tags (mirrors PyPI) - document the shared /auth/htpasswd credential model (npm + PyPI + docker) and Verdaccio's bcrypt requirement - point doc-weaver source/path at ../react/doc-weaver
1 line
86 KiB
JSON
1 line
86 KiB
JSON
{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Standards Reference","text":"<p>The reference half of the Standards description: the enforceable rules, patterns, and the internal library catalog that agents consult while coding.</p> <p>The reference is organized as an anatomy \u2014 one entry checklist and four pillars describing what an application is made of, plus delivery/meta pages and an optional pipelines layer.</p>"},{"location":"#anatomy","title":"Anatomy","text":"<ul> <li>Build Recipe \u2014 the ordered, agentic checklist for a new app; the entry point that walks the whole anatomy.</li> </ul>"},{"location":"#pillar-1-domains","title":"Pillar 1 \u2014 Domains","text":"<ul> <li>Layering Rules \u2014 the strict request \u2192 domain \u2192 repository layering and the rules that enforce it.</li> </ul>"},{"location":"#pillar-2-openapi-spec-the-contract","title":"Pillar 2 \u2014 OpenAPI Spec (the contract)","text":"<ul> <li>Composition Root \u2014 a single composition point wiring the whole app.</li> <li>Spec-First Routing \u2014 the OpenAPI contract as the routing source of truth.</li> <li>CRUD Template \u2014 the standardized generic resource endpoint scaffold.</li> <li>Auth Integration \u2014 consuming the shared auth-server service (spec-driven, no auth code).</li> </ul>"},{"location":"#pillar-3-interfaces-infra","title":"Pillar 3 \u2014 Interfaces & Infra","text":"<ul> <li>Infra Adapters \u2014 how infrastructure connects without leaking into domain.</li> </ul>"},{"location":"#pillar-4-ui","title":"Pillar 4 \u2014 UI","text":"<ul> <li>UI: Auto-Rendered Admin & Custom Pages \u2014 the <code>@aetoskia/react-openapi</code> admin rendered from the spec's x- extensions, and <code>@aetoskia/react-auth</code> for auth pages; the client template scaffold.</li> </ul>"},{"location":"#pipelines-optional-layer","title":"Pipelines (optional layer)","text":"<ul> <li>Pipeline Scaffold \u2014 the deterministic pipeline skeleton for multi-stage jobs.</li> <li>Worker & Progress \u2014 queued work, progress reporting, and visibility.</li> <li>Human-in-the-Loop \u2014 guardrails and approvals inside automations.</li> </ul>"},{"location":"#delivery-meta","title":"Delivery & Meta","text":"<ul> <li>Drone Pipelines \u2014 the three CI shapes: a PyPI package, an npm package, or a Docker deployment.</li> <li>Versioning & Changelog \u2014 SemVer, the tag\u2194version contract, and the release changelog.</li> <li>Repository Inventory \u2014 every covered repo, its type, version source, and Drone shape.</li> <li>Genericity Matrix \u2014 what is library-backed, template-provided, or authored.</li> <li>Library Registry \u2014 the catalog of internal libraries behind all of it.</li> </ul> <p>Read the Wiki for the philosophy and rationale; these pages are the reference that mirrors the <code>standards</code> MCP bundle.</p>"},{"location":"01_build_recipe/","title":"Build Recipe","text":""},{"location":"01_build_recipe/#summary","title":"Summary","text":"<p>The ordered, deterministic checklist for generating a new application of the standard shape. Every step is fixed; only the resource model (step 2), the codegen pass (step 3), and the pipeline family (step 8) need app-specific work.</p> <p>The reference pages behind this recipe are organized as an anatomy: the domains layer, the OpenAPI contract that drives routing and the admin UI, the interfaces/infra seam, the auto-rendered UI client, and an optional pipelines layer \u2014 see the Standards Reference index.</p> <p>This page is served by the <code>standards</code> MCP as <code>docs://modules/build_recipe</code>. The <code>start_app</code> prompt on that server collects the two axes and hands off here; pull exact files with <code>get_app_component</code>.</p>"},{"location":"01_build_recipe/#bootstrap","title":"\ud83d\ude80 Bootstrap","text":"<p>From the standards docs checkout:</p> <pre><code>cp -r <docs>/standards/src/templates/app apps/<your-app>\ncd apps/<your-app>\n</code></pre> <p>Pair it with the React client template when a UI is needed (auto-rendered admin from the spec \u2014 see 40 \u2013 UI):</p> <pre><code>cp -r <docs>/standards/src/templates/client apps/<your-app>-client\ncd apps/<your-app>-client && npm install\n</code></pre> <p>Install the internal libraries \u2014 editable local checkouts (preferred in development) or the private index (CI / fresh machines). Credentials are environment-only; never commit them.</p> <pre><code># local checkouts\npip install -e ../aetos/openapi-first -e ../aetos/mongo-ops -e ../aetos/hexa \\\n -e ../aetos/dagpipe -e ../aetos/doc-forge\n# private index\npip install --extra-index-url \\\n \"https://$PIP_REGISTRY_USERNAME:$PIP_REGISTRY_PASSWORD@pip.aetoskia.com/simple\" .\n</code></pre>"},{"location":"01_build_recipe/#agent-prompt-copyable","title":"\ud83d\udccb Agent prompt (copyable)","text":"<pre><code># Build a new application from the Aetos Standards blueprint\n\nYou are coding a complete new backend application conforming to the Aetos\nstandard app blueprint. Execute every step in order. Only steps 2\u20134 use the\napp-specific details provided by the user; everything else is deterministic.\n\n## Before writing any code\n\nConnect to the `standards` MCP server and read:\n- `docs://nav` (the module list)\n- `docs://modules/genericity_matrix` (Tier A library / Tier B template / Tier C authored)\n- `docs://modules/layering_rules` (the dependency rules)\n\nThen:\n- call `find_library` with `db`, `di`, `routing`, `ingestion`, `pipeline`, `docs`\n and note the Tier A libraries that already exist \u2014 never reimplement one;\n- call `list_app_components()` and pull each file you need with\n `get_app_component(kind)` instead of writing it from scratch.\n\n## Steps\n\n1. **Scaffold** \u2014 copy `standards/src/templates/app/` into `apps/<your-app>/`\n (or pull the same tree via `get_app_template`). Keep the tree: `domain/`,\n `infra/`, `interfaces/`, `pipelines/`, `tests/`, `scripts/`, plus\n `pyproject.toml`, `openapi.yaml`, `config.py`, `init.py`, `main.py`,\n `worker.py`, `Dockerfile`, `docker-compose.yaml`, `.env.example`.\n\n2. **Author the spec (app-specific)** \u2014 write `openapi.yaml`: resources,\n schemas, `operationId`s, security, and extension metadata. This is the\n contract and drives all routing. Every operation MUST have an `operationId`.\n\n3. **Generate the scaffold (codegen)** \u2014 from the spec, generate the wire\n models and the exact handler signatures:\n ```bash\n openapi-first models openapi.yaml -o _gen/models.py\n openapi-first routes openapi.yaml -o _gen/routes \\\n --use-models --models-module _gen.models --inject-request --package --template-style\n ```\n `_gen/models.py` is the **wire contract** (Pydantic, one class per\n `components.schemas` entry). `_gen/routes/<resource>/routes.py` lists every\n `operationId` with its exact parameter signature. `--template-style` makes\n those stubs `async def` with a `body` parameter, matching the handlers you\n keep in `interfaces/` (without it they are sync `def` stubs taking\n `payload`; both bind the same way). Treat `_gen/` as generated\n (never hand-edit it) \u2014 the canonical handlers live in `interfaces/`.\n\n4. **Write each CRUD slice (app-specific)** \u2014 per resource create the four-file\n slice: `models.py` (domain model, **no `id`**), `ports/db.py`\n (`class XDocument(BaseDocument, X)` \u2014 id comes from `BaseDocument`),\n `ports/service.py`, and optional `converters.py`. Follow the CRUD template.\n\n5. **Wire the composition root** \u2014 in `init.py`, build leaf repositories, then\n dependent repositories, then services; add infra adapters (db, cache, stores,\n progress bus, fetchers). Expose `get_svc` overloads.\n\n6. **Add routes** \u2014 create `interfaces/<resource>/routes.py` thin handlers whose\n names match the `operationId`s (use the signatures from step 3 as your\n checklist) and re-export them in `interfaces/openapi.py`. Add the cache\n decorator to read endpoints.\n\n7. **Bootstrap app and worker** \u2014 confirm `main.py` runs the OpenAPI-first app\n with its lifespan. If the app has long-running jobs, adapt `worker.py` to poll\n the job collection, run the pipeline, and publish progress.\n\n8. **Add pipeline families (only if data-driven)** \u2014 copy the worked family\n `pipelines/appointment_reminder/` to `pipelines/<family>/` with `abc.py`,\n `impl.py`, `pipeline.yaml`, `resources/`, and reuse `pipelines/shared/`\n (loader + registry). Register the root, add the `job.kind` handler in\n `worker.py`. Add fixtures and a parity test.\n\n9. **Add infrastructure only where needed** \u2014 Tier A capability: use the\n library. Tier B capability: copy the template. Never hand-write a Tier A\n library.\n\n10. **Tests** \u2014 integration tests with fakes at the HTTP edge; unit tests for\n domain and pipeline logic; a config-consistency test for rule-driven\n pipelines. Include `tests/integration/test_openapi_app_builds.py` (imports\n the app, asserts `import main` binds every operation).\n\n11. **Scripts and seeds** \u2014 operator scripts under `scripts/` using\n `init()` / `get_svc()` / `shutdown()`; run as `python -m scripts.<name>`.\n\n12. **Docs** \u2014 author GSDFC docstrings, generate `.pyi` stubs, mark the package\n `py.typed`, and build the docs.\n\n**Definition of done:**\n- `python -c \"import main\"` succeeds.\n- `pytest` passes.\n- The app serves every operation in the spec; the worker drains its queue.\n- No domain module imports a web framework.\n- No Tier A library was reimplemented.\n</code></pre>"},{"location":"01_build_recipe/#where-the-steps-land","title":"\ud83e\udded Where the steps land","text":"<p>The steps map onto the layers as follows:</p> Step Package / file Depends on 1 \u00b7 Scaffold whole <code>templates/app/</code> \u2014 2 \u00b7 Spec <code>openapi.yaml</code> \u2014 3 \u00b7 Codegen <code>_gen/</code> (generated) spec, openapi-first 4 \u00b7 CRUD slices <code>domain/<resource>/</code> spec, CRUD template 5 \u00b7 Wiring <code>init.py</code> every slice 6 \u00b7 Routes <code>interfaces/<resource>/</code> services, openapi route registry 7 \u00b7 Bootstrap <code>main.py</code>, <code>worker.py</code> composition root 8 \u00b7 Pipelines <code>pipelines/<family>/</code>, <code>shared/</code> library pipeline engine 9 \u00b7 Infra <code>infra/</code> domain ports 10 \u00b7 Tests <code>tests/</code> app + fixtures 11 \u00b7 Scripts <code>scripts/</code> composition root 12 \u00b7 Docs docstrings + docs build code, Doc Forge"},{"location":"01_build_recipe/#the-codegen-fast-path-in-detail","title":"\ud83d\udd27 The codegen fast path, in detail","text":"<p><code>openapi-first</code> is spec \u2192 code only (it never generates a spec). Use it to erase the two mechanical parts of the build:</p> Generated artifact Replaces Notes <code>_gen/models.py</code> hand-writing Pydantic wire models one class per <code>components.schemas</code> entry <code>_gen/routes/<resource>/routes.py</code> hand-typing handler signatures names == <code>operationId</code>s; <code>request: Request</code> injected <p>Rules of the road:</p> <ul> <li>Wire models are not domain models. <code>_gen/models.py</code> is the HTTP contract; <code>domain/<resource>/models.py</code> is your business shape. If the spec's schema has a required <code>id: str</code>, keep it off the domain model and let the document get its id from <code>BaseDocument</code>.</li> <li>Use <code>--template-style</code>. Stubs are then <code>async def ...(request, body: ...)</code> matching <code>interfaces/</code>; without it they are sync <code>def ...(payload: ...)</code>. Either binds (the router introspects the signature), but <code>--template-style</code> keeps the scaffold consistent with the code you keep.</li> <li>Resource names are sanitized. <code>/fetch-requests</code> \u2192 <code>fetch_requests.py</code>.</li> <li>Multipart bodies expand to <code>UploadFile = File(...)</code> / <code>Form(...)</code> params.</li> <li><code>_gen/</code> is disposable. Regenerate on every spec change; <code>git diff</code> shows exactly what moved.</li> </ul>"},{"location":"01_build_recipe/#definition-of-done","title":"\u26a0\ufe0f Definition of done","text":"<ul> <li><code>python -c \"import main\"</code> succeeds.</li> <li><code>pytest</code> passes.</li> <li>The app serves the spec's operations; the worker drains its queue.</li> <li>No domain module imports a web framework.</li> <li>No Tier A library was reimplemented.</li> </ul>"},{"location":"01_build_recipe/#related","title":"Related","text":"<ul> <li>10 \u2013 Layering Rules \u00b7 21 \u2013 CRUD Template \u00b7 22 \u2013 Auth Integration \u00b7 40 \u2013 UI \u00b7 50 \u2013 Pipeline Scaffold \u00b7 92 \u2013 Genericity Matrix</li> </ul>"},{"location":"02_composition_root/","title":"Composition Root","text":""},{"location":"02_composition_root/#summary","title":"Summary","text":"<p>All wiring happens in exactly one module. It builds a flat resource registry and exposes it to the HTTP layer and to scripts.</p>"},{"location":"02_composition_root/#the-resource-registry","title":"The resource registry","text":"<p><code>init.py</code> returns a plain <code>dict[str, Any]</code>. Keys follow two conventions:</p> <ul> <li><code><resource>_repo</code> \u2014 a repository adapter for a resource.</li> <li><code><resource>_svc</code> \u2014 a service for a resource.</li> </ul> <p>Plus named infrastructure adaptors:</p> <ul> <li><code>http_cache_store</code> \u2014 response cache backend.</li> <li><code>progress_bus</code> \u2014 progress/event bus.</li> <li><code><vendor>_<kind></code> \u2014 fetchers, clients, stores.</li> </ul> <p>The API stores the dict on <code>app.state.db_resources</code> during lifespan startup. Handlers read from it; nothing imports concrete adapters.</p>"},{"location":"02_composition_root/#shape","title":"Shape","text":"<pre><code>_resources: dict = {}\n_initialized = False\n\nasync def init(settings=None) -> dict:\n global _initialized\n if _initialized:\n return _resources\n\n await setup_database(uri=settings.mongo_uri, db_name=settings.mongo_db)\n\n # 1. leaf repositories\n user_repo = create_user_repo()\n\n # 2. dependent repositories (dependencies passed by keyword)\n wallet_repo = create_wallet_repo(user_repo=user_repo)\n\n # 3. services (build via the DI library, injecting the repo)\n _resources[\"user_repo\"] = user_repo\n _resources[\"wallet_repo\"] = wallet_repo\n _resources[\"user_svc\"] = get_user_service(user_repo)\n _resources[\"wallet_svc\"] = get_wallet_service(wallet_repo)\n\n # 4. infrastructure adaptors (optional)\n _resources[\"http_cache_store\"] = HttpCacheStore(...)\n _resources[\"progress_bus\"] = ProgressEventBus()\n\n _initialized = True\n return _resources\n\nasync def shutdown() -> None:\n global _initialized\n await close_stores(_resources)\n await shutdown_database()\n _resources.clear()\n _initialized = False\n</code></pre>"},{"location":"02_composition_root/#typed-accessors","title":"Typed accessors","text":"<p>Expose a <code>get_svc(name)</code> helper with one <code>@overload</code> per key so scripts get typing without importing concrete classes:</p> <pre><code>@overload\ndef get_svc(name: Literal[\"user_svc\"]) -> UserService: ...\n@overload\ndef get_svc(name: Literal[\"wallet_svc\"]) -> WalletService: ...\n\ndef get_svc(name: str):\n return _resources[name]\n</code></pre> <p>Also expose <code>get_db()</code> (raw database handle) and pure helpers such as <code>compute_fingerprint(payload)</code> (canonical hash used for deduplication).</p>"},{"location":"02_composition_root/#rules","title":"Rules","text":"<ul> <li><code>init()</code> is idempotent.</li> <li>The composition root is the only place that imports concrete adapters.</li> <li>Handlers never call <code>init()</code>; they read <code>request.app.state.db_resources</code>.</li> <li>Scripts call <code>init()</code> / <code>shutdown()</code> around their work.</li> </ul>"},{"location":"10_layering_rules/","title":"Layering Rules","text":""},{"location":"10_layering_rules/#summary","title":"Summary","text":"<p>The dependency direction is fixed. Violating it is a design defect.</p>"},{"location":"10_layering_rules/#allowed-imports","title":"Allowed imports","text":"<pre><code>interfaces/* -> domain/* (services, ports, models)\ninterfaces/* -> infra/adapters/* (only read-only value objects)\ndomain/* -> domain/* (same layer)\ndomain/* -> pipelines/* (call a pipeline via a port)\ndomain/*/ports/* -> infra/* (bind adapters + libraries at the seam)\ninfra/adapters/* -> domain/ports/* (implement the interface)\nmain.py/init.py -> everything (composition root only)\n</code></pre>"},{"location":"10_layering_rules/#forbidden-imports","title":"Forbidden imports","text":"<pre><code>domain/* -> interfaces/* (never)\ndomain/* -> fastapi/starlette (never)\ninfra/adapters/* -> interfaces/* (never)\ninterfaces/* -> infra/db/* (state lives in resources, not imports)\n</code></pre>"},{"location":"10_layering_rules/#why","title":"Why","text":"<ul> <li>The business logic is testable without a server or an event loop.</li> <li>Adapters can be swapped (in-memory fake, real database) without touching business logic.</li> <li>The HTTP edge stays thin and replaceable.</li> </ul>"},{"location":"10_layering_rules/#the-portadapter-contract","title":"The port/adapter contract","text":"<p>The <code>ports/</code> package is the seam where the domain meets infrastructure. Each <code>domain/<resource>/ports/</code> module declares the resource's models/document shape and exposes factory functions that bind a concrete adapter \u2014 so a port module is allowed to import from <code>infra/</code> and from the persistence libraries. Everything outside <code>ports/</code> in <code>domain/</code> stays framework- and driver-free.</p> <pre><code>domain/<resource>/ports/db.py -> document model + create_<resource>_repo()\ndomain/<resource>/ports/service.py -> service + get_<resource>_service()\ninfra/db/repos/<vendor>.py -> repository adapter factory\ninfra/adapters/service.py -> generic CRUD service base\n</code></pre> <p>The composition root calls the port factories; nothing else constructs an adapter.</p>"},{"location":"10_layering_rules/#enforcement","title":"Enforcement","text":"<ul> <li><code>domain/</code> must never import the web framework (<code>fastapi</code> / <code>starlette</code>) or <code>interfaces/</code>. Only the <code>ports/</code> seam, <code>infra/</code>, and the composition root touch drivers.</li> <li>Reviewers (and CI) reject <code>import fastapi</code> inside <code>domain/</code> or <code>infra/</code>.</li> </ul>"},{"location":"20_spec_first_routing/","title":"Spec-First Routing","text":""},{"location":"20_spec_first_routing/#summary","title":"Summary","text":"<p>The OpenAPI document is the single source of truth for the HTTP contract. Handlers are bound to it by name; there is no decorator router.</p>"},{"location":"20_spec_first_routing/#the-contract","title":"The contract","text":"<ul> <li><code>openapi.yaml</code> (or <code>.json</code>) declares paths, schemas, parameters, responses, and security.</li> <li>A custom <code>operationId</code> names the handler function.</li> <li>The framework (<code>openapi_first</code>) validates requests/responses against the spec and dispatches to the handler whose name equals <code>operationId</code>.</li> </ul>"},{"location":"20_spec_first_routing/#app-bootstrap","title":"App bootstrap","text":"<pre><code># main.py\nfrom openapi_first.app import OpenAPIFirstApp\n\napp = OpenAPIFirstApp(\n openapi_path=\"openapi.yaml\",\n routes_module=interfaces.openapi, # module whose namespace holds handlers\n title=\"<App> API\",\n version=\"0.1.0\",\n lifespan=lifespan,\n)\n</code></pre> <p><code>lifespan</code> calls the composition root:</p> <pre><code>@asynccontextmanager\nasync def lifespan(app):\n app.state.db_resources = await init()\n yield\n await shutdown()\n</code></pre>"},{"location":"20_spec_first_routing/#handler-registry","title":"Handler registry","text":"<p><code>interfaces/openapi.py</code> is a flat re-export of every handler, so the framework can resolve <code>operationId</code> across the whole app:</p> <pre><code>from interfaces.item.routes import *\nfrom interfaces.wallet.routes import *\nfrom interfaces.progress.sse import *\n</code></pre>"},{"location":"20_spec_first_routing/#handler-shape","title":"Handler shape","text":"<p>Handlers are thin and synchronous with respect to business logic:</p> <pre><code>async def create_item(request: Request, body: dict):\n svc = request.app.state.db_resources[\"item_svc\"]\n return await svc.create(Item.model_validate(body))\n</code></pre> <p>Generated stubs. <code>openapi-first routes</code> emits this exact shape (<code>async def</code> handlers taking <code>body</code>) when given <code>--template-style</code>; without that flag the stub is <code>def create_item(payload: ItemInput)</code> \u2014 sync, with a <code>payload</code> parameter. Both shapes bind identically: the router resolves handlers by <code>operationId</code> and introspects the FastAPI signature, and it never inspects the request body. Pass <code>--template-style</code> so the generated signatures match the handlers you keep in <code>interfaces/</code>.</p>"},{"location":"20_spec_first_routing/#conventions","title":"Conventions","text":"<ul> <li><code>operationId</code> naming: <code>list_<resource></code>, <code>create_<resource></code>, <code>get_<resource></code>, <code>update_<resource></code>, <code>delete_<resource></code>.</li> <li>Read-only endpoints may use a cache decorator keyed on path + query + user.</li> <li>Validation errors -> <code>400</code>; not found -> <code>404</code>; delete -> <code>204</code>.</li> <li>Security is declared once in the spec and enforced by the framework; handlers contain no auth code.</li> </ul>"},{"location":"20_spec_first_routing/#authless-apps","title":"Authless apps","text":"<p>Auth is deferred in the standard: the same spec-first app runs with or without it, because security lives entirely in the spec, never in handlers. For an app that needs no auth yet, do not invent per-handler auth checks \u2014 just omit the security keys:</p> <ul> <li>Remove the top-level <code>security:</code> block.</li> <li>Remove <code>components.securitySchemes</code>.</li> <li>Do not add <code>security: []</code> to operations. An explicit empty array is a per-operation override that means \"public\", and it only matters when a global <code>security</code> block exists. Leaving every operation without a <code>security</code> key is the cleanest authless spec.</li> </ul> <p>Profile endpoints (<code>/users/me</code>) and profile headers (<code>x-profile</code>) still exist in an authless app. Define a stand-in identity in the composition root \u2014 for example <code>get_current_user</code> returns or lazily creates a fixed \"Demo User\" \u2014 so handlers stay auth-agnostic and only the wiring changes when real auth arrives.</p> <p>The handlers, services, and composition root are unchanged. When auth is needed later, add the global <code>security</code> block and an introspecting dependency; only handlers that must opt out get <code>security: []</code>.</p>"},{"location":"20_spec_first_routing/#codegen","title":"Codegen","text":"<p><code>openapi_first</code> scaffolds models and route stubs from the spec (see the Build Recipe, step 3):</p> <pre><code>openapi-first models openapi.yaml -o _gen/models.py\nopenapi-first routes openapi.yaml -o _gen/routes \\\n --use-models --models-module _gen.models --inject-request --package --template-style\n</code></pre> <p>Prefer codegen over hand-writing boilerplate. The result is the wire contract and a set of <code>operationId</code>-named signatures \u2014 not the canonical handlers.</p>"},{"location":"21_crud_template/","title":"CRUD Template","text":""},{"location":"21_crud_template/#summary","title":"Summary","text":"<p>Every resource follows the same four-file slice. Repetition is intentional: it makes the codebase predictable and generatable.</p>"},{"location":"21_crud_template/#the-slice","title":"The slice","text":"<pre><code>domain/<resource>/\n models.py # pure Pydantic model (no DB fields)\n ports/\n db.py # document model + repository factory\n service.py # service class + factory\n converters.py # (optional) transforms\n</code></pre>"},{"location":"21_crud_template/#the-id-rule-model-vs-document","title":"The id rule (model vs. document)","text":"<p>Two different objects, two different jobs:</p> Object Lives in Has <code>id</code>? Purpose Domain model <code>domain/<resource>/models.py</code> No business shape, framework-free Document <code>domain/<resource>/ports/db.py</code> Yes (from <code>BaseDocument</code>) Mongo persistence shape <p><code>BaseDocument</code> (mongo-ops) supplies <code>id: Optional[PyObjectId]</code> (aliased to <code>_id</code>) plus <code>created_at</code> / <code>updated_at</code>. The domain model must not declare <code>id</code>; Mongo owns it.</p> <p>Trap \u2014 generated wire models. <code>openapi-first models</code> produces the HTTP contract (see the Build Recipe, step 3). Response schemas frequently declare a required <code>id: str</code>, so a generated class is not a domain model. If you write <code>class ItemDocument(Item, BaseDocument)</code> and <code>Item</code> re-exports a required <code>id: str</code>, that field shadows <code>BaseDocument.id</code> and no Mongo <code>_id</code> can ever be set. Fix it one of three ways:</p> <ol> <li>Preferred: keep the domain model id-free (hand-author it from the CRUD template), and keep the required <code>id</code> only on the wire model.</li> <li>Order the bases so the document wins: <code>class ItemDocument(BaseDocument, Item)</code>.</li> <li>Keep a separate document type and translate with <code>ports/converters.py</code>.</li> </ol>"},{"location":"21_crud_template/#modelspy","title":"models.py","text":"<p>A framework-free Pydantic model describing the business shape (no <code>id</code>).</p> <pre><code>class Item(BaseModel):\n name: str\n quantity: int = 0\n</code></pre>"},{"location":"21_crud_template/#portsdbpy","title":"ports/db.py","text":"<p>The persistence model extends the business model and adds database concerns. The repository factory selects the right repository flavour.</p> <pre><code># Item is id-free; BaseDocument adds id/created_at/updated_at.\nclass ItemDocument(Item, BaseDocument):\n pass\n\ndef create_item_repo(*, owner_repo=None) -> ItemRepository:\n return repo_factory(\n collection=\"items\",\n document=ItemDocument,\n populate_rules=[PopulateRule(field_name=\"owner\", collection_name=\"users\")],\n cache=None, # or a CachedBaseRepository config\n )\n</code></pre>"},{"location":"21_crud_template/#nested-references","title":"Nested references","text":"<p>The wire contract often declares a reference as an expanded object (<code>expense.entity: Entity</code>), while the domain model stays id-free. Pick one convention and apply it consistently:</p> <ul> <li>Embed (default for small, stable refs) \u2014 store the object as a <code>dict</code> / <code>list[dict]</code> on the document; the API returns it verbatim. No populate wiring.</li> <li>Populate \u2014 store an id and declare a <code>PopulateRule</code>; mongo-ops expands it on read. The referenced repo must be registered with the <code>PopulationEngine</code>.</li> <li>Id only \u2014 store the id and let the client resolve it.</li> </ul> <p>For populate-backed repositories change a reference with <code>update</code>, not <code>patch</code>: <code>patch</code> rejects changes to populated FK fields by design.</p>"},{"location":"21_crud_template/#portsservicepy","title":"ports/service.py","text":"<p>The service is built through the DI library, injecting the repository. The generic CRUD base provides <code>create/read/update/patch/delete/search/...</code>.</p> <pre><code>class ItemService(CrudsServiceAdaptor[Item, ItemDocument]):\n model = Item\n schema = ItemDocument\n\ndef get_item_service(repo) -> ItemService:\n return build(ItemService, instances={\"repo\": repo})\n</code></pre>"},{"location":"21_crud_template/#naming-conventions","title":"Naming conventions","text":"Concept Convention collection plural snake case (<code>items</code>) document <code><Resource>Document</code> repository factory <code>create_<resource>_repo</code> service <code><Resource>Service</code> service factory <code>get_<resource>_service</code> resource key <code><resource>_svc</code> operationId <code>list_items</code>, <code>create_item</code>, ..."},{"location":"21_crud_template/#optional-files","title":"Optional files","text":"<ul> <li><code>converters.py</code> \u2014 pure functions mapping between domain types.</li> <li><code>ports/enrichment.py</code> \u2014 adapts a pipeline into an async callable the service can invoke (used when a resource is populated by a pipeline).</li> </ul>"},{"location":"22_auth_integration/","title":"Auth Integration","text":""},{"location":"22_auth_integration/#summary","title":"Summary","text":"<p>Authentication is a service, not a library. Every Aetos application shares a single identity service \u2014 the auth-server \u2014 and consumes it through its HTTP contract. An app never implements registration, login, password hashing, or token issuance itself.</p> <p>The agent does not need an MCP bundle for the auth-server. It reads the pinned OpenAPI contract (below) and declares a bearer security scheme in its own spec; <code>openapi_first</code> then generates the verifying dependency. No auth code is written.</p>"},{"location":"22_auth_integration/#read-the-contract","title":"Read the contract","text":"<p>The auth-server OpenAPI document is pinned in the standards bundle and served as an MCP resource:</p> <pre><code>docs://spec\n</code></pre> <p>It is the authoritative client contract (OpenAPI 3.1). Read it to learn the endpoints and the request/response schemas:</p> Endpoint Purpose <code>POST /register</code> Create a user (201, returns <code>PublicUser</code>) <code>POST /login</code> Authenticate and issue a JWT (<code>LoginResponse</code>) <code>GET /me</code> Current user's <code>PublicUser</code> (bearer required) <code>POST /logout</code> Stateless logout (<code>LogoutResponse</code>) <code>POST /introspect</code> Internal token introspection (<code>active</code>, <code>user</code>) <p>The pinned snapshot is versioned (currently <code>0.0.5</code>). If the caller has a live auth-server and needs the current contract, they can pass the full endpoint (<code>{AUTH_SERVER}/openapi.json</code>) and read it directly instead of the pin.</p>"},{"location":"22_auth_integration/#wire-it-up-declare-dont-code","title":"Wire it up: declare, don't code","text":"<p>Authentication is added by editing the app's own <code>openapi.yaml</code> \u2014 nothing else. Declare a global bearer requirement and point the scheme at the auth server (values come from the environment):</p> <pre><code>security:\n - bearerAuth: []\npaths:\n # ...\ncomponents:\n securitySchemes:\n bearerAuth:\n type: http\n scheme: bearer\n bearerFormat: JWT\n x-server-url: \"{AUTH_SERVER}\"\n x-introspect-path: \"/introspect\"\n x-login-path: \"/login\"\n x-register-path: \"/register\"\n x-logout-path: \"/logout\"\n x-me-path: \"/me\"\n</code></pre> <p><code>openapi_first</code> reads <code>securitySchemes</code> + <code>security</code> and generates a FastAPI dependency that extracts the bearer token, POSTs <code>{\"token\": <jwt>}</code> to <code>{AUTH_SERVER}/introspect</code>, and returns the user when <code>active == true</code>. It raises <code>401</code> on an invalid or inactive token and <code>503</code> when the auth server is unreachable. The generated handlers receive the authenticated user \u2014 you author no authentication code.</p>"},{"location":"22_auth_integration/#public-operations","title":"Public operations","text":"<p>A route that must stay open opts out by overriding <code>security</code> with an empty list:</p> <pre><code>paths:\n /health:\n get:\n operationId: health\n security: [] # overrides the global requirement\n responses:\n \"200\":\n description: OK\n</code></pre>"},{"location":"22_auth_integration/#environment","title":"Environment","text":"<pre><code>AUTH_SERVER=http://localhost:8000 # auth-server base URL\nJWT_SERVER=http://localhost:8000 # jwtlib client base URL (seeding/CLI)\n</code></pre>"},{"location":"22_auth_integration/#path-1-jwtlib-client-no-generator","title":"Path 1: <code>jwtlib</code> client (no generator)","text":"<p>Apps that do not use a generated dependency verify requests with the <code>py-jwt</code> client:</p> <pre><code>from jwtlib.introspection import authenticate_request\n\nauthorized = await authenticate_request(\n should_skip_authentication, # allow public paths\n method,\n path,\n authorization_token, # \"Bearer <jwt>\" or None\n)\n</code></pre> <p><code>authenticate_request</code> POSTs to <code>{JWT_SERVER}/introspect</code> (3 second timeout) and treats <code>{\"active\": true, \"user\": {...}}</code> as valid. Use this for seeding scripts and tests (<code>jwtlib.register_user</code>, <code>jwtlib.login_user</code>).</p>"},{"location":"22_auth_integration/#golden-rules","title":"Golden rules","text":"<ol> <li>Never trust an unverified token. Decode through the generated dependency or <code>jwtlib</code> before handling the request.</li> <li>Treat <code>active: false</code> as unauthenticated, even if the JWT decodes.</li> <li>Do not implement your own JWT parsing. Revocation and issuer changes must stay centralized in the auth-server.</li> <li>Do not add auth until the app's outcome is validated. Ship the app authless first; layer the security scheme once the core behaviour works.</li> </ol>"},{"location":"22_auth_integration/#notes","title":"Notes","text":"<ul> <li>The default template is authless on purpose. Add the <code>securitySchemes</code> block above only when auth is required. Pull the shared slice with <code>get_app_component(\"auth\")</code>.</li> <li>The auth-server is deployed as a service (no importable package); the client surface is the spec, plus <code>jwtlib</code> for the low-level path.</li> </ul>"},{"location":"30_infra_adaptors/","title":"Infra Adapters","text":""},{"location":"30_infra_adaptors/#summary","title":"Summary","text":"<p>All external systems are reached through adapters in <code>infra/</code>. Prefer an existing internal library over writing a new adapter; where no library exists, copy the Tier B scaffold from <code>templates/app</code>. See the Genericity Matrix for the Tier A / B / C split.</p>"},{"location":"30_infra_adaptors/#database-mongodb","title":"Database (MongoDB)","text":"<ul> <li>A single connection manager owns the client lifecycle. <code>setup_database(uri, db_name)</code> / <code>shutdown_database()</code>.</li> <li>A repository factory selects the repository flavour:</li> </ul> <pre><code>repo_factory(collection, document, engine=None,\n populate_rules=None, cache=None)\n</code></pre> <ul> <li>base repository \u2014 plain CRUD;</li> <li>populating repository \u2014 resolves referenced documents (foreign keys);</li> <li> <p>cached repository \u2014 an LRU/TTL cache in front of reads.</p> </li> <li> <p>Referenced documents are described declaratively:</p> </li> </ul> <pre><code>PopulateRule(field_name=\"owner\", collection_name=\"users\",\n nested_rules=[], max_depth=2, projection=None)\n</code></pre> <ul> <li>Transactions use a session context manager provided by the library.</li> </ul> <p>Tier A \u2014 Library (Mongo Ops). Do not write raw Motor calls in the domain; consult the registry and the library's MCP before touching the driver.</p>"},{"location":"30_infra_adaptors/#http-response-cache","title":"HTTP response cache","text":"<p>Tier B \u2014 template. Copy from <code>templates/app</code>; not a standalone library.</p> <p>A generic GET cache sits behind a decorator.</p> <pre><code>@cache_response(ttl=60)\nasync def list_items(request): ...\n</code></pre> <ul> <li>Keyed on path + query + user.</li> <li>No-op when no store is wired.</li> <li><code>invalidate(*resources)</code> bumps a generation counter for write-triggered invalidation.</li> </ul>"},{"location":"30_infra_adaptors/#event-progress-bus","title":"Event / progress bus","text":"<p>Tier B \u2014 template. Copy from <code>templates/app</code>.</p> <p>A port with an in-memory default adapter (<code>publish</code>, <code>get_events</code>). Swap for a durable broker in production (see Worker & Progress).</p>"},{"location":"30_infra_adaptors/#external-clients-ingestion","title":"External clients (ingestion)","text":"<p>External fetchers (files, mail, HTTP) are adapters that wrap ingestion libraries. Construct them in the composition root and inject into orchestrators.</p> <p>Tier A \u2014 Libraries: Omniread (formats) and Mail Intake (email).</p>"},{"location":"30_infra_adaptors/#llm-clients","title":"LLM clients","text":"<p>Tier B \u2014 template. Copy the pool scaffold from <code>templates/app</code>.</p> <p>When a pipeline calls a model provider, route through a pool adaptor that handles provider selection, retries, and client lifecycle:</p> <pre><code>pool = setup_async_pool(provider=settings.llm_provider)\nresult = await pool.complete(model=\"...\", messages=[...])\n</code></pre> <p>Providers are selected by <code>LLM_PROVIDER</code> and configured by environment.</p>"},{"location":"30_infra_adaptors/#key-value-store","title":"Key-value store","text":"<p>Tier B \u2014 template. Copy from <code>templates/app</code>.</p> <p>For report/snapshot storage, provide a pluggable store port with a Redis adapter and an in-memory fallback.</p>"},{"location":"30_infra_adaptors/#rule","title":"Rule","text":"<ul> <li>Tier A (libraries: DB, DI, ingestion, mail, auth, docs, pipelines) \u2014 use the library; writing a private copy is a defect. Query <code>find_library</code> first.</li> <li>Tier B (cache, progress bus, LLM pool, KV store, composition shape, worker, tests) \u2014 these are template code, not libraries: copy them from <code>templates/app</code> rather than searching the registry.</li> <li>Tier C (models, business logic, pipeline families, rules) \u2014 author them yourself.</li> </ul>"},{"location":"40_ui_admin/","title":"UI: Auto-Rendered Admin & Custom Pages","text":""},{"location":"40_ui_admin/#summary","title":"Summary","text":"<p>The UI axis of an Aetos app is a published React client, not code written per application. Two npm packages carry the whole surface:</p> Package Role <code>@aetoskia/react-openapi</code> Renders a complete admin panel (CRUD tables, forms, filters, sorting, pagination, uploads) from the app's <code>openapi.yaml</code> \u2014 zero UI code. <code>@aetoskia/react-auth</code> Auth contexts and pages (login/register/me) wired to the auth-server, plus the axios client with token handling. <p>Both publish to the private registry <code>https://npm.aetoskia.com</code>. The server spec is the single source of truth: the same <code>openapi.yaml</code> that drives <code>openapi-first</code> routing drives the admin UI, through x- extension fields (see below).</p>"},{"location":"40_ui_admin/#scaffold","title":"Scaffold","text":"<pre><code>cp -r <docs>/standards/src/templates/client apps/<your-app>-client\ncd apps/<your-app>-client\nnpm install # deps come from npm.aetoskia.com via .npmrc\nnpm run dev\n</code></pre> <p>The client template is a Vite + MUI app with:</p> <ul> <li><code>src/openapi-config.ts</code> \u2014 points the renderer at the app's spec and the auth-server's spec;</li> <li><code>.env.example</code> \u2014 <code>VITE_API_BASE_URL</code> (app API) and <code>VITE_AUTH_BASE_URL</code> (auth-server);</li> <li>a <code>Home</code> page that auto-lists every resource the spec declares, and a <code>Header</code>/<code>Footer</code>/theme scaffold.</li> </ul> <p>Custom pages are ordinary React routes layered on top; the auto-rendered admin is available per resource at <code>#/resource/<name></code>.</p>"},{"location":"40_ui_admin/#x-extensions-how-the-spec-steers-the-ui","title":"x- extensions: how the spec steers the UI","text":"<p>The admin renderer reads a catalog of OpenAPI extension fields (<code>x-resource</code>, <code>x-list-columns</code>, <code>x-display-format</code>, <code>x-label</code>, <code>x-order</code>, <code>x-hidden</code>, <code>x-sortable</code>, <code>x-filterable</code>, <code>x-autocomplete</code>, <code>x-fk</code>, <code>x-ui-type</code>, <code>x-upload</code>, <code>x-sse</code>, <code>x-profile</code>, and the auth wiring <code>x-server-url</code> / <code>x-login-path</code> / <code>x-me-path</code> / \u2026) declared on schemas and properties.</p> <p>The catalog is canonical in the react-openapi source, not in this doc. Read it from the MCP bundle:</p> <pre><code>docs://modules/react-openapi # module: specExtensions \u2014 the full catalog\n</code></pre> <p>The app's own spec carries the per-resource values; the catalog explains what each field does. The template's <code>openapi.yaml</code> shows the convention on the example <code>Job</code> schema.</p>"},{"location":"40_ui_admin/#golden-rules","title":"Golden rules","text":"<ol> <li>The spec owns the UI. Add a resource or rename a column in <code>openapi.yaml</code>; the admin follows. Never fork the renderer per app.</li> <li>Custom pages for workflows, auto-render for CRUD. Anything that is \"list / create / edit / delete an entity\" is a spec concern; anything that is a bespoke workflow is a React page.</li> <li>Auth is consumed, never re-implemented. <code>@aetoskia/react-auth</code> holds tokens and calls the auth-server endpoints declared by the x- wiring; pages read the context.</li> <li>Publish, don't vendor. UI shared across apps belongs in <code>@aetoskia/react-openapi</code> or <code>@aetoskia/react-auth</code> (SemVer, tag-triggered npm release), not copy-pasted into app trees.</li> </ol>"},{"location":"40_ui_admin/#notes","title":"Notes","text":"<ul> <li>The server template ships authless by default (see 22 \u2013 Auth Integration); the client template pairs with it and degrades to \"no login\" until a security scheme is declared.</li> <li>The standards app + client templates are the full-stack reference: the server spec carries the x- extensions, and the client renders the admin from it. The wiki's Reference App walkthrough (<code>06_reference_app</code>) documents the pairing end to end.</li> </ul>"},{"location":"50_pipeline_scaffold/","title":"Pipeline Scaffold","text":""},{"location":"50_pipeline_scaffold/#summary","title":"Summary","text":"<p>Data-driven processing is expressed as a composition graph. The graph is built by a DI library; execution is driven by a DAG engine. Stages are pure and individually testable.</p>"},{"location":"50_pipeline_scaffold/#when-to-use-a-pipeline","title":"When to use a pipeline","text":"<p>Use a pipeline when a resource is produced by a multi-step transformation of an input (a document, a file, a message) rather than by a simple CRUD write.</p>"},{"location":"50_pipeline_scaffold/#structure-of-a-family","title":"Structure of a family","text":"<p>A family is one input shape (e.g. a file format or a message kind). Each family has the same skeleton:</p> <pre><code>pipelines/<family>/\n abc.py # port contracts (stage interfaces + root interface)\n impl.py # concrete stages + root implementation\n pipeline.yaml # graph spec: nodes, edges, roots, schema\n resources/\n pipeline.yaml # runtime spec consumed by the loader\n patterns/*.yaml # rule fragments (pattern registry)\n banks/<variant>.py # optional variants that re-pin a few slots\n</code></pre>"},{"location":"50_pipeline_scaffold/#shared-scaffolding","title":"Shared scaffolding","text":"<pre><code>pipelines/shared/\n ports.py # RegistryPort, FinalizePort, etc.\n loader.py # build(root_cls) -> wires the graph + engine\n registry.py # resolve_root(...) -> root class for a (kind, variant)\n</code></pre>"},{"location":"50_pipeline_scaffold/#the-loader","title":"The loader","text":"<pre><code>def build(root_cls, *, instances=None, config=None):\n # merge resources/patterns/*.yaml into a pattern registry\n # bind registry/config into each graph node by id\n # build the schema (state) class and the engine\n root = root_cls()\n root._engine = Engine(graph, state_cls, payload_cls)\n return root\n</code></pre>"},{"location":"50_pipeline_scaffold/#the-registry","title":"The registry","text":"<p>A single lookup replaces hand-rolled <code>if/else</code> selection:</p> <pre><code>_FAMILIES = {\n \"csv\": {\"acme\": {\"heuristic\": AcmeCsvPipeline}},\n \"json\": {\"globex\": {\"heuristic\": GlobexJsonPipeline}},\n}\n\ndef resolve_root(kind, variant, mode=\"heuristic\"):\n return _FAMILIES[kind][variant][mode]\n</code></pre> <p>Adding a new variant means adding one class and one registry entry.</p>"},{"location":"50_pipeline_scaffold/#driving-a-pipeline","title":"Driving a pipeline","text":"<p>Stages are driven explicitly so intermediate state can be persisted:</p> <pre><code>pipeline = build(resolve_root(kind, variant))\ncontent = pipeline.ingest.materialize(raw_bytes)\nrecords = pipeline.reader.read(content)\nitems = pipeline.normalize.build(records)\n</code></pre>"},{"location":"50_pipeline_scaffold/#testing","title":"Testing","text":"<p>Because stages are pure, test each stage in isolation and test the assembled root against fixture inputs whose expected outputs are stored as golden files. A parity test replays fixtures headlessly.</p>"},{"location":"51_worker_and_progress/","title":"Worker & Progress","text":""},{"location":"51_worker_and_progress/#summary","title":"Summary","text":"<p>Long-running work runs in a background worker. Jobs are durable documents that move through an explicit status machine. Progress is streamed to clients.</p>"},{"location":"51_worker_and_progress/#the-status-machine","title":"The status machine","text":"<pre><code>pending -> processing -> completed\n \\-> failed (retried up to N times)\n \\-> paused (waiting for human input)\n</code></pre> <ul> <li><code>pending</code> \u2014 queued, eligible for pickup.</li> <li><code>processing</code> \u2014 claimed by a worker.</li> <li><code>paused</code> \u2014 blocked on a human-in-the-loop question.</li> <li><code>failed</code> \u2014 error; <code>retry_count</code> increments until a max, then skipped.</li> <li><code>completed</code> \u2014 terminal success.</li> </ul>"},{"location":"51_worker_and_progress/#the-poll-loop","title":"The poll loop","text":"<pre><code>async def poll(resources):\n backoff = MIN_POLL\n while True:\n jobs = await job_svc.search(filter={\"status\": \"pending\"}, limit=100)\n backoff = MIN_POLL if jobs else min(backoff * 2, MAX_POLL)\n for job in jobs:\n if (job.retry_count or 0) >= MAX_RETRIES:\n continue\n await job_svc.update_status(job.id, \"processing\")\n try:\n await process(job, resources)\n except PauseForResolution:\n await job_svc.update_status(job.id, \"paused\")\n except Exception as e:\n await job_svc.update_status(\n job.id, \"failed\", error=str(e),\n retry_count=(job.retry_count or 0) + 1,\n )\n await asyncio.sleep(backoff)\n</code></pre> <p>Tunables live in environment variables: <code>WORKER_POLL_MIN</code>, <code>WORKER_POLL_MAX</code>, <code>WORKER_MAX_RETRIES</code>.</p>"},{"location":"51_worker_and_progress/#progress-events","title":"Progress events","text":"<p>A worker publishes structured progress events to a bus (a port). The HTTP edge exposes a streaming endpoint that replays events for a given job id.</p> <pre><code>class ProgressEvent(BaseModel):\n job_id: str\n step: str\n status: str # started | progress | completed | skipped | paused\n message: dict | None = None\n</code></pre> <p>The bus is an in-memory adapter by default; because it is a port, it can be swapped for a durable broker. Emit from the orchestrator at each stage:</p> <pre><code>await self.bus.publish(ProgressEvent(job_id=jid, step=\"normalize\",\n status=\"completed\", message={\"count\": n}))\n</code></pre>"},{"location":"51_worker_and_progress/#streaming-endpoint","title":"Streaming endpoint","text":"<pre><code>async def stream_job_events(request, id):\n bus = request.app.state.db_resources.get(\"progress_bus\")\n if bus is None:\n return {\"events\": []}\n\n async def gen():\n sent = 0\n while True:\n events = bus.get_events(id)\n for e in events[sent:]:\n yield f\"data: {e.model_dump_json(exclude_none=True)}\\n\\n\"\n sent += 1\n await asyncio.sleep(0.5)\n\n return StreamingResponse(gen(), media_type=\"text/event-stream\")\n</code></pre> <p>Known limitation. The default in-memory bus is per-process. If the worker and the API run as separate processes, events published by the worker are not visible to the API. For production, back the bus with a shared broker. The port boundary makes this a one-adapter change.</p>"},{"location":"52_human_in_loop/","title":"Human-in-the-Loop","text":""},{"location":"52_human_in_loop/#summary","title":"Summary","text":"<p>Some inputs are ambiguous and must be resolved by a human before processing can continue. The blueprint models this as a durable pause: an exception is raised, a question is recorded, the job pauses, and later resumes.</p>"},{"location":"52_human_in_loop/#the-mechanism","title":"The mechanism","text":"<ol> <li>During processing, a stage raises a domain exception carrying the context needed to ask the question:</li> </ol> <pre><code>raise PauseForResolution(\n job_id=jid,\n reference=\"<stable key for this decision>\",\n options=[...],\n context={...},\n)\n</code></pre> <ol> <li>The orchestrator catches it, records a question document, and re-raises so the worker marks the job <code>paused</code>.</li> <li>A client lists pending questions and answers one:</li> </ol> <pre><code>GET /jobs/{id}/questions -> list pending questions\nPOST /questions/{qid}/resolve {choice} -> record the choice\n</code></pre> <ol> <li>When the last pending question is resolved, the resolve endpoint flips the parent job back to <code>pending</code>.</li> <li>The worker picks it up again, rebuilds a resolution map from previously answered questions, and seeds it into the pipeline so the same decision is not asked twice.</li> </ol>"},{"location":"52_human_in_loop/#the-question-document","title":"The question document","text":"<pre><code>class Question(BaseDocument):\n job_id: str\n reference: str # stable key: stage|field|value\n status: str # pending | resolved\n options: list[dict]\n context: dict\n chosen: dict | None = None\n resolved_at: str | None = None\n</code></pre>"},{"location":"52_human_in_loop/#resume-map","title":"Resume map","text":"<p>The worker turns answered questions into a lookup keyed by <code>reference</code>, then injects it into the pipeline so that stage can skip re-raising:</p> <pre><code>resolved = {q.reference: q.chosen for q in await svc.search(\n filter={\"job_id\": jid, \"status\": \"resolved\"})}\n\npipeline = build(root, instances={\"pause\": handler}, config={\"resolved\": resolved})\n</code></pre>"},{"location":"52_human_in_loop/#resolve-endpoint","title":"Resolve endpoint","text":"<pre><code>async def resolve_question(request, id, body):\n svc = request.app.state.db_resources[\"question_svc\"]\n q = await svc.read(id)\n await svc.update(id, {\"chosen\": body[\"chosen\"], \"status\": \"resolved\",\n \"resolved_at\": now()})\n\n pending = await svc.search(filter={\"job_id\": q.job_id, \"status\": \"pending\"},\n limit=1)\n if not pending:\n jobs = request.app.state.db_resources[\"job_svc\"]\n job = await jobs.read(q.job_id)\n if job and job.status == \"paused\":\n await jobs.update_status(q.job_id, \"pending\")\n return q\n</code></pre>"},{"location":"52_human_in_loop/#cli-parity","title":"CLI parity","text":"<p>The same flow is available through scripts/CLI: <code>list</code>, <code>get</code>, <code>create</code>, <code>questions</code>, <code>resolve</code>, <code>retry</code>.</p>"},{"location":"90_drone_pipelines/","title":"Drone Pipelines","text":""},{"location":"90_drone_pipelines/#summary","title":"Summary","text":"<p>Every Aetos repository ships a Drone CI definition at <code>.drone.yml</code>. There is exactly one standard per repository, and there are three types of file, chosen by what the repository produces:</p> Type File Produces Repos A \u2014 PyPI package <code>templates/drone/python-lib.yaml</code> A wheel + sdist on the private index <code>dagpipe</code>, <code>doc-forge</code>, <code>hexa</code>, <code>mail-intake</code>, <code>mongo-ops</code>, <code>omniread</code>, <code>openapi-first</code>, <code>py-jwt</code> B \u2014 npm package <code>templates/drone/npm-lib.yaml</code> A package published to the private npm registry <code>doc-weaver</code>, <code>react-auth</code>, <code>react-openapi</code> C \u2014 Docker deployment <code>templates/drone/service-docker.yaml</code> An image pushed to the registry and run as a container <code>auth-server</code> <p>All templates run on <code>platform: linux / arm64</code> with the workspace at <code>/drone/src</code>. Copy the matching template to <code><repo>/.drone.yml</code>; the Type A and Type B files need no edits, the Type C file needs its placeholders filled in (below).</p> <p>Conventions: <code>.drone.yml</code> (not <code>.yaml</code>), one file per repo, secrets injected with <code>from_secret</code> \u2014 never commit credentials.</p>"},{"location":"90_drone_pipelines/#type-a-pypi-package-python-libyaml","title":"Type A \u2014 PyPI package (<code>python-lib.yaml</code>)","text":"<p>For internal Python libraries. The file contains two pipelines:</p> <ol> <li><code>build-and-publish-pypi</code> \u2014 triggered on <code>tag</code>:</li> <li><code>check-version</code> \u2014 guard: fail unless <code>$DRONE_TAG</code> equals the version in <code>pyproject.toml</code>; then read <code>name</code> / <code>version</code> and query <code>$PIP_REPO_URL/simple/<package>/</code>; if that version already exists, exit <code>78</code> and skip the whole build (see 91 \u2013 Versioning & Changelog).</li> <li><code>quality-gate</code> \u2014 install the package with its <code>[dev]</code> extras from the private index, then <code>black --check</code>, <code>ruff check</code>, <code>mypy</code>, <code>pydoclint</code>, <code>pytest</code>.</li> <li><code>build-package</code> \u2014 <code>python -m build</code> into <code>dist/</code>.</li> <li><code>upload-to-private-pypi</code> \u2014 <code>twine upload</code> to <code>$PIP_REPO_URL</code>.</li> <li><code>backfill-pypi-from-tags</code> \u2014 triggered on <code>custom</code>:</li> <li><code>fetch-tags</code> \u2014 <code>git fetch --tags --force</code>.</li> <li><code>build-and-upload-missing</code> \u2014 walk every tag in version order, skip the ones already on the index, <code>git checkout</code> the rest, verify <code>tag == pyproject version</code> at that commit, build, and upload. This is the disaster-recovery / replay path.</li> </ol> <p>Secrets: <code>PIP_REPO_URL</code>, <code>PIP_USERNAME</code>, <code>PIP_PASSWORD</code>.</p> <ul> <li>Ship as-is. Package name and version come from <code>pyproject.toml</code>, so the same file is reused verbatim across every library.</li> </ul>"},{"location":"90_drone_pipelines/#type-b-npm-package-npm-libyaml","title":"Type B \u2014 npm package (<code>npm-lib.yaml</code>)","text":"<p>For internal npm libraries (the React packages). The file contains two pipelines:</p> <ol> <li><code>build-and-publish-npm</code> \u2014 triggered on <code>tag</code>:</li> <li><code>check-version</code> \u2014 guard: fail unless <code>$DRONE_TAG</code> equals the version in <code>package.json</code>; query <code>$NPM_REPO_URL</code> for <code>name@version</code>; if it already exists, exit <code>78</code> and skip the whole build.</li> <li><code>quality-gate</code> \u2014 write <code>.npmrc</code> from the secrets, <code>npm install</code>, then <code>npm run quality-gate</code>.</li> <li><code>build-package</code> \u2014 <code>npm install && npm run build</code> into <code>dist/</code>.</li> <li><code>publish-to-private-npm</code> \u2014 <code>npm publish --access restricted</code> to <code>$NPM_REPO_URL</code>.</li> <li><code>backfill-npm-from-tags</code> \u2014 triggered on <code>custom</code>:</li> <li><code>fetch-tags</code> \u2014 <code>git fetch --tags --force</code>.</li> <li><code>build-and-publish-missing</code> \u2014 walk every tag in version order, skip the ones already on the registry, <code>git checkout</code> the rest, verify <code>tag == package.json version</code> at that commit, build, and publish. This is the disaster-recovery / replay path.</li> </ol> <p>Secrets: <code>NPM_REPO_URL</code>, <code>NPM_USERNAME</code>, <code>NPM_PASSWORD</code>. Auth is a generated <code>.npmrc</code> (<code>registry=</code> + <code>_auth=</code> base64 of <code>user:pass</code>) written from secrets at runtime \u2014 the committed <code>.npmrc</code> in a repo only points at the registry URL.</p> <ul> <li>Ship as-is. Name and version come from <code>package.json</code>, so the same file is reused verbatim across every npm package.</li> </ul> <p>Registry credentials are shared. The private npm registry (Verdaccio at <code>npm.aetoskia.com</code>) authenticates against the same <code>htpasswd</code> file as the private PyPI index and the Docker registry (<code>/auth/htpasswd</code>), with registration disabled (<code>max_users: -1</code>). <code>NPM_USERNAME</code> / <code>NPM_PASSWORD</code> are therefore the same credentials as <code>PIP_USERNAME</code> / <code>PIP_PASSWORD</code> \u2014 no separate npm account exists. Verdaccio's htpasswd plugin accepts bcrypt hashes (<code>$2y$\u2026</code>); non-bcrypt entries that pypiserver tolerates will not authenticate against npm. See 93 \u2013 Library Registry.</p>"},{"location":"90_drone_pipelines/#type-c-docker-deployment-service-dockeryaml","title":"Type C \u2014 Docker deployment (<code>service-docker.yaml</code>)","text":"<p>For a service that ships a container image \u2014 a long-running API/worker, e.g. <code>auth-server</code>. The file contains one pipeline, <code>build-and-deploy-service</code>, triggered on <code>tag</code>:</p> <ul> <li><code>fetch-tags</code> \u2014 resolve the latest tag into <code>LATEST_TAG.txt</code>; fail if there are no tags.</li> <li><code>check-version</code> \u2014 guard: fail unless <code>$DRONE_TAG</code> equals the repo-root <code>VERSION</code> file (see 91 \u2013 Versioning & Changelog).</li> <li><code>check-remote-image</code> \u2014 if <code>aetos/<app>:<tag></code> already exists locally, exit <code>78</code> and skip the build.</li> <li><code>quality-gate</code> \u2014 install from <code>requirements.txt</code> and run <code>black</code> / <code>ruff</code> / <code>mypy</code> / <code>pytest</code>.</li> <li><code>build-image</code> \u2014 <code>docker build</code> with the private-index build args, tagged <code>aetos/<app>:<tag></code> and <code>:latest</code>.</li> <li><code>push-image</code> \u2014 login, re-tag with <code>$REGISTRY_HOST</code>, push both tags.</li> <li><code>stop-old</code> \u2014 <code>docker rm -f <app></code>.</li> <li><code>run-container</code> \u2014 <code>docker run -d --name <app> -p <host>:<container> --restart always</code> with the service's <code>-e</code> environment.</li> </ul> <p>Secrets: <code>PIP_REPO_URL</code>, <code>PIP_USERNAME</code>, <code>PIP_PASSWORD</code> (service build args), <code>REGISTRY_HOST</code>, <code>REGISTRY_USER</code>, <code>REGISTRY_PASS</code>, plus the service runtime env (for example <code>MONGO_*</code>, <code>JWT_SECRET</code>).</p> <p>Placeholders to fill in per repo:</p> Placeholder Meaning Example <code><app></code> service image / container name <code>auth-server</code> <code>aetos/<app></code> image repository on the registry <code>aetos/auth-server</code> <code><host-port></code> / <code><container-port></code> published : in-image port <code>9003:8000</code> <code><ENV_VAR></code> runtime environment variable <code>MONGO_HOST</code>"},{"location":"90_drone_pipelines/#what-not-to-do","title":"What not to do","text":"<ul> <li>Do not hand-roll a fourth shape. A repo is exactly one of Type A, B, or C, never combinations.</li> <li>Do not inline secrets; always use <code>from_secret</code>.</li> <li>Do not drop the <code>check-version</code> / <code>check-remote-image</code> skip \u2014 it guards the index and the running container against duplicate publishes.</li> <li>Do not drop the tag\u2194version guard. A Python package needs its <code>pyproject.toml</code> version to equal the tag; an npm package needs its <code>package.json</code> version; a service needs a root <code>VERSION</code> file. See 91 \u2013 Versioning & Changelog.</li> <li>Do not rename the pipelines; tooling and runbooks key off <code>build-and-publish-pypi</code>, <code>build-and-publish-npm</code>, <code>build-and-deploy-service</code>, and their <code>backfill-*-from-tags</code> companions.</li> </ul>"},{"location":"90_drone_pipelines/#related","title":"Related","text":"<ul> <li>01 \u2013 Build Recipe \u00b7 30 \u2013 Infra Adapters \u00b7 91 \u2013 Versioning & Changelog \u00b7 93 \u2013 Library Registry</li> </ul>"},{"location":"91_versioning_and_changelog/","title":"Versioning & Changelog","text":""},{"location":"91_versioning_and_changelog/#summary","title":"Summary","text":"<p>Every Aetos artifact \u2014 library or service \u2014 is versioned with Semantic Versioning and documents its releases in a Keep a Changelog file. The version file is the source of truth, the git tag must equal it, and a Drone guard enforces that equality before anything is published. Releases are cut manually; a tag is what triggers the pipeline.</p> <p>This page pairs with 90 \u2013 Drone Pipelines: the version and tag contract is exactly what those three CI shapes consume.</p>"},{"location":"91_versioning_and_changelog/#the-version-contract","title":"The version contract","text":"Artifact Source of truth Notes Library (PyPI package) <code>version</code> in <code>pyproject.toml</code> <code>check-version</code> also reads it to skip duplicates Library (npm package) <code>version</code> in <code>package.json</code> <code>check-version</code> also queries the registry to skip duplicates Service (Docker) <code>VERSION</code> file at the repo root one line, <code>x.y.z</code>; the service reads it (build-arg) \u2014 never hardcode the version <p>Rules:</p> <ol> <li>One version per repository. No per-module versions.</li> <li>Tag = version. The git tag is bare SemVer (<code>0.0.6</code>), no <code>v</code> prefix, and it MUST match the version file byte-for-byte. The Drone guard fails the build otherwise \u2014 this is what stops a tag from silently publishing nothing.</li> <li>A release is a tag, created after the change is merged. The version bump and the changelog promotion travel inside the merge request (MR); the tag is added on the merged commit and pushed. See Tagging strategy.</li> <li>Never reuse a tag. If a release is wrong, cut a new patch version.</li> </ol>"},{"location":"91_versioning_and_changelog/#pre-10-0yz","title":"Pre-1.0 (0.y.z)","text":"<p>While a package is <code>0.y.z</code>, <code>0.y</code> is treated as the major line:</p> <ul> <li>breaking change \u2192 <code>0.(y+1).0</code></li> <li>new feature \u2192 <code>0.y.(z+1)</code></li> <li>bug fix \u2192 <code>0.y.(z+1)</code></li> </ul> <p>Once a package reaches <code>1.0.0</code>, normal SemVer applies. Do not ship <code>1.0.0</code> until the public API is considered stable.</p>"},{"location":"91_versioning_and_changelog/#the-changelog","title":"The changelog","text":"<p><code>CHANGELOG.md</code> at the repo root, in Keep a Changelog form. It always has exactly one open <code>[Unreleased]</code> section with the change categories that apply:</p> <pre><code>## [Unreleased]\n\n### Added\n### Changed\n### Deprecated\n### Removed\n### Fixed\n### Security\n</code></pre> <p>On release you promote it \u2014 never delete history:</p> <pre><code>## [0.0.6] - 2026-03-07\n\n### Added\n- ...\n</code></pre> <p>and add a link reference at the bottom:</p> <pre><code>[Unreleased]: https://git.aetoskia.com/aetos/<repo>/compare/0.0.6...HEAD\n[0.0.6]: https://git.aetoskia.com/aetos/<repo>/compare/0.0.5...0.0.6\n</code></pre> <p>Conventions:</p> <ul> <li>Write entries for humans: one line each, imperative, referencing the user impact, not the commit hash.</li> <li>Categories: Added, Changed, Deprecated, Removed, Fixed, Security. Omit empty categories.</li> <li>Security fixes get their own entry even if the change is otherwise a bug fix.</li> <li>Keep <code>[Unreleased]</code> current: add an entry in the same change that introduces the behaviour, not later.</li> <li>The <code>[Unreleased]</code> heading is never renamed; copy its content into the new version heading and clear it.</li> </ul>"},{"location":"91_versioning_and_changelog/#reconcile-before-releasing","title":"Reconcile before releasing","text":"<p><code>[Unreleased]</code> is a draft, not the source of truth. It is only trustworthy when it was maintained alongside each change; a long-running branch or a change that skipped the changelog leaves it stale, so it can silently under-report a release. Verify it against the change before you promote:</p> <ol> <li>List what actually changed since the last release: <code>git log <last-tag>..HEAD</code> and <code>git diff --stat <last-tag>..HEAD</code>.</li> <li>Compare that against <code>[Unreleased]</code>; add an entry for every user-facing change that is missing.</li> <li>Always show the proposed additions to the user and wait for approval before promoting. If an entry is ambiguous, ask rather than guess.</li> </ol> <p>Only then promote. Never cut a release with a known gap between the diff and the changelog.</p>"},{"location":"91_versioning_and_changelog/#tagging-strategy-mr-based","title":"Tagging strategy (MR-based)","text":"<p>The release version is known while the change is still a merge request, so the bump travels with the change and only the tag is added afterwards:</p> <ol> <li>In the MR \u2014 reconcile <code>[Unreleased]</code> against the change (see Reconcile before releasing) and get the user's approval for any additions. Then set the intended release version: <code>version</code> in <code>pyproject.toml</code> or <code>package.json</code> (or the <code>VERSION</code> file) to <code>x.y.z</code>, and promote the changelog (below). Changelog entries and link references name the version/tag, never a commit id.</li> <li>Merge or squash the MR into <code>main</code>.</li> <li>Create the tag <code>x.y.z</code> on the resulting commit (the merge/squash commit).</li> <li>Push the tag (<code>git push origin x.y.z</code>) \u2014 Drone builds the release.</li> </ol> <p>Because the version bump is inside the tagged commit, the guard's <code>tag == version</code> check can only fail if the wrong commit was tagged.</p>"},{"location":"91_versioning_and_changelog/#agent-assisted-tagging","title":"Agent-assisted tagging","text":"<p>The agent neither authors nor merges the MR, and never tags before the merge. After the user confirms the MR is merged/squashed and says to proceed:</p> <ol> <li>Wait for that explicit go-ahead.</li> <li>Find the commit that introduced the version bump:</li> <li>Python packages: <code>git log -S 'version = \"x.y.z\"' -n 1 --format='%H %s' -- pyproject.toml</code></li> <li>npm packages: <code>git log -S '\"version\": \"x.y.z\"' -n 1 --format='%H %s' -- package.json</code></li> <li>services: <code>git log -S 'x.y.z' -n 1 --format='%H %s' -- VERSION</code></li> <li>Confirm that commit's version file equals <code>x.y.z</code>, then tag only that commit, lightweight and bare: <code>git tag x.y.z <sha></code>.</li> <li>Do not push. Ask the user to push the tag: <code>git push origin x.y.z</code>.</li> </ol>"},{"location":"91_versioning_and_changelog/#release-checklist","title":"Release checklist","text":"<ol> <li>Confirm the quality gate is green and <code>main</code> is up to date.</li> <li>In the MR: reconcile <code>[Unreleased]</code> against the change being released \u2014 <code>git log <last-tag>..HEAD</code> and <code>git diff --stat <last-tag>..HEAD</code> \u2014 add any missing user-facing entries, and get the user's approval for the additions. Then move every <code>[Unreleased]</code> entry under the new version heading with the target date (<code>## [x.y.z] - YYYY-MM-DD</code>) and restore an empty <code>[Unreleased]</code> on top.</li> <li>In the MR: add the two compare-link references at the bottom of the changelog, naming the version/tag (never a commit id).</li> <li>In the MR: set the version file \u2014 <code>pyproject.toml</code> (<code>version = \"x.y.z\"</code>), <code>package.json</code> (<code>\"version\": \"x.y.z\"</code>), or <code>VERSION</code> \u2014 to the intended release version.</li> <li>Merge or squash the MR into <code>main</code>.</li> <li>Create the tag on the resulting commit: lightweight <code>git tag x.y.z</code>.</li> <li>Push the tag (<code>git push origin x.y.z</code>). Watch Drone: Type A publishes the wheel/sdist, Type B publishes the npm package, Type C builds and deploys the image; all three first assert <code>tag == version</code>.</li> <li>Do not reuse the tag if the run fails \u2014 fix and cut <code>x.y.(z+1)</code>.</li> </ol>"},{"location":"91_versioning_and_changelog/#why-the-guard-matters","title":"Why the guard matters","text":"<p>The version file and the tag are two facts that must agree; nothing in git enforces that by itself. A mismatch is not a theoretical risk \u2014 a package can be tagged <code>0.0.2</code> while its <code>pyproject.toml</code> still says <code>0.0.1</code>, and the publish step (which compares the file against the index) then skips the build entirely, so <code>0.0.2</code> never reaches the index while looking like a release. The Drone guard turns that silent no-op into a hard failure.</p>"},{"location":"91_versioning_and_changelog/#related","title":"Related","text":"<ul> <li>90 \u2013 Drone Pipelines \u00b7 94 \u2013 Repository Inventory \u00b7 01 \u2013 Build Recipe \u00b7 93 \u2013 Library Registry</li> </ul>"},{"location":"92_genericity_matrix/","title":"Genericity Matrix","text":""},{"location":"92_genericity_matrix/#summary","title":"Summary","text":"<p>This page states precisely what is library-backed, what is template-provided, and what is app-specific and must be authored. Use it to scope a new build and to know, for any given capability, whether to call a library, copy template code, or write it yourself.</p>"},{"location":"92_genericity_matrix/#tier-a-library-backed-use-never-reimplement","title":"Tier A \u2014 Library-backed (use; never reimplement)","text":"<p>These capabilities already exist as internal libraries. Reach them through the library registry MCP (<code>find_library(capability)</code>) and read the library's own MCP for its API. Writing a private copy of any of these is a defect.</p> Capability Library MCP Persistence: repositories, cache, populate, transactions Mongo Ops (<code>mongo_ops</code>) <code>:8007</code> Dependency injection / composition Hexa (<code>hexa</code>) <code>:8008</code> Deterministic multi-step transformation engine DAG Pipe (<code>dagpipe</code>) <code>:8006</code> Spec-first HTTP edge, client, codegen OpenAPI First (<code>openapi_first</code>) <code>:8004</code> Format-agnostic fetch + parse (HTML, PDF, CSV, XLSX) Omniread (<code>omniread</code>) <code>:8003</code> Read-only email ingestion Mail Intake (<code>mail_intake</code>) <code>:8002</code> JWT auth primitives (register, login, introspect) Py JWT (<code>jwtlib</code>) <code>:8005</code> Documentation build (MkDocs + MCP) Doc Forge (<code>docforge</code>) <code>:8001</code>"},{"location":"92_genericity_matrix/#tier-b-template-provided-copy-from-templatesapp-then-adapt","title":"Tier B \u2014 Template-provided (copy from <code>templates/app</code>, then adapt)","text":"<p>These are reusable internal components shipped inside the template \u2014 they are not independent libraries. Copy them as-is, then rename/populate only what is app-specific. They give every app the same wiring shape as the reference app. Fetch them by kind with <code>get_app_component</code>, or the whole tree with <code>get_app_template</code>.</p> Component Template location Action Shipped Layering & dependency rule the package layout + import rules keep \u2705 Composition root shape <code>init.py</code> (<code>get_svc</code>/<code>get_db</code>, idempotency) keep \u2705 Spec-first web edge <code>interfaces/openapi.py</code>, thin handlers bound to <code>operationId</code> keep \u2705 CRUD slice pattern <code>domain/<resource>/{models.py, ports/db.py, ports/service.py, converters.py}</code> rename + fill \u2705 Generic CRUD service <code>infra/adapters/service.py</code> (<code>CrudsServiceAdaptor</code>) keep \u2705 Repository factory <code>infra/db/repos/mongo.py</code> (<code>_repo_factory</code>) keep \u2705 Worker poll + status loop <code>worker.py</code> keep \u2705 Progress bus <code>infra/adapters/sse.py</code> (keyed in-memory bus) + an app SSE endpoint keep \u2705 HTTP response cache <code>infra/http_cache.py</code> (<code>cache_response</code> + generation invalidation) keep \u2705 Key-value store <code>infra/adapters/kv.py</code> (port + in-memory + Redis adaptor) keep \u2705 Pipeline scaffold <code>pipelines/shared/{ports,loader,registry}.py</code> + <code>pipelines/<family>/</code> rename + fill \u2705 Operator scripts <code>scripts/</code> (<code>init()</code> / <code>get_svc()</code> / <code>shutdown()</code>) rename \u2705 Testing harness integration fakes + fixture-parity pattern keep \u2705 Project scaffolding <code>pyproject.toml</code>, <code>Dockerfile</code>, <code>docker-compose.yaml</code>, <code>.env.example</code>, <code>openapi.yaml</code> rename + fill \u2705"},{"location":"92_genericity_matrix/#tier-b-patterns-port-from-the-reference-app","title":"Tier B (patterns) \u2014 port from the reference app","text":"<p>A few Tier B capabilities are real and reusable, but are shaped by the app or the provider, so the template does not ship them verbatim \u2014 copying them blindly would carry app-specific names. Treat them as patterns: lift the shape from the reference app and rename per app.</p> Component Reference location Action Shipped Human-in-the-loop durable pause + question document + resolve endpoint + resume map port + reshape \u2717 LLM client pool provider-select + retry/rotation pool scaffold port + reshape \u2717"},{"location":"92_genericity_matrix/#tier-c-app-specific-must-be-authored","title":"Tier C \u2014 App-specific (must be authored)","text":"Artifact Why Resource models the business shape Repository populate/cache rules the relationships Service business logic domain behaviour Pipeline families the transformation graph Pattern/rule resources data-driven policy Fixtures & golden outputs test data"},{"location":"92_genericity_matrix/#the-two-axes","title":"The two axes","text":"<p>Everything app-specific reduces to two axes:</p> <ol> <li>Resource model \u2014 the set of resources, their fields, relationships, and the OpenAPI contract.</li> <li>Pipeline family \u2014 only if inputs require multi-step transformation.</li> </ol> <p>If you can specify those two axes, the rest of the app follows mechanically: Tier A is imported, Tier B is copied, and Tier C is the only thing you author.</p>"},{"location":"93_library_registry/","title":"Library Registry","text":""},{"location":"93_library_registry/#summary","title":"Summary","text":"<p>The Aetos ecosystem provides internal libraries for the shared concerns of every application. Use them. Writing a private copy of any capability below is a defect.</p> <p>Each library also exposes its own documentation as an MCP server. An AI agent can read a library's full API by connecting to its MCP and reading <code>docs://index</code>, <code>docs://nav</code>, and <code>docs://modules/{module}</code>.</p>"},{"location":"93_library_registry/#catalog","title":"Catalog","text":"Library Import Purpose Use instead of MCP DAG Pipe <code>dagpipe</code> Deterministic state-transformation DAG engine hand-rolled orchestration/topological ordering <code>:8006</code> Hexa <code>hexa</code> Declarative DI / composition-root tooling manual wiring, hand-written factories <code>:8008</code> Mongo Ops <code>mongo_ops</code> Async MongoDB repositories, cache, populate, transactions raw Motor CRUD and connection lifecycle <code>:8007</code> Omniread <code>omniread</code> Format-agnostic fetch + parse (HTML, PDF, CSV, XLSX) custom scrapers/parsers <code>:8003</code> OpenAPI First <code>openapi_first</code> Spec-first FastAPI bootstrap, client, codegen decorator routers and hand-written clients <code>:8004</code> Mail Intake <code>mail_intake</code> Provider-agnostic read-only email ingestion Gmail OAuth + MIME parsing <code>:8002</code> Py JWT <code>jwtlib</code> JWT auth primitives (register/login/introspect) password hashing + token code <code>:8005</code> Doc Forge <code>docforge</code> Docstring compiler \u2192 MkDocs + MCP docs custom doc generators <code>:8001</code> Doc Weaver <code>doc-weaver</code> JSDoc compiler \u2192 TypeDoc + MCP docs (React/TS) custom TypeScript doc generators <code>:8010</code> React Auth <code>react-auth</code> JWT-backed React auth UI: provider, login page, profile screens, bearer axios client hand-rolled auth contexts and token handling in React apps <code>:8011</code> React OpenAPI <code>react-openapi</code> Spec-driven React admin UI (CRUD, tables, forms, SSE) derived from an OpenAPI document per-resource hand-written admin screens <code>:8012</code> Auth Server (service) Central identity service (register/login/me/logout/introspect) building auth endpoints yourself spec: <code>docs://spec</code>"},{"location":"93_library_registry/#how-to-choose","title":"How to choose","text":"You need to\u2026 Use persist and query documents Mongo Ops compose dependencies Hexa run a multi-step transformation DAG Pipe read/parse an arbitrary input format Omniread expose a contract-first HTTP API OpenAPI First ingest email Mail Intake authenticate users / verify tokens Py JWT (+ Auth Server) generate documentation Doc Forge generate React/TypeScript documentation Doc Weaver authenticate a React front-end React Auth (+ Auth Server) build a React admin UI from an OpenAPI spec React OpenAPI"},{"location":"93_library_registry/#discovery","title":"Discovery","text":"<p>A single standards MCP exposes <code>find_library(capability)</code>, which returns the matching library, its import name, its \"use instead of\" guidance, and its MCP endpoint (or, for the auth-server service, a spec pointer). It also serves the pinned auth-server OpenAPI contract at <code>docs://spec</code>. Query it before writing any infrastructure.</p>"},{"location":"93_library_registry/#publishing-registry-auth","title":"Publishing & registry auth","text":"<p>Python packages publish to the private index (<code>pip.aetoskia.com</code>, <code>pypiserver</code>) and npm packages to the private registry (<code>npm.aetoskia.com</code>, Verdaccio). Both authenticate against the same <code>htpasswd</code> file (<code>/auth/htpasswd</code>) used by the Docker registry and the Traefik basic-auth middleware; registration is disabled, so the CI credentials (<code>PIP_USERNAME</code> / <code>PIP_PASSWORD</code>, <code>NPM_USERNAME</code> / <code>NPM_PASSWORD</code>) are one and the same account \u2014 there is no separate npm user. Verdaccio requires bcrypt htpasswd entries; see 90 \u2013 Drone Pipelines for the CI wiring.</p>"},{"location":"93_library_registry/#notes","title":"Notes","text":"<ul> <li>Hexa is a full library with its own MCP server on <code>:8008</code>: read <code>docs://index</code>, <code>docs://nav</code>, and <code>docs://modules/{module}</code> (e.g. <code>docs://modules/hexa.container</code>) to discover the DI surface \u2014 <code>build(cls, *, instances, config)</code> and the <code>PortNode</code> model.</li> <li>Auth Server is a service, not an importable library: it has no MCP bundle by design. Read its client contract from the pinned OpenAPI document via <code>docs://spec</code> and wire it by declaring a <code>bearerAuth</code> security scheme in the app's own spec (see Auth Integration); <code>py-jwt</code> covers the low-level token path.</li> <li>Doc Weaver is the TypeScript counterpart to Doc Forge: it reads JSDoc comments and emits a TypeDoc site plus the same doc-forge-shaped MCP bundle for React packages. Its convention is RDFC (the React Doc-forge Convention), the JSDoc parallel to GSDFC.</li> <li>React Auth and React OpenAPI are front-end libraries (TypeScript, React 18 + MUI + React Router peers), not Python services \u2014 they consume the Auth Server and an OpenAPI document rather than serving one. Each ships a wiki, a TypeDoc site and an MCP bundle like every other library above.</li> <li>Library MCP servers expose documentation resources (not executable tools): they tell an agent the available API surface so it does not invent code.</li> </ul>"},{"location":"94_repository_inventory/","title":"Repository Inventory","text":""},{"location":"94_repository_inventory/#summary","title":"Summary","text":"<p>The canonical list of Aetos repositories covered by the CI and versioning standards, with the artifact each one produces, its version source, and its Drone shape. Every entry maps to the contract in 91 \u2013 Versioning & Changelog and the three CI files in 90 \u2013 Drone Pipelines.</p> <p>The same data is published machine-readably as the <code>registry://repos</code> MCP resource (and the <code>list_repos</code> tool), sourced from <code>standards/src/repos.json</code>. Paths are relative to the <code>aetos/</code> workspace root.</p>"},{"location":"94_repository_inventory/#packages-type-a-pypi","title":"Packages (Type A \u2014 PyPI)","text":"<p>A wheel + sdist published to the private index. Version source is <code>pyproject.toml</code>; Drone file is <code>python-lib.yaml</code> (tag-triggered).</p> Repo Path Docs MCP server <code>dagpipe</code> <code>dagpipe</code> lib, wiki <code>dagpipe:8006</code> <code>doc-forge</code> <code>doc-forge</code> lib, wiki <code>docforge:8001</code> <code>hexa</code> <code>hexa</code> lib, wiki <code>hexa:8008</code> <code>mail-intake</code> <code>mail-intake</code> lib, wiki <code>mail_intake:8002</code> <code>mongo-ops</code> <code>mongo-ops</code> lib, wiki <code>mongo_ops:8007</code> <code>omniread</code> <code>omniread</code> lib, wiki <code>omniread:8003</code> <code>openapi-first</code> <code>openapi-first</code> lib, wiki <code>openapi_first:8004</code> <code>py-jwt</code> <code>py-jwt</code> lib, wiki <code>jwtlib:8005</code>"},{"location":"94_repository_inventory/#packages-type-b-npm","title":"Packages (Type B \u2014 npm)","text":"<p>A package published to the private npm registry (<code>npm.aetoskia.com</code>). Version source is <code>package.json</code>; Drone file is <code>npm-lib.yaml</code> (tag-triggered).</p> Repo Path Docs MCP server <code>doc-weaver</code> <code>doc-weaver</code> lib, wiki <code>doc-weaver:8010</code> <code>react-auth</code> <code>react/react-auth</code> lib, wiki <code>react-auth:8011</code> <code>react-openapi</code> <code>react/react-openapi</code> lib, wiki <code>react-openapi:8012</code>"},{"location":"94_repository_inventory/#services-type-c","title":"Services (Type C)","text":"<p>A container image pushed to the registry and run on the Docker host. Version source is the <code>VERSION</code> file; Drone file is <code>service-docker.yaml</code>.</p> Repo Path Trigger Docs <code>auth-server</code> <code>auth-server</code> tag api, lib, wiki <p>Tag-triggered and enforces <code>tag == VERSION</code>.</p>"},{"location":"94_repository_inventory/#required-files-per-repo","title":"Required files per repo","text":"Type Version source Changelog Drone package (PyPI) <code>pyproject.toml</code> <code>CHANGELOG.md</code> <code>python-lib.yaml</code> package (npm) <code>package.json</code> <code>CHANGELOG.md</code> <code>npm-lib.yaml</code> service <code>VERSION</code> <code>CHANGELOG.md</code> <code>service-docker.yaml</code> <p>Every covered repository must be able to satisfy <code>tag == version</code>: the version file (never a commit id) is bumped in the release MR, the MR is merged, and the tag is created on the merged commit.</p>"},{"location":"94_repository_inventory/#reading-this-at-runtime","title":"Reading this at runtime","text":"<ul> <li>Resource: <code>registry://repos</code></li> <li>Tool: <code>list_repos()</code></li> <li>Humans: this page (mirror of <code>standards/src/repos.json</code>).</li> </ul>"},{"location":"94_repository_inventory/#related","title":"Related","text":"<ul> <li>90 \u2013 Drone Pipelines \u00b7 91 \u2013 Versioning & Changelog \u00b7 93 \u2013 Library Registry</li> </ul>"}]} |