aetos/docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(standards): codify codegen, ship generic Tier-B infra, add get_app_template + prompts; fix docs:// regressions

Browse Source 
P0: fix doc:// -> docs:// across standards src/server/registry and regenerate stale mongo-ops bundle. P1: build recipe + spec-first routing now document the openapi-first codegen fast path (models/routes with --inject-request/--package). P4: CRUD template documents the model-vs-document id rule (BaseDocument owns id). P5: authless guidance (omit security keys). P6: template gains generic infra (http_cache, keyed SSE bus, KV port+in-memory+Redis), interfaces/openapi.py + .dockerignore + generic build test registered as components, and a get_app_template tool; genericity matrix marks LLM pool + human-in-loop as reference-app patterns. P7: intake/build/start_app/trial prompts. P8: template ships tests/integration/test_openapi_app_builds.py. Rebuilt standards + openapi-first bundles and re-collected.
This commit is contained in:
Vishesh 'ironeagle' Bangotra
2026-10-13 03:26:51 +05:30
parent ef951a1913
commit fced40af59
109 changed files with 4082 additions and 5885 deletions
Download Patch File Download Diff File Expand all files Collapse all files

110
standards/src/lib/01_build_recipe.md
View File

@@ -3,10 +3,10 @@
## Summary
The ordered, deterministic checklist for generating a new application of the
standard shape. Every step is fixed; only the resource model (step 2) and the
pipeline family (step 7) need app-specific authoring.
standard shape. Every step is fixed; only the resource model (step 2), the
codegen pass (step 3), and the pipeline family (step 8) need app-specific work.
This page is served by the `standards` MCP as `doc://modules/build_recipe`. The
This page is served by the `standards` MCP as `docs://modules/build_recipe`. The
`start_app` prompt on that server collects the two axes and hands off here; pull
exact files with `get_app_component`.
@@ -42,15 +42,15 @@ pip install --extra-index-url \
# Build a new application from the Aetos Standards blueprint
You are coding a complete new backend application conforming to the Aetos
standard app blueprint. Execute every step in order. Only steps 2 and 3 use the
standard app blueprint. Execute every step in order. Only steps 2–4 use the
app-specific details provided by the user; everything else is deterministic.
## Before writing any code
Connect to the `standards` MCP server and read:
- `doc://nav` (the module list)
- `doc://modules/genericity_matrix` (Tier A library / Tier B template / Tier C authored)
- `doc://modules/layering_rules` (the dependency rules)
- `docs://nav` (the module list)
- `docs://modules/genericity_matrix` (Tier A library / Tier B template / Tier C authored)
- `docs://modules/layering_rules` (the dependency rules)
Then:
- call `find_library` with `db`, `di`, `routing`, `ingestion`, `pipeline`, `docs`
@@ -61,48 +61,64 @@ Then:
## Steps
1. **Scaffold** — copy `standards/src/templates/app/` into `apps/<your-app>/`.
Keep the tree: `domain/`, `infra/`, `interfaces/`, `pipelines/`, `tests/`,
`scripts/`, plus `pyproject.toml`, `openapi.yaml`, `config.py`, `init.py`,
`main.py`, `worker.py`, `Dockerfile`, `docker-compose.yaml`, `.env.example`.
1. **Scaffold** — copy `standards/src/templates/app/` into `apps/<your-app>/`
(or pull the same tree via `get_app_template`). Keep the tree: `domain/`,
`infra/`, `interfaces/`, `pipelines/`, `tests/`, `scripts/`, plus
`pyproject.toml`, `openapi.yaml`, `config.py`, `init.py`, `main.py`,
`worker.py`, `Dockerfile`, `docker-compose.yaml`, `.env.example`.
2. **Author the spec (app-specific)** — write `openapi.yaml`: resources,
schemas, `operationId`s, security, and extension metadata. This is the
contract and drives all routing.
contract and drives all routing. Every operation MUST have an `operationId`.
3. **Write each CRUD slice (app-specific)** — per resource create the four-file
slice: `models.py`, `ports/db.py`, `ports/service.py`, and optional
`converters.py`. Follow the CRUD template.
3. **Generate the scaffold (codegen)** — from the spec, generate the wire
models and the exact handler signatures:
```bash
openapi-first models openapi.yaml -o _gen/models.py
openapi-first routes openapi.yaml -o _gen/routes \
--use-models --models-module _gen.models --inject-request --package
```
`_gen/models.py` is the **wire contract** (Pydantic, one class per
`components.schemas` entry). `_gen/routes/<resource>/routes.py` lists every
`operationId` with its exact parameter signature. Treat `_gen/` as generated
(never hand-edit it) — the canonical handlers live in `interfaces/`.
4. **Wire the composition root** — in `init.py`, build leaf repositories, then
4. **Write each CRUD slice (app-specific)** — per resource create the four-file
slice: `models.py` (domain model, **no `id`**), `ports/db.py`
(`class XDocument(BaseDocument, X)` — id comes from `BaseDocument`),
`ports/service.py`, and optional `converters.py`. Follow the CRUD template.
5. **Wire the composition root** — in `init.py`, build leaf repositories, then
dependent repositories, then services; add infra adapters (db, cache, stores,
progress bus, fetchers). Expose `get_svc` overloads.
5. **Add routes** — create `interfaces/<resource>/routes.py` thin handlers bound
to `operationId`s and re-export them in `interfaces/openapi.py`. Add the cache
6. **Add routes** — create `interfaces/<resource>/routes.py` thin handlers whose
names match the `operationId`s (use the signatures from step 3 as your
checklist) and re-export them in `interfaces/openapi.py`. Add the cache
decorator to read endpoints.
6. **Bootstrap app and worker** — confirm `main.py` runs the OpenAPI-first app
7. **Bootstrap app and worker** — confirm `main.py` runs the OpenAPI-first app
with its lifespan. If the app has long-running jobs, adapt `worker.py` to poll
the job collection, run the pipeline, and publish progress.
7. **Add pipeline families (only if data-driven)** — copy `pipelines/example/`
8. **Add pipeline families (only if data-driven)** — copy `pipelines/example/`
to `pipelines/<family>/` with `abc.py`, `impl.py`, `pipeline.yaml`,
`resources/`, and reuse `pipelines/shared/` (loader + registry). Register the
root. Add fixtures and a parity test.
8. **Add infrastructure only where needed** — Tier A capability: use the
9. **Add infrastructure only where needed** — Tier A capability: use the
library. Tier B capability: copy the template. Never hand-write a Tier A
library.
9. **Tests** — integration tests with fakes at the HTTP edge; unit tests for
domain and pipeline logic; a config-consistency test for rule-driven
pipelines.
10. **Tests** — integration tests with fakes at the HTTP edge; unit tests for
domain and pipeline logic; a config-consistency test for rule-driven
pipelines. Include `tests/integration/test_openapi_app_builds.py` (imports
the app, asserts `import main` binds every operation).
10. **Scripts and seeds** — operator scripts under `scripts/` using
11. **Scripts and seeds** — operator scripts under `scripts/` using
`init()` / `get_svc()` / `shutdown()`; run as `python -m scripts.<name>`.
11. **Docs** — author GSDFC docstrings, generate `.pyi` stubs, mark the package
12. **Docs** — author GSDFC docstrings, generate `.pyi` stubs, mark the package
`py.typed`, and build the docs.
**Definition of done:**
@@ -123,15 +139,39 @@ The steps map onto the layers as follows:
|---------------------|-----------------------------------|---------------------------------------------|
| 1 · Scaffold | whole `templates/app/` | — |
| 2 · Spec | `openapi.yaml` | — |
| 3 · CRUD slices | `domain/<resource>/` | spec, CRUD template |
| 4 · Wiring | `init.py` | every slice |
| 5 · Routes | `interfaces/<resource>/` | services, openapi route registry |
| 6 · Bootstrap | `main.py`, `worker.py` | composition root |
| 7 · Pipelines | `pipelines/<family>/`, `shared/` | library pipeline engine |
| 8 · Infra | `infra/` | domain ports |
| 9 · Tests | `tests/` | app + fixtures |
| 10 · Scripts | `scripts/` | composition root |
| 11 · Docs | docstrings + docs build | code, Doc Forge |
| 3 · Codegen | `_gen/` (generated) | spec, openapi-first |
| 4 · CRUD slices | `domain/<resource>/` | spec, CRUD template |
| 5 · Wiring | `init.py` | every slice |
| 6 · Routes | `interfaces/<resource>/` | services, openapi route registry |
| 7 · Bootstrap | `main.py`, `worker.py` | composition root |
| 8 · Pipelines | `pipelines/<family>/`, `shared/` | library pipeline engine |
| 9 · Infra | `infra/` | domain ports |
| 10 · Tests | `tests/` | app + fixtures |
| 11 · Scripts | `scripts/` | composition root |
| 12 · Docs | docstrings + docs build | code, Doc Forge |
---
## 🔧 The codegen fast path, in detail
`openapi-first` is spec → code only (it never generates a spec). Use it to erase
the two mechanical parts of the build:
| Generated artifact | Replaces | Notes |
|--------------------|----------|-------|
| `_gen/models.py` | hand-writing Pydantic wire models | one class per `components.schemas` entry |
| `_gen/routes/<resource>/routes.py` | hand-typing handler signatures | names == `operationId`s; `request: Request` injected |
Rules of the road:
- **Wire models are not domain models.** `_gen/models.py` is the HTTP contract;
`domain/<resource>/models.py` is your business shape. If the spec's schema has
a required `id: str`, keep it off the domain model and let the document get its
id from `BaseDocument`.
- **Resource names are sanitized.** `/fetch-requests` → `fetch_requests.py`.
- **Multipart bodies expand** to `UploadFile = File(...)` / `Form(...)` params.
- **`_gen/` is disposable.** Regenerate on every spec change; `git diff` shows
exactly what moved.
---

32
standards/src/lib/04_spec_first_routing.md
View File

@@ -70,8 +70,34 @@ async def create_item(request: Request, body: dict):
- Security is declared once in the spec and enforced by the framework; handlers
contain no auth code.
## Authless apps
Auth is **deferred** in the standard: the same spec-first app runs with or
without it, because security lives entirely in the spec, never in handlers.
For an app that needs no auth yet, do **not** invent per-handler auth checks —
just omit the security keys:
- Remove the top-level `security:` block.
- Remove `components.securitySchemes`.
- Do **not** add `security: []` to operations. An explicit empty array is a
per-operation *override* that means "public", and it only matters when a
global `security` block exists. Leaving every operation without a `security`
key is the cleanest authless spec.
The handlers, services, and composition root are unchanged. When auth is needed
later, add the global `security` block and an introspecting dependency; only
handlers that must opt out get `security: []`.
## Codegen
The `openapi_first` library can scaffold models and route stubs from the spec,
and ships app templates (CRUD app, model app, service). Prefer codegen over
hand-writing boilerplate.
`openapi_first` scaffolds models and route stubs from the spec (see the
[Build Recipe](01_build_recipe.md), step 3):
```bash
openapi-first models openapi.yaml -o _gen/models.py
openapi-first routes openapi.yaml -o _gen/routes \
--use-models --models-module _gen.models --inject-request --package
```
Prefer codegen over hand-writing boilerplate. The result is the wire contract
and a set of `operationId`-named signatures — not the canonical handlers.

32
standards/src/lib/05_crud_template.md
View File

@@ -18,9 +18,36 @@ domain/<resource>/
converters.py # (optional) transforms
```
## The id rule (model vs. document)
Two different objects, two different jobs:
| Object | Lives in | Has `id`? | Purpose |
|---|---|---|---|
| **Domain model** | `domain/<resource>/models.py` | **No** | business shape, framework-free |
| **Document** | `domain/<resource>/ports/db.py` | **Yes** (from `BaseDocument`) | Mongo persistence shape |
`BaseDocument` (mongo-ops) supplies `id: Optional[PyObjectId]` (aliased to
`_id`) plus `created_at` / `updated_at`. The domain model must **not** declare
`id`; Mongo owns it.
> **Trap — generated wire models.** `openapi-first models` produces the HTTP
> *contract* (see the [Build Recipe](01_build_recipe.md), step 3). Response
> schemas frequently declare a required `id: str`, so a generated class is
> **not** a domain model. If you write
> `class ItemDocument(Item, BaseDocument)` and `Item` re-exports a required
> `id: str`, that field shadows `BaseDocument.id` and no Mongo `_id` can ever be
> set. Fix it one of three ways:
>
> 1. **Preferred:** keep the domain model id-free (hand-author it from the CRUD
> template), and keep the required `id` only on the wire model.
> 2. Order the bases so the document wins:
> `class ItemDocument(BaseDocument, Item)`.
> 3. Keep a separate document type and translate with `ports/converters.py`.
## models.py
A framework-free Pydantic model describing the business shape.
A framework-free Pydantic model describing the business shape (no `id`).
```python
class Item(BaseModel):
@@ -34,7 +61,8 @@ The persistence model extends the business model and adds database concerns.
The repository factory selects the right repository flavour.
```python
class ItemDocument(Item, BaseDocument): # BaseDocument adds id/created_at/...
# Item is id-free; BaseDocument adds id/created_at/updated_at.
class ItemDocument(Item, BaseDocument):
pass
def create_item_repo(*, owner_repo=None) -> ItemRepository:

48
standards/src/lib/12_genericity_matrix.md
View File

@@ -31,25 +31,37 @@ for its API. Writing a private copy of any of these is a defect.
These are reusable internal components shipped **inside** the template — they are
not independent libraries. Copy them as-is, then rename/populate only what is
app-specific. They give every app the same wiring shape as the reference app.
Fetch them by kind with `get_app_component`, or the whole tree with
`get_app_template`.
| Component | Template location | Action |
|---|---|---|
| Layering & dependency rule | the package layout + import rules | keep |
| Composition root shape | `init.py` (`get_svc`/`get_db`, idempotency) | keep |
| Spec-first web edge | `interfaces/openapi.py`, thin handlers bound to `operationId` | keep |
| CRUD slice pattern | `domain/<resource>/{models.py, ports/db.py, ports/service.py, converters.py}` | rename + fill |
| Generic CRUD service | `infra/adapters/service.py` (`CrudsServiceAdaptor`) | keep |
| Repository factory | `infra/db/repos/mongo.py` (`_repo_factory`) | keep |
| Worker poll + status loop | `worker.py` | keep |
| Progress bus + streaming | bus port + in-memory adapter + SSE endpoint | keep |
| Human-in-the-loop | durable pause + question document + resolve endpoint + resume map | keep |
| HTTP response cache | `cache_response` decorator + generation invalidation | keep |
| Key-value store | KV store port + Redis adapter + in-memory fallback | keep |
| LLM client pool | provider-select + retry pool scaffold | keep |
| Pipeline scaffold | `pipelines/shared/{ports,loader,registry}.py` + `pipelines/<family>/` | rename + fill |
| Operator scripts | `scripts/` (`init()` / `get_svc()` / `shutdown()`) | rename |
| Testing harness | integration fakes + fixture-parity pattern | keep |
| Project scaffolding | `pyproject.toml`, `Dockerfile`, `docker-compose.yaml`, `.env.example`, `openapi.yaml` | rename + fill |
| Component | Template location | Action | Shipped |
|---|---|---|---|
| Layering & dependency rule | the package layout + import rules | keep | ✅ |
| Composition root shape | `init.py` (`get_svc`/`get_db`, idempotency) | keep | ✅ |
| Spec-first web edge | `interfaces/openapi.py`, thin handlers bound to `operationId` | keep | ✅ |
| CRUD slice pattern | `domain/<resource>/{models.py, ports/db.py, ports/service.py, converters.py}` | rename + fill | ✅ |
| Generic CRUD service | `infra/adapters/service.py` (`CrudsServiceAdaptor`) | keep | ✅ |
| Repository factory | `infra/db/repos/mongo.py` (`_repo_factory`) | keep | ✅ |
| Worker poll + status loop | `worker.py` | keep | ✅ |
| Progress bus | `infra/adapters/sse.py` (keyed in-memory bus) + an app SSE endpoint | keep | ✅ |
| HTTP response cache | `infra/http_cache.py` (`cache_response` + generation invalidation) | keep | ✅ |
| Key-value store | `infra/adapters/kv.py` (port + in-memory + Redis adaptor) | keep | ✅ |
| Pipeline scaffold | `pipelines/shared/{ports,loader,registry}.py` + `pipelines/<family>/` | rename + fill | ✅ |
| Operator scripts | `scripts/` (`init()` / `get_svc()` / `shutdown()`) | rename | ✅ |
| Testing harness | integration fakes + fixture-parity pattern | keep | ✅ |
| Project scaffolding | `pyproject.toml`, `Dockerfile`, `docker-compose.yaml`, `.env.example`, `openapi.yaml` | rename + fill | ✅ |
## Tier B (patterns) — port from the reference app
A few Tier B capabilities are real and reusable, but are **shaped by the app or
the provider**, so the template does not ship them verbatim — copying them
blindly would carry app-specific names. Treat them as patterns: lift the shape
from the reference app and rename per app.
| Component | Reference location | Action | Shipped |
|---|---|---|---|
| Human-in-the-loop | durable pause + question document + resolve endpoint + resume map | port + reshape | ✗ |
| LLM client pool | provider-select + retry/rotation pool scaffold | port + reshape | ✗ |
## Tier C — App-specific (must be authored)

6
standards/src/lib/13_library_registry.md
View File

@@ -8,7 +8,7 @@ is a defect.
Each library also exposes its own documentation as an MCP server. An AI agent
can read a library's full API by connecting to its MCP and reading
`doc://index`, `doc://nav`, and `doc://modules/{module}`.
`docs://index`, `docs://nav`, and `docs://modules/{module}`.
---
@@ -48,8 +48,8 @@ endpoint. Query it before writing any infrastructure.
## Notes
- **Hexa** is a full library with its own MCP server on `:8008`: read
`doc://index`, `doc://nav`, and `doc://modules/{module}` (e.g.
`doc://modules/hexa.container`) to discover the DI surface — `build(cls, *,
`docs://index`, `docs://nav`, and `docs://modules/{module}` (e.g.
`docs://modules/hexa.container`) to discover the DI surface — `build(cls, *,
instances, config)` and the `PortNode` model.
- **Auth Server** is deferred: it has no `pyproject.toml`/importable package and
no MCP bundle. Do not reach for it yet; `py-jwt` covers token primitives.