aetos/docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(standards): codify codegen, ship generic Tier-B infra, add get_app_template + prompts; fix docs:// regressions

Browse Source 
P0: fix doc:// -> docs:// across standards src/server/registry and regenerate stale mongo-ops bundle. P1: build recipe + spec-first routing now document the openapi-first codegen fast path (models/routes with --inject-request/--package). P4: CRUD template documents the model-vs-document id rule (BaseDocument owns id). P5: authless guidance (omit security keys). P6: template gains generic infra (http_cache, keyed SSE bus, KV port+in-memory+Redis), interfaces/openapi.py + .dockerignore + generic build test registered as components, and a get_app_template tool; genericity matrix marks LLM pool + human-in-loop as reference-app patterns. P7: intake/build/start_app/trial prompts. P8: template ships tests/integration/test_openapi_app_builds.py. Rebuilt standards + openapi-first bundles and re-collected.
This commit is contained in:
Vishesh 'ironeagle' Bangotra
2026-10-13 03:26:51 +05:30
parent ef951a1913
commit fced40af59
109 changed files with 4082 additions and 5885 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
standards/mcp/modules/build_recipe.json
View File

File diff suppressed because one or more lines are too long

2
standards/mcp/modules/crud_template.json
View File

@@ -2,7 +2,7 @@
"module": "crud_template",
"content": {
"path": "crud_template",
"docstring": "# CRUD Template\n\n## Summary\n\nEvery resource follows the same four-file slice. Repetition is intentional: it\nmakes the codebase predictable and generatable.\n\n---\n\n## The slice\n\n```\ndomain/<resource>/\n models.py # pure Pydantic model (no DB fields)\n ports/\n db.py # document model + repository factory\n service.py # service class + factory\n converters.py # (optional) transforms\n```\n\n## models.py\n\nA framework-free Pydantic model describing the business shape.\n\n```python\nclass Item(BaseModel):\n name: str\n quantity: int = 0\n```\n\n## ports/db.py\n\nThe persistence model extends the business model and adds database concerns.\nThe repository factory selects the right repository flavour.\n\n```python\nclass ItemDocument(Item, BaseDocument): # BaseDocument adds id/created_at/...\n pass\n\ndef create_item_repo(*, owner_repo=None) -> ItemRepository:\n return repo_factory(\n collection=\"items\",\n document=ItemDocument,\n populate_rules=[PopulateRule(field=\"owner\", collection=\"users\")],\n cache=None, # or a CachedBaseRepository config\n )\n```\n\n## ports/service.py\n\nThe service is built through the DI library, injecting the repository. The\ngeneric CRUD base provides `create/read/update/patch/delete/search/...`.\n\n```python\nclass ItemService(CrudsServiceAdaptor[Item, ItemDocument]):\n model = Item\n schema = ItemDocument\n\ndef get_item_service(repo) -> ItemService:\n return build(ItemService, instances={\"repo\": repo})\n```\n\n## Naming conventions\n\n| Concept | Convention |\n|---|---|\n| collection | plural snake case (`items`) |\n| document | `<Resource>Document` |\n| repository factory | `create_<resource>_repo` |\n| service | `<Resource>Service` |\n| service factory | `get_<resource>_service` |\n| resource key | `<resource>_svc` |\n| operationId | `list_items`, `create_item`, ... |\n\n## Optional files\n\n- `converters.py` — pure functions mapping between domain types.\n- `ports/enrichment.py` — adapts a pipeline into an async callable the service\n can invoke (used when a resource is populated by a pipeline).\n",
"docstring": "# CRUD Template\n\n## Summary\n\nEvery resource follows the same four-file slice. Repetition is intentional: it\nmakes the codebase predictable and generatable.\n\n---\n\n## The slice\n\n```\ndomain/<resource>/\n models.py # pure Pydantic model (no DB fields)\n ports/\n db.py # document model + repository factory\n service.py # service class + factory\n converters.py # (optional) transforms\n```\n\n## The id rule (model vs. document)\n\nTwo different objects, two different jobs:\n\n| Object | Lives in | Has `id`? | Purpose |\n|---|---|---|---|\n| **Domain model** | `domain/<resource>/models.py` | **No** | business shape, framework-free |\n| **Document** | `domain/<resource>/ports/db.py` | **Yes** (from `BaseDocument`) | Mongo persistence shape |\n\n`BaseDocument` (mongo-ops) supplies `id: Optional[PyObjectId]` (aliased to\n`_id`) plus `created_at` / `updated_at`. The domain model must **not** declare\n`id`; Mongo owns it.\n\n> **Trap — generated wire models.** `openapi-first models` produces the HTTP\n> *contract* (see the [Build Recipe](01_build_recipe.md), step 3). Response\n> schemas frequently declare a required `id: str`, so a generated class is\n> **not** a domain model. If you write\n> `class ItemDocument(Item, BaseDocument)` and `Item` re-exports a required\n> `id: str`, that field shadows `BaseDocument.id` and no Mongo `_id` can ever be\n> set. Fix it one of three ways:\n>\n> 1. **Preferred:** keep the domain model id-free (hand-author it from the CRUD\n> template), and keep the required `id` only on the wire model.\n> 2. Order the bases so the document wins:\n> `class ItemDocument(BaseDocument, Item)`.\n> 3. Keep a separate document type and translate with `ports/converters.py`.\n\n## models.py\n\nA framework-free Pydantic model describing the business shape (no `id`).\n\n```python\nclass Item(BaseModel):\n name: str\n quantity: int = 0\n```\n\n## ports/db.py\n\nThe persistence model extends the business model and adds database concerns.\nThe repository factory selects the right repository flavour.\n\n```python\n# Item is id-free; BaseDocument adds id/created_at/updated_at.\nclass ItemDocument(Item, BaseDocument):\n pass\n\ndef create_item_repo(*, owner_repo=None) -> ItemRepository:\n return repo_factory(\n collection=\"items\",\n document=ItemDocument,\n populate_rules=[PopulateRule(field=\"owner\", collection=\"users\")],\n cache=None, # or a CachedBaseRepository config\n )\n```\n\n## ports/service.py\n\nThe service is built through the DI library, injecting the repository. The\ngeneric CRUD base provides `create/read/update/patch/delete/search/...`.\n\n```python\nclass ItemService(CrudsServiceAdaptor[Item, ItemDocument]):\n model = Item\n schema = ItemDocument\n\ndef get_item_service(repo) -> ItemService:\n return build(ItemService, instances={\"repo\": repo})\n```\n\n## Naming conventions\n\n| Concept | Convention |\n|---|---|\n| collection | plural snake case (`items`) |\n| document | `<Resource>Document` |\n| repository factory | `create_<resource>_repo` |\n| service | `<Resource>Service` |\n| service factory | `get_<resource>_service` |\n| resource key | `<resource>_svc` |\n| operationId | `list_items`, `create_item`, ... |\n\n## Optional files\n\n- `converters.py` — pure functions mapping between domain types.\n- `ports/enrichment.py` — adapts a pipeline into an async callable the service\n can invoke (used when a resource is populated by a pipeline).\n",
"objects": {}
}
}

2
standards/mcp/modules/genericity_matrix.json
View File

@@ -2,7 +2,7 @@
"module": "genericity_matrix",
"content": {
"path": "genericity_matrix",
"docstring": "# Genericity Matrix\n\n## Summary\n\nThis page states precisely what is **library-backed**, what is\n**template-provided**, and what is **app-specific and must be authored**. Use it\nto scope a new build and to know, for any given capability, whether to call a\nlibrary, copy template code, or write it yourself.\n\n---\n\n## Tier A — Library-backed (use; never reimplement)\n\nThese capabilities already exist as internal libraries. Reach them through the\nlibrary registry MCP (`find_library(capability)`) and read the library's own MCP\nfor its API. Writing a private copy of any of these is a defect.\n\n| Capability | Library | MCP |\n|---|---|---|\n| Persistence: repositories, cache, populate, transactions | Mongo Ops (`mongo_ops`) | `:8007` |\n| Dependency injection / composition | Hexa (`hexa`) | `:8008` |\n| Deterministic multi-step transformation engine | DAG Pipe (`dagpipe`) | `:8006` |\n| Spec-first HTTP edge, client, codegen | OpenAPI First (`openapi_first`) | `:8004` |\n| Format-agnostic fetch + parse (HTML, PDF, CSV, XLSX) | Omniread (`omniread`) | `:8003` |\n| Read-only email ingestion | Mail Intake (`mail_intake`) | `:8002` |\n| JWT auth primitives (register, login, introspect) | Py JWT (`jwtlib`) | `:8005` |\n| Documentation build (MkDocs + MCP) | Doc Forge (`docforge`) | `:8001` |\n\n## Tier B — Template-provided (copy from `templates/app`, then adapt)\n\nThese are reusable internal components shipped **inside** the template — they are\nnot independent libraries. Copy them as-is, then rename/populate only what is\napp-specific. They give every app the same wiring shape as the reference app.\n\n| Component | Template location | Action |\n|---|---|---|\n| Layering & dependency rule | the package layout + import rules | keep |\n| Composition root shape | `init.py` (`get_svc`/`get_db`, idempotency) | keep |\n| Spec-first web edge | `interfaces/openapi.py`, thin handlers bound to `operationId` | keep |\n| CRUD slice pattern | `domain/<resource>/{models.py, ports/db.py, ports/service.py, converters.py}` | rename + fill |\n| Generic CRUD service | `infra/adapters/service.py` (`CrudsServiceAdaptor`) | keep |\n| Repository factory | `infra/db/repos/mongo.py` (`_repo_factory`) | keep |\n| Worker poll + status loop | `worker.py` | keep |\n| Progress bus + streaming | bus port + in-memory adapter + SSE endpoint | keep |\n| Human-in-the-loop | durable pause + question document + resolve endpoint + resume map | keep |\n| HTTP response cache | `cache_response` decorator + generation invalidation | keep |\n| Key-value store | KV store port + Redis adapter + in-memory fallback | keep |\n| LLM client pool | provider-select + retry pool scaffold | keep |\n| Pipeline scaffold | `pipelines/shared/{ports,loader,registry}.py` + `pipelines/<family>/` | rename + fill |\n| Operator scripts | `scripts/` (`init()` / `get_svc()` / `shutdown()`) | rename |\n| Testing harness | integration fakes + fixture-parity pattern | keep |\n| Project scaffolding | `pyproject.toml`, `Dockerfile`, `docker-compose.yaml`, `.env.example`, `openapi.yaml` | rename + fill |\n\n## Tier C — App-specific (must be authored)\n\n| Artifact | Why |\n|---|---|\n| Resource models | the business shape |\n| Repository populate/cache rules | the relationships |\n| Service business logic | domain behaviour |\n| Pipeline families | the transformation graph |\n| Pattern/rule resources | data-driven policy |\n| Fixtures & golden outputs | test data |\n\n---\n\n## The two axes\n\nEverything app-specific reduces to **two axes**:\n\n1. **Resource model** — the set of resources, their fields, relationships, and\n the OpenAPI contract.\n2. **Pipeline family** — only if inputs require multi-step transformation.\n\nIf you can specify those two axes, the rest of the app follows mechanically:\nTier A is imported, Tier B is copied, and Tier C is the only thing you author.\n",
"docstring": "# Genericity Matrix\n\n## Summary\n\nThis page states precisely what is **library-backed**, what is\n**template-provided**, and what is **app-specific and must be authored**. Use it\nto scope a new build and to know, for any given capability, whether to call a\nlibrary, copy template code, or write it yourself.\n\n---\n\n## Tier A — Library-backed (use; never reimplement)\n\nThese capabilities already exist as internal libraries. Reach them through the\nlibrary registry MCP (`find_library(capability)`) and read the library's own MCP\nfor its API. Writing a private copy of any of these is a defect.\n\n| Capability | Library | MCP |\n|---|---|---|\n| Persistence: repositories, cache, populate, transactions | Mongo Ops (`mongo_ops`) | `:8007` |\n| Dependency injection / composition | Hexa (`hexa`) | `:8008` |\n| Deterministic multi-step transformation engine | DAG Pipe (`dagpipe`) | `:8006` |\n| Spec-first HTTP edge, client, codegen | OpenAPI First (`openapi_first`) | `:8004` |\n| Format-agnostic fetch + parse (HTML, PDF, CSV, XLSX) | Omniread (`omniread`) | `:8003` |\n| Read-only email ingestion | Mail Intake (`mail_intake`) | `:8002` |\n| JWT auth primitives (register, login, introspect) | Py JWT (`jwtlib`) | `:8005` |\n| Documentation build (MkDocs + MCP) | Doc Forge (`docforge`) | `:8001` |\n\n## Tier B — Template-provided (copy from `templates/app`, then adapt)\n\nThese are reusable internal components shipped **inside** the template — they are\nnot independent libraries. Copy them as-is, then rename/populate only what is\napp-specific. They give every app the same wiring shape as the reference app.\nFetch them by kind with `get_app_component`, or the whole tree with\n`get_app_template`.\n\n| Component | Template location | Action | Shipped |\n|---|---|---|---|\n| Layering & dependency rule | the package layout + import rules | keep | ✅ |\n| Composition root shape | `init.py` (`get_svc`/`get_db`, idempotency) | keep | ✅ |\n| Spec-first web edge | `interfaces/openapi.py`, thin handlers bound to `operationId` | keep | ✅ |\n| CRUD slice pattern | `domain/<resource>/{models.py, ports/db.py, ports/service.py, converters.py}` | rename + fill | ✅ |\n| Generic CRUD service | `infra/adapters/service.py` (`CrudsServiceAdaptor`) | keep | ✅ |\n| Repository factory | `infra/db/repos/mongo.py` (`_repo_factory`) | keep | ✅ |\n| Worker poll + status loop | `worker.py` | keep | ✅ |\n| Progress bus | `infra/adapters/sse.py` (keyed in-memory bus) + an app SSE endpoint | keep | ✅ |\n| HTTP response cache | `infra/http_cache.py` (`cache_response` + generation invalidation) | keep | ✅ |\n| Key-value store | `infra/adapters/kv.py` (port + in-memory + Redis adaptor) | keep | ✅ |\n| Pipeline scaffold | `pipelines/shared/{ports,loader,registry}.py` + `pipelines/<family>/` | rename + fill | ✅ |\n| Operator scripts | `scripts/` (`init()` / `get_svc()` / `shutdown()`) | rename | ✅ |\n| Testing harness | integration fakes + fixture-parity pattern | keep | ✅ |\n| Project scaffolding | `pyproject.toml`, `Dockerfile`, `docker-compose.yaml`, `.env.example`, `openapi.yaml` | rename + fill | ✅ |\n\n## Tier B (patterns) — port from the reference app\n\nA few Tier B capabilities are real and reusable, but are **shaped by the app or\nthe provider**, so the template does not ship them verbatim — copying them\nblindly would carry app-specific names. Treat them as patterns: lift the shape\nfrom the reference app and rename per app.\n\n| Component | Reference location | Action | Shipped |\n|---|---|---|---|\n| Human-in-the-loop | durable pause + question document + resolve endpoint + resume map | port + reshape | ✗ |\n| LLM client pool | provider-select + retry/rotation pool scaffold | port + reshape | ✗ |\n\n## Tier C — App-specific (must be authored)\n\n| Artifact | Why |\n|---|---|\n| Resource models | the business shape |\n| Repository populate/cache rules | the relationships |\n| Service business logic | domain behaviour |\n| Pipeline families | the transformation graph |\n| Pattern/rule resources | data-driven policy |\n| Fixtures & golden outputs | test data |\n\n---\n\n## The two axes\n\nEverything app-specific reduces to **two axes**:\n\n1. **Resource model** — the set of resources, their fields, relationships, and\n the OpenAPI contract.\n2. **Pipeline family** — only if inputs require multi-step transformation.\n\nIf you can specify those two axes, the rest of the app follows mechanically:\nTier A is imported, Tier B is copied, and Tier C is the only thing you author.\n",
"objects": {}
}
}

2
standards/mcp/modules/spec_first_routing.json
View File

@@ -2,7 +2,7 @@
"module": "spec_first_routing",
"content": {
"path": "spec_first_routing",
"docstring": "# Spec-First Routing\n\n## Summary\n\nThe OpenAPI document is the single source of truth for the HTTP contract.\nHandlers are bound to it by name; there is no decorator router.\n\n---\n\n## The contract\n\n- `openapi.yaml` (or `.json`) declares paths, schemas, parameters, responses,\n and security.\n- A custom `operationId` names the handler function.\n- The framework (`openapi_first`) validates requests/responses against the\n spec and dispatches to the handler whose name equals `operationId`.\n\n## App bootstrap\n\n```python\n# main.py\nfrom openapi_first.app import OpenAPIFirstApp\n\napp = OpenAPIFirstApp(\n openapi_path=\"openapi.yaml\",\n routes_module=interfaces.openapi, # module whose namespace holds handlers\n title=\"<App> API\",\n version=\"0.1.0\",\n lifespan=lifespan,\n)\n```\n\n`lifespan` calls the composition root:\n\n```python\n@asynccontextmanager\nasync def lifespan(app):\n app.state.db_resources = await init()\n yield\n await shutdown()\n```\n\n## Handler registry\n\n`interfaces/openapi.py` is a flat re-export of every handler, so the framework\ncan resolve `operationId` across the whole app:\n\n```python\nfrom interfaces.item.routes import *\nfrom interfaces.wallet.routes import *\nfrom interfaces.progress.sse import *\n```\n\n## Handler shape\n\nHandlers are thin and synchronous with respect to business logic:\n\n```python\nasync def create_item(request: Request, body: dict):\n svc = request.app.state.db_resources[\"item_svc\"]\n return await svc.create(Item.model_validate(body))\n```\n\n## Conventions\n\n- `operationId` naming: `list_<resource>`, `create_<resource>`,\n `get_<resource>`, `update_<resource>`, `delete_<resource>`.\n- Read-only endpoints may use a cache decorator keyed on path + query + user.\n- Validation errors -> `400`; not found -> `404`; delete -> `204`.\n- Security is declared once in the spec and enforced by the framework; handlers\n contain no auth code.\n\n## Codegen\n\nThe `openapi_first` library can scaffold models and route stubs from the spec,\nand ships app templates (CRUD app, model app, service). Prefer codegen over\nhand-writing boilerplate.\n",
"docstring": "# Spec-First Routing\n\n## Summary\n\nThe OpenAPI document is the single source of truth for the HTTP contract.\nHandlers are bound to it by name; there is no decorator router.\n\n---\n\n## The contract\n\n- `openapi.yaml` (or `.json`) declares paths, schemas, parameters, responses,\n and security.\n- A custom `operationId` names the handler function.\n- The framework (`openapi_first`) validates requests/responses against the\n spec and dispatches to the handler whose name equals `operationId`.\n\n## App bootstrap\n\n```python\n# main.py\nfrom openapi_first.app import OpenAPIFirstApp\n\napp = OpenAPIFirstApp(\n openapi_path=\"openapi.yaml\",\n routes_module=interfaces.openapi, # module whose namespace holds handlers\n title=\"<App> API\",\n version=\"0.1.0\",\n lifespan=lifespan,\n)\n```\n\n`lifespan` calls the composition root:\n\n```python\n@asynccontextmanager\nasync def lifespan(app):\n app.state.db_resources = await init()\n yield\n await shutdown()\n```\n\n## Handler registry\n\n`interfaces/openapi.py` is a flat re-export of every handler, so the framework\ncan resolve `operationId` across the whole app:\n\n```python\nfrom interfaces.item.routes import *\nfrom interfaces.wallet.routes import *\nfrom interfaces.progress.sse import *\n```\n\n## Handler shape\n\nHandlers are thin and synchronous with respect to business logic:\n\n```python\nasync def create_item(request: Request, body: dict):\n svc = request.app.state.db_resources[\"item_svc\"]\n return await svc.create(Item.model_validate(body))\n```\n\n## Conventions\n\n- `operationId` naming: `list_<resource>`, `create_<resource>`,\n `get_<resource>`, `update_<resource>`, `delete_<resource>`.\n- Read-only endpoints may use a cache decorator keyed on path + query + user.\n- Validation errors -> `400`; not found -> `404`; delete -> `204`.\n- Security is declared once in the spec and enforced by the framework; handlers\n contain no auth code.\n\n## Authless apps\n\nAuth is **deferred** in the standard: the same spec-first app runs with or\nwithout it, because security lives entirely in the spec, never in handlers.\nFor an app that needs no auth yet, do **not** invent per-handler auth checks —\njust omit the security keys:\n\n- Remove the top-level `security:` block.\n- Remove `components.securitySchemes`.\n- Do **not** add `security: []` to operations. An explicit empty array is a\n per-operation *override* that means \"public\", and it only matters when a\n global `security` block exists. Leaving every operation without a `security`\n key is the cleanest authless spec.\n\nThe handlers, services, and composition root are unchanged. When auth is needed\nlater, add the global `security` block and an introspecting dependency; only\nhandlers that must opt out get `security: []`.\n\n## Codegen\n\n`openapi_first` scaffolds models and route stubs from the spec (see the\n[Build Recipe](01_build_recipe.md), step 3):\n\n```bash\nopenapi-first models openapi.yaml -o _gen/models.py\nopenapi-first routes openapi.yaml -o _gen/routes \\\n --use-models --models-module _gen.models --inject-request --package\n```\n\nPrefer codegen over hand-writing boilerplate. The result is the wire contract\nand a set of `operationId`-named signatures — not the canonical handlers.\n",
"objects": {}
}
}