docs(standards): mirror doc-forge wiki style; mermaid + copyable build prompt
- move templates into src/templates; renumber wiki pages; add app-components page - embed agentic build prompt as a copyable markdown block; add mermaid diagrams - build.py writes <name>/mcp; config static bundle -> standards/mcp - registry reads libraries.json as bytes (committed file was UTF-16LE); normalize to UTF-8 - Dockerfile: drop stale blog copy; per-repo <name>/mcp copies - drop nested standards/mcp/standards duplicate and empty root mcp/
This commit is contained in:
Binary file not shown.
@@ -2,7 +2,7 @@
|
||||
"module": "library_registry",
|
||||
"content": {
|
||||
"path": "library_registry",
|
||||
"docstring": "# Library Registry\n\n## Summary\n\nThe Aetos ecosystem provides internal libraries for the shared concerns of\nevery application. **Use them.** Writing a private copy of any capability below\nis a defect.\n\nEach library also exposes its own documentation as an MCP server. An AI agent\ncan read a library's full API by connecting to its MCP and reading\n`docs://index`, `docs://nav`, and `docs://modules/{module}`.\n\n---\n\n## Catalog\n\n| Library | Import | Purpose | Use instead of | MCP |\n|---|---|---|---|---|\n| DAG Pipe | `dagpipe` | Deterministic state-transformation DAG engine | hand-rolled orchestration/topological ordering | `:8006` |\n| Hexa | `hexa` | Declarative DI / composition-root tooling | manual wiring, hand-written factories | *(see note)* |\n| Mongo Ops | `mongo_ops` | Async MongoDB repositories, cache, populate, transactions | raw Motor CRUD and connection lifecycle | `:8007` |\n| Omniread | `omniread` | Format-agnostic fetch + parse (HTML, PDF, CSV, XLSX) | custom scrapers/parsers | `:8003` |\n| OpenAPI First | `openapi_first` | Spec-first FastAPI bootstrap, client, codegen | decorator routers and hand-written clients | `:8004` |\n| Mail Intake | `mail_intake` | Provider-agnostic read-only email ingestion | Gmail OAuth + MIME parsing | `:8002` |\n| Py JWT | `jwtlib` | JWT auth primitives (register/login/introspect) | password hashing + token code | `:8005` |\n| Doc Forge | `docforge` | Docstring compiler → MkDocs + MCP docs | custom doc generators | `:8001` |\n| Auth Server | *(service)* | Central identity service (register/login/me/introspect) | building auth endpoints yourself | — |\n\n## How to choose\n\n| You need to… | Use |\n|---|---|\n| persist and query documents | Mongo Ops |\n| compose dependencies | Hexa |\n| run a multi-step transformation | DAG Pipe |\n| read/parse an arbitrary input format | Omniread |\n| expose a contract-first HTTP API | OpenAPI First |\n| ingest email | Mail Intake |\n| authenticate users / verify tokens | Py JWT (+ Auth Server) |\n| generate documentation | Doc Forge |\n\n## Discovery\n\nA single **standards MCP** exposes `find_library(capability)`, which returns the\nmatching library, its import name, its \"use instead of\" guidance, and its MCP\nendpoint. Query it before writing any infrastructure.\n\n## Notes\n\n- **Hexa** ships an MCP bundle but is not yet assigned a dedicated port in the\n hub config; it is reachable via the standards registry entry and will be\n wired to a port.\n- Library MCP servers expose **documentation resources** (not executable tools):\n they tell an agent the available API surface so it does not invent code.\n",
|
||||
"docstring": "# Library Registry\n\n## Summary\n\nThe Aetos ecosystem provides internal libraries for the shared concerns of\nevery application. **Use them.** Writing a private copy of any capability below\nis a defect.\n\nEach library also exposes its own documentation as an MCP server. An AI agent\ncan read a library's full API by connecting to its MCP and reading\n`doc://index`, `doc://nav`, and `doc://modules/{module}`.\n\n---\n\n## Catalog\n\n| Library | Import | Purpose | Use instead of | MCP |\n|---|---|---|---|---|\n| DAG Pipe | `dagpipe` | Deterministic state-transformation DAG engine | hand-rolled orchestration/topological ordering | `:8006` |\n| Hexa | `hexa` | Declarative DI / composition-root tooling | manual wiring, hand-written factories | *(see note)* |\n| Mongo Ops | `mongo_ops` | Async MongoDB repositories, cache, populate, transactions | raw Motor CRUD and connection lifecycle | `:8007` |\n| Omniread | `omniread` | Format-agnostic fetch + parse (HTML, PDF, CSV, XLSX) | custom scrapers/parsers | `:8003` |\n| OpenAPI First | `openapi_first` | Spec-first FastAPI bootstrap, client, codegen | decorator routers and hand-written clients | `:8004` |\n| Mail Intake | `mail_intake` | Provider-agnostic read-only email ingestion | Gmail OAuth + MIME parsing | `:8002` |\n| Py JWT | `jwtlib` | JWT auth primitives (register/login/introspect) | password hashing + token code | `:8005` |\n| Doc Forge | `docforge` | Docstring compiler → MkDocs + MCP docs | custom doc generators | `:8001` |\n| Auth Server | *(service)* | Central identity service (register/login/me/introspect) | building auth endpoints yourself | — |\n\n## How to choose\n\n| You need to… | Use |\n|---|---|\n| persist and query documents | Mongo Ops |\n| compose dependencies | Hexa |\n| run a multi-step transformation | DAG Pipe |\n| read/parse an arbitrary input format | Omniread |\n| expose a contract-first HTTP API | OpenAPI First |\n| ingest email | Mail Intake |\n| authenticate users / verify tokens | Py JWT (+ Auth Server) |\n| generate documentation | Doc Forge |\n\n## Discovery\n\nA single **standards MCP** exposes `find_library(capability)`, which returns the\nmatching library, its import name, its \"use instead of\" guidance, and its MCP\nendpoint. Query it before writing any infrastructure.\n\n## Notes\n\n- **Hexa** ships an MCP bundle but is not yet assigned a dedicated port in the\n hub config; it is reachable via the standards registry entry and will be\n wired to a port.\n- Library MCP servers expose **documentation resources** (not executable tools):\n they tell an agent the available API surface so it does not invent code.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,6 +0,0 @@
|
||||
{
|
||||
"project": "standards",
|
||||
"type": "standards",
|
||||
"modules_count": 11,
|
||||
"source": "hand-authored"
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "composition_root",
|
||||
"content": {
|
||||
"path": "composition_root",
|
||||
"docstring": "# Composition Root\n\n## Summary\n\nAll wiring happens in exactly one module. It builds a **flat resource\nregistry** and exposes it to the HTTP layer and to scripts.\n\n---\n\n## The resource registry\n\n`init.py` returns a plain `dict[str, Any]`. Keys follow two conventions:\n\n- `<resource>_repo` — a repository adapter for a resource.\n- `<resource>_svc` — a service for a resource.\n\nPlus named infrastructure adaptors:\n\n- `http_cache_store` — response cache backend.\n- `progress_bus` — progress/event bus.\n- `<vendor>_<kind>` — fetchers, clients, stores.\n\nThe API stores the dict on `app.state.resources` during lifespan startup.\nHandlers read from it; nothing imports concrete adapters.\n\n## Shape\n\n```python\n_resources: dict = {}\n_initialized = False\n\nasync def init(settings=None) -> dict:\n global _initialized\n if _initialized:\n return _resources\n\n await setup_database(uri=settings.mongo_uri, db_name=settings.mongo_db)\n\n # 1. leaf repositories\n user_repo = create_user_repo()\n\n # 2. dependent repositories (dependencies passed by keyword)\n wallet_repo = create_wallet_repo(user_repo=user_repo)\n\n # 3. services (build via the DI library, injecting the repo)\n _resources[\"user_repo\"] = user_repo\n _resources[\"wallet_repo\"] = wallet_repo\n _resources[\"user_svc\"] = get_user_service(user_repo)\n _resources[\"wallet_svc\"] = get_wallet_service(wallet_repo)\n\n # 4. infrastructure adaptors (optional)\n _resources[\"http_cache_store\"] = HttpCacheStore(...)\n _resources[\"progress_bus\"] = ProgressEventBus()\n\n _initialized = True\n return _resources\n\nasync def shutdown() -> None:\n global _initialized\n await close_stores(_resources)\n await shutdown_database()\n _resources.clear()\n _initialized = False\n```\n\n## Typed accessors\n\nExpose a `get_svc(name)` helper with one `@overload` per key so scripts get\ntyping without importing concrete classes:\n\n```python\n@overload\ndef get_svc(name: Literal[\"user_svc\"]) -> UserService: ...\n@overload\ndef get_svc(name: Literal[\"wallet_svc\"]) -> WalletService: ...\n\ndef get_svc(name: str):\n return _resources[name]\n```\n\nAlso expose `get_db()` (raw database handle) and pure helpers such as\n`compute_fingerprint(payload)` (canonical hash used for deduplication).\n\n## Rules\n\n- `init()` is **idempotent**.\n- The composition root is the only place that imports concrete adapters.\n- Handlers never call `init()`; they read `request.app.state.resources`.\n- Scripts call `init()` / `shutdown()` around their work.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "crud_template",
|
||||
"content": {
|
||||
"path": "crud_template",
|
||||
"docstring": "# CRUD Template\n\n## Summary\n\nEvery resource follows the same four-file slice. Repetition is intentional: it\nmakes the codebase predictable and generatable.\n\n---\n\n## The slice\n\n```\ndomain/<resource>/\n models.py # pure Pydantic model (no DB fields)\n ports/\n db.py # document model + repository factory\n service.py # service class + factory\n converters.py # (optional) transforms\n```\n\n## models.py\n\nA framework-free Pydantic model describing the business shape.\n\n```python\nclass Item(BaseModel):\n name: str\n quantity: int = 0\n```\n\n## ports/db.py\n\nThe persistence model extends the business model and adds database concerns.\nThe repository factory selects the right repository flavour.\n\n```python\nclass ItemDocument(Item, BaseDocument): # BaseDocument adds id/created_at/...\n pass\n\ndef create_item_repo(*, owner_repo=None) -> ItemRepository:\n return repo_factory(\n collection=\"items\",\n document=ItemDocument,\n populate_rules=[PopulateRule(field=\"owner\", collection=\"users\")],\n cache=None, # or a CachedBaseRepository config\n )\n```\n\n## ports/service.py\n\nThe service is built through the DI library, injecting the repository. The\ngeneric CRUD base provides `create/read/update/patch/delete/search/...`.\n\n```python\nclass ItemService(CrudsServiceAdaptor[Item, ItemDocument]):\n model = Item\n schema = ItemDocument\n\ndef get_item_service(repo) -> ItemService:\n return build(ItemService, instances={\"repo\": repo})\n```\n\n## Naming conventions\n\n| Concept | Convention |\n|---|---|\n| collection | plural snake case (`items`) |\n| document | `<Resource>Document` |\n| repository factory | `create_<resource>_repo` |\n| service | `<Resource>Service` |\n| service factory | `get_<resource>_service` |\n| resource key | `<resource>_svc` |\n| operationId | `list_items`, `create_item`, ... |\n\n## Optional files\n\n- `converters.py` — pure functions mapping between domain types.\n- `ports/enrichment.py` — adapts a pipeline into an async callable the service\n can invoke (used when a resource is populated by a pipeline).\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "genericity_matrix",
|
||||
"content": {
|
||||
"path": "genericity_matrix",
|
||||
"docstring": "# Genericity Matrix\n\n## Summary\n\nThis page states precisely what is **reusable as-is**, what is a **template to\nadapt**, and what is **app-specific and must be authored**. Use it to scope a\nnew build.\n\n---\n\n## Reusable as-is (do not rewrite)\n\n| Area | Artifact |\n|---|---|\n| Layering & dependency rule | the package layout and import rules |\n| Composition root | the `init.py` shape, idempotency, `get_svc` overloading |\n| Web edge | `OpenAPIFirstApp` bootstrap, handler-name = `operationId`, thin handlers |\n| CRUD slice | the four-file pattern (model / ports-db / ports-service / converters) |\n| Generic CRUD service | the base service adaptor |\n| Repository factory | base / populating / cached selection |\n| Worker | the poll loop, backoff, retry, status machine |\n| Progress bus | the bus port + in-memory adapter + streaming endpoint |\n| Human-in-the-loop | the durable pause + question document + resolve endpoint + resume map |\n| Infra adaptors | HTTP cache decorator, KV store port, LLM pool scaffold |\n| Testing | integration harness with fakes, fixture-parity pattern |\n| Docs | GSDFC + generated stubs + MCP bundle pipeline |\n\n## Template to adapt (copy, then change names)\n\n| Artifact | Change |\n|---|---|\n| `pyproject.toml` | project name, dependencies |\n| `Dockerfile` / `docker-compose.yaml` | service name, ports, services |\n| `.env.example` | connection strings, provider keys |\n| `openapi.yaml` | every path/schema/operationId |\n| `scripts/` | seed and operator tasks |\n\n## App-specific (must be authored)\n\n| Artifact | Why |\n|---|---|\n| Resource models | the business shape |\n| Repository populate/cache rules | the relationships |\n| Service business logic | domain behaviour |\n| Pipeline families (if any) | the transformation graph |\n| Pattern/rule resources | data-driven policy |\n| Fixtures & golden outputs | test data |\n\n## The two axes\n\nEverything app-specific reduces to **two axes**:\n\n1. **Resource model** — the set of resources, their fields, relationships, and\n the OpenAPI contract.\n2. **Pipeline family** — only if inputs require multi-step transformation.\n\nIf you can specify those two axes, the rest of the app follows mechanically.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "human_in_loop",
|
||||
"content": {
|
||||
"path": "human_in_loop",
|
||||
"docstring": "# Human-in-the-Loop\n\n## Summary\n\nSome inputs are ambiguous and must be resolved by a human before processing can\ncontinue. The blueprint models this as a **durable pause**: an exception is\nraised, a question is recorded, the job pauses, and later resumes.\n\n---\n\n## The mechanism\n\n1. During processing, a stage raises a domain exception carrying the context\n needed to ask the question:\n\n ```python\n raise PauseForResolution(\n job_id=jid,\n reference=\"<stable key for this decision>\",\n options=[...],\n context={...},\n )\n ```\n\n2. The orchestrator catches it, records a **question document**, and re-raises\n so the worker marks the job `paused`.\n3. A client lists pending questions and answers one:\n\n ```\n GET /jobs/{id}/questions -> list pending questions\n POST /questions/{qid}/resolve {choice} -> record the choice\n ```\n\n4. When the **last** pending question is resolved, the resolve endpoint flips\n the parent job back to `pending`.\n5. The worker picks it up again, rebuilds a **resolution map** from previously\n answered questions, and seeds it into the pipeline so the same decision is\n not asked twice.\n\n## The question document\n\n```python\nclass Question(BaseDocument):\n job_id: str\n reference: str # stable key: stage|field|value\n status: str # pending | resolved\n options: list[dict]\n context: dict\n chosen: dict | None = None\n resolved_at: str | None = None\n```\n\n## Resume map\n\nThe worker turns answered questions into a lookup keyed by `reference`, then\ninjects it into the pipeline so that stage can skip re-raising:\n\n```python\nresolved = {q.reference: q.chosen for q in await svc.search(\n filter={\"job_id\": jid, \"status\": \"resolved\"})}\n\npipeline = build(root, instances={\"pause\": handler}, config={\"resolved\": resolved})\n```\n\n## Resolve endpoint\n\n```python\nasync def resolve_question(request, id, body):\n svc = request.app.state.resources[\"question_svc\"]\n q = await svc.read(id)\n await svc.update(id, {\"chosen\": body[\"chosen\"], \"status\": \"resolved\",\n \"resolved_at\": now()})\n\n pending = await svc.search(filter={\"job_id\": q.job_id, \"status\": \"pending\"},\n limit=1)\n if not pending:\n jobs = request.app.state.resources[\"job_svc\"]\n job = await jobs.read(q.job_id)\n if job and job.status == \"paused\":\n await jobs.update_status(q.job_id, \"pending\")\n return q\n```\n\n## CLI parity\n\nThe same flow is available through scripts/CLI: `list`, `get`, `create`,\n`questions`, `resolve`, `retry`.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "index",
|
||||
"content": {
|
||||
"path": "index",
|
||||
"docstring": "# Standards Reference\n\nThe reference half of the Standards description: the enforceable rules,\npatterns, and the internal library catalog that agents consult while coding.\n\n- **Layering Rules** — the strict request → domain → repository layering and\n the rules that enforce it.\n- **Composition Root** — a single composition point wiring the whole app.\n- **Spec-First Routing** — the OpenAPI contract as the routing source of truth.\n- **CRUD Template** — the standardized generic resource endpoint scaffold.\n- **Pipeline Scaffold** — the deterministic pipeline skeleton for multi-stage\n jobs.\n- **Worker & Progress** — queued work, progress reporting, and visibility.\n- **Human-in-the-Loop** — guardrails and approvals inside automations.\n- **Infra Adapters** — how infrastructure connects without leaking into domain.\n- **Genericity Matrix** — what is generic vs app-specific.\n- **Library Registry** — the catalog of internal libraries behind all of it.\n\nRead the **Wiki** for the philosophy and rationale; these pages are the\nreference that mirrors the `standards` MCP bundle.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "infra_adaptors",
|
||||
"content": {
|
||||
"path": "infra_adaptors",
|
||||
"docstring": "# Infra Adapters\n\n## Summary\n\nAll external systems are reached through adapters in `infra/`. Prefer an\nexisting internal library over writing a new adapter.\n\n---\n\n## Database (MongoDB)\n\n- A single connection manager owns the client lifecycle.\n `setup_database(uri, db_name)` / `shutdown_database()`.\n- A repository **factory** selects the repository flavour:\n\n ```python\n repo_factory(collection, document, engine=None,\n populate_rules=None, cache=None)\n ```\n\n - base repository — plain CRUD;\n - populating repository — resolves referenced documents (foreign keys);\n - cached repository — an LRU/TTL cache in front of reads.\n\n- Referenced documents are described declaratively:\n\n ```python\n PopulateRule(field=\"owner\", collection=\"users\",\n nested_rules=[], max_depth=2, projection=None)\n ```\n\n- Transactions use a session context manager provided by the library.\n\n**Library:** the Mongo operations library (see registry). Do not write raw\nMotor calls in the domain.\n\n## HTTP response cache\n\nA generic GET cache sits behind a decorator.\n\n```python\n@cache_response(ttl=60)\nasync def list_items(request): ...\n```\n\n- Keyed on path + query + user.\n- No-op when no store is wired.\n- `invalidate(*resources)` bumps a generation counter for write-triggered\n invalidation.\n\n## Event / progress bus\n\nA port with an in-memory default adapter (`publish`, `get_events`). Swap for a\ndurable broker in production (see Worker & Progress).\n\n## External clients (ingestion)\n\nExternal fetchers (files, mail, HTTP) are adapters that wrap ingestion\nlibraries. Construct them in the composition root and inject into\norchestrators.\n\n**Libraries:** the ingestion library (formats) and the mail library (email).\n\n## LLM clients\n\nWhen a pipeline calls a model provider, route through a pool adaptor that\nhandles provider selection, retries, and client lifecycle:\n\n```python\npool = setup_async_pool(provider=settings.llm_provider)\nresult = await pool.complete(model=\"...\", messages=[...])\n```\n\nProviders are selected by `LLM_PROVIDER` and configured by environment.\n\n## Key-value store\n\nFor report/snapshot storage, provide a pluggable store port with a Redis\nadapter and an in-memory fallback.\n\n## Rule\n\nIf a capability already exists as a library (DB, DI, ingestion, mail, auth,\ndocs), **use it**. Writing a private copy of a library is a defect.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "layering_rules",
|
||||
"content": {
|
||||
"path": "layering_rules",
|
||||
"docstring": "# Layering Rules\n\n## Summary\n\nThe dependency direction is fixed. Violating it is a design defect.\n\n---\n\n## Allowed imports\n\n```\ninterfaces/* -> domain/* (services, ports, models)\ninterfaces/* -> infra/adapters/* (only read-only value objects)\ndomain/* -> domain/* (same layer)\ndomain/* -> pipelines/* (call a pipeline via a port)\ninfra/adapters/* -> domain/ports/* (implement the interface)\nmain.py/init.py -> everything (composition root only)\n```\n\n## Forbidden imports\n\n```\ndomain/* -> interfaces/* (never)\ndomain/* -> fastapi/starlette (never)\ninfra/adapters/* -> interfaces/* (never)\ninterfaces/* -> infra/db/* (state lives in resources, not imports)\n```\n\n## Why\n\n- A pure domain is testable without a server, a database, or an event loop.\n- Adapters can be swapped (in-memory fake, real database) without touching\n business logic.\n- The HTTP edge stays thin and replaceable.\n\n## The port/adapter contract\n\nA **port** is an abstract class or protocol declared in `domain/<x>/ports/`.\nAn **adapter** implements it in `infra/adapters/`. The composition root binds\nthe two.\n\n```\ndomain/<resource>/ports/db.py -> repository port + document model\ndomain/<resource>/ports/service.py -> service + factory\ninfra/db/repos/<vendor>.py -> repository adapter factory\ninfra/adapters/service.py -> generic CRUD service base\n```\n\n## Enforcement\n\n- Keep every domain module importable with only the standard library and\n Pydantic. If a domain import pulls in `fastapi`, the layering is broken.\n- Reviewers (and CI) reject `import fastapi` inside `domain/` or `infra/`.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "library_registry",
|
||||
"content": {
|
||||
"path": "library_registry",
|
||||
"docstring": "# Library Registry\n\n## Summary\n\nThe Aetos ecosystem provides internal libraries for the shared concerns of\nevery application. **Use them.** Writing a private copy of any capability below\nis a defect.\n\nEach library also exposes its own documentation as an MCP server. An AI agent\ncan read a library's full API by connecting to its MCP and reading\n`docs://index`, `docs://nav`, and `docs://modules/{module}`.\n\n---\n\n## Catalog\n\n| Library | Import | Purpose | Use instead of | MCP |\n|---|---|---|---|---|\n| DAG Pipe | `dagpipe` | Deterministic state-transformation DAG engine | hand-rolled orchestration/topological ordering | `:8006` |\n| Hexa | `hexa` | Declarative DI / composition-root tooling | manual wiring, hand-written factories | *(see note)* |\n| Mongo Ops | `mongo_ops` | Async MongoDB repositories, cache, populate, transactions | raw Motor CRUD and connection lifecycle | `:8007` |\n| Omniread | `omniread` | Format-agnostic fetch + parse (HTML, PDF, CSV, XLSX) | custom scrapers/parsers | `:8003` |\n| OpenAPI First | `openapi_first` | Spec-first FastAPI bootstrap, client, codegen | decorator routers and hand-written clients | `:8004` |\n| Mail Intake | `mail_intake` | Provider-agnostic read-only email ingestion | Gmail OAuth + MIME parsing | `:8002` |\n| Py JWT | `jwtlib` | JWT auth primitives (register/login/introspect) | password hashing + token code | `:8005` |\n| Doc Forge | `docforge` | Docstring compiler → MkDocs + MCP docs | custom doc generators | `:8001` |\n| Auth Server | *(service)* | Central identity service (register/login/me/introspect) | building auth endpoints yourself | — |\n\n## How to choose\n\n| You need to… | Use |\n|---|---|\n| persist and query documents | Mongo Ops |\n| compose dependencies | Hexa |\n| run a multi-step transformation | DAG Pipe |\n| read/parse an arbitrary input format | Omniread |\n| expose a contract-first HTTP API | OpenAPI First |\n| ingest email | Mail Intake |\n| authenticate users / verify tokens | Py JWT (+ Auth Server) |\n| generate documentation | Doc Forge |\n\n## Discovery\n\nA single **standards MCP** exposes `find_library(capability)`, which returns the\nmatching library, its import name, its \"use instead of\" guidance, and its MCP\nendpoint. Query it before writing any infrastructure.\n\n## Notes\n\n- **Hexa** ships an MCP bundle but is not yet assigned a dedicated port in the\n hub config; it is reachable via the standards registry entry and will be\n wired to a port.\n- Library MCP servers expose **documentation resources** (not executable tools):\n they tell an agent the available API surface so it does not invent code.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "pipeline_scaffold",
|
||||
"content": {
|
||||
"path": "pipeline_scaffold",
|
||||
"docstring": "# Pipeline Scaffold\n\n## Summary\n\nData-driven processing is expressed as a **composition graph**. The graph is\nbuilt by a DI library; execution is driven by a DAG engine. Stages are pure and\nindividually testable.\n\n---\n\n## When to use a pipeline\n\nUse a pipeline when a resource is produced by a **multi-step transformation**\nof an input (a document, a file, a message) rather than by a simple CRUD write.\n\n## Structure of a family\n\nA **family** is one input shape (e.g. a file format or a message kind). Each\nfamily has the same skeleton:\n\n```\npipelines/<family>/\n abc.py # port contracts (stage interfaces + root interface)\n impl.py # concrete stages + root implementation\n pipeline.yaml # graph spec: nodes, edges, roots, schema\n resources/\n pipeline.yaml # runtime spec consumed by the loader\n patterns/*.yaml # rule fragments (pattern registry)\n banks/<variant>.py # optional variants that re-pin a few slots\n```\n\n## Shared scaffolding\n\n```\npipelines/shared/\n ports.py # RegistryPort, FinalizePort, etc.\n loader.py # build(root_cls) -> wires the graph + engine\n registry.py # resolve_root(...) -> root class for a (kind, variant)\n```\n\n## The loader\n\n```python\ndef build(root_cls, *, instances=None, config=None):\n # merge resources/patterns/*.yaml into a pattern registry\n # bind registry/config into each graph node by id\n # build the schema (state) class and the engine\n root = root_cls()\n root._engine = Engine(graph, state_cls, payload_cls)\n return root\n```\n\n## The registry\n\nA single lookup replaces hand-rolled `if/else` selection:\n\n```python\n_FAMILIES = {\n \"csv\": {\"acme\": {\"heuristic\": AcmeCsvPipeline}},\n \"json\": {\"globex\": {\"heuristic\": GlobexJsonPipeline}},\n}\n\ndef resolve_root(kind, variant, mode=\"heuristic\"):\n return _FAMILIES[kind][variant][mode]\n```\n\nAdding a new variant means adding one class and one registry entry.\n\n## Driving a pipeline\n\nStages are driven explicitly so intermediate state can be persisted:\n\n```python\npipeline = build(resolve_root(kind, variant))\ncontent = pipeline.ingest.materialize(raw_bytes)\nrecords = pipeline.reader.read(content)\nitems = pipeline.normalize.build(records)\n```\n\n## Testing\n\nBecause stages are pure, test each stage in isolation and test the assembled\nroot against **fixture inputs** whose expected outputs are stored as golden\nfiles. A parity test replays fixtures headlessly.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "spec_first_routing",
|
||||
"content": {
|
||||
"path": "spec_first_routing",
|
||||
"docstring": "# Spec-First Routing\n\n## Summary\n\nThe OpenAPI document is the single source of truth for the HTTP contract.\nHandlers are bound to it by name; there is no decorator router.\n\n---\n\n## The contract\n\n- `openapi.yaml` (or `.json`) declares paths, schemas, parameters, responses,\n and security.\n- A custom `operationId` names the handler function.\n- The framework (`openapi_first`) validates requests/responses against the\n spec and dispatches to the handler whose name equals `operationId`.\n\n## App bootstrap\n\n```python\n# main.py\nfrom openapi_first.app import OpenAPIFirstApp\n\napp = OpenAPIFirstApp(\n openapi_path=\"openapi.yaml\",\n routes_module=interfaces.openapi, # module whose namespace holds handlers\n title=\"<App> API\",\n version=\"0.1.0\",\n lifespan=lifespan,\n)\n```\n\n`lifespan` calls the composition root:\n\n```python\n@asynccontextmanager\nasync def lifespan(app):\n app.state.resources = await init()\n yield\n await shutdown()\n```\n\n## Handler registry\n\n`interfaces/openapi.py` is a flat re-export of every handler, so the framework\ncan resolve `operationId` across the whole app:\n\n```python\nfrom interfaces.item.routes import *\nfrom interfaces.wallet.routes import *\nfrom interfaces.progress.sse import *\n```\n\n## Handler shape\n\nHandlers are thin and synchronous with respect to business logic:\n\n```python\nasync def create_item(request: Request, body: dict):\n svc = request.app.state.resources[\"item_svc\"]\n return await svc.create(Item.model_validate(body))\n```\n\n## Conventions\n\n- `operationId` naming: `list_<resource>`, `create_<resource>`,\n `get_<resource>`, `update_<resource>`, `delete_<resource>`.\n- Read-only endpoints may use a cache decorator keyed on path + query + user.\n- Validation errors -> `400`; not found -> `404`; delete -> `204`.\n- Security is declared once in the spec and enforced by the framework; handlers\n contain no auth code.\n\n## Codegen\n\nThe `openapi_first` library can scaffold models and route stubs from the spec,\nand ships app templates (CRUD app, model app, service). Prefer codegen over\nhand-writing boilerplate.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"module": "worker_and_progress",
|
||||
"content": {
|
||||
"path": "worker_and_progress",
|
||||
"docstring": "# Worker & Progress\n\n## Summary\n\nLong-running work runs in a background worker. Jobs are durable documents that\nmove through an explicit status machine. Progress is streamed to clients.\n\n---\n\n## The status machine\n\n```\npending -> processing -> completed\n \\-> failed (retried up to N times)\n \\-> paused (waiting for human input)\n```\n\n- `pending` — queued, eligible for pickup.\n- `processing` — claimed by a worker.\n- `paused` — blocked on a human-in-the-loop question.\n- `failed` — error; `retry_count` increments until a max, then skipped.\n- `completed` — terminal success.\n\n## The poll loop\n\n```python\nasync def poll(resources):\n backoff = MIN_POLL\n while True:\n jobs = await job_svc.search(filter={\"status\": \"pending\"}, limit=100)\n backoff = MIN_POLL if jobs else min(backoff * 2, MAX_POLL)\n for job in jobs:\n if (job.retry_count or 0) >= MAX_RETRIES:\n continue\n await job_svc.update_status(job.id, \"processing\")\n try:\n await process(job, resources)\n except PauseForResolution:\n await job_svc.update_status(job.id, \"paused\")\n except Exception as e:\n await job_svc.update_status(\n job.id, \"failed\", error=str(e),\n retry_count=(job.retry_count or 0) + 1,\n )\n await asyncio.sleep(backoff)\n```\n\nTunables live in environment variables: `WORKER_POLL_MIN`, `WORKER_POLL_MAX`,\n`WORKER_MAX_RETRIES`.\n\n## Progress events\n\nA worker publishes structured progress events to a **bus** (a port). The HTTP\nedge exposes a streaming endpoint that replays events for a given job id.\n\n```python\nclass ProgressEvent(BaseModel):\n job_id: str\n step: str\n status: str # started | progress | completed | skipped | paused\n message: dict | None = None\n```\n\nThe bus is an in-memory adapter by default; because it is a port, it can be\nswapped for a durable broker. Emit from the orchestrator at each stage:\n\n```python\nawait self.bus.publish(ProgressEvent(job_id=jid, step=\"normalize\",\n status=\"completed\", message={\"count\": n}))\n```\n\n## Streaming endpoint\n\n```python\nasync def stream_job_events(request, id):\n bus = request.app.state.resources.get(\"progress_bus\")\n if bus is None:\n return {\"events\": []}\n\n async def gen():\n sent = 0\n while True:\n events = bus.get_events(id)\n for e in events[sent:]:\n yield f\"data: {e.model_dump_json(exclude_none=True)}\\n\\n\"\n sent += 1\n await asyncio.sleep(0.5)\n\n return StreamingResponse(gen(), media_type=\"text/event-stream\")\n```\n\n> **Known limitation.** The default in-memory bus is per-process. If the worker\n> and the API run as separate processes, events published by the worker are not\n> visible to the API. For production, back the bus with a shared broker. The\n> port boundary makes this a one-adapter change.\n",
|
||||
"objects": {}
|
||||
}
|
||||
}
|
||||
@@ -1,57 +0,0 @@
|
||||
[
|
||||
{
|
||||
"module": "layering_rules",
|
||||
"resource": "doc://modules/layering_rules",
|
||||
"title": "Layering Rules"
|
||||
},
|
||||
{
|
||||
"module": "composition_root",
|
||||
"resource": "doc://modules/composition_root",
|
||||
"title": "Composition Root"
|
||||
},
|
||||
{
|
||||
"module": "spec_first_routing",
|
||||
"resource": "doc://modules/spec_first_routing",
|
||||
"title": "Spec-First Routing"
|
||||
},
|
||||
{
|
||||
"module": "crud_template",
|
||||
"resource": "doc://modules/crud_template",
|
||||
"title": "CRUD Template"
|
||||
},
|
||||
{
|
||||
"module": "pipeline_scaffold",
|
||||
"resource": "doc://modules/pipeline_scaffold",
|
||||
"title": "Pipeline Scaffold"
|
||||
},
|
||||
{
|
||||
"module": "worker_and_progress",
|
||||
"resource": "doc://modules/worker_and_progress",
|
||||
"title": "Worker & Progress"
|
||||
},
|
||||
{
|
||||
"module": "human_in_loop",
|
||||
"resource": "doc://modules/human_in_loop",
|
||||
"title": "Human-in-the-Loop"
|
||||
},
|
||||
{
|
||||
"module": "infra_adaptors",
|
||||
"resource": "doc://modules/infra_adaptors",
|
||||
"title": "Infra Adapters"
|
||||
},
|
||||
{
|
||||
"module": "genericity_matrix",
|
||||
"resource": "doc://modules/genericity_matrix",
|
||||
"title": "Genericity Matrix"
|
||||
},
|
||||
{
|
||||
"module": "library_registry",
|
||||
"resource": "doc://modules/library_registry",
|
||||
"title": "Library Registry"
|
||||
},
|
||||
{
|
||||
"module": "index",
|
||||
"resource": "doc://modules/index",
|
||||
"title": "Standards Reference"
|
||||
}
|
||||
]
|
||||
Reference in New Issue
Block a user