docs-and-mcps (#3 )

Reviewed-on: #3 Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com> Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
2026-01-22 11:28:14 +00:00
2 changed files with 23 additions and 327 deletions
--- a/ADS.llm.md
+++ b/ADS.llm.md
@@ -1,327 +0,0 @@
 # doc-forge — Architecture & Design Specification
 **doc-forge** is a renderer-agnostic Python documentation compiler. It converts Python source code and docstrings into a structured, semantic documentation model and then emits multiple downstream representations, including:
 * Human-facing documentation sites (MkDocs, Sphinx)
 * Machine-facing documentation bundles (MCP JSON)
 * Live documentation APIs (MCP servers)
 This document is the **authoritative design and codebase specification** for the library. It is written to be both **LLM-friendly** and **developer-facing**, and should be treated as the canonical reference for implementation decisions.
 ---
 ## 1. Design Goals
 1. **Single Source of Truth**
   Python source code and docstrings are the only authoritative input.
 2. **Renderer Agnosticism**
   MkDocs, Sphinx, MCP, or future renderers must not influence the core model.
 3. **Deterministic Output**
   Given the same codebase, outputs must be reproducible.
 4. **AI-Native Documentation**
   Documentation must be structured, queryable, and machine-consumable.
 5. **Library-First, CLI-Second**
   All functionality must be accessible as a Python API. The CLI is a thin wrapper.
 ---
 ## 2. Core Mental Model
 ### Fundamental Abstraction
 > **The atomic unit of documentation is a Python import path**
 Examples:
 * `mail_intake`
 * `mail_intake.config`
 * `mail_intake.adapters.base`
 Files, Markdown, HTML, and JSON are *representations*, not documentation units.
 ---
 ## 3. High-Level Architecture
 ```
 Python Source Code
        ↓
 Introspection Layer (Griffe)
        ↓
 Documentation Model (doc-forge core)
        ↓
 Renderer / Exporter Layer
   ├── MkDocs
   ├── Sphinx
   ├── MCP (static JSON)
   └── MCP Server (live)
 ```
 Only the **Documentation Model** is shared across all outputs.
 ---
 ## 4. Package Layout (Proposed)
 ```
 docforge/
 ├── __init__.py
 ├── model/
 │   ├── project.py
 │   ├── module.py
 │   ├── object.py
 │   └── nav.py
 ├── loader/
 │   └── griffe_loader.py
 ├── renderers/
 │   ├── base.py
 │   ├── mkdocs.py
 │   └── sphinx.py
 ├── exporters/
 │   └── mcp.py
 ├── server/
 │   └── mcp_server.py
 ├── cli/
 │   └── main.py
 └── utils/
 ```
 ---
 ## 5. Documentation Model (Core)
 The documentation model is renderer-neutral and must not contain any MkDocs-, Sphinx-, or MCP-specific logic.
 ### 5.1 Project
 ```python
 class Project:
    name: str
    version: str | None
    modules: dict[str, Module]
    nav: Navigation
 ```
 ---
 ### 5.2 Module
 ```python
 class Module:
    path: str                # import path
    docstring: str | None
    members: dict[str, DocObject]
 ```
 ---
 ### 5.3 DocObject
 Represents classes, functions, variables, etc.
 ```python
 class DocObject:
    name: str
    kind: str                # class, function, attribute, module
    path: str
    signature: str | None
    docstring: str | None
    members: dict[str, DocObject]
 ```
 Private members (`_name`) are excluded by default.
 ---
 ### 5.4 Navigation
 ```python
 class Navigation:
    entries: list[NavEntry]
 class NavEntry:
    title: str
    module: str
 ```
 Navigation is derived, not authored.
 ---
 ## 6. Introspection Layer
 ### 6.1 Griffe Loader
 Griffe is the **only supported introspection backend**.
 Responsibilities:
 * Load modules by import path
 * Resolve docstrings, signatures, and members
 * Tolerate alias resolution failures
 Output: fully populated `Project` and `Module` objects.
 ---
 ## 7. Renderer Interface
 Renderers consume the documentation model and emit renderer-specific source trees.
 ```python
 class DocRenderer(Protocol):
    name: str
    def generate_sources(self, project: Project, out_dir: Path) -> None:
        """Generate renderer-specific source files."""
    def build(self, config: RendererConfig) -> None:
        """Build final artifacts (HTML, site, etc.)."""
    def serve(self, config: RendererConfig) -> None:
        """Serve documentation locally (optional)."""
 ```
 ---
 ## 8. MkDocs Renderer
 ### Source Generation
 * Emits `.md` files
 * One file per module
 * Uses `mkdocstrings` directives exclusively
 ```md
 # Config
 ::: mail_intake.config
 ```
 ### Build
 * Uses `mkdocs.commands.build`
 ### Serve
 * Uses `mkdocs.commands.serve`
 MkDocs-specific configuration lives outside the core model.
 ---
 ## 9. Sphinx Renderer
 ### Source Generation
 * Emits `.rst` files
 * Uses `autodoc` directives
 ```rst
 mail_intake.config
 ==================
 .. automodule:: mail_intake.config
   :members:
   :undoc-members:
 ```
 ### Build
 * Uses `sphinx.application.Sphinx` directly
 ### Serve
 * Optional (static build is sufficient)
 ---
 ## 10. MCP Exporter (Static)
 The MCP exporter bypasses renderers entirely.
 ### Output Structure
 ```
 mcp/
 ├── index.json
 ├── nav.json
 └── modules/
    └── package.module.json
 ```
 ### Design Principles
 * Alias-safe
 * Deterministic
 * Fully self-contained
 * No Markdown, HTML, or templates
 ---
 ## 11. MCP Server (Live)
 The MCP server exposes documentation as queryable resources.
 ### Resources
 * `docs://index`
 * `docs://nav`
 * `docs://module/{module}`
 ### Characteristics
 * Read-only
 * Stateless
 * Backed by MCP JSON bundle
 ---
 ## 12. CLI Design
 The CLI is a thin orchestration layer.
 ```bash
 doc-forge generate --renderer mkdocs
 doc-forge generate --renderer sphinx
 doc-forge build --renderer mkdocs
 doc-forge serve --renderer mkdocs
 doc-forge export mcp
 ```
 Renderer choice never affects the core model.
 ---
 ## 13. Explicit Non-Goals
 * Markdown authoring
 * Theme design
 * Runtime code execution
 * Code formatting or linting
 ---
 ## 14. Invariants (Must Never Break)
 1. Import paths are canonical identifiers
 2. Core model contains no renderer logic
 3. MCP does not depend on MkDocs or Sphinx
 4. Renderers do not introspect Python directly
 5. All outputs trace back to the same model
 ---
 ## 15. One-Line Definition
 > **doc-forge is a documentation compiler that turns Python code into structured knowledge and emits it through multiple human and machine interfaces.**
 ---
 *End of specification.*
--- a/docforge/init.py
+++ b/docforge/init.py
@@ -6,6 +6,29 @@ speed, flexibility, and beautiful output. It decouples the introspection of
 your code from the rendering process, allowing you to generate documentation
 for various platforms (starting with MkDocs) from a single internal models.
 ## Core Philosophy
 `doc-forge` operates on two fundamental principles:
 1.  **The Atomic Unit is a Python Import Path**: Documentation is organized around the semantic structure of your code (e.g., `mypackage.utils`), not the filesystem.
 2.  **The Documentation Compiler Paradigm**: We separate documentation into three distinct phases:
    -   **Front-end (Introspection)**: Static analysis of source code and docstrings.
    -   **Middle-end (Semantic Model)**: A renderer-neutral internal representation.
    -   **Back-end (Renderers)**: Generation of human-facing (MkDocs) or machine-facing (MCP) outputs.
 ## Documentation Design
 `doc-forge` is an "AI-Native" documentation compiler. To get the most out of it, design your docstrings with both humans and LLMs in mind:
 ### For Humans (Readability & Structure)
 - **`__init__.py` as Landing Pages**: Use the docstring of your package's `__init__.py` as the home page. Include overviews, installation instructions, and high-level examples here.
 - **Single Source of Truth**: Keep all technical details in docstrings. This ensures your MkDocs/Sphinx sites stay in sync with the code.
 - **Semantic Hierarchy**: Use standard Markdown headers to structure complex module documentation.
 ### For LLMs (AI-Native Knowledge)
 - **Model Context Protocol (MCP)**: `doc-forge` exports your docs as structured JSON. This allows AI agents to "understand" your API surface area without layout noise.
 - **Canonical Paths**: Use dotted import paths as primary identifiers. AI tools use these to link code usage to documentation.
 - **Type Annotations**: While not in docstrings, `doc-forge` (via Griffe) extracts signatures. Clean type hints dramatically improve an LLM's ability to generate correct code using your library.
 ## Available Commands
 - **build**: Build documentation (MkDocs site or MCP resources).