--module-is-source Support (#4)
# Merge Request: Flatten MkDocs Structure + `--module-is-source` Support ## Summary This MR introduces structural improvements to the MkDocs generation pipeline to: 1. Ensure a root `docs/index.md` always exists 2. Flatten documentation structure (remove `docs/<module>/` nesting by default) 3. Add support for `--module-is-source` to treat the module as the documentation root 4. Align navigation (`docforge.nav.yml`) with the new flat layout 5. Regenerate MCP artifacts to reflect updated signatures and docstrings This resolves static hosting issues (e.g., Nginx 403 due to missing `site/index.html`) and makes each generated MkDocs site deployable as a standalone static website. --- ## Motivation Previously, documentation was generated under: ``` docs/<module>/... ``` Which resulted in: ``` site/<module>/index.html ``` When deployed at `/libs/<project>/`, this caused: * Missing `site/index.html` * Nginx returning 403 for root access * Inconsistent static hosting behavior This MR corrects the architecture so each MkDocs build is a valid static site with a root entry point. --- ## Key Changes ### 1️⃣ Flattened Docs Structure **Before** ``` docs/docforge/index.md ``` **After** ``` docs/index.md ``` All documentation paths were updated accordingly: * `docs/docforge/cli/...` → `docs/cli/...` * `docs/docforge/models/...` → `docs/models/...` * `docs/docforge/renderers/...` → `docs/renderers/...` Navigation updated to match the flat layout. --- ### 2️⃣ Root Index Enforcement `MkDocsRenderer` now guarantees: * `docs/index.md` is always created * Parent `index.md` files are auto-generated if missing * Parent indexes link to child modules (idempotent behavior) This ensures: ``` site/index.html ``` Always exists after `mkdocs build`. --- ### 3️⃣ New CLI Flag: `--module-is-source` Added option: ``` --module-is-source ``` Behavior: * Treats the provided module as the documentation root * Removes the top-level module folder from generated paths * Prevents redundant nesting when the module corresponds to the source root Updated components: * `cli.commands.build` * `mkdocs_utils.generate_sources` * `MkDocsRenderer.generate_sources` * Stub files (`.pyi`) * MCP JSON artifacts --- ### 4️⃣ Navigation Spec Update `docforge.nav.yml` updated: **Before** ```yaml home: docforge/index.md ``` **After** ```yaml home: index.md ``` All group paths adjusted to remove `docforge/` prefix. --- ### 5️⃣ MkDocs Config Update `mkdocs.yml` updated to: * Move `site_name` below theme/plugins * Use flat navigation paths * Point Home to `index.md` --- ### 6️⃣ MCP Artifact Regeneration Updated: * Function signatures (new parameter) * Docstrings (reflect `module_is_source`) * Renderer metadata * Line numbers Ensures MCP output matches updated API. --- ## Architectural Outcome Each project now produces a **valid standalone static site**: ``` site/ index.html assets/ search/ ``` Safe for deployment under: ``` /libs/<project>/ ``` No Nginx rewrites required. No directory-index issues. No nested-site ambiguity. --- ## Backward Compatibility * Existing CLI usage remains valid * `--module-is-source` is optional * Navigation spec format unchanged (only paths adjusted) --- ## Deployment Impact After merge: * Each library can be deployed independently * Sites can be merged under a shared root without internal conflicts * Static hosting is predictable and production-safe --- ## Testing * Verified MkDocs build produces `site/index.html` * Verified navigation renders correctly * Verified parent index generation is idempotent * Regenerated MCP docs and validated schema consistency --- ## Result The documentation compiler now: * Produces structurally correct static sites * Supports flat and source-root modes * Eliminates 403 root issues * Scales cleanly across multiple repositories This aligns doc-forge with proper static-site architectural invariants. Reviewed-on: #4 Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com> Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
doc-forge
A renderer-agnostic Python documentation compiler that converts Python source code and docstrings into a structured, semantic documentation model and emits multiple downstream representations.
Features
- Single Source of Truth: Python source code and docstrings are the only authoritative input
- Renderer Agnosticism: MkDocs, Sphinx, MCP, or future renderers don't influence the core model
- Deterministic Output: Given the same codebase, outputs are reproducible
- AI-Native Documentation: Structured, queryable, and machine-consumable
- Library-First Design: All functionality accessible as a Python API
Installation
pip install doc-forge
Optional Dependencies
# For MkDocs rendering
pip install doc-forge[mkdocs]
# For Sphinx rendering
pip install doc-forge[sphinx]
# For MCP support
pip install doc-forge[mcp]
# For development
pip install doc-forge[dev]
Quick Start
Command Line Interface
# Generate MkDocs documentation
doc-forge generate --renderer mkdocs mypackage
# Build final HTML documentation
doc-forge build --renderer mkdocs mypackage
# Serve documentation locally
doc-forge serve --renderer mkdocs mypackage
# Export to MCP format
doc-forge export mypackage
# Start live MCP server
doc-forge server mypackage
Python API
from docforge.loaders import GriffeLoader
from docforge.renderers import MkDocsRenderer
from pathlib import Path
# Load your project
loader = GriffeLoader()
project = loader.load_project(["mypackage", "mypackage.utils"])
# Generate MkDocs sources
renderer = MkDocsRenderer()
renderer.generate_sources(project, Path("docs"))
# Build final documentation
from docforge.renderers.base import RendererConfig
config = RendererConfig(Path("docs"), project)
renderer.build(config)
Architecture
doc-forge follows this high-level architecture:
Python Source Code
↓
Introspection Layer (Griffe)
↓
Documentation Model (doc-forge core)
↓
Renderer / Exporter Layer
├── MkDocs
├── Sphinx
├── MCP (static JSON)
└── MCP Server (live)
Core Components
Documentation Model
- Project: Root container for all documentation
- Module: Represents Python modules
- DocObject: Base class for classes, functions, variables, etc.
- Navigation: Hierarchical structure for browsing
Renderers
- MkDocs Renderer: Generates Markdown with mkdocstrings directives
- Sphinx Renderer: Generates reStructuredText with autodoc directives
Exporters
- MCP Exporter: Creates static JSON bundles for machine consumption
- MCP Server: Live server for real-time documentation access
CLI Commands
generate
Generate renderer-specific source files without building final artifacts.
doc-forge generate --renderer mkdocs --out-dir docs mypackage
build
Build final documentation artifacts (HTML, etc.).
doc-forge build --renderer sphinx mypackage
serve
Start a local development server.
doc-forge serve --renderer mkdocs --port 9000 mypackage
export
Export to MCP format for machine consumption.
doc-forge export --out-dir mcp mypackage
server
Start live MCP server for real-time access.
doc-forge server --host 0.0.0.0 --port 8080 mypackage
Configuration
doc-forge is designed to work with minimal configuration. Most settings are derived automatically from your Python code structure.
MkDocs Configuration
The MkDocs renderer automatically generates mkdocs.yml with sensible defaults:
site_name: Your Project
plugins:
- mkdocstrings
theme:
name: material
Sphinx Configuration
The Sphinx renderer automatically generates conf.py with standard extensions:
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.viewcode',
'sphinx.ext.napoleon',
]
MCP Integration
doc-forge provides two ways to integrate with MCP (Model Context Protocol):
Static Export
doc-forge export mypackage
Creates a static JSON bundle in mcp/ directory that can be loaded by MCP clients.
Live Server
doc-forge server mypackage
Starts a live MCP server providing real-time access to documentation resources:
docs://index- Project metadatadocs://nav- Navigation structuredocs://module/{module}- Individual module data
Development
Setup
git clone https://github.com/doc-forge/doc-forge
cd doc-forge
pip install -e ".[dev]"
Running Tests
pytest
Code Quality
black docforge/
ruff check docforge/
mypy docforge/
License
MIT License - see LICENSE file for details.
Contributing
Contributions are welcome! Please see CONTRIBUTING.md for guidelines.
Philosophy
doc-forge is built on these core principles:
- Single Source of Truth: Python source code and docstrings are the only authoritative input
- Renderer Agnosticism: The core model contains no renderer-specific logic
- Deterministic Output: Same input always produces same output
- AI-Native Documentation: Documentation must be structured, queryable, and machine-consumable
- Library-First: All functionality must be accessible as a Python API
doc-forge turns Python code into structured knowledge and emits it through multiple human and machine interfaces.