aetos/doc-forge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Flatten MkDocs Structure + --module-is-source Support (#4)

Browse Source 
# Merge Request: Flatten MkDocs Structure + `--module-is-source` Support

## Summary

This MR introduces structural improvements to the MkDocs generation pipeline to:

1. Ensure a root `docs/index.md` always exists
2. Flatten documentation structure (remove `docs/<module>/` nesting by default)
3. Add support for `--module-is-source` to treat the module as the documentation root
4. Align navigation (`docforge.nav.yml`) with the new flat layout
5. Regenerate MCP artifacts to reflect updated signatures and docstrings

This resolves static hosting issues (e.g., Nginx 403 due to missing `site/index.html`) and makes each generated MkDocs site deployable as a standalone static website.

---

## Motivation

Previously, documentation was generated under:

```
docs/<module>/...
```

Which resulted in:

```
site/<module>/index.html
```

When deployed at `/libs/<project>/`, this caused:

* Missing `site/index.html`
* Nginx returning 403 for root access
* Inconsistent static hosting behavior

This MR corrects the architecture so each MkDocs build is a valid static site with a root entry point.

---

## Key Changes

### 1️⃣ Flattened Docs Structure

**Before**

```
docs/docforge/index.md
```

**After**

```
docs/index.md
```

All documentation paths were updated accordingly:

* `docs/docforge/cli/...` → `docs/cli/...`
* `docs/docforge/models/...` → `docs/models/...`
* `docs/docforge/renderers/...` → `docs/renderers/...`

Navigation updated to match the flat layout.

---

### 2️⃣ Root Index Enforcement

`MkDocsRenderer` now guarantees:

* `docs/index.md` is always created
* Parent `index.md` files are auto-generated if missing
* Parent indexes link to child modules (idempotent behavior)

This ensures:

```
site/index.html
```

Always exists after `mkdocs build`.

---

### 3️⃣ New CLI Flag: `--module-is-source`

Added option:

```
--module-is-source
```

Behavior:

* Treats the provided module as the documentation root
* Removes the top-level module folder from generated paths
* Prevents redundant nesting when the module corresponds to the source root

Updated components:

* `cli.commands.build`
* `mkdocs_utils.generate_sources`
* `MkDocsRenderer.generate_sources`
* Stub files (`.pyi`)
* MCP JSON artifacts

---

### 4️⃣ Navigation Spec Update

`docforge.nav.yml` updated:

**Before**

```yaml
home: docforge/index.md
```

**After**

```yaml
home: index.md
```

All group paths adjusted to remove `docforge/` prefix.

---

### 5️⃣ MkDocs Config Update

`mkdocs.yml` updated to:

* Move `site_name` below theme/plugins
* Use flat navigation paths
* Point Home to `index.md`

---

### 6️⃣ MCP Artifact Regeneration

Updated:

* Function signatures (new parameter)
* Docstrings (reflect `module_is_source`)
* Renderer metadata
* Line numbers

Ensures MCP output matches updated API.

---

## Architectural Outcome

Each project now produces a **valid standalone static site**:

```
site/
  index.html
  assets/
  search/
```

Safe for deployment under:

```
/libs/<project>/
```

No Nginx rewrites required.
No directory-index issues.
No nested-site ambiguity.

---

## Backward Compatibility

* Existing CLI usage remains valid
* `--module-is-source` is optional
* Navigation spec format unchanged (only paths adjusted)

---

## Deployment Impact

After merge:

* Each library can be deployed independently
* Sites can be merged under a shared root without internal conflicts
* Static hosting is predictable and production-safe

---

## Testing

* Verified MkDocs build produces `site/index.html`
* Verified navigation renders correctly
* Verified parent index generation is idempotent
* Regenerated MCP docs and validated schema consistency

---

## Result

The documentation compiler now:

* Produces structurally correct static sites
* Supports flat and source-root modes
* Eliminates 403 root issues
* Scales cleanly across multiple repositories

This aligns doc-forge with proper static-site architectural invariants.

Reviewed-on: #4
Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
This commit is contained in:
Vishesh 'ironeagle' Bangotra
2026-02-21 16:16:36 +00:00
committed by aetos
parent 56fb39de08
commit 22fceef020
38 changed files with 306 additions and 218 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docforge/cli/commands.py
View File

@@ -18,6 +18,7 @@ def cli() -> None:
@cli.command()
@click.option("--mcp", is_flag=True, help="Build MCP resources")
@click.option("--mkdocs", is_flag=True, help="Build MkDocs site")
@click.option("--module-is-source", is_flag=True, help="Module is source folder and to be treated as root folder")
@click.option("--module", help="Python module to document")
@click.option("--project-name", help="Project name override")
# MkDocs specific
@@ -32,6 +33,7 @@ def cli() -> None:
def build(
mcp: bool,
mkdocs: bool,
module_is_source: bool,
module: Optional[str],
project_name: Optional[str],
site_name: Optional[str],
@@ -52,7 +54,8 @@ def build(
Args:
mcp: Use the MCP documentation builder.
mkdocs: Use the MkDocs documentation builder.
module: The dotted path of the module to document.
module_is_source: Module is the source folder and to be treated as the root folder.
module: The dotted path of the module to the document.
project_name: Optional override for the project name.
site_name: (MkDocs) The site display name. Defaults to module name.
docs_dir: (MkDocs) Target directory for Markdown sources.
@@ -71,7 +74,12 @@ def build(
site_name = module
click.echo(f"Generating MkDocs sources in {docs_dir}...")
mkdocs_utils.generate_sources(module, project_name, docs_dir)
mkdocs_utils.generate_sources(
module,
docs_dir,
project_name,
module_is_source,
)
click.echo(f"Generating MkDocs config {mkdocs_yml}...")
mkdocs_utils.generate_config(docs_dir, nav_file, template, mkdocs_yml, site_name)

1
docforge/cli/commands.pyi
View File

@@ -7,6 +7,7 @@ cli: Group
def build(
mcp: bool,
mkdocs: bool,
module_is_source: bool,
module: Optional[str],
project_name: Optional[str],
site_name: Optional[str],

14
docforge/cli/mkdocs_utils.py
View File

@@ -6,7 +6,12 @@ from docforge.loaders import GriffeLoader, discover_module_paths
from docforge.renderers import MkDocsRenderer
from docforge.nav import load_nav_spec, resolve_nav, MkDocsNavEmitter
def generate_sources(module: str, project_name: str | None, docs_dir: Path) -> None:
def generate_sources(
module: str,
docs_dir: Path,
project_name: str | None = None,
module_is_source: bool | None = None,
) -> None:
"""
Generate Markdown source files for the specified module.
@@ -14,13 +19,18 @@ def generate_sources(module: str, project_name: str | None, docs_dir: Path) -> N
module: The dotted path of the primary module to document.
project_name: Optional override for the project name.
docs_dir: Directory where the generated Markdown files will be written.
module_is_source: Module is the source folder and to be treated as the root folder.
"""
loader = GriffeLoader()
discovered_paths = discover_module_paths(module)
project = loader.load_project(discovered_paths, project_name)
renderer = MkDocsRenderer()
renderer.generate_sources(project, docs_dir)
renderer.generate_sources(
project,
docs_dir,
module_is_source,
)
def generate_config(docs_dir: Path, nav_file: Path, template: Path | None, out: Path, site_name: str) -> None:
"""

7
docforge/cli/mkdocs_utils.pyi
View File

@@ -1,6 +1,11 @@
from pathlib import Path
def generate_sources(module: str, project_name: str | None, docs_dir: Path) -> None: ...
def generate_sources(
module: str,
docs_dir: Path,
project_name: str | None = None,
module_is_source: bool | None = None,
) -> None: ...
def generate_config(docs_dir: Path, nav_file: Path, template: Path | None, out: Path, site_name: str) -> None: ...
def build(mkdocs_yml: Path) -> None: ...
def serve(mkdocs_yml: Path) -> None: ...

109
docforge/renderers/mkdocs_renderer.py
View File

@@ -1,11 +1,16 @@
"""
This module implements the MkDocsRenderer, which generates Markdown source files
compatible with the MkDocs 'material' theme and 'mkdocstrings' extension.
MkDocsRenderer
Generates Markdown source files compatible with MkDocs Material
and mkdocstrings, ensuring:
- Root index.md always exists
- Parent package indexes are created automatically
- Child modules are linked in parent index files
"""
from pathlib import Path
from docforge.models import Project
from docforge.models import Project, Module
class MkDocsRenderer:
@@ -16,7 +21,15 @@ class MkDocsRenderer:
name = "mkdocs"
def generate_sources(self, project: Project, out_dir: Path) -> None:
# -------------------------
# Public API
# -------------------------
def generate_sources(
self,
project: Project,
out_dir: Path,
module_is_source: bool | None = None,
) -> None:
"""
Produce a set of Markdown files in the output directory based on the
provided Project models.
@@ -24,7 +37,11 @@ class MkDocsRenderer:
Args:
project: The project models to render.
out_dir: Target directory for documentation files.
module_is_source: Module is the source folder and to be treated as the root folder.
"""
out_dir.mkdir(parents=True, exist_ok=True)
self._ensure_root_index(project, out_dir)
modules = list(project.get_all_modules())
paths = {m.path for m in modules}
@@ -35,12 +52,23 @@ class MkDocsRenderer:
}
for module in modules:
self._write_module(module, packages, out_dir)
self._write_module(
module,
packages,
out_dir,
module_is_source,
)
# -------------------------
# Internal helpers
# -------------------------
def _write_module(self, module, packages: set[str], out_dir: Path) -> None:
def _write_module(
self,
module: Module,
packages: set[str],
out_dir: Path,
module_is_source: bool | None = None,
) -> None:
"""
Write a single module's documentation file. Packages are written as
'index.md' inside their respective directories.
@@ -49,30 +77,36 @@ class MkDocsRenderer:
module: The module to write.
packages: A set of module paths that are identified as packages.
out_dir: The base output directory.
module_is_source: Module is the source folder and to be treated as the root folder.
"""
parts = module.path.split(".")
if module_is_source:
module_name, parts = parts[0], parts[1:]
else:
module_name, parts = parts[0], parts
if module.path in packages:
# package → index.md
# Package → directory/index.md
dir_path = out_dir.joinpath(*parts)
dir_path.mkdir(parents=True, exist_ok=True)
md_path = dir_path / "index.md"
title = parts[-1].replace("_", " ").title()
link_target = f"{parts[-1]}/" if parts else None
else:
# leaf module → <name>.md
# Leaf module → parent_dir/<name>.md
dir_path = out_dir.joinpath(*parts[:-1])
dir_path.mkdir(parents=True, exist_ok=True)
md_path = dir_path / f"{parts[-1]}.md"
title = parts[-1].replace("_", " ").title()
link_target = f"{parts[-1]}.md" if parts else None
title = parts[-1].replace("_", " ").title() if parts else module_name
content = self._render_markdown(title, module.path)
if md_path.exists() and md_path.read_text(encoding="utf-8") == content:
return
if not md_path.exists() or md_path.read_text(encoding="utf-8") != content:
md_path.write_text(content, encoding="utf-8")
md_path.write_text(content, encoding="utf-8")
if not module_is_source:
self._ensure_parent_index(parts, out_dir, link_target, title)
def _render_markdown(self, title: str, module_path: str) -> str:
"""
@@ -89,3 +123,46 @@ class MkDocsRenderer:
f"# {title}\n\n"
f"::: {module_path}\n"
)
def _ensure_root_index(
self,
project: Project,
out_dir: Path
) -> None:
root_index = out_dir / "index.md"
if not root_index.exists():
root_index.write_text(
f"# {project.name}\n\n"
"## Modules\n\n",
encoding="utf-8",
)
def _ensure_parent_index(
self,
parts: list[str],
out_dir: Path,
link_target: str,
title: str,
) -> None:
if len(parts) == 1:
parent_index = out_dir / "index.md"
link = f"- [{title}]({link_target})\n"
else:
parent_dir = out_dir.joinpath(*parts[:-1])
parent_dir.mkdir(parents=True, exist_ok=True)
parent_index = parent_dir / "index.md"
link = f"- [{title}]({link_target})\n"
if not parent_index.exists():
parent_title = parts[-2].replace("_", " ").title()
parent_index.write_text(
f"# {parent_title}\n\n",
encoding="utf-8",
)
content = parent_index.read_text(encoding="utf-8")
if link not in content:
parent_index.write_text(content + link, encoding="utf-8")

18
docforge/renderers/mkdocs_renderer.pyi
View File

@@ -1,17 +1,27 @@
from pathlib import Path
from typing import Set
from docforge.models import Project, Module
class MkDocsRenderer:
name: str
def generate_sources(self, project: Project, out_dir: Path) -> None: ...
def generate_sources(
self,
project: Project,
out_dir: Path,
module_is_source: bool | None = None,
) -> None: ...
def _write_module(
self,
module: Module,
packages: Set[str],
packages: set[str],
out_dir: Path,
module_is_source: bool | None = None,
) -> None: ...
def _render_markdown(self, title: str, module_path: str) -> str: ...
def _ensure_root_index(self, project, out_dir) -> None: ...
def _ensure_parent_index(self, parts, out_dir, link_target, title) -> None: ...