# Merge Request: Flatten MkDocs Structure + `--module-is-source` Support ## Summary This MR introduces structural improvements to the MkDocs generation pipeline to: 1. Ensure a root `docs/index.md` always exists 2. Flatten documentation structure (remove `docs/<module>/` nesting by default) 3. Add support for `--module-is-source` to treat the module as the documentation root 4. Align navigation (`docforge.nav.yml`) with the new flat layout 5. Regenerate MCP artifacts to reflect updated signatures and docstrings This resolves static hosting issues (e.g., Nginx 403 due to missing `site/index.html`) and makes each generated MkDocs site deployable as a standalone static website. --- ## Motivation Previously, documentation was generated under: ``` docs/<module>/... ``` Which resulted in: ``` site/<module>/index.html ``` When deployed at `/libs/<project>/`, this caused: * Missing `site/index.html` * Nginx returning 403 for root access * Inconsistent static hosting behavior This MR corrects the architecture so each MkDocs build is a valid static site with a root entry point. --- ## Key Changes ### 1️⃣ Flattened Docs Structure **Before** ``` docs/docforge/index.md ``` **After** ``` docs/index.md ``` All documentation paths were updated accordingly: * `docs/docforge/cli/...` → `docs/cli/...` * `docs/docforge/models/...` → `docs/models/...` * `docs/docforge/renderers/...` → `docs/renderers/...` Navigation updated to match the flat layout. --- ### 2️⃣ Root Index Enforcement `MkDocsRenderer` now guarantees: * `docs/index.md` is always created * Parent `index.md` files are auto-generated if missing * Parent indexes link to child modules (idempotent behavior) This ensures: ``` site/index.html ``` Always exists after `mkdocs build`. --- ### 3️⃣ New CLI Flag: `--module-is-source` Added option: ``` --module-is-source ``` Behavior: * Treats the provided module as the documentation root * Removes the top-level module folder from generated paths * Prevents redundant nesting when the module corresponds to the source root Updated components: * `cli.commands.build` * `mkdocs_utils.generate_sources` * `MkDocsRenderer.generate_sources` * Stub files (`.pyi`) * MCP JSON artifacts --- ### 4️⃣ Navigation Spec Update `docforge.nav.yml` updated: **Before** ```yaml home: docforge/index.md ``` **After** ```yaml home: index.md ``` All group paths adjusted to remove `docforge/` prefix. --- ### 5️⃣ MkDocs Config Update `mkdocs.yml` updated to: * Move `site_name` below theme/plugins * Use flat navigation paths * Point Home to `index.md` --- ### 6️⃣ MCP Artifact Regeneration Updated: * Function signatures (new parameter) * Docstrings (reflect `module_is_source`) * Renderer metadata * Line numbers Ensures MCP output matches updated API. --- ## Architectural Outcome Each project now produces a **valid standalone static site**: ``` site/ index.html assets/ search/ ``` Safe for deployment under: ``` /libs/<project>/ ``` No Nginx rewrites required. No directory-index issues. No nested-site ambiguity. --- ## Backward Compatibility * Existing CLI usage remains valid * `--module-is-source` is optional * Navigation spec format unchanged (only paths adjusted) --- ## Deployment Impact After merge: * Each library can be deployed independently * Sites can be merged under a shared root without internal conflicts * Static hosting is predictable and production-safe --- ## Testing * Verified MkDocs build produces `site/index.html` * Verified navigation renders correctly * Verified parent index generation is idempotent * Regenerated MCP docs and validated schema consistency --- ## Result The documentation compiler now: * Produces structurally correct static sites * Supports flat and source-root modes * Eliminates 403 root issues * Scales cleanly across multiple repositories This aligns doc-forge with proper static-site architectural invariants. Reviewed-on: #4 Co-authored-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com> Co-committed-by: Vishesh 'ironeagle' Bangotra <aetoskia@gmail.com>
179 lines
6.2 KiB
Python
179 lines
6.2 KiB
Python
import click
|
|
from pathlib import Path
|
|
from typing import Sequence, Optional
|
|
from docforge.loaders import GriffeLoader
|
|
from docforge.cli import mkdocs_utils
|
|
from docforge.cli import mcp_utils
|
|
|
|
|
|
@click.group()
|
|
def cli() -> None:
|
|
"""
|
|
doc-forge CLI: A tool for introspecting Python projects and generating
|
|
documentation.
|
|
"""
|
|
pass
|
|
|
|
|
|
@cli.command()
|
|
@click.option("--mcp", is_flag=True, help="Build MCP resources")
|
|
@click.option("--mkdocs", is_flag=True, help="Build MkDocs site")
|
|
@click.option("--module-is-source", is_flag=True, help="Module is source folder and to be treated as root folder")
|
|
@click.option("--module", help="Python module to document")
|
|
@click.option("--project-name", help="Project name override")
|
|
# MkDocs specific
|
|
@click.option("--site-name", help="MkDocs site name")
|
|
@click.option("--docs-dir", type=click.Path(path_type=Path), default=Path("docs"), help="Directory for MD sources")
|
|
@click.option("--nav", "nav_file", type=click.Path(path_type=Path), default=Path("docforge.nav.yml"),
|
|
help="Nav spec path")
|
|
@click.option("--template", type=click.Path(path_type=Path), help="MkDocs template path")
|
|
@click.option("--mkdocs-yml", type=click.Path(path_type=Path), default=Path("mkdocs.yml"), help="Output config path")
|
|
# MCP specific
|
|
@click.option("--out-dir", type=click.Path(path_type=Path), default=Path("mcp_docs"), help="MCP output directory")
|
|
def build(
|
|
mcp: bool,
|
|
mkdocs: bool,
|
|
module_is_source: bool,
|
|
module: Optional[str],
|
|
project_name: Optional[str],
|
|
site_name: Optional[str],
|
|
docs_dir: Path,
|
|
nav_file: Path,
|
|
template: Optional[Path],
|
|
mkdocs_yml: Path,
|
|
out_dir: Path,
|
|
) -> None:
|
|
"""
|
|
Build documentation (MkDocs site or MCP resources).
|
|
|
|
This command orchestrates the full build process:
|
|
1. Introspects the code (Griffe)
|
|
2. Renders sources (MkDocs Markdown or MCP JSON)
|
|
3. (MkDocs only) Generates config and runs the final site build.
|
|
|
|
Args:
|
|
mcp: Use the MCP documentation builder.
|
|
mkdocs: Use the MkDocs documentation builder.
|
|
module_is_source: Module is the source folder and to be treated as the root folder.
|
|
module: The dotted path of the module to the document.
|
|
project_name: Optional override for the project name.
|
|
site_name: (MkDocs) The site display name. Defaults to module name.
|
|
docs_dir: (MkDocs) Target directory for Markdown sources.
|
|
nav_file: (MkDocs) Path to the docforge.nav.yml specification.
|
|
template: (MkDocs) Optional custom mkdocs.yml template.
|
|
mkdocs_yml: (MkDocs) Target path for the generated mkdocs.yml.
|
|
out_dir: (MCP) Target directory for MCP JSON resources.
|
|
"""
|
|
if not mcp and not mkdocs:
|
|
raise click.UsageError("Must specify either --mcp or --mkdocs")
|
|
|
|
if mkdocs:
|
|
if not module:
|
|
raise click.UsageError("--module is required for MkDocs build")
|
|
if not site_name:
|
|
site_name = module
|
|
|
|
click.echo(f"Generating MkDocs sources in {docs_dir}...")
|
|
mkdocs_utils.generate_sources(
|
|
module,
|
|
docs_dir,
|
|
project_name,
|
|
module_is_source,
|
|
)
|
|
|
|
click.echo(f"Generating MkDocs config {mkdocs_yml}...")
|
|
mkdocs_utils.generate_config(docs_dir, nav_file, template, mkdocs_yml, site_name)
|
|
|
|
click.echo("Running MkDocs build...")
|
|
mkdocs_utils.build(mkdocs_yml)
|
|
click.echo("MkDocs build completed.")
|
|
|
|
if mcp:
|
|
if not module:
|
|
raise click.UsageError("--module is required for MCP build")
|
|
|
|
click.echo(f"Generating MCP resources in {out_dir}...")
|
|
mcp_utils.generate_resources(module, project_name, out_dir)
|
|
click.echo("MCP build completed.")
|
|
|
|
|
|
@cli.command()
|
|
@click.option("--mcp", is_flag=True, help="Serve MCP documentation")
|
|
@click.option("--mkdocs", is_flag=True, help="Serve MkDocs site")
|
|
@click.option("--module", help="Python module to serve")
|
|
@click.option("--mkdocs-yml", type=click.Path(path_type=Path), default=Path("mkdocs.yml"), help="MkDocs config path")
|
|
@click.option("--out-dir", type=click.Path(path_type=Path), default=Path("mcp_docs"), help="MCP root directory")
|
|
def serve(
|
|
mcp: bool,
|
|
mkdocs: bool,
|
|
module: Optional[str],
|
|
mkdocs_yml: Path,
|
|
out_dir: Path,
|
|
) -> None:
|
|
"""
|
|
Serve documentation (MkDocs or MCP).
|
|
|
|
Args:
|
|
mcp: Serve MCP resources via an MCP server.
|
|
mkdocs: Serve the MkDocs site using the built-in development server.
|
|
module: The dotted path of the module to serve.
|
|
mkdocs_yml: (MkDocs) Path to the mkdocs.yml configuration.
|
|
out_dir: (MCP) Path to the mcp_docs/ directory.
|
|
"""
|
|
if mcp and mkdocs:
|
|
raise click.UsageError("Cannot specify both --mcp and --mkdocs")
|
|
if not mcp and not mkdocs:
|
|
raise click.UsageError("Must specify either --mcp or --mkdocs")
|
|
if mcp and not module:
|
|
raise click.UsageError("--module is required for MCP serve")
|
|
|
|
if mkdocs:
|
|
mkdocs_utils.serve(mkdocs_yml)
|
|
elif mcp:
|
|
mcp_utils.serve(module, out_dir)
|
|
|
|
|
|
@cli.command()
|
|
@click.option(
|
|
"--module",
|
|
required=True,
|
|
help="Python module import path to introspect",
|
|
)
|
|
@click.option(
|
|
"--project-name",
|
|
help="Project name (defaults to specified module)",
|
|
)
|
|
def tree(
|
|
module: str,
|
|
project_name: Optional[str],
|
|
) -> None:
|
|
"""
|
|
Visualize the project structure in the terminal.
|
|
|
|
Args:
|
|
module: The module import path to recursively introspect.
|
|
project_name: Optional override for the project name shown at the root.
|
|
"""
|
|
loader = GriffeLoader()
|
|
project = loader.load_project([module], project_name)
|
|
|
|
click.echo(project.name)
|
|
|
|
for module in project.get_all_modules():
|
|
click.echo(f"├── {module.path}")
|
|
for obj in module.get_all_objects():
|
|
_print_object(obj, indent="│ ")
|
|
|
|
|
|
def _print_object(obj, indent: str) -> None:
|
|
"""
|
|
Recursive helper to print doc objects and their members to the console.
|
|
|
|
Args:
|
|
obj: The DocObject instance to print.
|
|
indent: Current line indentation (e.g., '│ ').
|
|
"""
|
|
click.echo(f"{indent}├── {obj.name}")
|
|
for member in obj.get_all_members():
|
|
_print_object(member, indent + "│ ")
|