Compare commits
3 Commits
b5379cd82e
...
f8ca6075fb
| Author | SHA1 | Date | |
|---|---|---|---|
| f8ca6075fb | |||
| 3f68e51e58 | |||
| 87a7f4eee1 |
@@ -1,270 +1,536 @@
|
||||
"""
|
||||
Renderer-agnostic Python documentation compiler that converts docstrings into
|
||||
structured documentation for both humans (MkDocs) and machines (MCP / AI agents).
|
||||
Renderer-agnostic Python documentation compiler that converts Python docstrings
|
||||
into structured documentation for both humans (MkDocs) and machines (MCP / AI agents).
|
||||
|
||||
`doc-forge` statically analyzes source code, builds a semantic model of modules,
|
||||
classes, functions, and attributes, and renders that model into documentation
|
||||
outputs without executing user code.
|
||||
|
||||
`doc-forge` statically analyzes your source code, builds a semantic model of
|
||||
modules and objects, and renders that model into documentation outputs without
|
||||
executing your code.
|
||||
---
|
||||
|
||||
# Core Philosophy
|
||||
## Installation
|
||||
|
||||
Install using pip:
|
||||
|
||||
pip install doc-forge
|
||||
|
||||
---
|
||||
|
||||
## Quick start
|
||||
|
||||
Generate a MkDocs site from a Python package:
|
||||
|
||||
doc-forge build --mkdocs --module my_package
|
||||
|
||||
Generate MCP JSON documentation:
|
||||
|
||||
doc-forge build --mcp --module my_package
|
||||
|
||||
Serve documentation locally:
|
||||
|
||||
doc-forge serve --mkdocs --module my_package
|
||||
|
||||
---
|
||||
|
||||
## Core concepts
|
||||
|
||||
**Loader**
|
||||
|
||||
Extracts symbols, signatures, and docstrings using static analysis.
|
||||
|
||||
**Semantic model**
|
||||
|
||||
Structured, renderer-agnostic representation of the API.
|
||||
|
||||
**Renderer**
|
||||
|
||||
Converts the semantic model into output formats such as MkDocs or MCP JSON.
|
||||
|
||||
**Symbol**
|
||||
|
||||
Any documentable object:
|
||||
|
||||
- module
|
||||
- class
|
||||
- function
|
||||
- method
|
||||
- property
|
||||
- attribute
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
`doc-forge` follows a compiler architecture:
|
||||
|
||||
1. **Front-end (Introspection)**
|
||||
Static analysis of modules, classes, functions, signatures, and docstrings.
|
||||
Front-end:
|
||||
|
||||
2. **Middle-end (Semantic Model)**
|
||||
Renderer-neutral structured representation of your API.
|
||||
Static analysis of modules, classes, functions, type hints, and docstrings.
|
||||
|
||||
3. **Back-end (Renderers)**
|
||||
Middle-end:
|
||||
|
||||
Builds a semantic model describing symbols and relationships.
|
||||
|
||||
Back-end:
|
||||
|
||||
Renders documentation using interchangeable renderers.
|
||||
|
||||
This architecture ensures deterministic documentation generation.
|
||||
|
||||
* MkDocs → human documentation
|
||||
* MCP JSON → AI-readable documentation
|
||||
---
|
||||
|
||||
# Docstring Writing Standard
|
||||
## Rendering pipeline
|
||||
|
||||
Docstrings are the single source of truth. `doc-forge` does not generate content.
|
||||
It compiles and renders what you write.
|
||||
Typical flow:
|
||||
|
||||
Python package
|
||||
↓
|
||||
Loader (static analysis)
|
||||
↓
|
||||
Semantic model
|
||||
↓
|
||||
Renderer
|
||||
↓
|
||||
MkDocs site or MCP JSON
|
||||
|
||||
Documentation follows the Python import hierarchy.
|
||||
---
|
||||
|
||||
# Package docstring (`package/__init__.py`) — Full user guide
|
||||
## CLI usage
|
||||
|
||||
This is the landing page. A developer must be able to install and use the
|
||||
package after reading only this docstring.
|
||||
Build MkDocs documentation:
|
||||
|
||||
## Example:
|
||||
doc-forge build --mkdocs --module my_package
|
||||
|
||||
'''
|
||||
Short description of what this package provides.
|
||||
Build MCP documentation:
|
||||
|
||||
# Installation
|
||||
doc-forge build --mcp --module my_package
|
||||
|
||||
```bash
|
||||
pip install my-package
|
||||
```
|
||||
Serve MkDocs locally:
|
||||
|
||||
# Quick start
|
||||
doc-forge serve --module my_package
|
||||
|
||||
```python
|
||||
from my_package.foo import Bar
|
||||
|
||||
bar = Bar()
|
||||
result = bar.process("example")
|
||||
```
|
||||
---
|
||||
|
||||
# Core concepts
|
||||
|
||||
## Bar
|
||||
- Primary object exposed by the package.
|
||||
|
||||
## foo module
|
||||
- Provides core functionality.
|
||||
---
|
||||
|
||||
# Typical workflow
|
||||
|
||||
1. Import public objects
|
||||
2. Initialize objects
|
||||
3. Call methods
|
||||
---
|
||||
|
||||
# Public API
|
||||
|
||||
foo.Bar
|
||||
foo.helper_function
|
||||
---
|
||||
'''
|
||||
---
|
||||
|
||||
# Submodule docstring (`package/foo/__init__.py`) — Subsystem guide
|
||||
## Public API
|
||||
|
||||
Explains a specific subsystem.
|
||||
Loaders:
|
||||
|
||||
## Example:
|
||||
GriffeLoader
|
||||
discover_module_paths
|
||||
|
||||
'''
|
||||
Provides core functionality.
|
||||
Renderers:
|
||||
|
||||
# Usage
|
||||
MkDocsRenderer
|
||||
MCPRenderer
|
||||
|
||||
```python
|
||||
from my_package.foo import Bar
|
||||
CLI:
|
||||
|
||||
main
|
||||
|
||||
Models:
|
||||
|
||||
models
|
||||
|
||||
bar = Bar()
|
||||
bar.process("example")
|
||||
```
|
||||
---
|
||||
'''
|
||||
---
|
||||
|
||||
# Class docstring — Object contract
|
||||
# Google-Styled Doc-Forge Convention (GSDFC)
|
||||
|
||||
Defines responsibility and behavior.
|
||||
GSDFC defines how docstrings must be written so they render correctly in MkDocs and remain machine-parsable by doc-forge and AI tooling.
|
||||
|
||||
## Example:
|
||||
- Docstrings are the single source of truth.
|
||||
- doc-forge compiles docstrings but does not generate documentation content.
|
||||
- Documentation follows the Python import hierarchy.
|
||||
- Every public symbol should have a complete and accurate docstring.
|
||||
|
||||
```python
|
||||
class Bar:
|
||||
'''
|
||||
Performs processing on input data.
|
||||
|
||||
Instances may be reused across multiple calls.
|
||||
---
|
||||
'''
|
||||
```
|
||||
|
||||
Include:
|
||||
|
||||
* Responsibility
|
||||
* Lifecycle expectations
|
||||
* Thread safety (if relevant)
|
||||
* Performance characteristics (if relevant)
|
||||
---
|
||||
|
||||
# Function and method docstrings — API specification
|
||||
## General rules
|
||||
|
||||
## Example:
|
||||
- Use **Markdown headings** at package and module level.
|
||||
- Use **Google-style structured sections** at class, function, and method level.
|
||||
- Indent section contents using four spaces.
|
||||
- Use type hints in signatures instead of duplicating types in prose.
|
||||
- Write summaries in imperative form.
|
||||
- Sections are separated by ```---```
|
||||
|
||||
```python
|
||||
def process(
|
||||
self,
|
||||
value1: str,
|
||||
value2: str | None = "default value",
|
||||
value3: str | None = None,
|
||||
) -> str:
|
||||
'''
|
||||
Process an input value.
|
||||
---
|
||||
---
|
||||
|
||||
## Package docstrings
|
||||
|
||||
- Package docstrings act as the documentation home page.
|
||||
|
||||
Recommended sections:
|
||||
|
||||
## Summary
|
||||
## Installation
|
||||
## Quick start
|
||||
## Core concepts
|
||||
## Architecture
|
||||
## Rendering pipeline
|
||||
## CLI usage
|
||||
## Public API
|
||||
## Examples
|
||||
## Notes
|
||||
|
||||
Example:
|
||||
Package Doc String:
|
||||
|
||||
'''
|
||||
Foo-bar processing framework.
|
||||
|
||||
Provides tools for defining Foo objects and executing Bar pipelines.
|
||||
|
||||
---
|
||||
|
||||
# Installation
|
||||
|
||||
pip install foo-bar
|
||||
|
||||
---
|
||||
|
||||
# Quick start
|
||||
|
||||
from foobar import Foo, BarEngine
|
||||
|
||||
foo = Foo("example")
|
||||
engine = BarEngine([foo])
|
||||
|
||||
result = engine.run()
|
||||
|
||||
---
|
||||
|
||||
# Public API
|
||||
|
||||
Foo
|
||||
Bar
|
||||
BarEngine
|
||||
|
||||
---
|
||||
'''
|
||||
|
||||
---
|
||||
|
||||
## Module docstrings
|
||||
|
||||
- Module docstrings describe a subsystem.
|
||||
|
||||
Recommended sections:
|
||||
|
||||
## Summary
|
||||
## Examples
|
||||
## Notes
|
||||
## Public API
|
||||
|
||||
Example:
|
||||
Module Doc String:
|
||||
|
||||
'''
|
||||
Foo execution subsystem.
|
||||
|
||||
Provides utilities for executing Foo objects through Bar stages.
|
||||
|
||||
---
|
||||
|
||||
Parameters
|
||||
----------
|
||||
value1 : str
|
||||
required: True
|
||||
value to be processed
|
||||
Example:
|
||||
'string'
|
||||
|
||||
value2 : str
|
||||
required: False
|
||||
default: "default value"
|
||||
value to be processed
|
||||
from foobar.engine import BarEngine
|
||||
from foobar.foo import Foo
|
||||
|
||||
foo = Foo("example")
|
||||
|
||||
engine = BarEngine([foo])
|
||||
engine.run()
|
||||
|
||||
---
|
||||
'''
|
||||
|
||||
---
|
||||
|
||||
## Class docstrings
|
||||
|
||||
- Class docstrings define object responsibility, lifecycle, and attributes.
|
||||
|
||||
Recommended sections:
|
||||
|
||||
Attributes:
|
||||
Notes:
|
||||
Example:
|
||||
Raises:
|
||||
|
||||
Example:
|
||||
Simple Foo:
|
||||
|
||||
class Foo:
|
||||
'''
|
||||
Represents a unit of work.
|
||||
|
||||
Attributes:
|
||||
name (str):
|
||||
Identifier of the foo instance.
|
||||
|
||||
value (int):
|
||||
Numeric value associated with foo.
|
||||
|
||||
Notes:
|
||||
Guarantees:
|
||||
|
||||
- instances are immutable after creation
|
||||
|
||||
Lifecycle:
|
||||
|
||||
- create instance
|
||||
- pass to processing engine
|
||||
|
||||
Example:
|
||||
Create and inspect a Foo:
|
||||
|
||||
foo = Foo("example", value=42)
|
||||
print(foo.name)
|
||||
'''
|
||||
|
||||
Complex Bar:
|
||||
|
||||
class BarEngine:
|
||||
'''
|
||||
Executes Foo objects through Bar stages.
|
||||
|
||||
Attributes:
|
||||
foos (tuple[Foo, ...]):
|
||||
Foo instances managed by the engine.
|
||||
|
||||
Notes:
|
||||
Guarantees:
|
||||
|
||||
- deterministic execution order
|
||||
|
||||
Example:
|
||||
Run engine:
|
||||
|
||||
foo1 = Foo("a")
|
||||
foo2 = Foo("b")
|
||||
|
||||
engine = BarEngine([foo1, foo2])
|
||||
engine.run()
|
||||
'''
|
||||
|
||||
---
|
||||
|
||||
## Function and method docstrings
|
||||
|
||||
- Function docstrings define API contracts.
|
||||
|
||||
Recommended sections:
|
||||
|
||||
Args:
|
||||
Returns:
|
||||
Raises:
|
||||
Yields:
|
||||
Notes:
|
||||
Example:
|
||||
|
||||
Example:
|
||||
Simple process method:
|
||||
|
||||
def process(foo: Foo, multiplier: int) -> int:
|
||||
'''
|
||||
Process a Foo instance.
|
||||
|
||||
Args:
|
||||
foo (Foo):
|
||||
Foo instance to process.
|
||||
|
||||
multiplier (int):
|
||||
Value used to scale foo.
|
||||
|
||||
Returns:
|
||||
int:
|
||||
Processed result.
|
||||
|
||||
Raises:
|
||||
ValueError:
|
||||
If multiplier is negative.
|
||||
|
||||
Notes:
|
||||
Guarantees:
|
||||
|
||||
- foo is not modified
|
||||
|
||||
Example:
|
||||
Process foo:
|
||||
|
||||
foo = Foo("example", value=10)
|
||||
|
||||
result = process(foo, multiplier=2)
|
||||
print(result)
|
||||
'''
|
||||
|
||||
Multiple Examples:
|
||||
|
||||
def combine(foo_a: Foo, foo_b: Foo) -> Foo:
|
||||
'''
|
||||
Combine two Foo instances.
|
||||
|
||||
Args:
|
||||
foo_a (Foo):
|
||||
First foo.
|
||||
|
||||
foo_b (Foo):
|
||||
Second foo.
|
||||
|
||||
Returns:
|
||||
Foo:
|
||||
Combined foo.
|
||||
|
||||
Example:
|
||||
Basic usage:
|
||||
|
||||
foo1 = Foo("a")
|
||||
foo2 = Foo("b")
|
||||
|
||||
combined = combine(foo1, foo2)
|
||||
|
||||
Pipeline usage:
|
||||
|
||||
engine = BarEngine([foo1, foo2])
|
||||
engine.run()
|
||||
'''
|
||||
|
||||
---
|
||||
|
||||
## Property docstrings
|
||||
|
||||
- Properties must document return values.
|
||||
|
||||
Example:
|
||||
Property Doc String:
|
||||
|
||||
@property
|
||||
def foos(self) -> tuple[Foo, ...]:
|
||||
'''
|
||||
Return contained Foo instances.
|
||||
|
||||
Returns:
|
||||
tuple[Foo, ...]:
|
||||
Stored foo objects.
|
||||
|
||||
Example:
|
||||
container = FooContainer()
|
||||
|
||||
foos = container.foos
|
||||
'''
|
||||
|
||||
---
|
||||
|
||||
## Attribute documentation
|
||||
|
||||
- Document attributes in class docstrings using Attributes:.
|
||||
|
||||
Example:
|
||||
Attribute Doc String:
|
||||
|
||||
'''
|
||||
Represents a processing stage.
|
||||
|
||||
Attributes:
|
||||
id (str):
|
||||
Unique identifier.
|
||||
|
||||
enabled (bool):
|
||||
Whether the stage is active.
|
||||
'''
|
||||
|
||||
---
|
||||
|
||||
## Notes subsection grouping
|
||||
|
||||
- Group related information using labeled subsections.
|
||||
|
||||
Example:
|
||||
Notes Example:
|
||||
|
||||
Notes:
|
||||
**Guarantees:**
|
||||
|
||||
- deterministic behavior
|
||||
|
||||
**Lifecycle:**
|
||||
|
||||
- created during initialization
|
||||
- reused across executions
|
||||
|
||||
**Thread safety:**
|
||||
|
||||
- safe for concurrent reads
|
||||
|
||||
---
|
||||
|
||||
## Example formatting
|
||||
|
||||
- Use indentation for examples.
|
||||
|
||||
Example:
|
||||
Single example:
|
||||
|
||||
Example:
|
||||
'string'
|
||||
|
||||
value3 : str
|
||||
required: False
|
||||
value to be processed
|
||||
foo = Foo("example")
|
||||
process(foo, multiplier=2)
|
||||
|
||||
Multiple examples:
|
||||
|
||||
Example:
|
||||
'string'
|
||||
Create foo:
|
||||
|
||||
foo = Foo("example")
|
||||
|
||||
Run engine:
|
||||
|
||||
engine = BarEngine([foo])
|
||||
engine.run()
|
||||
|
||||
Avoid fenced code blocks inside structured sections.
|
||||
|
||||
---
|
||||
|
||||
## Separator rules
|
||||
|
||||
Use horizontal separators only at docstring root level to separate sections:
|
||||
|
||||
---
|
||||
|
||||
Returns
|
||||
-------
|
||||
processed value : str
|
||||
result after processing value
|
||||
---
|
||||
Allowed locations:
|
||||
|
||||
- package docstrings
|
||||
- module docstrings
|
||||
- major documentation sections
|
||||
|
||||
Do not use separators inside code sections.
|
||||
|
||||
Behavior
|
||||
--------
|
||||
- behaviour 1
|
||||
- behaviour 2
|
||||
---
|
||||
'''
|
||||
```
|
||||
---
|
||||
|
||||
# Attribute docstrings (optional)
|
||||
## Parsing guarantees
|
||||
|
||||
## Example:
|
||||
GSDFC ensures doc-forge can deterministically extract:
|
||||
|
||||
```python
|
||||
class Class
|
||||
'''
|
||||
attribute1 : str
|
||||
required: True
|
||||
default: "default value"
|
||||
attribute description
|
||||
- symbol kind (module, class, function, property, attribute)
|
||||
- symbol name
|
||||
- parameters
|
||||
- return values
|
||||
- attributes
|
||||
- examples
|
||||
- structured Notes subsections
|
||||
|
||||
attribute2 : str
|
||||
required: False
|
||||
attribute description
|
||||
This enables:
|
||||
|
||||
attribute2 : str
|
||||
required: False
|
||||
default: "default value"
|
||||
attribute description
|
||||
'''
|
||||
- reliable MkDocs rendering
|
||||
- deterministic MCP export
|
||||
- accurate AI semantic interpretation
|
||||
|
||||
attribute1: str = "default value"
|
||||
attribute2: str | None = None
|
||||
attribute3: str | None = "default value"
|
||||
```
|
||||
---
|
||||
|
||||
# Writing Rules
|
||||
|
||||
**Heading hierarchy**
|
||||
|
||||
Module docstring
|
||||
|
||||
- Examples
|
||||
- Usage
|
||||
- Core concepts
|
||||
- Public API
|
||||
|
||||
Class docstring
|
||||
|
||||
- Attributes
|
||||
- Execution contract
|
||||
- Lifecycle
|
||||
- Thread safety
|
||||
- Notes
|
||||
|
||||
Method docstring
|
||||
|
||||
- Parameters
|
||||
- Returns
|
||||
- Raises
|
||||
- Yields
|
||||
- Behavior
|
||||
|
||||
**Required**
|
||||
|
||||
* Use Markdown headings
|
||||
* Use Markdown line separator `---`
|
||||
* Line separator should be followed by a blank line
|
||||
* Provide real import examples
|
||||
* Document all public APIs
|
||||
* Keep descriptions precise and factual
|
||||
|
||||
**Avoid**
|
||||
|
||||
* Plain-text separators like `====`
|
||||
* Duplicate external documentation
|
||||
* Informal or conversational language
|
||||
---
|
||||
|
||||
# How doc-forge uses these docstrings
|
||||
|
||||
## Build MkDocs site:
|
||||
|
||||
```bash
|
||||
doc-forge build --mkdocs --module my_package
|
||||
```
|
||||
|
||||
## Build MCP documentation:
|
||||
|
||||
```bash
|
||||
doc-forge build --mcp --module my_package
|
||||
```
|
||||
|
||||
Both outputs are generated directly from docstrings.
|
||||
---
|
||||
Notes:
|
||||
- doc-forge never executes analyzed modules.
|
||||
- Documentation is generated entirely through static analysis.
|
||||
"""
|
||||
|
||||
from .loaders import GriffeLoader, discover_module_paths
|
||||
|
||||
Reference in New Issue
Block a user