Vishesh 'ironeagle' Bangotra
b984dd5f42
- Add hand-written wiki (index, overview, how-to, extending, dev) with MkDocs config following the platform anatomy - Complete the library nav by registering the csv and xlsx groups in docforge.nav.yml and docs/mkdocs.lib.yml - Regenerate lib/MCP outputs with the rebuilt nav
87 lines
1.9 KiB
Markdown
87 lines
1.9 KiB
Markdown
# 🛠️ Development
|
|
|
|
Working on `omniread` itself.
|
|
|
|
---
|
|
|
|
## 📂 Repository layout
|
|
|
|
| Path | Purpose |
|
|
|---|---|
|
|
| `omniread/` | The library package (core + html, pdf, csv, xlsx modules) |
|
|
| `omniread/*.pyi` | Type stubs kept in sync with implementations |
|
|
| `tests/` | End-to-end and unit tests against mock transports/clients |
|
|
| `covers` | — |
|
|
| `docs/lib/` | Generated library reference (docforge, flat layout) |
|
|
| `docs/mcp/` | Machine-readable bundle served by the MCP server |
|
|
| `docs/wiki/` | This hand-written wiki |
|
|
|
|
---
|
|
|
|
## 🔧 Setup
|
|
|
|
```bash
|
|
python -m venv .venv
|
|
.venv/Scripts/pip install -e ".[dev]"
|
|
```
|
|
|
|
---
|
|
|
|
## 🧪 Tests
|
|
|
|
Run the suite (offline; a mock httpx transport and mock PDF client are used):
|
|
|
|
```bash
|
|
.venv/Scripts/pytest
|
|
```
|
|
|
|
Coverage spans end-to-end scrape → parse flows for HTML, PDF, and XLSX,
|
|
plus client validation and CSV/XLSX parsing edge cases.
|
|
|
|
---
|
|
|
|
## ✅ Quality gates
|
|
|
|
The CI quality gate runs, matching the Drone pipeline:
|
|
|
|
```bash
|
|
.venv/Scripts/black --check .
|
|
.venv/Scripts/ruff check .
|
|
.venv/Scripts/mypy
|
|
.venv/Scripts/pytest
|
|
```
|
|
|
|
---
|
|
|
|
## 📝 Building documentation (docforge)
|
|
|
|
The site is generated by [`docforge`](https://git.aetoskia.com/aetos/doc-forge)
|
|
and served per kind under `site/{kind}`:
|
|
|
|
```bash
|
|
doc-forge build \
|
|
--mkdocs --mcp \
|
|
--module-is-source --module omniread \
|
|
--site-name "OmniRead"
|
|
```
|
|
|
|
- `--module-is-source` renders the flat `docs/lib/` layout (no nesting under
|
|
`omniread/`), matching `docforge.nav.yml` and `docs/mkdocs.lib.yml`.
|
|
- `--mcp` regenerates the structured bundle in `docs/mcp/`.
|
|
- `--wiki` builds this wiki.
|
|
- `gen_api.py` (if present) regenerates the API docs.
|
|
|
|
Preview locally:
|
|
|
|
```bash
|
|
doc-forge serve --lib
|
|
doc-forge serve --wiki
|
|
doc-forge serve --mcp
|
|
```
|
|
|
|
---
|
|
|
|
## 📚 Read Next
|
|
|
|
- [Extending OmniRead](03_extending.md) — custom parsers, scrapers, clients.
|
|
- [Overview](01_overview.md) — the core architecture. |