docs: add wiki, complete lib nav, and rebuild mcp artifacts
- Add hand-written wiki (index, overview, how-to, extending, dev) with MkDocs config following the platform anatomy - Complete the library nav by registering the csv and xlsx groups in docforge.nav.yml and docs/mkdocs.lib.yml - Regenerate lib/MCP outputs with the rebuilt nav
This commit is contained in:
109
docs/wiki/02_how_to_use.md
Normal file
109
docs/wiki/02_how_to_use.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# 🖥️ How to Use
|
||||
|
||||
Every format follows the same pipeline: **scrape → `Content` → parse**.
|
||||
This page shows each supported format with working patterns.
|
||||
|
||||
---
|
||||
|
||||
## 🌐 HTML
|
||||
|
||||
```python
|
||||
from omniread import HTMLScraper, HTMLParser
|
||||
|
||||
class TitleParser(HTMLParser[str]):
|
||||
def parse(self) -> str:
|
||||
return self._soup.title.string
|
||||
|
||||
scraper = HTMLScraper() # httpx under the hood
|
||||
content = scraper.fetch("https://example.com")
|
||||
|
||||
title = TitleParser(content).parse()
|
||||
```
|
||||
|
||||
`HTMLScraper` accepts an optional `client` (an `httpx.Client`) for transport
|
||||
control — the test suite wires one to a mock transport.
|
||||
|
||||
---
|
||||
|
||||
## 📕 PDF
|
||||
|
||||
PDFs need a **client** to supply raw bytes before parsing:
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
from omniread import FileSystemPDFClient, PDFScraper, PDFParser
|
||||
|
||||
class TextPDFParser(PDFParser[str]):
|
||||
def parse(self) -> str:
|
||||
# implement your extraction logic
|
||||
return self.content.raw # bytes, decode as needed
|
||||
|
||||
client = FileSystemPDFClient()
|
||||
scraper = PDFScraper(client=client)
|
||||
content = scraper.fetch(Path("document.pdf"))
|
||||
|
||||
result = TextPDFParser(content).parse()
|
||||
```
|
||||
|
||||
`PDFParser` subclasses receive `self.content` and implement `parse()`.
|
||||
|
||||
---
|
||||
|
||||
## 📊 CSV
|
||||
|
||||
```python
|
||||
from omniread import FileSystemCsvClient, CsvScraper, CsvParser
|
||||
|
||||
scraper = CsvScraper(client=FileSystemCsvClient())
|
||||
content = scraper.fetch("data.csv")
|
||||
|
||||
parser = CsvParser(content)
|
||||
for row in parser.rows():
|
||||
print(row)
|
||||
```
|
||||
|
||||
`CsvParser.rows()` yields string rows trimmed of empties by default
|
||||
(`skip_empty=True`).
|
||||
|
||||
---
|
||||
|
||||
## 📑 XLSX
|
||||
|
||||
```python
|
||||
from omniread import FileSystemXlsxClient, XlsxScraper, XlsxParser
|
||||
|
||||
scraper = XlsxScraper(client=FileSystemXlsxClient())
|
||||
content = scraper.fetch("statement.xlsx")
|
||||
|
||||
parser = XlsxParser(content)
|
||||
print(parser.sheet_names) # e.g. ["Statement"]
|
||||
rows = parser.rows(sheet="Statement") # by name or index
|
||||
all_rows = parser.parse() # alias for rows()
|
||||
```
|
||||
|
||||
Key behaviors:
|
||||
|
||||
- `rows(skip_empty=False)` keeps blank rows (off by default).
|
||||
- Cells render as trimmed strings; date cells convert to ISO format
|
||||
(`2026-06-01T00:00:00`).
|
||||
- `rows(sheet="Missing")` raises for an unknown sheet.
|
||||
|
||||
---
|
||||
|
||||
## 🔀 End-to-end flow
|
||||
|
||||
```python
|
||||
content = scraper.fetch(source) # 1. acquire → Content
|
||||
assert isinstance(content.raw, bytes)
|
||||
assert content.content_type is not None
|
||||
|
||||
parser = MyParser(content) # 2. interpret → T
|
||||
result = parser.parse()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 Read Next
|
||||
|
||||
- [Overview](01_overview.md) — the full architecture.
|
||||
- [Extending OmniRead](03_extending.md) — custom clients and parsers.
|
||||
Reference in New Issue
Block a user