Vishesh 'ironeagle' Bangotra
b984dd5f42
- Add hand-written wiki (index, overview, how-to, extending, dev) with MkDocs config following the platform anatomy - Complete the library nav by registering the csv and xlsx groups in docforge.nav.yml and docs/mkdocs.lib.yml - Regenerate lib/MCP outputs with the rebuilt nav
109 lines
2.5 KiB
Markdown
109 lines
2.5 KiB
Markdown
# 🖥️ How to Use
|
|
|
|
Every format follows the same pipeline: **scrape → `Content` → parse**.
|
|
This page shows each supported format with working patterns.
|
|
|
|
---
|
|
|
|
## 🌐 HTML
|
|
|
|
```python
|
|
from omniread import HTMLScraper, HTMLParser
|
|
|
|
class TitleParser(HTMLParser[str]):
|
|
def parse(self) -> str:
|
|
return self._soup.title.string
|
|
|
|
scraper = HTMLScraper() # httpx under the hood
|
|
content = scraper.fetch("https://example.com")
|
|
|
|
title = TitleParser(content).parse()
|
|
```
|
|
|
|
`HTMLScraper` accepts an optional `client` (an `httpx.Client`) for transport
|
|
control — the test suite wires one to a mock transport.
|
|
|
|
---
|
|
|
|
## 📕 PDF
|
|
|
|
PDFs need a **client** to supply raw bytes before parsing:
|
|
|
|
```python
|
|
from pathlib import Path
|
|
from omniread import FileSystemPDFClient, PDFScraper, PDFParser
|
|
|
|
class TextPDFParser(PDFParser[str]):
|
|
def parse(self) -> str:
|
|
# implement your extraction logic
|
|
return self.content.raw # bytes, decode as needed
|
|
|
|
client = FileSystemPDFClient()
|
|
scraper = PDFScraper(client=client)
|
|
content = scraper.fetch(Path("document.pdf"))
|
|
|
|
result = TextPDFParser(content).parse()
|
|
```
|
|
|
|
`PDFParser` subclasses receive `self.content` and implement `parse()`.
|
|
|
|
---
|
|
|
|
## 📊 CSV
|
|
|
|
```python
|
|
from omniread import FileSystemCsvClient, CsvScraper, CsvParser
|
|
|
|
scraper = CsvScraper(client=FileSystemCsvClient())
|
|
content = scraper.fetch("data.csv")
|
|
|
|
parser = CsvParser(content)
|
|
for row in parser.rows():
|
|
print(row)
|
|
```
|
|
|
|
`CsvParser.rows()` yields string rows trimmed of empties by default
|
|
(`skip_empty=True`).
|
|
|
|
---
|
|
|
|
## 📑 XLSX
|
|
|
|
```python
|
|
from omniread import FileSystemXlsxClient, XlsxScraper, XlsxParser
|
|
|
|
scraper = XlsxScraper(client=FileSystemXlsxClient())
|
|
content = scraper.fetch("statement.xlsx")
|
|
|
|
parser = XlsxParser(content)
|
|
print(parser.sheet_names) # e.g. ["Statement"]
|
|
rows = parser.rows(sheet="Statement") # by name or index
|
|
all_rows = parser.parse() # alias for rows()
|
|
```
|
|
|
|
Key behaviors:
|
|
|
|
- `rows(skip_empty=False)` keeps blank rows (off by default).
|
|
- Cells render as trimmed strings; date cells convert to ISO format
|
|
(`2026-06-01T00:00:00`).
|
|
- `rows(sheet="Missing")` raises for an unknown sheet.
|
|
|
|
---
|
|
|
|
## 🔀 End-to-end flow
|
|
|
|
```python
|
|
content = scraper.fetch(source) # 1. acquire → Content
|
|
assert isinstance(content.raw, bytes)
|
|
assert content.content_type is not None
|
|
|
|
parser = MyParser(content) # 2. interpret → T
|
|
result = parser.parse()
|
|
```
|
|
|
|
---
|
|
|
|
## 📚 Read Next
|
|
|
|
- [Overview](01_overview.md) — the full architecture.
|
|
- [Extending OmniRead](03_extending.md) — custom clients and parsers. |