omniread/docs/wiki/index.md

📄 OmniRead — Format-Agnostic Content Acquisition and Parsing

OmniRead is a composable content ingestion framework with pluggable scrapers and parsers for HTML, PDF, CSV, and XLSX. It cleanly separates fetching (I/O) from interpretation (parsing) through a canonical Content exchange model, giving each format its own scraper and parser while sharing one uniform pipeline.

Doc model: this wiki is written for humans — how‑to guides, examples, and extension recipes. The authoritative API contracts live in the code (GSDFC docstrings) and the machine‑readable bundle under docs/mcp/.

---

🚀 Key Features

• 🧱 Content/Scraper/Parser architecture — fetch and interpret in independent, testable layers
• 📄 Canonical Content model — raw bytes + metadata passed between all layers via one dataclass
• 🌐 HTML — HTTP-based scraper with httpx; DOM parsing via BeautifulSoup
• 📕 PDF — client + scraper for raw PDF bytes; user-defined parsers
• 📊 CSV — standard-library CSV parser over rows() iterator
• 📑 XLSX — openpyxl-backed spreadsheet parser with sheet selection, empty-row control, and ISO date conversion

---

⚡ Quick Start

from omniread import HTMLScraper, HTMLParser

class TitleParser(HTMLParser[str]):
    def parse(self) -> str:
        return self._soup.title.string

content = HTMLScraper().fetch("https://example.com")
title = TitleParser(content).parse()
assert title == "Example Domain"

---

📁 Documentation Structure

Section	What you'll find
Overview	The three-layer architecture and the Content model
How to Use	Ready-to-run examples for HTML, PDF, CSV, and XLSX
Extending OmniRead	Subclassing scrapers, parsers, and clients
Development	Setup, tests, and regenerating docs

---

🔗 Related Resources

• Source Code: Gitea Repository
• Internal PyPI: pip.aetoskia.com/simple/omniread
• CI: Builds and publishes tagged releases, gated on black / ruff / mypy / pytest.

---

© Aetoskia Internal — omniread 0.0.1