Skip to content

📄 OmniRead — Format-Agnostic Content Acquisition and Parsing

OmniRead is a composable content ingestion framework with pluggable scrapers and parsers for HTML, PDF, CSV, and XLSX. It cleanly separates fetching (I/O) from interpretation (parsing) through a canonical Content exchange model, giving each format its own scraper and parser while sharing one uniform pipeline.

Doc model: this wiki is written for humans — how‑to guides, examples, and extension recipes. The authoritative API contracts live in the code (GSDFC docstrings) and the machine‑readable bundle under docs/mcp/.

🚀 Key Features

  • 🧱 Content/Scraper/Parser architecture — fetch and interpret in independent, testable layers
  • 📄 Canonical Content model — raw bytes + metadata passed between all layers via one dataclass
  • 🌐 HTML — HTTP-based scraper with httpx; DOM parsing via BeautifulSoup
  • 📕 PDF — client + scraper for raw PDF bytes; user-defined parsers
  • 📊 CSV — standard-library CSV parser over rows() iterator
  • 📑 XLSX — openpyxl-backed spreadsheet parser with sheet selection, empty-row control, and ISO date conversion

⚡ Quick Start

1
2
3
4
5
6
7
8
9
from omniread import HTMLScraper, HTMLParser

class TitleParser(HTMLParser[str]):
    def parse(self) -> str:
        return self._soup.title.string

content = HTMLScraper().fetch("https://example.com")
title = TitleParser(content).parse()
assert title == "Example Domain"

📁 Documentation Structure

Section What you'll find
Overview The three-layer architecture and the Content model
How to Use Ready-to-run examples for HTML, PDF, CSV, and XLSX
Extending OmniRead Subclassing scrapers, parsers, and clients
Development Setup, tests, and regenerating docs

© Aetoskia Internal — omniread 0.0.1