Files

Vishesh 'ironeagle' Bangotra 650d3a19e8 chore(release): 0.0.3

Squash-merge lib-standardisation: adopt repo standards across packaging,
tooling, docs, and CI.

- standardize pyproject.toml (canonical packaging, lint config, extras, GSDFC pydoclint)
- add py.typed marker, modernize type hints and Google-style docstrings
- nest generated docs under docs/lib and docs/mcp; add hand-written wiki
- adopt standard Type A PyPI CI pipeline (tag==version guard + quality gate)
- promote generated CHANGELOG.md; bump version to 0.0.3

2026-10-17 02:16:48 +05:30

2.5 KiB

Raw Blame History

🖥️ How to Use

Every format follows the same pipeline: scrape → Content → parse. This page shows each supported format with working patterns.

🌐 HTML

from omniread import HTMLScraper, HTMLParser

class TitleParser(HTMLParser[str]):
    def parse(self) -> str:
        return self._soup.title.string

scraper = HTMLScraper()                     # httpx under the hood
content = scraper.fetch("https://example.com")

title = TitleParser(content).parse()

HTMLScraper accepts an optional client (an httpx.Client) for transport control — the test suite wires one to a mock transport.

📕 PDF

PDFs need a client to supply raw bytes before parsing:

from pathlib import Path
from omniread import FileSystemPDFClient, PDFScraper, PDFParser

class TextPDFParser(PDFParser[str]):
    def parse(self) -> str:
        # implement your extraction logic
        return self.content.raw  # bytes, decode as needed

client = FileSystemPDFClient()
scraper = PDFScraper(client=client)
content = scraper.fetch(Path("document.pdf"))

result = TextPDFParser(content).parse()

PDFParser subclasses receive self.content and implement parse().

📊 CSV

from omniread import FileSystemCsvClient, CsvScraper, CsvParser

scraper = CsvScraper(client=FileSystemCsvClient())
content = scraper.fetch("data.csv")

parser = CsvParser(content)
for row in parser.rows():
    print(row)

CsvParser.rows() yields string rows trimmed of empties by default (skip_empty=True).

📑 XLSX

from omniread import FileSystemXlsxClient, XlsxScraper, XlsxParser

scraper = XlsxScraper(client=FileSystemXlsxClient())
content = scraper.fetch("statement.xlsx")

parser = XlsxParser(content)
print(parser.sheet_names)                 # e.g. ["Statement"]
rows = parser.rows(sheet="Statement")     # by name or index
all_rows = parser.parse()                 # alias for rows()

Key behaviors:

rows(skip_empty=False) keeps blank rows (off by default).
Cells render as trimmed strings; date cells convert to ISO format (2026-06-01T00:00:00).
rows(sheet="Missing") raises for an unknown sheet.

🔀 End-to-end flow

content = scraper.fetch(source)   # 1. acquire → Content
assert isinstance(content.raw, bytes)
assert content.content_type is not None

parser = MyParser(content)        # 2. interpret → T
result = parser.parse()

📚 Read Next

Overview — the full architecture.
Extending OmniRead — custom clients and parsers.

2.5 KiB Raw Blame History