Files
omniread/mcp_docs/modules/omniread.json
2026-03-08 00:41:28 +05:30

1639 lines
112 KiB
JSON

{
"module": "omniread",
"content": {
"path": "omniread",
"docstring": "OmniRead — format-agnostic content acquisition and parsing framework.\n\n---\n\n## Summary\n\nOmniRead provides a **cleanly layered architecture** for fetching, parsing,\nand normalizing content from heterogeneous sources such as HTML documents\nand PDF files.\n\nThe library is structured around three core concepts:\n\n1. **Content**: A canonical, format-agnostic container representing raw content bytes and minimal contextual metadata.\n2. **Scrapers**: Components responsible for *acquiring* raw content from a source (HTTP, filesystem, object storage, etc.). Scrapers never interpret content.\n3. **Parsers**: Components responsible for *interpreting* acquired content and converting it into structured, typed representations.\n\nOmniRead deliberately separates these responsibilities to ensure:\n- Clear boundaries between IO and interpretation\n- Replaceable implementations per format\n- Predictable, testable behavior\n\n---\n\n## Installation\n\nInstall OmniRead using pip:\n\n pip install omniread\n\nOr with Poetry:\n\n poetry add omniread\n\n---\n\n## Quick start\n\nHTML example:\n\n from omniread import HTMLScraper, HTMLParser\n\n scraper = HTMLScraper()\n content = scraper.fetch(\"https://example.com\")\n\n class TitleParser(HTMLParser[str]):\n def parse(self) -> str:\n return self._soup.title.string\n\n parser = TitleParser(content)\n title = parser.parse()\n\nPDF example:\n\n from omniread import FileSystemPDFClient, PDFScraper, PDFParser\n from pathlib import Path\n\n client = FileSystemPDFClient()\n scraper = PDFScraper(client=client)\n content = scraper.fetch(Path(\"document.pdf\"))\n\n class TextPDFParser(PDFParser[str]):\n def parse(self) -> str:\n # implement PDF text extraction\n ...\n\n parser = TextPDFParser(content)\n result = parser.parse()\n\n---\n\n## Public API\n\nThis module re-exports the **recommended public entry points** of OmniRead.\nConsumers are encouraged to import from this namespace rather than from\nformat-specific submodules directly, unless advanced customization is\nrequired.\n\n**Core:**\n- Content\n- ContentType\n\n**HTML:**\n- HTMLScraper\n- HTMLParser\n\n**PDF:**\n- FileSystemPDFClient\n- PDFScraper\n- PDFParser\n\n**Core Philosophy:**\n`OmniRead` is designed as a **decoupled content engine**:\n1. **Separation of Concerns**: Scrapers *fetch*, Parsers *interpret*. Neither knows about the other.\n2. **Normalized Exchange**: All components communicate via the `Content` model, ensuring a consistent contract.\n3. **Format Agnosticism**: The core logic is independent of whether the input is HTML, PDF, or JSON.\n\n---",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.Content",
"signature": "<bound method Alias.signature of Alias('Content', 'omniread.core.Content')>",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with minimal contextual metadata describing its origin and type\n - This class is the primary exchange format between Scrapers, Parsers, and Downstream consumers",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.Content.raw",
"signature": "<bound method Alias.signature of Alias('raw', 'omniread.core.content.Content.raw')>",
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.Content.source",
"signature": "<bound method Alias.signature of Alias('source', 'omniread.core.content.Content.source')>",
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.Content.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.core.content.Content.content_type')>",
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.Content.metadata",
"signature": "<bound method Alias.signature of Alias('metadata', 'omniread.core.content.Content.metadata')>",
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.ContentType",
"signature": "<bound method Alias.signature of Alias('ContentType', 'omniread.core.ContentType')>",
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the content source\n - It is primarily used for routing content to the appropriate parser or downstream consumer",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.ContentType.HTML",
"signature": "<bound method Alias.signature of Alias('HTML', 'omniread.core.content.ContentType.HTML')>",
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.ContentType.PDF",
"signature": "<bound method Alias.signature of Alias('PDF', 'omniread.core.content.ContentType.PDF')>",
"docstring": "PDF document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.ContentType.JSON",
"signature": "<bound method Alias.signature of Alias('JSON', 'omniread.core.content.ContentType.JSON')>",
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.ContentType.XML",
"signature": "<bound method Alias.signature of Alias('XML', 'omniread.core.content.ContentType.XML')>",
"docstring": "XML document content."
}
}
},
"HTMLScraper": {
"name": "HTMLScraper",
"kind": "class",
"path": "omniread.HTMLScraper",
"signature": "<bound method Alias.signature of Alias('HTMLScraper', 'omniread.html.HTMLScraper')>",
"docstring": "Base HTML scraper using httpx.\n\nNotes:\n **Responsibilities:**\n\n - This scraper retrieves HTML documents over HTTP(S) and returns them as raw content wrapped in a `Content` object\n - Fetches raw bytes and metadata only. The scraper uses `httpx.Client` for HTTP requests, enforces an HTML content type, preserves HTTP response metadata\n\n **Constraints:**\n \n - The scraper does not: Parse HTML, perform retries or backoff, handle non-HTML responses",
"members": {
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.HTMLScraper.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.html.scraper.HTMLScraper.content_type')>",
"docstring": null
},
"validate_content_type": {
"name": "validate_content_type",
"kind": "function",
"path": "omniread.HTMLScraper.validate_content_type",
"signature": "<bound method Alias.signature of Alias('validate_content_type', 'omniread.html.scraper.HTMLScraper.validate_content_type')>",
"docstring": "Validate that the HTTP response contains HTML content.\n\nArgs:\n response (httpx.Response):\n HTTP response returned by `httpx`.\n\nRaises:\n ValueError:\n If the `Content-Type` header is missing or does not indicate HTML content."
},
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.HTMLScraper.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.html.scraper.HTMLScraper.fetch')>",
"docstring": "Fetch an HTML document from the given source.\n\nArgs:\n source (str):\n URL of the HTML document.\n metadata (Optional[Mapping[str, Any]], optional):\n Optional metadata to be merged into the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw HTML bytes, source URL, HTML content type, and HTTP response metadata.\n\nRaises:\n httpx.HTTPError:\n If the HTTP request fails.\n ValueError:\n If the response is not valid HTML."
}
}
},
"HTMLParser": {
"name": "HTMLParser",
"kind": "class",
"path": "omniread.HTMLParser",
"signature": "<bound method Alias.signature of Alias('HTMLParser', 'omniread.html.HTMLParser')>",
"docstring": "Base HTML parser.\n\nNotes:\n **Responsibilities:**\n\n - This class extends the core `BaseParser` with HTML-specific behavior, including DOM parsing via BeautifulSoup and reusable extraction helpers\n - Provides reusable helpers for HTML extraction. Concrete parsers must explicitly define the return type\n\n **Guarantees:**\n\n - Characteristics: Accepts only HTML content, owns a parsed BeautifulSoup DOM tree, provides pure helper utilities for common HTML structures\n\n **Constraints:**\n \n - Concrete subclasses must define the output type `T` and implement the `parse()` method",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.HTMLParser.supported_types",
"signature": "<bound method Alias.signature of Alias('supported_types', 'omniread.html.parser.HTMLParser.supported_types')>",
"docstring": "Set of content types supported by this parser (HTML only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.HTMLParser.parse",
"signature": "<bound method Alias.signature of Alias('parse', 'omniread.html.parser.HTMLParser.parse')>",
"docstring": "Fully parse the HTML content into structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the HTML DOM and return a deterministic, structured output"
},
"parse_div": {
"name": "parse_div",
"kind": "function",
"path": "omniread.HTMLParser.parse_div",
"signature": "<bound method Alias.signature of Alias('parse_div', 'omniread.html.parser.HTMLParser.parse_div')>",
"docstring": "Extract normalized text from a `<div>` element.\n\nArgs:\n div (Tag):\n BeautifulSoup tag representing a `<div>`.\n separator (str, optional):\n String used to separate text nodes.\n\nReturns:\n str:\n Flattened, whitespace-normalized text content."
},
"parse_link": {
"name": "parse_link",
"kind": "function",
"path": "omniread.HTMLParser.parse_link",
"signature": "<bound method Alias.signature of Alias('parse_link', 'omniread.html.parser.HTMLParser.parse_link')>",
"docstring": "Extract the hyperlink reference from an `<a>` element.\n\nArgs:\n a (Tag):\n BeautifulSoup tag representing an anchor.\n\nReturns:\n Optional[str]:\n The value of the `href` attribute, or None if absent."
},
"parse_table": {
"name": "parse_table",
"kind": "function",
"path": "omniread.HTMLParser.parse_table",
"signature": "<bound method Alias.signature of Alias('parse_table', 'omniread.html.parser.HTMLParser.parse_table')>",
"docstring": "Parse an HTML table into a 2D list of strings.\n\nArgs:\n table (Tag):\n BeautifulSoup tag representing a `<table>`.\n\nReturns:\n list[list[str]]:\n A list of rows, where each row is a list of cell text values."
},
"parse_meta": {
"name": "parse_meta",
"kind": "function",
"path": "omniread.HTMLParser.parse_meta",
"signature": "<bound method Alias.signature of Alias('parse_meta', 'omniread.html.parser.HTMLParser.parse_meta')>",
"docstring": "Extract high-level metadata from the HTML document.\n\nReturns:\n dict[str, Any]:\n Dictionary containing extracted metadata.\n\nNotes:\n **Responsibilities:**\n\n - Extract high-level metadata from the HTML document\n - This includes: Document title, `<meta>` tag name/property → content mappings"
}
}
},
"FileSystemPDFClient": {
"name": "FileSystemPDFClient",
"kind": "class",
"path": "omniread.FileSystemPDFClient",
"signature": "<bound method Alias.signature of Alias('FileSystemPDFClient', 'omniread.pdf.FileSystemPDFClient')>",
"docstring": "PDF client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads PDF files directly from the disk and returns their raw binary contents",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.FileSystemPDFClient.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.pdf.client.FileSystemPDFClient.fetch')>",
"docstring": "Read a PDF file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the PDF file.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
},
"PDFScraper": {
"name": "PDFScraper",
"kind": "class",
"path": "omniread.PDFScraper",
"signature": "<bound method Alias.signature of Alias('PDFScraper', 'omniread.pdf.PDFScraper')>",
"docstring": "Scraper for PDF sources.\n\nNotes:\n **Responsibilities:**\n\n - Delegates byte retrieval to a PDF client and normalizes output into Content\n - Preserves caller-provided metadata\n\n **Constraints:**\n \n - The scraper: Does not perform parsing or interpretation, does not assume a specific storage backend",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.PDFScraper.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.pdf.scraper.PDFScraper.fetch')>",
"docstring": "Fetch a PDF document from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF source as understood by the configured PDF client.\n metadata (Optional[Mapping[str, Any]], optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw PDF bytes, source identifier, PDF content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the PDF client."
}
}
},
"PDFParser": {
"name": "PDFParser",
"kind": "class",
"path": "omniread.PDFParser",
"signature": "<bound method Alias.signature of Alias('PDFParser', 'omniread.pdf.PDFParser')>",
"docstring": "Base PDF parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces PDF content-type compatibility and provides the extension point for implementing concrete PDF parsing strategies\n\n **Constraints:**\n\n - Concrete implementations must: Define the output type `T`, implement the `parse()` method",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.PDFParser.supported_types",
"signature": "<bound method Alias.signature of Alias('supported_types', 'omniread.pdf.parser.PDFParser.supported_types')>",
"docstring": "Set of content types supported by this parser (PDF only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.PDFParser.parse",
"signature": "<bound method Alias.signature of Alias('parse', 'omniread.pdf.parser.PDFParser.parse')>",
"docstring": "Parse PDF content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the PDF binary payload and return a deterministic, structured output"
}
}
},
"core": {
"name": "core",
"kind": "module",
"path": "omniread.core",
"signature": null,
"docstring": "Core domain contracts for OmniRead.\n\n---\n\n## Summary\n\nThis package defines the **format-agnostic domain layer** of OmniRead.\nIt exposes canonical content models and abstract interfaces that are\nimplemented by format-specific modules (HTML, PDF, etc.).\n\nPublic exports from this package are considered **stable contracts** and\nare safe for downstream consumers to depend on.\n\nSubmodules:\n- content: Canonical content models and enums\n- parser: Abstract parsing contracts\n- scraper: Abstract scraping contracts\n\nFormat-specific behavior must not be introduced at this layer.\n\n---\n\n## Public API\n\n Content\n ContentType\n\n---",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.Content",
"signature": "<bound method Alias.signature of Alias('Content', 'omniread.core.content.Content')>",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with minimal contextual metadata describing its origin and type\n - This class is the primary exchange format between Scrapers, Parsers, and Downstream consumers",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.Content.raw",
"signature": "<bound method Alias.signature of Alias('raw', 'omniread.core.content.Content.raw')>",
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.Content.source",
"signature": "<bound method Alias.signature of Alias('source', 'omniread.core.content.Content.source')>",
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.Content.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.core.content.Content.content_type')>",
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.Content.metadata",
"signature": "<bound method Alias.signature of Alias('metadata', 'omniread.core.content.Content.metadata')>",
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.core.ContentType",
"signature": "<bound method Alias.signature of Alias('ContentType', 'omniread.core.content.ContentType')>",
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the content source\n - It is primarily used for routing content to the appropriate parser or downstream consumer",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.core.ContentType.HTML",
"signature": "<bound method Alias.signature of Alias('HTML', 'omniread.core.content.ContentType.HTML')>",
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.core.ContentType.PDF",
"signature": "<bound method Alias.signature of Alias('PDF', 'omniread.core.content.ContentType.PDF')>",
"docstring": "PDF document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.core.ContentType.JSON",
"signature": "<bound method Alias.signature of Alias('JSON', 'omniread.core.content.ContentType.JSON')>",
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.core.ContentType.XML",
"signature": "<bound method Alias.signature of Alias('XML', 'omniread.core.content.ContentType.XML')>",
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.core.BaseParser",
"signature": "<bound method Alias.signature of Alias('BaseParser', 'omniread.core.parser.BaseParser')>",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the Content it is responsible for interpreting\n - Consumers may rely on early validation of content compatibility and type-stable return values from `parse()`\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`\n - Implementations must raise parsing-specific exceptions from `parse()`\n - Implementations must remain deterministic for a given input",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.core.BaseParser.supported_types",
"signature": "<bound method Alias.signature of Alias('supported_types', 'omniread.core.parser.BaseParser.supported_types')>",
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.core.BaseParser.content",
"signature": "<bound method Alias.signature of Alias('content', 'omniread.core.parser.BaseParser.content')>",
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.core.BaseParser.parse",
"signature": "<bound method Alias.signature of Alias('parse', 'omniread.core.parser.BaseParser.parse')>",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and return a deterministic, structured output"
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.core.BaseParser.supports",
"signature": "<bound method Alias.signature of Alias('supports', 'omniread.core.parser.BaseParser.supports')>",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.core.BaseScraper",
"signature": "<bound method Alias.signature of Alias('BaseScraper', 'omniread.core.scraper.BaseScraper')>",
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes) from a source. It must not interpret or parse it\n - A scraper is a stateless acquisition component that retrieves raw content from a source and returns it as a `Content` object\n - Scrapers define how content is obtained, not what the content means\n - Implementations may vary in transport mechanism, authentication strategy, retry and backoff behavior\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics, or couple scraping logic to a specific parser",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.core.BaseScraper.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.core.scraper.BaseScraper.fetch')>",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.)\n metadata (Optional[Mapping[str, Any]], optional):\n Optional hints for the scraper (headers, auth, etc.)\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source` and return it as raw bytes wrapped in a `Content` object"
}
}
},
"content": {
"name": "content",
"kind": "module",
"path": "omniread.core.content",
"signature": null,
"docstring": "Canonical content models for OmniRead.\n\n---\n\n## Summary\n\nThis module defines the **format-agnostic content representation** used across\nall parsers and scrapers in OmniRead.\n\nThe models defined here represent *what* was extracted, not *how* it was\nretrieved or parsed. Format-specific behavior and metadata must not alter\nthe semantic meaning of these models.",
"members": {
"Enum": {
"name": "Enum",
"kind": "alias",
"path": "omniread.core.content.Enum",
"signature": "<bound method Alias.signature of Alias('Enum', 'enum.Enum')>",
"docstring": null
},
"dataclass": {
"name": "dataclass",
"kind": "alias",
"path": "omniread.core.content.dataclass",
"signature": "<bound method Alias.signature of Alias('dataclass', 'dataclasses.dataclass')>",
"docstring": null
},
"Any": {
"name": "Any",
"kind": "alias",
"path": "omniread.core.content.Any",
"signature": "<bound method Alias.signature of Alias('Any', 'typing.Any')>",
"docstring": null
},
"Mapping": {
"name": "Mapping",
"kind": "alias",
"path": "omniread.core.content.Mapping",
"signature": "<bound method Alias.signature of Alias('Mapping', 'typing.Mapping')>",
"docstring": null
},
"Optional": {
"name": "Optional",
"kind": "alias",
"path": "omniread.core.content.Optional",
"signature": "<bound method Alias.signature of Alias('Optional', 'typing.Optional')>",
"docstring": null
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.core.content.ContentType",
"signature": "<bound method Class.signature of Class('ContentType', 21, 42)>",
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the content source\n - It is primarily used for routing content to the appropriate parser or downstream consumer",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.core.content.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.core.content.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.core.content.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.core.content.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.content.Content",
"signature": "<bound method Class.signature of Class('Content', 45, 75)>",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with minimal contextual metadata describing its origin and type\n - This class is the primary exchange format between Scrapers, Parsers, and Downstream consumers",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.content.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.content.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.content.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.content.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
}
}
},
"parser": {
"name": "parser",
"kind": "module",
"path": "omniread.core.parser",
"signature": null,
"docstring": "Abstract parsing contracts for OmniRead.\n\n---\n\n## Summary\n\nThis module defines the **format-agnostic parser interface** used to transform\nraw content into structured, typed representations.\n\nParsers are responsible for:\n- Interpreting a single `Content` instance\n- Validating compatibility with the content type\n- Producing a structured output suitable for downstream consumers\n\nParsers are not responsible for:\n- Fetching or acquiring content\n- Performing retries or error recovery\n- Managing multiple content sources",
"members": {
"ABC": {
"name": "ABC",
"kind": "alias",
"path": "omniread.core.parser.ABC",
"signature": "<bound method Alias.signature of Alias('ABC', 'abc.ABC')>",
"docstring": null
},
"abstractmethod": {
"name": "abstractmethod",
"kind": "alias",
"path": "omniread.core.parser.abstractmethod",
"signature": "<bound method Alias.signature of Alias('abstractmethod', 'abc.abstractmethod')>",
"docstring": null
},
"Generic": {
"name": "Generic",
"kind": "alias",
"path": "omniread.core.parser.Generic",
"signature": "<bound method Alias.signature of Alias('Generic', 'typing.Generic')>",
"docstring": null
},
"TypeVar": {
"name": "TypeVar",
"kind": "alias",
"path": "omniread.core.parser.TypeVar",
"signature": "<bound method Alias.signature of Alias('TypeVar', 'typing.TypeVar')>",
"docstring": null
},
"Set": {
"name": "Set",
"kind": "alias",
"path": "omniread.core.parser.Set",
"signature": "<bound method Alias.signature of Alias('Set', 'typing.Set')>",
"docstring": null
},
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.parser.Content",
"signature": "<bound method Alias.signature of Alias('Content', 'omniread.core.content.Content')>",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with minimal contextual metadata describing its origin and type\n - This class is the primary exchange format between Scrapers, Parsers, and Downstream consumers",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.parser.Content.raw",
"signature": "<bound method Alias.signature of Alias('raw', 'omniread.core.content.Content.raw')>",
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.parser.Content.source",
"signature": "<bound method Alias.signature of Alias('source', 'omniread.core.content.Content.source')>",
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.parser.Content.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.core.content.Content.content_type')>",
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.parser.Content.metadata",
"signature": "<bound method Alias.signature of Alias('metadata', 'omniread.core.content.Content.metadata')>",
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.core.parser.ContentType",
"signature": "<bound method Alias.signature of Alias('ContentType', 'omniread.core.content.ContentType')>",
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the content source\n - It is primarily used for routing content to the appropriate parser or downstream consumer",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.HTML",
"signature": "<bound method Alias.signature of Alias('HTML', 'omniread.core.content.ContentType.HTML')>",
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.PDF",
"signature": "<bound method Alias.signature of Alias('PDF', 'omniread.core.content.ContentType.PDF')>",
"docstring": "PDF document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.JSON",
"signature": "<bound method Alias.signature of Alias('JSON', 'omniread.core.content.ContentType.JSON')>",
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.XML",
"signature": "<bound method Alias.signature of Alias('XML', 'omniread.core.content.ContentType.XML')>",
"docstring": "XML document content."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.core.parser.T",
"signature": null,
"docstring": null
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.core.parser.BaseParser",
"signature": "<bound method Class.signature of Class('BaseParser', 30, 108)>",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the Content it is responsible for interpreting\n - Consumers may rely on early validation of content compatibility and type-stable return values from `parse()`\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`\n - Implementations must raise parsing-specific exceptions from `parse()`\n - Implementations must remain deterministic for a given input",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.core.parser.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.core.parser.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.core.parser.BaseParser.parse",
"signature": "<bound method Function.signature of Function('parse', 73, 91)>",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and return a deterministic, structured output"
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.core.parser.BaseParser.supports",
"signature": "<bound method Function.signature of Function('supports', 93, 108)>",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
}
}
},
"scraper": {
"name": "scraper",
"kind": "module",
"path": "omniread.core.scraper",
"signature": null,
"docstring": "Abstract scraping contracts for OmniRead.\n\n---\n\n## Summary\n\nThis module defines the **format-agnostic scraper interface** responsible for\nacquiring raw content from external sources.\n\nScrapers are responsible for:\n- Locating and retrieving raw content bytes\n- Attaching minimal contextual metadata\n- Returning normalized `Content` objects\n\nScrapers are explicitly NOT responsible for:\n- Parsing or interpreting content\n- Inferring structure or semantics\n- Performing content-type specific processing\n\nAll interpretation must be delegated to parsers.",
"members": {
"ABC": {
"name": "ABC",
"kind": "alias",
"path": "omniread.core.scraper.ABC",
"signature": "<bound method Alias.signature of Alias('ABC', 'abc.ABC')>",
"docstring": null
},
"abstractmethod": {
"name": "abstractmethod",
"kind": "alias",
"path": "omniread.core.scraper.abstractmethod",
"signature": "<bound method Alias.signature of Alias('abstractmethod', 'abc.abstractmethod')>",
"docstring": null
},
"Any": {
"name": "Any",
"kind": "alias",
"path": "omniread.core.scraper.Any",
"signature": "<bound method Alias.signature of Alias('Any', 'typing.Any')>",
"docstring": null
},
"Mapping": {
"name": "Mapping",
"kind": "alias",
"path": "omniread.core.scraper.Mapping",
"signature": "<bound method Alias.signature of Alias('Mapping', 'typing.Mapping')>",
"docstring": null
},
"Optional": {
"name": "Optional",
"kind": "alias",
"path": "omniread.core.scraper.Optional",
"signature": "<bound method Alias.signature of Alias('Optional', 'typing.Optional')>",
"docstring": null
},
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.scraper.Content",
"signature": "<bound method Alias.signature of Alias('Content', 'omniread.core.content.Content')>",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with minimal contextual metadata describing its origin and type\n - This class is the primary exchange format between Scrapers, Parsers, and Downstream consumers",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.scraper.Content.raw",
"signature": "<bound method Alias.signature of Alias('raw', 'omniread.core.content.Content.raw')>",
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.scraper.Content.source",
"signature": "<bound method Alias.signature of Alias('source', 'omniread.core.content.Content.source')>",
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.scraper.Content.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.core.content.Content.content_type')>",
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.scraper.Content.metadata",
"signature": "<bound method Alias.signature of Alias('metadata', 'omniread.core.content.Content.metadata')>",
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.core.scraper.BaseScraper",
"signature": "<bound method Class.signature of Class('BaseScraper', 30, 76)>",
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes) from a source. It must not interpret or parse it\n - A scraper is a stateless acquisition component that retrieves raw content from a source and returns it as a `Content` object\n - Scrapers define how content is obtained, not what the content means\n - Implementations may vary in transport mechanism, authentication strategy, retry and backoff behavior\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics, or couple scraping logic to a specific parser",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.core.scraper.BaseScraper.fetch",
"signature": "<bound method Function.signature of Function('fetch', 47, 76)>",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.)\n metadata (Optional[Mapping[str, Any]], optional):\n Optional hints for the scraper (headers, auth, etc.)\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source` and return it as raw bytes wrapped in a `Content` object"
}
}
}
}
}
}
},
"html": {
"name": "html",
"kind": "module",
"path": "omniread.html",
"signature": null,
"docstring": "HTML format implementation for OmniRead.\n\n---\n\n## Summary\n\nThis package provides **HTML-specific implementations** of the core OmniRead\ncontracts defined in `omniread.core`.\n\nIt includes:\n- HTML parsers that interpret HTML content\n- HTML scrapers that retrieve HTML documents\n\nThis package:\n- Implements, but does not redefine, core contracts\n- May contain HTML-specific behavior and edge-case handling\n- Produces canonical content models defined in `omniread.core.content`\n\nConsumers should depend on `omniread.core` interfaces wherever possible and\nuse this package only when HTML-specific behavior is required.\n\n---\n\n## Public API\n\n HTMLScraper\n HTMLParser\n\n---",
"members": {
"HTMLScraper": {
"name": "HTMLScraper",
"kind": "class",
"path": "omniread.html.HTMLScraper",
"signature": "<bound method Alias.signature of Alias('HTMLScraper', 'omniread.html.scraper.HTMLScraper')>",
"docstring": "Base HTML scraper using httpx.\n\nNotes:\n **Responsibilities:**\n\n - This scraper retrieves HTML documents over HTTP(S) and returns them as raw content wrapped in a `Content` object\n - Fetches raw bytes and metadata only. The scraper uses `httpx.Client` for HTTP requests, enforces an HTML content type, preserves HTTP response metadata\n\n **Constraints:**\n \n - The scraper does not: Parse HTML, perform retries or backoff, handle non-HTML responses",
"members": {
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.HTMLScraper.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.html.scraper.HTMLScraper.content_type')>",
"docstring": null
},
"validate_content_type": {
"name": "validate_content_type",
"kind": "function",
"path": "omniread.html.HTMLScraper.validate_content_type",
"signature": "<bound method Alias.signature of Alias('validate_content_type', 'omniread.html.scraper.HTMLScraper.validate_content_type')>",
"docstring": "Validate that the HTTP response contains HTML content.\n\nArgs:\n response (httpx.Response):\n HTTP response returned by `httpx`.\n\nRaises:\n ValueError:\n If the `Content-Type` header is missing or does not indicate HTML content."
},
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.html.HTMLScraper.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.html.scraper.HTMLScraper.fetch')>",
"docstring": "Fetch an HTML document from the given source.\n\nArgs:\n source (str):\n URL of the HTML document.\n metadata (Optional[Mapping[str, Any]], optional):\n Optional metadata to be merged into the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw HTML bytes, source URL, HTML content type, and HTTP response metadata.\n\nRaises:\n httpx.HTTPError:\n If the HTTP request fails.\n ValueError:\n If the response is not valid HTML."
}
}
},
"HTMLParser": {
"name": "HTMLParser",
"kind": "class",
"path": "omniread.html.HTMLParser",
"signature": "<bound method Alias.signature of Alias('HTMLParser', 'omniread.html.parser.HTMLParser')>",
"docstring": "Base HTML parser.\n\nNotes:\n **Responsibilities:**\n\n - This class extends the core `BaseParser` with HTML-specific behavior, including DOM parsing via BeautifulSoup and reusable extraction helpers\n - Provides reusable helpers for HTML extraction. Concrete parsers must explicitly define the return type\n\n **Guarantees:**\n\n - Characteristics: Accepts only HTML content, owns a parsed BeautifulSoup DOM tree, provides pure helper utilities for common HTML structures\n\n **Constraints:**\n \n - Concrete subclasses must define the output type `T` and implement the `parse()` method",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.html.HTMLParser.supported_types",
"signature": "<bound method Alias.signature of Alias('supported_types', 'omniread.html.parser.HTMLParser.supported_types')>",
"docstring": "Set of content types supported by this parser (HTML only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.html.HTMLParser.parse",
"signature": "<bound method Alias.signature of Alias('parse', 'omniread.html.parser.HTMLParser.parse')>",
"docstring": "Fully parse the HTML content into structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the HTML DOM and return a deterministic, structured output"
},
"parse_div": {
"name": "parse_div",
"kind": "function",
"path": "omniread.html.HTMLParser.parse_div",
"signature": "<bound method Alias.signature of Alias('parse_div', 'omniread.html.parser.HTMLParser.parse_div')>",
"docstring": "Extract normalized text from a `<div>` element.\n\nArgs:\n div (Tag):\n BeautifulSoup tag representing a `<div>`.\n separator (str, optional):\n String used to separate text nodes.\n\nReturns:\n str:\n Flattened, whitespace-normalized text content."
},
"parse_link": {
"name": "parse_link",
"kind": "function",
"path": "omniread.html.HTMLParser.parse_link",
"signature": "<bound method Alias.signature of Alias('parse_link', 'omniread.html.parser.HTMLParser.parse_link')>",
"docstring": "Extract the hyperlink reference from an `<a>` element.\n\nArgs:\n a (Tag):\n BeautifulSoup tag representing an anchor.\n\nReturns:\n Optional[str]:\n The value of the `href` attribute, or None if absent."
},
"parse_table": {
"name": "parse_table",
"kind": "function",
"path": "omniread.html.HTMLParser.parse_table",
"signature": "<bound method Alias.signature of Alias('parse_table', 'omniread.html.parser.HTMLParser.parse_table')>",
"docstring": "Parse an HTML table into a 2D list of strings.\n\nArgs:\n table (Tag):\n BeautifulSoup tag representing a `<table>`.\n\nReturns:\n list[list[str]]:\n A list of rows, where each row is a list of cell text values."
},
"parse_meta": {
"name": "parse_meta",
"kind": "function",
"path": "omniread.html.HTMLParser.parse_meta",
"signature": "<bound method Alias.signature of Alias('parse_meta', 'omniread.html.parser.HTMLParser.parse_meta')>",
"docstring": "Extract high-level metadata from the HTML document.\n\nReturns:\n dict[str, Any]:\n Dictionary containing extracted metadata.\n\nNotes:\n **Responsibilities:**\n\n - Extract high-level metadata from the HTML document\n - This includes: Document title, `<meta>` tag name/property → content mappings"
}
}
},
"parser": {
"name": "parser",
"kind": "module",
"path": "omniread.html.parser",
"signature": null,
"docstring": "HTML parser base implementations for OmniRead.\n\n---\n\n## Summary\n\nThis module provides reusable HTML parsing utilities built on top of\nthe abstract parser contracts defined in `omniread.core.parser`.\n\nIt supplies:\n- Content-type enforcement for HTML inputs\n- BeautifulSoup initialization and lifecycle management\n- Common helper methods for extracting structured data from HTML elements\n\nConcrete parsers must subclass `HTMLParser` and implement the `parse()` method\nto return a structured representation appropriate for their use case.",
"members": {
"Any": {
"name": "Any",
"kind": "alias",
"path": "omniread.html.parser.Any",
"signature": "<bound method Alias.signature of Alias('Any', 'typing.Any')>",
"docstring": null
},
"Generic": {
"name": "Generic",
"kind": "alias",
"path": "omniread.html.parser.Generic",
"signature": "<bound method Alias.signature of Alias('Generic', 'typing.Generic')>",
"docstring": null
},
"TypeVar": {
"name": "TypeVar",
"kind": "alias",
"path": "omniread.html.parser.TypeVar",
"signature": "<bound method Alias.signature of Alias('TypeVar', 'typing.TypeVar')>",
"docstring": null
},
"Optional": {
"name": "Optional",
"kind": "alias",
"path": "omniread.html.parser.Optional",
"signature": "<bound method Alias.signature of Alias('Optional', 'typing.Optional')>",
"docstring": null
},
"abstractmethod": {
"name": "abstractmethod",
"kind": "alias",
"path": "omniread.html.parser.abstractmethod",
"signature": "<bound method Alias.signature of Alias('abstractmethod', 'abc.abstractmethod')>",
"docstring": null
},
"BeautifulSoup": {
"name": "BeautifulSoup",
"kind": "alias",
"path": "omniread.html.parser.BeautifulSoup",
"signature": "<bound method Alias.signature of Alias('BeautifulSoup', 'bs4.BeautifulSoup')>",
"docstring": null
},
"Tag": {
"name": "Tag",
"kind": "alias",
"path": "omniread.html.parser.Tag",
"signature": "<bound method Alias.signature of Alias('Tag', 'bs4.Tag')>",
"docstring": null
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.html.parser.ContentType",
"signature": "<bound method Alias.signature of Alias('ContentType', 'omniread.core.content.ContentType')>",
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the content source\n - It is primarily used for routing content to the appropriate parser or downstream consumer",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.HTML",
"signature": "<bound method Alias.signature of Alias('HTML', 'omniread.core.content.ContentType.HTML')>",
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.PDF",
"signature": "<bound method Alias.signature of Alias('PDF', 'omniread.core.content.ContentType.PDF')>",
"docstring": "PDF document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.JSON",
"signature": "<bound method Alias.signature of Alias('JSON', 'omniread.core.content.ContentType.JSON')>",
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.XML",
"signature": "<bound method Alias.signature of Alias('XML', 'omniread.core.content.ContentType.XML')>",
"docstring": "XML document content."
}
}
},
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.html.parser.Content",
"signature": "<bound method Alias.signature of Alias('Content', 'omniread.core.content.Content')>",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with minimal contextual metadata describing its origin and type\n - This class is the primary exchange format between Scrapers, Parsers, and Downstream consumers",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.html.parser.Content.raw",
"signature": "<bound method Alias.signature of Alias('raw', 'omniread.core.content.Content.raw')>",
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.html.parser.Content.source",
"signature": "<bound method Alias.signature of Alias('source', 'omniread.core.content.Content.source')>",
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.parser.Content.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.core.content.Content.content_type')>",
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.html.parser.Content.metadata",
"signature": "<bound method Alias.signature of Alias('metadata', 'omniread.core.content.Content.metadata')>",
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.html.parser.BaseParser",
"signature": "<bound method Alias.signature of Alias('BaseParser', 'omniread.core.parser.BaseParser')>",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the Content it is responsible for interpreting\n - Consumers may rely on early validation of content compatibility and type-stable return values from `parse()`\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`\n - Implementations must raise parsing-specific exceptions from `parse()`\n - Implementations must remain deterministic for a given input",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.html.parser.BaseParser.supported_types",
"signature": "<bound method Alias.signature of Alias('supported_types', 'omniread.core.parser.BaseParser.supported_types')>",
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.html.parser.BaseParser.content",
"signature": "<bound method Alias.signature of Alias('content', 'omniread.core.parser.BaseParser.content')>",
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.html.parser.BaseParser.parse",
"signature": "<bound method Alias.signature of Alias('parse', 'omniread.core.parser.BaseParser.parse')>",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and return a deterministic, structured output"
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.html.parser.BaseParser.supports",
"signature": "<bound method Alias.signature of Alias('supports', 'omniread.core.parser.BaseParser.supports')>",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.html.parser.T",
"signature": null,
"docstring": null
},
"HTMLParser": {
"name": "HTMLParser",
"kind": "class",
"path": "omniread.html.parser.HTMLParser",
"signature": "<bound method Class.signature of Class('HTMLParser', 31, 199)>",
"docstring": "Base HTML parser.\n\nNotes:\n **Responsibilities:**\n\n - This class extends the core `BaseParser` with HTML-specific behavior, including DOM parsing via BeautifulSoup and reusable extraction helpers\n - Provides reusable helpers for HTML extraction. Concrete parsers must explicitly define the return type\n\n **Guarantees:**\n\n - Characteristics: Accepts only HTML content, owns a parsed BeautifulSoup DOM tree, provides pure helper utilities for common HTML structures\n\n **Constraints:**\n \n - Concrete subclasses must define the output type `T` and implement the `parse()` method",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.html.parser.HTMLParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (HTML only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse",
"signature": "<bound method Function.signature of Function('parse', 77, 91)>",
"docstring": "Fully parse the HTML content into structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the HTML DOM and return a deterministic, structured output"
},
"parse_div": {
"name": "parse_div",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_div",
"signature": "<bound method Function.signature of Function('parse_div', 97, 112)>",
"docstring": "Extract normalized text from a `<div>` element.\n\nArgs:\n div (Tag):\n BeautifulSoup tag representing a `<div>`.\n separator (str, optional):\n String used to separate text nodes.\n\nReturns:\n str:\n Flattened, whitespace-normalized text content."
},
"parse_link": {
"name": "parse_link",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_link",
"signature": "<bound method Function.signature of Function('parse_link', 114, 127)>",
"docstring": "Extract the hyperlink reference from an `<a>` element.\n\nArgs:\n a (Tag):\n BeautifulSoup tag representing an anchor.\n\nReturns:\n Optional[str]:\n The value of the `href` attribute, or None if absent."
},
"parse_table": {
"name": "parse_table",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_table",
"signature": "<bound method Function.signature of Function('parse_table', 129, 150)>",
"docstring": "Parse an HTML table into a 2D list of strings.\n\nArgs:\n table (Tag):\n BeautifulSoup tag representing a `<table>`.\n\nReturns:\n list[list[str]]:\n A list of rows, where each row is a list of cell text values."
},
"parse_meta": {
"name": "parse_meta",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_meta",
"signature": "<bound method Function.signature of Function('parse_meta', 172, 199)>",
"docstring": "Extract high-level metadata from the HTML document.\n\nReturns:\n dict[str, Any]:\n Dictionary containing extracted metadata.\n\nNotes:\n **Responsibilities:**\n\n - Extract high-level metadata from the HTML document\n - This includes: Document title, `<meta>` tag name/property → content mappings"
}
}
},
"list": {
"name": "list",
"kind": "alias",
"path": "omniread.html.parser.list",
"signature": "<bound method Alias.signature of Alias('list', 'typing.list')>",
"docstring": null
},
"dict": {
"name": "dict",
"kind": "alias",
"path": "omniread.html.parser.dict",
"signature": "<bound method Alias.signature of Alias('dict', 'typing.dict')>",
"docstring": null
}
}
},
"scraper": {
"name": "scraper",
"kind": "module",
"path": "omniread.html.scraper",
"signature": null,
"docstring": "HTML scraping implementation for OmniRead.\n\n---\n\n## Summary\n\nThis module provides an HTTP-based scraper for retrieving HTML documents.\nIt implements the core `BaseScraper` contract using `httpx` as the transport\nlayer.\n\nThis scraper is responsible for:\n- Fetching raw HTML bytes over HTTP(S)\n- Validating response content type\n- Attaching HTTP metadata to the returned content\n\nThis scraper is not responsible for:\n- Parsing or interpreting HTML\n- Retrying failed requests\n- Managing crawl policies or rate limiting",
"members": {
"httpx": {
"name": "httpx",
"kind": "alias",
"path": "omniread.html.scraper.httpx",
"signature": "<bound method Alias.signature of Alias('httpx', 'httpx')>",
"docstring": null
},
"Any": {
"name": "Any",
"kind": "alias",
"path": "omniread.html.scraper.Any",
"signature": "<bound method Alias.signature of Alias('Any', 'typing.Any')>",
"docstring": null
},
"Mapping": {
"name": "Mapping",
"kind": "alias",
"path": "omniread.html.scraper.Mapping",
"signature": "<bound method Alias.signature of Alias('Mapping', 'typing.Mapping')>",
"docstring": null
},
"Optional": {
"name": "Optional",
"kind": "alias",
"path": "omniread.html.scraper.Optional",
"signature": "<bound method Alias.signature of Alias('Optional', 'typing.Optional')>",
"docstring": null
},
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.html.scraper.Content",
"signature": "<bound method Alias.signature of Alias('Content', 'omniread.core.content.Content')>",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with minimal contextual metadata describing its origin and type\n - This class is the primary exchange format between Scrapers, Parsers, and Downstream consumers",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.html.scraper.Content.raw",
"signature": "<bound method Alias.signature of Alias('raw', 'omniread.core.content.Content.raw')>",
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.html.scraper.Content.source",
"signature": "<bound method Alias.signature of Alias('source', 'omniread.core.content.Content.source')>",
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.scraper.Content.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.core.content.Content.content_type')>",
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.html.scraper.Content.metadata",
"signature": "<bound method Alias.signature of Alias('metadata', 'omniread.core.content.Content.metadata')>",
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.html.scraper.ContentType",
"signature": "<bound method Alias.signature of Alias('ContentType', 'omniread.core.content.ContentType')>",
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the content source\n - It is primarily used for routing content to the appropriate parser or downstream consumer",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.HTML",
"signature": "<bound method Alias.signature of Alias('HTML', 'omniread.core.content.ContentType.HTML')>",
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.PDF",
"signature": "<bound method Alias.signature of Alias('PDF', 'omniread.core.content.ContentType.PDF')>",
"docstring": "PDF document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.JSON",
"signature": "<bound method Alias.signature of Alias('JSON', 'omniread.core.content.ContentType.JSON')>",
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.XML",
"signature": "<bound method Alias.signature of Alias('XML', 'omniread.core.content.ContentType.XML')>",
"docstring": "XML document content."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.html.scraper.BaseScraper",
"signature": "<bound method Alias.signature of Alias('BaseScraper', 'omniread.core.scraper.BaseScraper')>",
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes) from a source. It must not interpret or parse it\n - A scraper is a stateless acquisition component that retrieves raw content from a source and returns it as a `Content` object\n - Scrapers define how content is obtained, not what the content means\n - Implementations may vary in transport mechanism, authentication strategy, retry and backoff behavior\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics, or couple scraping logic to a specific parser",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.html.scraper.BaseScraper.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.core.scraper.BaseScraper.fetch')>",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.)\n metadata (Optional[Mapping[str, Any]], optional):\n Optional hints for the scraper (headers, auth, etc.)\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source` and return it as raw bytes wrapped in a `Content` object"
}
}
},
"HTMLScraper": {
"name": "HTMLScraper",
"kind": "class",
"path": "omniread.html.scraper.HTMLScraper",
"signature": "<bound method Class.signature of Class('HTMLScraper', 30, 139)>",
"docstring": "Base HTML scraper using httpx.\n\nNotes:\n **Responsibilities:**\n\n - This scraper retrieves HTML documents over HTTP(S) and returns them as raw content wrapped in a `Content` object\n - Fetches raw bytes and metadata only. The scraper uses `httpx.Client` for HTTP requests, enforces an HTML content type, preserves HTTP response metadata\n\n **Constraints:**\n \n - The scraper does not: Parse HTML, perform retries or backoff, handle non-HTML responses",
"members": {
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.scraper.HTMLScraper.content_type",
"signature": null,
"docstring": null
},
"validate_content_type": {
"name": "validate_content_type",
"kind": "function",
"path": "omniread.html.scraper.HTMLScraper.validate_content_type",
"signature": "<bound method Function.signature of Function('validate_content_type', 74, 98)>",
"docstring": "Validate that the HTTP response contains HTML content.\n\nArgs:\n response (httpx.Response):\n HTTP response returned by `httpx`.\n\nRaises:\n ValueError:\n If the `Content-Type` header is missing or does not indicate HTML content."
},
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.html.scraper.HTMLScraper.fetch",
"signature": "<bound method Function.signature of Function('fetch', 100, 139)>",
"docstring": "Fetch an HTML document from the given source.\n\nArgs:\n source (str):\n URL of the HTML document.\n metadata (Optional[Mapping[str, Any]], optional):\n Optional metadata to be merged into the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw HTML bytes, source URL, HTML content type, and HTTP response metadata.\n\nRaises:\n httpx.HTTPError:\n If the HTTP request fails.\n ValueError:\n If the response is not valid HTML."
}
}
}
}
}
}
},
"pdf": {
"name": "pdf",
"kind": "module",
"path": "omniread.pdf",
"signature": null,
"docstring": "PDF format implementation for OmniRead.\n\n---\n\n## Summary\n\nThis package provides **PDF-specific implementations** of the core OmniRead\ncontracts defined in `omniread.core`.\n\nUnlike HTML, PDF handling requires an explicit client layer for document\naccess. This package therefore includes:\n- PDF clients for acquiring raw PDF data\n- PDF scrapers that coordinate client access\n- PDF parsers that extract structured content from PDF binaries\n\nPublic exports from this package represent the supported PDF pipeline\nand are safe for consumers to import directly when working with PDFs.\n\n---\n\n## Public API\n\n FileSystemPDFClient\n PDFScraper\n PDFParser\n\n---",
"members": {
"FileSystemPDFClient": {
"name": "FileSystemPDFClient",
"kind": "class",
"path": "omniread.pdf.FileSystemPDFClient",
"signature": "<bound method Alias.signature of Alias('FileSystemPDFClient', 'omniread.pdf.client.FileSystemPDFClient')>",
"docstring": "PDF client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads PDF files directly from the disk and returns their raw binary contents",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.FileSystemPDFClient.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.pdf.client.FileSystemPDFClient.fetch')>",
"docstring": "Read a PDF file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the PDF file.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
},
"PDFScraper": {
"name": "PDFScraper",
"kind": "class",
"path": "omniread.pdf.PDFScraper",
"signature": "<bound method Alias.signature of Alias('PDFScraper', 'omniread.pdf.scraper.PDFScraper')>",
"docstring": "Scraper for PDF sources.\n\nNotes:\n **Responsibilities:**\n\n - Delegates byte retrieval to a PDF client and normalizes output into Content\n - Preserves caller-provided metadata\n\n **Constraints:**\n \n - The scraper: Does not perform parsing or interpretation, does not assume a specific storage backend",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.PDFScraper.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.pdf.scraper.PDFScraper.fetch')>",
"docstring": "Fetch a PDF document from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF source as understood by the configured PDF client.\n metadata (Optional[Mapping[str, Any]], optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw PDF bytes, source identifier, PDF content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the PDF client."
}
}
},
"PDFParser": {
"name": "PDFParser",
"kind": "class",
"path": "omniread.pdf.PDFParser",
"signature": "<bound method Alias.signature of Alias('PDFParser', 'omniread.pdf.parser.PDFParser')>",
"docstring": "Base PDF parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces PDF content-type compatibility and provides the extension point for implementing concrete PDF parsing strategies\n\n **Constraints:**\n\n - Concrete implementations must: Define the output type `T`, implement the `parse()` method",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.pdf.PDFParser.supported_types",
"signature": "<bound method Alias.signature of Alias('supported_types', 'omniread.pdf.parser.PDFParser.supported_types')>",
"docstring": "Set of content types supported by this parser (PDF only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.pdf.PDFParser.parse",
"signature": "<bound method Alias.signature of Alias('parse', 'omniread.pdf.parser.PDFParser.parse')>",
"docstring": "Parse PDF content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the PDF binary payload and return a deterministic, structured output"
}
}
},
"client": {
"name": "client",
"kind": "module",
"path": "omniread.pdf.client",
"signature": null,
"docstring": "PDF client abstractions for OmniRead.\n\n---\n\n## Summary\n\nThis module defines the **client layer** responsible for retrieving raw PDF\nbytes from a concrete backing store.\n\nClients provide low-level access to PDF binaries and are intentionally\ndecoupled from scraping and parsing logic. They do not perform validation,\ninterpretation, or content extraction.\n\nTypical backing stores include:\n- Local filesystems\n- Object storage (S3, GCS, etc.)\n- Network file systems",
"members": {
"Any": {
"name": "Any",
"kind": "alias",
"path": "omniread.pdf.client.Any",
"signature": "<bound method Alias.signature of Alias('Any', 'typing.Any')>",
"docstring": null
},
"ABC": {
"name": "ABC",
"kind": "alias",
"path": "omniread.pdf.client.ABC",
"signature": "<bound method Alias.signature of Alias('ABC', 'abc.ABC')>",
"docstring": null
},
"abstractmethod": {
"name": "abstractmethod",
"kind": "alias",
"path": "omniread.pdf.client.abstractmethod",
"signature": "<bound method Alias.signature of Alias('abstractmethod', 'abc.abstractmethod')>",
"docstring": null
},
"Path": {
"name": "Path",
"kind": "alias",
"path": "omniread.pdf.client.Path",
"signature": "<bound method Alias.signature of Alias('Path', 'pathlib.Path')>",
"docstring": null
},
"BasePDFClient": {
"name": "BasePDFClient",
"kind": "class",
"path": "omniread.pdf.client.BasePDFClient",
"signature": "<bound method Class.signature of Class('BasePDFClient', 26, 54)>",
"docstring": "Abstract client responsible for retrieving PDF bytes\nfrom a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to the backing store, return the full PDF binary payload, and raise retrieval-specific errors on failure",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.client.BasePDFClient.fetch",
"signature": "<bound method Function.signature of Function('fetch', 37, 54)>",
"docstring": "Fetch raw PDF bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF location, such as a file path, object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemPDFClient": {
"name": "FileSystemPDFClient",
"kind": "class",
"path": "omniread.pdf.client.FileSystemPDFClient",
"signature": "<bound method Class.signature of Class('FileSystemPDFClient', 57, 92)>",
"docstring": "PDF client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads PDF files directly from the disk and returns their raw binary contents",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.client.FileSystemPDFClient.fetch",
"signature": "<bound method Function.signature of Function('fetch', 67, 92)>",
"docstring": "Read a PDF file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the PDF file.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
}
}
},
"parser": {
"name": "parser",
"kind": "module",
"path": "omniread.pdf.parser",
"signature": null,
"docstring": "PDF parser base implementations for OmniRead.\n\n---\n\n## Summary\n\nThis module defines the **PDF-specific parser contract**, extending the\nformat-agnostic `BaseParser` with constraints appropriate for PDF content.\n\nPDF parsers are responsible for interpreting binary PDF data and producing\nstructured representations suitable for downstream consumption.",
"members": {
"Generic": {
"name": "Generic",
"kind": "alias",
"path": "omniread.pdf.parser.Generic",
"signature": "<bound method Alias.signature of Alias('Generic', 'typing.Generic')>",
"docstring": null
},
"TypeVar": {
"name": "TypeVar",
"kind": "alias",
"path": "omniread.pdf.parser.TypeVar",
"signature": "<bound method Alias.signature of Alias('TypeVar', 'typing.TypeVar')>",
"docstring": null
},
"abstractmethod": {
"name": "abstractmethod",
"kind": "alias",
"path": "omniread.pdf.parser.abstractmethod",
"signature": "<bound method Alias.signature of Alias('abstractmethod', 'abc.abstractmethod')>",
"docstring": null
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.pdf.parser.ContentType",
"signature": "<bound method Alias.signature of Alias('ContentType', 'omniread.core.content.ContentType')>",
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the content source\n - It is primarily used for routing content to the appropriate parser or downstream consumer",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.HTML",
"signature": "<bound method Alias.signature of Alias('HTML', 'omniread.core.content.ContentType.HTML')>",
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.PDF",
"signature": "<bound method Alias.signature of Alias('PDF', 'omniread.core.content.ContentType.PDF')>",
"docstring": "PDF document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.JSON",
"signature": "<bound method Alias.signature of Alias('JSON', 'omniread.core.content.ContentType.JSON')>",
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.XML",
"signature": "<bound method Alias.signature of Alias('XML', 'omniread.core.content.ContentType.XML')>",
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.pdf.parser.BaseParser",
"signature": "<bound method Alias.signature of Alias('BaseParser', 'omniread.core.parser.BaseParser')>",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the Content it is responsible for interpreting\n - Consumers may rely on early validation of content compatibility and type-stable return values from `parse()`\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`\n - Implementations must raise parsing-specific exceptions from `parse()`\n - Implementations must remain deterministic for a given input",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.pdf.parser.BaseParser.supported_types",
"signature": "<bound method Alias.signature of Alias('supported_types', 'omniread.core.parser.BaseParser.supported_types')>",
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.pdf.parser.BaseParser.content",
"signature": "<bound method Alias.signature of Alias('content', 'omniread.core.parser.BaseParser.content')>",
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.pdf.parser.BaseParser.parse",
"signature": "<bound method Alias.signature of Alias('parse', 'omniread.core.parser.BaseParser.parse')>",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and return a deterministic, structured output"
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.pdf.parser.BaseParser.supports",
"signature": "<bound method Alias.signature of Alias('supports', 'omniread.core.parser.BaseParser.supports')>",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.pdf.parser.T",
"signature": null,
"docstring": null
},
"PDFParser": {
"name": "PDFParser",
"kind": "class",
"path": "omniread.pdf.parser.PDFParser",
"signature": "<bound method Class.signature of Class('PDFParser', 24, 61)>",
"docstring": "Base PDF parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces PDF content-type compatibility and provides the extension point for implementing concrete PDF parsing strategies\n\n **Constraints:**\n\n - Concrete implementations must: Define the output type `T`, implement the `parse()` method",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.pdf.parser.PDFParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (PDF only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.pdf.parser.PDFParser.parse",
"signature": "<bound method Function.signature of Function('parse', 43, 61)>",
"docstring": "Parse PDF content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the PDF binary payload and return a deterministic, structured output"
}
}
}
}
},
"scraper": {
"name": "scraper",
"kind": "module",
"path": "omniread.pdf.scraper",
"signature": null,
"docstring": "PDF scraping implementation for OmniRead.\n\n---\n\n## Summary\n\nThis module provides a PDF-specific scraper that coordinates PDF byte\nretrieval via a client and normalizes the result into a `Content` object.\n\nThe scraper implements the core `BaseScraper` contract while delegating\nall storage and access concerns to a `BasePDFClient` implementation.",
"members": {
"Any": {
"name": "Any",
"kind": "alias",
"path": "omniread.pdf.scraper.Any",
"signature": "<bound method Alias.signature of Alias('Any', 'typing.Any')>",
"docstring": null
},
"Mapping": {
"name": "Mapping",
"kind": "alias",
"path": "omniread.pdf.scraper.Mapping",
"signature": "<bound method Alias.signature of Alias('Mapping', 'typing.Mapping')>",
"docstring": null
},
"Optional": {
"name": "Optional",
"kind": "alias",
"path": "omniread.pdf.scraper.Optional",
"signature": "<bound method Alias.signature of Alias('Optional', 'typing.Optional')>",
"docstring": null
},
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.pdf.scraper.Content",
"signature": "<bound method Alias.signature of Alias('Content', 'omniread.core.content.Content')>",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with minimal contextual metadata describing its origin and type\n - This class is the primary exchange format between Scrapers, Parsers, and Downstream consumers",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.raw",
"signature": "<bound method Alias.signature of Alias('raw', 'omniread.core.content.Content.raw')>",
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.source",
"signature": "<bound method Alias.signature of Alias('source', 'omniread.core.content.Content.source')>",
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.content_type",
"signature": "<bound method Alias.signature of Alias('content_type', 'omniread.core.content.Content.content_type')>",
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.metadata",
"signature": "<bound method Alias.signature of Alias('metadata', 'omniread.core.content.Content.metadata')>",
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.pdf.scraper.ContentType",
"signature": "<bound method Alias.signature of Alias('ContentType', 'omniread.core.content.ContentType')>",
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the content source\n - It is primarily used for routing content to the appropriate parser or downstream consumer",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.HTML",
"signature": "<bound method Alias.signature of Alias('HTML', 'omniread.core.content.ContentType.HTML')>",
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.PDF",
"signature": "<bound method Alias.signature of Alias('PDF', 'omniread.core.content.ContentType.PDF')>",
"docstring": "PDF document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.JSON",
"signature": "<bound method Alias.signature of Alias('JSON', 'omniread.core.content.ContentType.JSON')>",
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.XML",
"signature": "<bound method Alias.signature of Alias('XML', 'omniread.core.content.ContentType.XML')>",
"docstring": "XML document content."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.pdf.scraper.BaseScraper",
"signature": "<bound method Alias.signature of Alias('BaseScraper', 'omniread.core.scraper.BaseScraper')>",
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes) from a source. It must not interpret or parse it\n - A scraper is a stateless acquisition component that retrieves raw content from a source and returns it as a `Content` object\n - Scrapers define how content is obtained, not what the content means\n - Implementations may vary in transport mechanism, authentication strategy, retry and backoff behavior\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics, or couple scraping logic to a specific parser",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.BaseScraper.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.core.scraper.BaseScraper.fetch')>",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.)\n metadata (Optional[Mapping[str, Any]], optional):\n Optional hints for the scraper (headers, auth, etc.)\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source` and return it as raw bytes wrapped in a `Content` object"
}
}
},
"BasePDFClient": {
"name": "BasePDFClient",
"kind": "class",
"path": "omniread.pdf.scraper.BasePDFClient",
"signature": "<bound method Alias.signature of Alias('BasePDFClient', 'omniread.pdf.client.BasePDFClient')>",
"docstring": "Abstract client responsible for retrieving PDF bytes\nfrom a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to the backing store, return the full PDF binary payload, and raise retrieval-specific errors on failure",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.BasePDFClient.fetch",
"signature": "<bound method Alias.signature of Alias('fetch', 'omniread.pdf.client.BasePDFClient.fetch')>",
"docstring": "Fetch raw PDF bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF location, such as a file path, object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"PDFScraper": {
"name": "PDFScraper",
"kind": "class",
"path": "omniread.pdf.scraper.PDFScraper",
"signature": "<bound method Class.signature of Class('PDFScraper', 22, 77)>",
"docstring": "Scraper for PDF sources.\n\nNotes:\n **Responsibilities:**\n\n - Delegates byte retrieval to a PDF client and normalizes output into Content\n - Preserves caller-provided metadata\n\n **Constraints:**\n \n - The scraper: Does not perform parsing or interpretation, does not assume a specific storage backend",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.PDFScraper.fetch",
"signature": "<bound method Function.signature of Function('fetch', 47, 77)>",
"docstring": "Fetch a PDF document from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF source as understood by the configured PDF client.\n metadata (Optional[Mapping[str, Any]], optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw PDF bytes, source identifier, PDF content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the PDF client."
}
}
}
}
}
}
}
}
}
}