aetos/omniread
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore(release): 0.0.3

Browse Source 
Squash-merge lib-standardisation: adopt repo standards across packaging,
tooling, docs, and CI.

- standardize pyproject.toml (canonical packaging, lint config, extras, GSDFC pydoclint)
- add py.typed marker, modernize type hints and Google-style docstrings
- nest generated docs under docs/lib and docs/mcp; add hand-written wiki
- adopt standard Type A PyPI CI pipeline (tag==version guard + quality gate)
- promote generated CHANGELOG.md; bump version to 0.0.3
This commit is contained in:
Vishesh 'ironeagle' Bangotra
2026-10-17 02:16:48 +05:30
parent f92d40ed7f
commit 650d3a19e8
106 changed files with 6175 additions and 2716 deletions
Download Patch File Download Diff File Expand all files Collapse all files

0
docs/core/content.md → docs/lib/core/content.md
View File

0
docs/core/index.md → docs/lib/core/index.md
View File

0
docs/core/parser.md → docs/lib/core/parser.md
View File

0
docs/core/scraper.md → docs/lib/core/scraper.md
View File

3
docs/lib/csv/client.md Normal file
View File

@@ -0,0 +1,3 @@
# Client
::: omniread.csv.client

3
docs/lib/csv/index.md Normal file
View File

@@ -0,0 +1,3 @@
# Csv
::: omniread.csv

3
docs/lib/csv/parser.md Normal file
View File

@@ -0,0 +1,3 @@
# Parser
::: omniread.csv.parser

3
docs/lib/csv/parser_base.md Normal file
View File

@@ -0,0 +1,3 @@
# Parser Base
::: omniread.csv.parser_base

3
docs/lib/csv/scraper.md Normal file
View File

@@ -0,0 +1,3 @@
# Scraper
::: omniread.csv.scraper

0
docs/html/index.md → docs/lib/html/index.md
View File

0
docs/html/parser.md → docs/lib/html/parser.md
View File

0
docs/html/scraper.md → docs/lib/html/scraper.md
View File

0
docs/index.md → docs/lib/index.md
View File

0
docs/pdf/client.md → docs/lib/pdf/client.md
View File

0
docs/pdf/index.md → docs/lib/pdf/index.md
View File

0
docs/pdf/parser.md → docs/lib/pdf/parser.md
View File

0
docs/pdf/scraper.md → docs/lib/pdf/scraper.md
View File

3
docs/lib/xlsx/client.md Normal file
View File

@@ -0,0 +1,3 @@
# Client
::: omniread.xlsx.client

3
docs/lib/xlsx/index.md Normal file
View File

@@ -0,0 +1,3 @@
# Xlsx
::: omniread.xlsx

3
docs/lib/xlsx/parser.md Normal file
View File

@@ -0,0 +1,3 @@
# Parser
::: omniread.xlsx.parser

3
docs/lib/xlsx/parser_base.md Normal file
View File

@@ -0,0 +1,3 @@
# Parser Base
::: omniread.xlsx.parser_base

3
docs/lib/xlsx/scraper.md Normal file
View File

@@ -0,0 +1,3 @@
# Scraper
::: omniread.xlsx.scraper

6
docs/mcp/index.json Normal file
View File

@@ -0,0 +1,6 @@
{
"project": "OmniRead",
"type": "docforge-model",
"modules_count": 22,
"source": "docforge"
}

97
docs/mcp/modules/omniread.core.content.json Normal file
View File

@@ -0,0 +1,97 @@
{
"module": "omniread.core.content",
"content": {
"path": "omniread.core.content",
"docstring": "# Summary\n\nCanonical content models for OmniRead.\n\nThis module defines the **format-agnostic content representation** used across\nall parsers and scrapers in OmniRead.\n\nThe models defined here represent *what* was extracted, not *how* it was\nretrieved or parsed. Format-specific behavior and metadata must not alter\nthe semantic meaning of these models.",
"objects": {
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.core.content.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.core.content.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.core.content.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.core.content.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.core.content.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.core.content.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.core.content.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.content.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.content.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.content.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.content.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.content.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
}
}
}
}

450
docs/mcp/modules/omniread.core.json Normal file
View File

@@ -0,0 +1,450 @@
{
"module": "omniread.core",
"content": {
"path": "omniread.core",
"docstring": "# Summary\n\nCore domain contracts for OmniRead.\n\nThis package defines the **format-agnostic domain layer** of OmniRead.\nIt exposes canonical content models and abstract interfaces that are\nimplemented by format-specific modules (HTML, PDF, etc.).\n\nPublic exports from this package are considered **stable contracts** and\nare safe for downstream consumers to depend on.\n\nSubmodules:\n\n- `content`: Canonical content models and enums.\n- `parser`: Abstract parsing contracts.\n- `scraper`: Abstract scraping contracts.\n\nFormat-specific behavior must not be introduced at this layer.\n\n---\n\n# Public API\n\n- `Content`\n- `ContentType`\n\n---",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.core.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.core.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.core.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.core.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.core.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.core.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.core.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.core.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.core.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.core.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.core.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.core.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.core.BaseScraper",
"signature": null,
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes)\n from a source. It must not interpret or parse it.\n - A scraper is a stateless acquisition component that retrieves raw\n content from a source and returns it as a `Content` object.\n - Scrapers define how content is obtained, not what the content means.\n - Implementations may vary in transport mechanism, authentication\n strategy, retry and backoff behavior.\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics,\n or couple scraping logic to a specific parser.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.core.BaseScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.).\n\n metadata (Mapping[str, Any] | None, optional):\n Optional hints for the scraper (headers, auth, etc.).\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source`\n and return it as raw bytes wrapped in a `Content` object."
}
}
},
"content": {
"name": "content",
"kind": "module",
"path": "omniread.core.content",
"signature": null,
"docstring": "# Summary\n\nCanonical content models for OmniRead.\n\nThis module defines the **format-agnostic content representation** used across\nall parsers and scrapers in OmniRead.\n\nThe models defined here represent *what* was extracted, not *how* it was\nretrieved or parsed. Format-specific behavior and metadata must not alter\nthe semantic meaning of these models.",
"members": {
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.core.content.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.core.content.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.core.content.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.core.content.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.core.content.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.core.content.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.core.content.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.content.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.content.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.content.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.content.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.content.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
}
}
},
"parser": {
"name": "parser",
"kind": "module",
"path": "omniread.core.parser",
"signature": null,
"docstring": "# Summary\n\nAbstract parsing contracts for OmniRead.\n\nThis module defines the **format-agnostic parser interface** used to transform\nraw content into structured, typed representations.\n\nParsers are responsible for:\n\n- Interpreting a single `Content` instance\n- Validating compatibility with the content type\n- Producing a structured output suitable for downstream consumers\n\nParsers are not responsible for:\n\n- Fetching or acquiring content\n- Performing retries or error recovery\n- Managing multiple content sources",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.parser.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.parser.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.parser.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.parser.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.parser.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.core.parser.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.core.parser.T",
"signature": null,
"docstring": null
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.core.parser.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.core.parser.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.core.parser.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.core.parser.BaseParser.parse",
"signature": "parse() -> T",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.core.parser.BaseParser.supports",
"signature": "supports() -> bool",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
}
}
},
"scraper": {
"name": "scraper",
"kind": "module",
"path": "omniread.core.scraper",
"signature": null,
"docstring": "# Summary\n\nAbstract scraping contracts for OmniRead.\n\nThis module defines the **format-agnostic scraper interface** responsible for\nacquiring raw content from external sources.\n\nScrapers are responsible for:\n\n- Locating and retrieving raw content bytes\n- Attaching minimal contextual metadata\n- Returning normalized `Content` objects\n\nScrapers are explicitly NOT responsible for:\n\n- Parsing or interpreting content\n- Inferring structure or semantics\n- Performing content-type specific processing\n\nAll interpretation must be delegated to parsers.",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.core.scraper.BaseScraper",
"signature": null,
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes)\n from a source. It must not interpret or parse it.\n - A scraper is a stateless acquisition component that retrieves raw\n content from a source and returns it as a `Content` object.\n - Scrapers define how content is obtained, not what the content means.\n - Implementations may vary in transport mechanism, authentication\n strategy, retry and backoff behavior.\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics,\n or couple scraping logic to a specific parser.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.core.scraper.BaseScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.).\n\n metadata (Mapping[str, Any] | None, optional):\n Optional hints for the scraper (headers, auth, etc.).\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source`\n and return it as raw bytes wrapped in a `Content` object."
}
}
}
}
}
}
}
}

141
docs/mcp/modules/omniread.core.parser.json Normal file
View File

@@ -0,0 +1,141 @@
{
"module": "omniread.core.parser",
"content": {
"path": "omniread.core.parser",
"docstring": "# Summary\n\nAbstract parsing contracts for OmniRead.\n\nThis module defines the **format-agnostic parser interface** used to transform\nraw content into structured, typed representations.\n\nParsers are responsible for:\n\n- Interpreting a single `Content` instance\n- Validating compatibility with the content type\n- Producing a structured output suitable for downstream consumers\n\nParsers are not responsible for:\n\n- Fetching or acquiring content\n- Performing retries or error recovery\n- Managing multiple content sources",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.parser.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.parser.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.parser.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.parser.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.parser.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.core.parser.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.core.parser.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.core.parser.T",
"signature": null,
"docstring": null
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.core.parser.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.core.parser.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.core.parser.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.core.parser.BaseParser.parse",
"signature": "parse() -> T",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.core.parser.BaseParser.supports",
"signature": "supports() -> bool",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
}
}
}
}

62
docs/mcp/modules/omniread.core.scraper.json Normal file
View File

@@ -0,0 +1,62 @@
{
"module": "omniread.core.scraper",
"content": {
"path": "omniread.core.scraper",
"docstring": "# Summary\n\nAbstract scraping contracts for OmniRead.\n\nThis module defines the **format-agnostic scraper interface** responsible for\nacquiring raw content from external sources.\n\nScrapers are responsible for:\n\n- Locating and retrieving raw content bytes\n- Attaching minimal contextual metadata\n- Returning normalized `Content` objects\n\nScrapers are explicitly NOT responsible for:\n\n- Parsing or interpreting content\n- Inferring structure or semantics\n- Performing content-type specific processing\n\nAll interpretation must be delegated to parsers.",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.core.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.core.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.core.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.core.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.core.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.core.scraper.BaseScraper",
"signature": null,
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes)\n from a source. It must not interpret or parse it.\n - A scraper is a stateless acquisition component that retrieves raw\n content from a source and returns it as a `Content` object.\n - Scrapers define how content is obtained, not what the content means.\n - Implementations may vary in transport mechanism, authentication\n strategy, retry and backoff behavior.\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics,\n or couple scraping logic to a specific parser.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.core.scraper.BaseScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.).\n\n metadata (Mapping[str, Any] | None, optional):\n Optional hints for the scraper (headers, auth, etc.).\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source`\n and return it as raw bytes wrapped in a `Content` object."
}
}
}
}
}
}

41
docs/mcp/modules/omniread.csv.client.json Normal file
View File

@@ -0,0 +1,41 @@
{
"module": "omniread.csv.client",
"content": {
"path": "omniread.csv.client",
"docstring": "# Summary\n\nCSV client abstractions for OmniRead.\n\nThis module defines the **client layer** responsible for retrieving raw\ncomma-separated-value document bytes from a concrete backing store.\n\nClients provide low-level access to csv binaries and are intentionally\ndecoupled from scraping and parsing logic. They do not perform validation,\ninterpretation, or content extraction.\n\nTypical backing stores include:\n\n- Local filesystems\n- Object storage (S3, GCS, etc.)\n- Network file systems",
"objects": {
"BaseCsvClient": {
"name": "BaseCsvClient",
"kind": "class",
"path": "omniread.csv.client.BaseCsvClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving csv bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full csv binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.client.BaseCsvClient.fetch",
"signature": "fetch(source: Any) -> bytes",
"docstring": "Fetch raw csv bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the csv location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw csv bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemCsvClient": {
"name": "FileSystemCsvClient",
"kind": "class",
"path": "omniread.csv.client.FileSystemCsvClient",
"signature": null,
"docstring": "CSV client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads csv files directly from the disk and\n returns their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.client.FileSystemCsvClient.fetch",
"signature": "fetch(path: Path) -> bytes",
"docstring": "Read a csv file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the csv file.\n\nReturns:\n bytes:\n Raw csv bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
}
}
}
}

492
docs/mcp/modules/omniread.csv.json Normal file
View File

@@ -0,0 +1,492 @@
{
"module": "omniread.csv",
"content": {
"path": "omniread.csv",
"docstring": "# Summary\n\nCSV subpackage for OmniRead.\n\nProvides acquisition and parsing of comma-separated-value content:\n\n- `BaseCsvClient`: abstract backing-store client for csv bytes.\n- `FileSystemCsvClient`: local filesystem implementation.\n- `CsvScraper`: wraps fetched bytes into canonical `Content`.\n- `CsvParserBase`: content-type-enforcing parser contract.\n- `CsvParser`: generic string-row parser built on the standard csv module.",
"objects": {
"BaseCsvClient": {
"name": "BaseCsvClient",
"kind": "class",
"path": "omniread.csv.BaseCsvClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving csv bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full csv binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.BaseCsvClient.fetch",
"signature": "fetch(source: Any)",
"docstring": "Fetch raw csv bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the csv location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw csv bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemCsvClient": {
"name": "FileSystemCsvClient",
"kind": "class",
"path": "omniread.csv.FileSystemCsvClient",
"signature": null,
"docstring": "CSV client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads csv files directly from the disk and\n returns their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.FileSystemCsvClient.fetch",
"signature": "fetch(path: Path)",
"docstring": "Read a csv file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the csv file.\n\nReturns:\n bytes:\n Raw csv bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
},
"CsvParser": {
"name": "CsvParser",
"kind": "class",
"path": "omniread.csv.CsvParser",
"signature": "CsvParser(content: Content)",
"docstring": "Generic csv parser producing string rows from the document.\n\nNotes:\n **Responsibilities:**\n\n - Decode the payload (UTF-8 with BOM support, Latin-1 fallback).\n - Detect the delimiter from a leading sample (`,` `;` tab `|`),\n defaulting to `,`.\n - Normalize cells into deterministic stripped string values.\n - Expose row extraction helpers mirroring `XlsxParser.rows`.\n\n **Constraints:**\n\n - All values are strings; consumers requiring typed values must\n convert on their side.\n - Quoted fields containing delimiters/newlines are handled by\n the standard ``csv`` module.",
"members": {
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.CsvParser.parse",
"signature": "parse()",
"docstring": "Parse the document into normalized string rows.\n\nReturns:\n list[list[str]]:\n Rows of the document."
},
"rows": {
"name": "rows",
"kind": "function",
"path": "omniread.csv.CsvParser.rows",
"signature": "rows(*, skip_empty: bool = True)",
"docstring": "Extract normalized string rows from the document.\n\nArgs:\n skip_empty (bool):\n When True (default), rows whose cells are all blank are\n omitted.\n\nReturns:\n list[list[str]]:\n Normalized rows; trailing blank cells are trimmed per row."
}
}
},
"CsvParserBase": {
"name": "CsvParserBase",
"kind": "class",
"path": "omniread.csv.CsvParserBase",
"signature": null,
"docstring": "Base csv parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces csv content-type compatibility and provides\n the extension point for implementing concrete csv parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.csv.CsvParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (CSV only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.CsvParserBase.parse",
"signature": "parse()",
"docstring": "Parse csv content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
},
"CsvScraper": {
"name": "CsvScraper",
"kind": "class",
"path": "omniread.csv.CsvScraper",
"signature": "CsvScraper(*, client: BaseCsvClient)",
"docstring": "Scraper for csv documents.\n\nNotes:\n **Responsibilities:**\n\n - Fetch raw csv bytes via the configured client.\n - Wrap the payload in a canonical `Content` instance with the\n CSV content type and source identifier.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.CsvScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch a csv document from the given source.\n\nArgs:\n source (Any):\n Identifier of the csv source as understood by the\n configured client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw csv bytes, source\n identifier, CSV content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the client."
}
}
},
"client": {
"name": "client",
"kind": "module",
"path": "omniread.csv.client",
"signature": null,
"docstring": "# Summary\n\nCSV client abstractions for OmniRead.\n\nThis module defines the **client layer** responsible for retrieving raw\ncomma-separated-value document bytes from a concrete backing store.\n\nClients provide low-level access to csv binaries and are intentionally\ndecoupled from scraping and parsing logic. They do not perform validation,\ninterpretation, or content extraction.\n\nTypical backing stores include:\n\n- Local filesystems\n- Object storage (S3, GCS, etc.)\n- Network file systems",
"members": {
"BaseCsvClient": {
"name": "BaseCsvClient",
"kind": "class",
"path": "omniread.csv.client.BaseCsvClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving csv bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full csv binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.client.BaseCsvClient.fetch",
"signature": "fetch(source: Any) -> bytes",
"docstring": "Fetch raw csv bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the csv location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw csv bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemCsvClient": {
"name": "FileSystemCsvClient",
"kind": "class",
"path": "omniread.csv.client.FileSystemCsvClient",
"signature": null,
"docstring": "CSV client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads csv files directly from the disk and\n returns their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.client.FileSystemCsvClient.fetch",
"signature": "fetch(path: Path) -> bytes",
"docstring": "Read a csv file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the csv file.\n\nReturns:\n bytes:\n Raw csv bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
}
}
},
"parser": {
"name": "parser",
"kind": "module",
"path": "omniread.csv.parser",
"signature": null,
"docstring": "# Summary\n\nCSV parser implementations for OmniRead.\n\nThis module provides a concrete, generic parser for comma-separated-value\ndocuments. It exposes records as lists of string cells so downstream\nconsumers can interpret tabular content without depending on the ``csv``\nmodule directly.\n\nThe parser is intentionally statement-agnostic: it performs no header\ndetection or column interpretation beyond basic cell normalization and\ndelimiter detection.",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.csv.parser.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.csv.parser.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.csv.parser.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.csv.parser.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.csv.parser.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"CsvParserBase": {
"name": "CsvParserBase",
"kind": "class",
"path": "omniread.csv.parser.CsvParserBase",
"signature": null,
"docstring": "Base csv parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces csv content-type compatibility and provides\n the extension point for implementing concrete csv parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.csv.parser.CsvParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (CSV only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.parser.CsvParserBase.parse",
"signature": "parse()",
"docstring": "Parse csv content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
},
"CsvParser": {
"name": "CsvParser",
"kind": "class",
"path": "omniread.csv.parser.CsvParser",
"signature": "CsvParser(content: Content)",
"docstring": "Generic csv parser producing string rows from the document.\n\nNotes:\n **Responsibilities:**\n\n - Decode the payload (UTF-8 with BOM support, Latin-1 fallback).\n - Detect the delimiter from a leading sample (`,` `;` tab `|`),\n defaulting to `,`.\n - Normalize cells into deterministic stripped string values.\n - Expose row extraction helpers mirroring `XlsxParser.rows`.\n\n **Constraints:**\n\n - All values are strings; consumers requiring typed values must\n convert on their side.\n - Quoted fields containing delimiters/newlines are handled by\n the standard ``csv`` module.",
"members": {
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.parser.CsvParser.parse",
"signature": "parse() -> list[list[str]]",
"docstring": "Parse the document into normalized string rows.\n\nReturns:\n list[list[str]]:\n Rows of the document."
},
"rows": {
"name": "rows",
"kind": "function",
"path": "omniread.csv.parser.CsvParser.rows",
"signature": "rows(*, skip_empty: bool = True) -> list[list[str]]",
"docstring": "Extract normalized string rows from the document.\n\nArgs:\n skip_empty (bool):\n When True (default), rows whose cells are all blank are\n omitted.\n\nReturns:\n list[list[str]]:\n Normalized rows; trailing blank cells are trimmed per row."
}
}
}
}
},
"parser_base": {
"name": "parser_base",
"kind": "module",
"path": "omniread.csv.parser_base",
"signature": null,
"docstring": "# Summary\n\nCSV parser base implementation for OmniRead.\n\nThis module defines the **CSV-specific parser contract**, extending the\nformat-agnostic `BaseParser` with constraints appropriate for\ncomma-separated-value documents.",
"members": {
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.csv.parser_base.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.csv.parser_base.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.csv.parser_base.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.csv.parser_base.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.parser_base.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.csv.parser_base.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.csv.parser_base.T",
"signature": null,
"docstring": null
},
"CsvParserBase": {
"name": "CsvParserBase",
"kind": "class",
"path": "omniread.csv.parser_base.CsvParserBase",
"signature": null,
"docstring": "Base csv parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces csv content-type compatibility and provides\n the extension point for implementing concrete csv parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.csv.parser_base.CsvParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (CSV only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.parser_base.CsvParserBase.parse",
"signature": "parse() -> T",
"docstring": "Parse csv content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
}
}
},
"scraper": {
"name": "scraper",
"kind": "module",
"path": "omniread.csv.scraper",
"signature": null,
"docstring": "# Summary\n\nCSV scraper for OmniRead.\n\nThis module defines the scraper responsible for acquiring raw\ncomma-separated-value document content from a backing store via a\nconfigured client.\n\nThe scraper does not interpret or parse the acquired bytes; it wraps them in\nthe canonical `Content` model.",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.csv.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.csv.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.csv.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.csv.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.csv.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.csv.scraper.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseCsvClient": {
"name": "BaseCsvClient",
"kind": "class",
"path": "omniread.csv.scraper.BaseCsvClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving csv bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full csv binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.scraper.BaseCsvClient.fetch",
"signature": "fetch(source: Any)",
"docstring": "Fetch raw csv bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the csv location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw csv bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"CsvScraper": {
"name": "CsvScraper",
"kind": "class",
"path": "omniread.csv.scraper.CsvScraper",
"signature": "CsvScraper(*, client: BaseCsvClient)",
"docstring": "Scraper for csv documents.\n\nNotes:\n **Responsibilities:**\n\n - Fetch raw csv bytes via the configured client.\n - Wrap the payload in a canonical `Content` instance with the\n CSV content type and source identifier.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.scraper.CsvScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch a csv document from the given source.\n\nArgs:\n source (Any):\n Identifier of the csv source as understood by the\n configured client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw csv bytes, source\n identifier, CSV content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the client."
}
}
}
}
}
}
}
}

92
docs/mcp/modules/omniread.csv.parser.json Normal file
View File

@@ -0,0 +1,92 @@
{
"module": "omniread.csv.parser",
"content": {
"path": "omniread.csv.parser",
"docstring": "# Summary\n\nCSV parser implementations for OmniRead.\n\nThis module provides a concrete, generic parser for comma-separated-value\ndocuments. It exposes records as lists of string cells so downstream\nconsumers can interpret tabular content without depending on the ``csv``\nmodule directly.\n\nThe parser is intentionally statement-agnostic: it performs no header\ndetection or column interpretation beyond basic cell normalization and\ndelimiter detection.",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.csv.parser.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.csv.parser.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.csv.parser.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.csv.parser.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.csv.parser.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"CsvParserBase": {
"name": "CsvParserBase",
"kind": "class",
"path": "omniread.csv.parser.CsvParserBase",
"signature": null,
"docstring": "Base csv parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces csv content-type compatibility and provides\n the extension point for implementing concrete csv parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.csv.parser.CsvParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (CSV only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.parser.CsvParserBase.parse",
"signature": "parse()",
"docstring": "Parse csv content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
},
"CsvParser": {
"name": "CsvParser",
"kind": "class",
"path": "omniread.csv.parser.CsvParser",
"signature": "CsvParser(content: Content)",
"docstring": "Generic csv parser producing string rows from the document.\n\nNotes:\n **Responsibilities:**\n\n - Decode the payload (UTF-8 with BOM support, Latin-1 fallback).\n - Detect the delimiter from a leading sample (`,` `;` tab `|`),\n defaulting to `,`.\n - Normalize cells into deterministic stripped string values.\n - Expose row extraction helpers mirroring `XlsxParser.rows`.\n\n **Constraints:**\n\n - All values are strings; consumers requiring typed values must\n convert on their side.\n - Quoted fields containing delimiters/newlines are handled by\n the standard ``csv`` module.",
"members": {
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.parser.CsvParser.parse",
"signature": "parse() -> list[list[str]]",
"docstring": "Parse the document into normalized string rows.\n\nReturns:\n list[list[str]]:\n Rows of the document."
},
"rows": {
"name": "rows",
"kind": "function",
"path": "omniread.csv.parser.CsvParser.rows",
"signature": "rows(*, skip_empty: bool = True) -> list[list[str]]",
"docstring": "Extract normalized string rows from the document.\n\nArgs:\n skip_empty (bool):\n When True (default), rows whose cells are all blank are\n omitted.\n\nReturns:\n list[list[str]]:\n Normalized rows; trailing blank cells are trimmed per row."
}
}
}
}
}
}

127
docs/mcp/modules/omniread.csv.parser_base.json Normal file
View File

@@ -0,0 +1,127 @@
{
"module": "omniread.csv.parser_base",
"content": {
"path": "omniread.csv.parser_base",
"docstring": "# Summary\n\nCSV parser base implementation for OmniRead.\n\nThis module defines the **CSV-specific parser contract**, extending the\nformat-agnostic `BaseParser` with constraints appropriate for\ncomma-separated-value documents.",
"objects": {
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.csv.parser_base.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.csv.parser_base.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.csv.parser_base.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.csv.parser_base.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.csv.parser_base.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.parser_base.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.csv.parser_base.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.csv.parser_base.T",
"signature": null,
"docstring": null
},
"CsvParserBase": {
"name": "CsvParserBase",
"kind": "class",
"path": "omniread.csv.parser_base.CsvParserBase",
"signature": null,
"docstring": "Base csv parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces csv content-type compatibility and provides\n the extension point for implementing concrete csv parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.csv.parser_base.CsvParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (CSV only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.csv.parser_base.CsvParserBase.parse",
"signature": "parse() -> T",
"docstring": "Parse csv content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
}
}
}
}

129
docs/mcp/modules/omniread.csv.scraper.json Normal file
View File

@@ -0,0 +1,129 @@
{
"module": "omniread.csv.scraper",
"content": {
"path": "omniread.csv.scraper",
"docstring": "# Summary\n\nCSV scraper for OmniRead.\n\nThis module defines the scraper responsible for acquiring raw\ncomma-separated-value document content from a backing store via a\nconfigured client.\n\nThe scraper does not interpret or parse the acquired bytes; it wraps them in\nthe canonical `Content` model.",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.csv.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.csv.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.csv.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.csv.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.csv.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.csv.scraper.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.csv.scraper.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseCsvClient": {
"name": "BaseCsvClient",
"kind": "class",
"path": "omniread.csv.scraper.BaseCsvClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving csv bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full csv binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.scraper.BaseCsvClient.fetch",
"signature": "fetch(source: Any)",
"docstring": "Fetch raw csv bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the csv location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw csv bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"CsvScraper": {
"name": "CsvScraper",
"kind": "class",
"path": "omniread.csv.scraper.CsvScraper",
"signature": "CsvScraper(*, client: BaseCsvClient)",
"docstring": "Scraper for csv documents.\n\nNotes:\n **Responsibilities:**\n\n - Fetch raw csv bytes via the configured client.\n - Wrap the payload in a canonical `Content` instance with the\n CSV content type and source identifier.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.csv.scraper.CsvScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch a csv document from the given source.\n\nArgs:\n source (Any):\n Identifier of the csv source as understood by the\n configured client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw csv bytes, source\n identifier, CSV content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the client."
}
}
}
}
}
}

425
docs/mcp/modules/omniread.html.json Normal file
View File

@@ -0,0 +1,425 @@
{
"module": "omniread.html",
"content": {
"path": "omniread.html",
"docstring": "# Summary\n\nHTML format implementation for OmniRead.\n\nThis package provides **HTML-specific implementations** of the core OmniRead\ncontracts defined in `omniread.core`.\n\nIt includes:\n\n- HTML parsers that interpret HTML content.\n- HTML scrapers that retrieve HTML documents.\n\nKey characteristics:\n\n- Implements, but does not redefine, core contracts.\n- May contain HTML-specific behavior and edge-case handling.\n- Produces canonical content models defined in `omniread.core.content`.\n\nConsumers should depend on `omniread.core` interfaces wherever possible and\nuse this package only when HTML-specific behavior is required.\n\n---\n\n# Public API\n\n- `HTMLScraper`\n- `HTMLParser`\n\n---",
"objects": {
"HTMLScraper": {
"name": "HTMLScraper",
"kind": "class",
"path": "omniread.html.HTMLScraper",
"signature": "HTMLScraper(*, client: httpx.Client | None = None, timeout: float = 15.0, headers: Mapping[str, str] | None = None, follow_redirects: bool = True)",
"docstring": "Base HTML scraper using `httpx`.\n\nNotes:\n **Responsibilities:**\n\n - This scraper retrieves HTML documents over HTTP(S) and returns\n them as raw content wrapped in a `Content` object.\n - Fetches raw bytes and metadata only.\n - The scraper uses `httpx.Client` for HTTP requests, enforces an\n HTML content type, and preserves HTTP response metadata.\n\n **Constraints:**\n\n - The scraper does not: Parse HTML, perform retries or backoff,\n handle non-HTML responses.",
"members": {
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.HTMLScraper.content_type",
"signature": null,
"docstring": null
},
"validate_content_type": {
"name": "validate_content_type",
"kind": "function",
"path": "omniread.html.HTMLScraper.validate_content_type",
"signature": "validate_content_type(response: httpx.Response)",
"docstring": "Validate that the HTTP response contains HTML content.\n\nArgs:\n response (httpx.Response):\n HTTP response returned by `httpx`.\n\nRaises:\n ValueError:\n If the `Content-Type` header is missing or does not indicate HTML content."
},
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.html.HTMLScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch an HTML document from the given source.\n\nArgs:\n source (str):\n URL of the HTML document.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to be merged into the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw HTML bytes, source URL, HTML content type, and HTTP response metadata.\n\nRaises:\n httpx.HTTPError:\n If the HTTP request fails.\n ValueError:\n If the response is not valid HTML."
}
}
},
"HTMLParser": {
"name": "HTMLParser",
"kind": "class",
"path": "omniread.html.HTMLParser",
"signature": "HTMLParser(content: Content, features: str = 'html.parser')",
"docstring": "Base HTML parser.\n\nNotes:\n **Responsibilities:**\n\n - This class extends the core `BaseParser` with HTML-specific behavior,\n including DOM parsing via BeautifulSoup and reusable extraction helpers.\n - Provides reusable helpers for HTML extraction. Concrete parsers must\n explicitly define the return type.\n\n **Guarantees:**\n\n - Accepts only HTML content.\n - Owns a parsed BeautifulSoup DOM tree.\n - Provides pure helper utilities for common HTML structures.\n\n **Constraints:**\n\n - Concrete subclasses must define the output type `T` and implement\n the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.html.HTMLParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (HTML only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.html.HTMLParser.parse",
"signature": "parse()",
"docstring": "Fully parse the HTML content into structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the HTML DOM and return a\n deterministic, structured output."
},
"parse_div": {
"name": "parse_div",
"kind": "function",
"path": "omniread.html.HTMLParser.parse_div",
"signature": "parse_div(div: Tag, *, separator: str = ' ')",
"docstring": "Extract normalized text from a `<div>` element.\n\nArgs:\n div (Tag):\n BeautifulSoup tag representing a `<div>`.\n separator (str, optional):\n String used to separate text nodes.\n\nReturns:\n str:\n Flattened, whitespace-normalized text content."
},
"parse_link": {
"name": "parse_link",
"kind": "function",
"path": "omniread.html.HTMLParser.parse_link",
"signature": "parse_link(a: Tag)",
"docstring": "Extract the hyperlink reference from an `<a>` element.\n\nArgs:\n a (Tag):\n BeautifulSoup tag representing an anchor.\n\nReturns:\n str | None:\n The value of the `href` attribute, or None if absent."
},
"parse_table": {
"name": "parse_table",
"kind": "function",
"path": "omniread.html.HTMLParser.parse_table",
"signature": "parse_table(table: Tag)",
"docstring": "Parse an HTML table into a 2D list of strings.\n\nArgs:\n table (Tag):\n BeautifulSoup tag representing a `<table>`.\n\nReturns:\n list[list[str]]:\n A list of rows, where each row is a list of cell text values."
},
"parse_meta": {
"name": "parse_meta",
"kind": "function",
"path": "omniread.html.HTMLParser.parse_meta",
"signature": "parse_meta()",
"docstring": "Extract high-level metadata from the HTML document.\n\nReturns:\n dict[str, Any]:\n Dictionary containing extracted metadata.\n\nNotes:\n **Responsibilities:**\n\n - Extract high-level metadata from the HTML document.\n - This includes: Document title, `<meta>` tag name/property to\n content mappings."
}
}
},
"parser": {
"name": "parser",
"kind": "module",
"path": "omniread.html.parser",
"signature": null,
"docstring": "# Summary\n\nHTML parser base implementations for OmniRead.\n\nThis module provides reusable HTML parsing utilities built on top of\nthe abstract parser contracts defined in `omniread.core.parser`.\n\nIt supplies:\n\n- Content-type enforcement for HTML inputs\n- BeautifulSoup initialization and lifecycle management\n- Common helper methods for extracting structured data from HTML elements\n\nConcrete parsers must subclass `HTMLParser` and implement the `parse()` method\nto return a structured representation appropriate for their use case.",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.html.parser.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.html.parser.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.html.parser.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.parser.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.html.parser.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.html.parser.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.html.parser.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.html.parser.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.html.parser.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.html.parser.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.html.parser.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.html.parser.T",
"signature": null,
"docstring": null
},
"HTMLParser": {
"name": "HTMLParser",
"kind": "class",
"path": "omniread.html.parser.HTMLParser",
"signature": "HTMLParser(content: Content, features: str = 'html.parser')",
"docstring": "Base HTML parser.\n\nNotes:\n **Responsibilities:**\n\n - This class extends the core `BaseParser` with HTML-specific behavior,\n including DOM parsing via BeautifulSoup and reusable extraction helpers.\n - Provides reusable helpers for HTML extraction. Concrete parsers must\n explicitly define the return type.\n\n **Guarantees:**\n\n - Accepts only HTML content.\n - Owns a parsed BeautifulSoup DOM tree.\n - Provides pure helper utilities for common HTML structures.\n\n **Constraints:**\n\n - Concrete subclasses must define the output type `T` and implement\n the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.html.parser.HTMLParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (HTML only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse",
"signature": "parse() -> T",
"docstring": "Fully parse the HTML content into structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the HTML DOM and return a\n deterministic, structured output."
},
"parse_div": {
"name": "parse_div",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_div",
"signature": "parse_div(div: Tag, *, separator: str = ' ') -> str",
"docstring": "Extract normalized text from a `<div>` element.\n\nArgs:\n div (Tag):\n BeautifulSoup tag representing a `<div>`.\n separator (str, optional):\n String used to separate text nodes.\n\nReturns:\n str:\n Flattened, whitespace-normalized text content."
},
"parse_link": {
"name": "parse_link",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_link",
"signature": "parse_link(a: Tag) -> str | None",
"docstring": "Extract the hyperlink reference from an `<a>` element.\n\nArgs:\n a (Tag):\n BeautifulSoup tag representing an anchor.\n\nReturns:\n str | None:\n The value of the `href` attribute, or None if absent."
},
"parse_table": {
"name": "parse_table",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_table",
"signature": "parse_table(table: Tag) -> list[list[str]]",
"docstring": "Parse an HTML table into a 2D list of strings.\n\nArgs:\n table (Tag):\n BeautifulSoup tag representing a `<table>`.\n\nReturns:\n list[list[str]]:\n A list of rows, where each row is a list of cell text values."
},
"parse_meta": {
"name": "parse_meta",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_meta",
"signature": "parse_meta() -> dict[str, Any]",
"docstring": "Extract high-level metadata from the HTML document.\n\nReturns:\n dict[str, Any]:\n Dictionary containing extracted metadata.\n\nNotes:\n **Responsibilities:**\n\n - Extract high-level metadata from the HTML document.\n - This includes: Document title, `<meta>` tag name/property to\n content mappings."
}
}
}
}
},
"scraper": {
"name": "scraper",
"kind": "module",
"path": "omniread.html.scraper",
"signature": null,
"docstring": "# Summary\n\nHTML scraping implementation for OmniRead.\n\nThis module provides an HTTP-based scraper for retrieving HTML documents.\nIt implements the core `BaseScraper` contract using `httpx` as the transport\nlayer.\n\nThis scraper is responsible for:\n\n- Fetching raw HTML bytes over HTTP(S)\n- Validating response content type\n- Attaching HTTP metadata to the returned content\n\nThis scraper is not responsible for:\n\n- Parsing or interpreting HTML\n- Retrying failed requests\n- Managing crawl policies or rate limiting",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.html.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.html.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.html.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.html.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.html.scraper.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.html.scraper.BaseScraper",
"signature": null,
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes)\n from a source. It must not interpret or parse it.\n - A scraper is a stateless acquisition component that retrieves raw\n content from a source and returns it as a `Content` object.\n - Scrapers define how content is obtained, not what the content means.\n - Implementations may vary in transport mechanism, authentication\n strategy, retry and backoff behavior.\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics,\n or couple scraping logic to a specific parser.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.html.scraper.BaseScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.).\n\n metadata (Mapping[str, Any] | None, optional):\n Optional hints for the scraper (headers, auth, etc.).\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source`\n and return it as raw bytes wrapped in a `Content` object."
}
}
},
"HTMLScraper": {
"name": "HTMLScraper",
"kind": "class",
"path": "omniread.html.scraper.HTMLScraper",
"signature": "HTMLScraper(*, client: httpx.Client | None = None, timeout: float = 15.0, headers: Mapping[str, str] | None = None, follow_redirects: bool = True)",
"docstring": "Base HTML scraper using `httpx`.\n\nNotes:\n **Responsibilities:**\n\n - This scraper retrieves HTML documents over HTTP(S) and returns\n them as raw content wrapped in a `Content` object.\n - Fetches raw bytes and metadata only.\n - The scraper uses `httpx.Client` for HTTP requests, enforces an\n HTML content type, and preserves HTTP response metadata.\n\n **Constraints:**\n\n - The scraper does not: Parse HTML, perform retries or backoff,\n handle non-HTML responses.",
"members": {
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.scraper.HTMLScraper.content_type",
"signature": null,
"docstring": null
},
"validate_content_type": {
"name": "validate_content_type",
"kind": "function",
"path": "omniread.html.scraper.HTMLScraper.validate_content_type",
"signature": "validate_content_type(response: httpx.Response) -> None",
"docstring": "Validate that the HTTP response contains HTML content.\n\nArgs:\n response (httpx.Response):\n HTTP response returned by `httpx`.\n\nRaises:\n ValueError:\n If the `Content-Type` header is missing or does not indicate HTML content."
},
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.html.scraper.HTMLScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch an HTML document from the given source.\n\nArgs:\n source (str):\n URL of the HTML document.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to be merged into the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw HTML bytes, source URL, HTML content type, and HTTP response metadata.\n\nRaises:\n httpx.HTTPError:\n If the HTTP request fails.\n ValueError:\n If the response is not valid HTML."
}
}
}
}
}
}
}
}

192
docs/mcp/modules/omniread.html.parser.json Normal file
View File

@@ -0,0 +1,192 @@
{
"module": "omniread.html.parser",
"content": {
"path": "omniread.html.parser",
"docstring": "# Summary\n\nHTML parser base implementations for OmniRead.\n\nThis module provides reusable HTML parsing utilities built on top of\nthe abstract parser contracts defined in `omniread.core.parser`.\n\nIt supplies:\n\n- Content-type enforcement for HTML inputs\n- BeautifulSoup initialization and lifecycle management\n- Common helper methods for extracting structured data from HTML elements\n\nConcrete parsers must subclass `HTMLParser` and implement the `parse()` method\nto return a structured representation appropriate for their use case.",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.html.parser.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.html.parser.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.html.parser.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.parser.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.html.parser.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.html.parser.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.html.parser.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.html.parser.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.html.parser.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.html.parser.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.html.parser.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.html.parser.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.html.parser.T",
"signature": null,
"docstring": null
},
"HTMLParser": {
"name": "HTMLParser",
"kind": "class",
"path": "omniread.html.parser.HTMLParser",
"signature": "HTMLParser(content: Content, features: str = 'html.parser')",
"docstring": "Base HTML parser.\n\nNotes:\n **Responsibilities:**\n\n - This class extends the core `BaseParser` with HTML-specific behavior,\n including DOM parsing via BeautifulSoup and reusable extraction helpers.\n - Provides reusable helpers for HTML extraction. Concrete parsers must\n explicitly define the return type.\n\n **Guarantees:**\n\n - Accepts only HTML content.\n - Owns a parsed BeautifulSoup DOM tree.\n - Provides pure helper utilities for common HTML structures.\n\n **Constraints:**\n\n - Concrete subclasses must define the output type `T` and implement\n the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.html.parser.HTMLParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (HTML only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse",
"signature": "parse() -> T",
"docstring": "Fully parse the HTML content into structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the HTML DOM and return a\n deterministic, structured output."
},
"parse_div": {
"name": "parse_div",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_div",
"signature": "parse_div(div: Tag, *, separator: str = ' ') -> str",
"docstring": "Extract normalized text from a `<div>` element.\n\nArgs:\n div (Tag):\n BeautifulSoup tag representing a `<div>`.\n separator (str, optional):\n String used to separate text nodes.\n\nReturns:\n str:\n Flattened, whitespace-normalized text content."
},
"parse_link": {
"name": "parse_link",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_link",
"signature": "parse_link(a: Tag) -> str | None",
"docstring": "Extract the hyperlink reference from an `<a>` element.\n\nArgs:\n a (Tag):\n BeautifulSoup tag representing an anchor.\n\nReturns:\n str | None:\n The value of the `href` attribute, or None if absent."
},
"parse_table": {
"name": "parse_table",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_table",
"signature": "parse_table(table: Tag) -> list[list[str]]",
"docstring": "Parse an HTML table into a 2D list of strings.\n\nArgs:\n table (Tag):\n BeautifulSoup tag representing a `<table>`.\n\nReturns:\n list[list[str]]:\n A list of rows, where each row is a list of cell text values."
},
"parse_meta": {
"name": "parse_meta",
"kind": "function",
"path": "omniread.html.parser.HTMLParser.parse_meta",
"signature": "parse_meta() -> dict[str, Any]",
"docstring": "Extract high-level metadata from the HTML document.\n\nReturns:\n dict[str, Any]:\n Dictionary containing extracted metadata.\n\nNotes:\n **Responsibilities:**\n\n - Extract high-level metadata from the HTML document.\n - This includes: Document title, `<meta>` tag name/property to\n content mappings."
}
}
}
}
}
}

143
docs/mcp/modules/omniread.html.scraper.json Normal file
View File

@@ -0,0 +1,143 @@
{
"module": "omniread.html.scraper",
"content": {
"path": "omniread.html.scraper",
"docstring": "# Summary\n\nHTML scraping implementation for OmniRead.\n\nThis module provides an HTTP-based scraper for retrieving HTML documents.\nIt implements the core `BaseScraper` contract using `httpx` as the transport\nlayer.\n\nThis scraper is responsible for:\n\n- Fetching raw HTML bytes over HTTP(S)\n- Validating response content type\n- Attaching HTTP metadata to the returned content\n\nThis scraper is not responsible for:\n\n- Parsing or interpreting HTML\n- Retrying failed requests\n- Managing crawl policies or rate limiting",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.html.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.html.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.html.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.html.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.html.scraper.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.html.scraper.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.html.scraper.BaseScraper",
"signature": null,
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes)\n from a source. It must not interpret or parse it.\n - A scraper is a stateless acquisition component that retrieves raw\n content from a source and returns it as a `Content` object.\n - Scrapers define how content is obtained, not what the content means.\n - Implementations may vary in transport mechanism, authentication\n strategy, retry and backoff behavior.\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics,\n or couple scraping logic to a specific parser.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.html.scraper.BaseScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.).\n\n metadata (Mapping[str, Any] | None, optional):\n Optional hints for the scraper (headers, auth, etc.).\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source`\n and return it as raw bytes wrapped in a `Content` object."
}
}
},
"HTMLScraper": {
"name": "HTMLScraper",
"kind": "class",
"path": "omniread.html.scraper.HTMLScraper",
"signature": "HTMLScraper(*, client: httpx.Client | None = None, timeout: float = 15.0, headers: Mapping[str, str] | None = None, follow_redirects: bool = True)",
"docstring": "Base HTML scraper using `httpx`.\n\nNotes:\n **Responsibilities:**\n\n - This scraper retrieves HTML documents over HTTP(S) and returns\n them as raw content wrapped in a `Content` object.\n - Fetches raw bytes and metadata only.\n - The scraper uses `httpx.Client` for HTTP requests, enforces an\n HTML content type, and preserves HTTP response metadata.\n\n **Constraints:**\n\n - The scraper does not: Parse HTML, perform retries or backoff,\n handle non-HTML responses.",
"members": {
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.html.scraper.HTMLScraper.content_type",
"signature": null,
"docstring": null
},
"validate_content_type": {
"name": "validate_content_type",
"kind": "function",
"path": "omniread.html.scraper.HTMLScraper.validate_content_type",
"signature": "validate_content_type(response: httpx.Response) -> None",
"docstring": "Validate that the HTTP response contains HTML content.\n\nArgs:\n response (httpx.Response):\n HTTP response returned by `httpx`.\n\nRaises:\n ValueError:\n If the `Content-Type` header is missing or does not indicate HTML content."
},
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.html.scraper.HTMLScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch an HTML document from the given source.\n\nArgs:\n source (str):\n URL of the HTML document.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to be merged into the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw HTML bytes, source URL, HTML content type, and HTTP response metadata.\n\nRaises:\n httpx.HTTPError:\n If the HTTP request fails.\n ValueError:\n If the response is not valid HTML."
}
}
}
}
}
}

2699
docs/mcp/modules/omniread.json Normal file
View File

File diff suppressed because it is too large Load Diff

41
docs/mcp/modules/omniread.pdf.client.json Normal file
View File

@@ -0,0 +1,41 @@
{
"module": "omniread.pdf.client",
"content": {
"path": "omniread.pdf.client",
"docstring": "# Summary\n\nPDF client abstractions for OmniRead.\n\nThis module defines the **client layer** responsible for retrieving raw PDF\nbytes from a concrete backing store.\n\nClients provide low-level access to PDF binaries and are intentionally\ndecoupled from scraping and parsing logic. They do not perform validation,\ninterpretation, or content extraction.\n\nTypical backing stores include:\n\n- Local filesystems\n- Object storage (S3, GCS, etc.)\n- Network file systems",
"objects": {
"BasePDFClient": {
"name": "BasePDFClient",
"kind": "class",
"path": "omniread.pdf.client.BasePDFClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving PDF bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full PDF binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.client.BasePDFClient.fetch",
"signature": "fetch(source: Any) -> bytes",
"docstring": "Fetch raw PDF bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF location, such as a file path, object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemPDFClient": {
"name": "FileSystemPDFClient",
"kind": "class",
"path": "omniread.pdf.client.FileSystemPDFClient",
"signature": null,
"docstring": "PDF client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads PDF files directly from the disk and returns\n their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.client.FileSystemPDFClient.fetch",
"signature": "fetch(path: Path) -> bytes",
"docstring": "Read a PDF file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the PDF file.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
}
}
}
}

377
docs/mcp/modules/omniread.pdf.json Normal file
View File

@@ -0,0 +1,377 @@
{
"module": "omniread.pdf",
"content": {
"path": "omniread.pdf",
"docstring": "# Summary\n\nPDF format implementation for OmniRead.\n\nThis package provides **PDF-specific implementations** of the core OmniRead\ncontracts defined in `omniread.core`.\n\nUnlike HTML, PDF handling requires an explicit client layer for document\naccess. This package therefore includes:\n\n- PDF clients for acquiring raw PDF data.\n- PDF scrapers that coordinate client access.\n- PDF parsers that extract structured content from PDF binaries.\n\nPublic exports from this package represent the supported PDF pipeline\nand are safe for consumers to import directly when working with PDFs.\n\n---\n\n# Public API\n\n- `FileSystemPDFClient`\n- `PDFScraper`\n- `PDFParser`\n\n---",
"objects": {
"FileSystemPDFClient": {
"name": "FileSystemPDFClient",
"kind": "class",
"path": "omniread.pdf.FileSystemPDFClient",
"signature": null,
"docstring": "PDF client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads PDF files directly from the disk and returns\n their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.FileSystemPDFClient.fetch",
"signature": "fetch(path: Path)",
"docstring": "Read a PDF file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the PDF file.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
},
"PDFScraper": {
"name": "PDFScraper",
"kind": "class",
"path": "omniread.pdf.PDFScraper",
"signature": "PDFScraper(*, client: BasePDFClient)",
"docstring": "Scraper for PDF sources.\n\nNotes:\n **Responsibilities:**\n\n - Delegates byte retrieval to a PDF client and normalizes output\n into `Content`.\n - Preserves caller-provided metadata.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.PDFScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch a PDF document from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF source as understood by the configured PDF client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw PDF bytes, source identifier, PDF content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the PDF client."
}
}
},
"PDFParser": {
"name": "PDFParser",
"kind": "class",
"path": "omniread.pdf.PDFParser",
"signature": null,
"docstring": "Base PDF parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces PDF content-type compatibility and provides\n the extension point for implementing concrete PDF parsing strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.pdf.PDFParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (PDF only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.pdf.PDFParser.parse",
"signature": "parse()",
"docstring": "Parse PDF content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the PDF binary payload and\n return a deterministic, structured output."
}
}
},
"client": {
"name": "client",
"kind": "module",
"path": "omniread.pdf.client",
"signature": null,
"docstring": "# Summary\n\nPDF client abstractions for OmniRead.\n\nThis module defines the **client layer** responsible for retrieving raw PDF\nbytes from a concrete backing store.\n\nClients provide low-level access to PDF binaries and are intentionally\ndecoupled from scraping and parsing logic. They do not perform validation,\ninterpretation, or content extraction.\n\nTypical backing stores include:\n\n- Local filesystems\n- Object storage (S3, GCS, etc.)\n- Network file systems",
"members": {
"BasePDFClient": {
"name": "BasePDFClient",
"kind": "class",
"path": "omniread.pdf.client.BasePDFClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving PDF bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full PDF binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.client.BasePDFClient.fetch",
"signature": "fetch(source: Any) -> bytes",
"docstring": "Fetch raw PDF bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF location, such as a file path, object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemPDFClient": {
"name": "FileSystemPDFClient",
"kind": "class",
"path": "omniread.pdf.client.FileSystemPDFClient",
"signature": null,
"docstring": "PDF client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads PDF files directly from the disk and returns\n their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.client.FileSystemPDFClient.fetch",
"signature": "fetch(path: Path) -> bytes",
"docstring": "Read a PDF file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the PDF file.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
}
}
},
"parser": {
"name": "parser",
"kind": "module",
"path": "omniread.pdf.parser",
"signature": null,
"docstring": "# Summary\n\nPDF parser base implementations for OmniRead.\n\nThis module defines the **PDF-specific parser contract**, extending the\nformat-agnostic `BaseParser` with constraints appropriate for PDF content.\n\nPDF parsers are responsible for interpreting binary PDF data and producing\nstructured representations suitable for downstream consumption.",
"members": {
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.pdf.parser.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.pdf.parser.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.pdf.parser.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.pdf.parser.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.pdf.parser.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.pdf.parser.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.pdf.parser.T",
"signature": null,
"docstring": null
},
"PDFParser": {
"name": "PDFParser",
"kind": "class",
"path": "omniread.pdf.parser.PDFParser",
"signature": null,
"docstring": "Base PDF parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces PDF content-type compatibility and provides\n the extension point for implementing concrete PDF parsing strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.pdf.parser.PDFParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (PDF only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.pdf.parser.PDFParser.parse",
"signature": "parse() -> T",
"docstring": "Parse PDF content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the PDF binary payload and\n return a deterministic, structured output."
}
}
}
}
},
"scraper": {
"name": "scraper",
"kind": "module",
"path": "omniread.pdf.scraper",
"signature": null,
"docstring": "# Summary\n\nPDF scraping implementation for OmniRead.\n\nThis module provides a PDF-specific scraper that coordinates PDF byte\nretrieval via a client and normalizes the result into a `Content` object.\n\nThe scraper implements the core `BaseScraper` contract while delegating\nall storage and access concerns to a `BasePDFClient` implementation.",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.pdf.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.pdf.scraper.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.pdf.scraper.BaseScraper",
"signature": null,
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes)\n from a source. It must not interpret or parse it.\n - A scraper is a stateless acquisition component that retrieves raw\n content from a source and returns it as a `Content` object.\n - Scrapers define how content is obtained, not what the content means.\n - Implementations may vary in transport mechanism, authentication\n strategy, retry and backoff behavior.\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics,\n or couple scraping logic to a specific parser.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.BaseScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.).\n\n metadata (Mapping[str, Any] | None, optional):\n Optional hints for the scraper (headers, auth, etc.).\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source`\n and return it as raw bytes wrapped in a `Content` object."
}
}
},
"BasePDFClient": {
"name": "BasePDFClient",
"kind": "class",
"path": "omniread.pdf.scraper.BasePDFClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving PDF bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full PDF binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.BasePDFClient.fetch",
"signature": "fetch(source: Any)",
"docstring": "Fetch raw PDF bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF location, such as a file path, object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"PDFScraper": {
"name": "PDFScraper",
"kind": "class",
"path": "omniread.pdf.scraper.PDFScraper",
"signature": "PDFScraper(*, client: BasePDFClient)",
"docstring": "Scraper for PDF sources.\n\nNotes:\n **Responsibilities:**\n\n - Delegates byte retrieval to a PDF client and normalizes output\n into `Content`.\n - Preserves caller-provided metadata.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.PDFScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch a PDF document from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF source as understood by the configured PDF client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw PDF bytes, source identifier, PDF content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the PDF client."
}
}
}
}
}
}
}
}

127
docs/mcp/modules/omniread.pdf.parser.json Normal file
View File

@@ -0,0 +1,127 @@
{
"module": "omniread.pdf.parser",
"content": {
"path": "omniread.pdf.parser",
"docstring": "# Summary\n\nPDF parser base implementations for OmniRead.\n\nThis module defines the **PDF-specific parser contract**, extending the\nformat-agnostic `BaseParser` with constraints appropriate for PDF content.\n\nPDF parsers are responsible for interpreting binary PDF data and producing\nstructured representations suitable for downstream consumption.",
"objects": {
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.pdf.parser.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.pdf.parser.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.pdf.parser.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.pdf.parser.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.pdf.parser.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.pdf.parser.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.pdf.parser.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.pdf.parser.T",
"signature": null,
"docstring": null
},
"PDFParser": {
"name": "PDFParser",
"kind": "class",
"path": "omniread.pdf.parser.PDFParser",
"signature": null,
"docstring": "Base PDF parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces PDF content-type compatibility and provides\n the extension point for implementing concrete PDF parsing strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.pdf.parser.PDFParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (PDF only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.pdf.parser.PDFParser.parse",
"signature": "parse() -> T",
"docstring": "Parse PDF content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully interpret the PDF binary payload and\n return a deterministic, structured output."
}
}
}
}
}
}

145
docs/mcp/modules/omniread.pdf.scraper.json Normal file
View File

@@ -0,0 +1,145 @@
{
"module": "omniread.pdf.scraper",
"content": {
"path": "omniread.pdf.scraper",
"docstring": "# Summary\n\nPDF scraping implementation for OmniRead.\n\nThis module provides a PDF-specific scraper that coordinates PDF byte\nretrieval via a client and normalizes the result into a `Content` object.\n\nThe scraper implements the core `BaseScraper` contract while delegating\nall storage and access concerns to a `BasePDFClient` implementation.",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.pdf.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.pdf.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.pdf.scraper.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.pdf.scraper.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseScraper": {
"name": "BaseScraper",
"kind": "class",
"path": "omniread.pdf.scraper.BaseScraper",
"signature": null,
"docstring": "Base interface for all scrapers.\n\nNotes:\n **Responsibilities:**\n\n - A scraper is responsible ONLY for fetching raw content (bytes)\n from a source. It must not interpret or parse it.\n - A scraper is a stateless acquisition component that retrieves raw\n content from a source and returns it as a `Content` object.\n - Scrapers define how content is obtained, not what the content means.\n - Implementations may vary in transport mechanism, authentication\n strategy, retry and backoff behavior.\n\n **Constraints:**\n\n - Implementations must not parse content, modify content semantics,\n or couple scraping logic to a specific parser.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.BaseScraper.fetch",
"signature": "fetch(source: str, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch raw content from the given source.\n\nArgs:\n source (str):\n Location identifier (URL, file path, S3 URI, etc.).\n\n metadata (Mapping[str, Any] | None, optional):\n Optional hints for the scraper (headers, auth, etc.).\n\nReturns:\n Content:\n Content object containing raw bytes and metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must retrieve the content referenced by `source`\n and return it as raw bytes wrapped in a `Content` object."
}
}
},
"BasePDFClient": {
"name": "BasePDFClient",
"kind": "class",
"path": "omniread.pdf.scraper.BasePDFClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving PDF bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full PDF binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.BasePDFClient.fetch",
"signature": "fetch(source: Any)",
"docstring": "Fetch raw PDF bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF location, such as a file path, object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw PDF bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"PDFScraper": {
"name": "PDFScraper",
"kind": "class",
"path": "omniread.pdf.scraper.PDFScraper",
"signature": "PDFScraper(*, client: BasePDFClient)",
"docstring": "Scraper for PDF sources.\n\nNotes:\n **Responsibilities:**\n\n - Delegates byte retrieval to a PDF client and normalizes output\n into `Content`.\n - Preserves caller-provided metadata.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.pdf.scraper.PDFScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch a PDF document from the given source.\n\nArgs:\n source (Any):\n Identifier of the PDF source as understood by the configured PDF client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw PDF bytes, source identifier, PDF content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the PDF client."
}
}
}
}
}
}

41
docs/mcp/modules/omniread.xlsx.client.json Normal file
View File

@@ -0,0 +1,41 @@
{
"module": "omniread.xlsx.client",
"content": {
"path": "omniread.xlsx.client",
"docstring": "# Summary\n\nXLSX client abstractions for OmniRead.\n\nThis module defines the **client layer** responsible for retrieving raw\nOffice Open XML spreadsheet bytes from a concrete backing store.\n\nClients provide low-level access to xlsx binaries and are intentionally\ndecoupled from scraping and parsing logic. They do not perform validation,\ninterpretation, or content extraction.\n\nTypical backing stores include:\n\n- Local filesystems\n- Object storage (S3, GCS, etc.)\n- Network file systems",
"objects": {
"BaseXlsxClient": {
"name": "BaseXlsxClient",
"kind": "class",
"path": "omniread.xlsx.client.BaseXlsxClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving spreadsheet bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full xlsx binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.client.BaseXlsxClient.fetch",
"signature": "fetch(source: Any) -> bytes",
"docstring": "Fetch raw xlsx bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the spreadsheet location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw xlsx bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemXlsxClient": {
"name": "FileSystemXlsxClient",
"kind": "class",
"path": "omniread.xlsx.client.FileSystemXlsxClient",
"signature": null,
"docstring": "XLSX client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads spreadsheet files directly from the disk and\n returns their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.client.FileSystemXlsxClient.fetch",
"signature": "fetch(path: Path) -> bytes",
"docstring": "Read an xlsx file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the spreadsheet file.\n\nReturns:\n bytes:\n Raw xlsx bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
}
}
}
}

520
docs/mcp/modules/omniread.xlsx.json Normal file
View File

@@ -0,0 +1,520 @@
{
"module": "omniread.xlsx",
"content": {
"path": "omniread.xlsx",
"docstring": "# Summary\n\nXLSX subpackage for OmniRead.\n\nProvides acquisition and parsing of Office Open XML spreadsheet (xlsx)\ncontent:\n\n- `BaseXlsxClient`: abstract backing-store client for xlsx bytes.\n- `FileSystemXlsxClient`: local filesystem implementation.\n- `XlsxScraper`: wraps fetched bytes into canonical `Content`.\n- `XlsxParserBase`: content-type-enforcing parser contract.\n- `XlsxParser`: generic string-row parser built on openpyxl.",
"objects": {
"BaseXlsxClient": {
"name": "BaseXlsxClient",
"kind": "class",
"path": "omniread.xlsx.BaseXlsxClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving spreadsheet bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full xlsx binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.BaseXlsxClient.fetch",
"signature": "fetch(source: Any)",
"docstring": "Fetch raw xlsx bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the spreadsheet location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw xlsx bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemXlsxClient": {
"name": "FileSystemXlsxClient",
"kind": "class",
"path": "omniread.xlsx.FileSystemXlsxClient",
"signature": null,
"docstring": "XLSX client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads spreadsheet files directly from the disk and\n returns their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.FileSystemXlsxClient.fetch",
"signature": "fetch(path: Path)",
"docstring": "Read an xlsx file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the spreadsheet file.\n\nReturns:\n bytes:\n Raw xlsx bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
},
"XlsxParser": {
"name": "XlsxParser",
"kind": "class",
"path": "omniread.xlsx.XlsxParser",
"signature": "XlsxParser(content: Content, *, data_only: bool = True, read_only: bool = True)",
"docstring": "Generic xlsx parser producing string rows from a worksheet.\n\nNotes:\n **Responsibilities:**\n\n - Lazily load the workbook owned by the parser's content.\n - Normalize cells (including dates and numeric values) into\n deterministic string representations.\n - Expose sheet discovery and row extraction helpers.\n\n **Constraints:**\n\n - Cells are rendered with ``str(value)`` after trimming; date and\n datetime values are rendered in ISO format. Consumers requiring\n locale-specific formatting must convert on their side.",
"members": {
"workbook": {
"name": "workbook",
"kind": "attribute",
"path": "omniread.xlsx.XlsxParser.workbook",
"signature": null,
"docstring": "The lazily loaded workbook backing this parser's content."
},
"sheet_names": {
"name": "sheet_names",
"kind": "attribute",
"path": "omniread.xlsx.XlsxParser.sheet_names",
"signature": null,
"docstring": "Names of all worksheets contained in the workbook."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.XlsxParser.parse",
"signature": "parse()",
"docstring": "Parse the first worksheet into normalized string rows.\n\nReturns:\n list[list[str]]:\n Rows of the default (first) worksheet."
},
"rows": {
"name": "rows",
"kind": "function",
"path": "omniread.xlsx.XlsxParser.rows",
"signature": "rows(sheet: int | str | None = None, *, skip_empty: bool = True)",
"docstring": "Extract normalized string rows from a worksheet.\n\nArgs:\n sheet (int | str | None):\n Worksheet index or title; defaults to the first worksheet.\n skip_empty (bool):\n When True (default), rows whose cells are all blank are omitted.\n\nReturns:\n list[list[str]]:\n Normalized rows; trailing blank cells are trimmed per row.\n\nRaises:\n ValueError:\n If the requested sheet does not exist."
}
}
},
"XlsxParserBase": {
"name": "XlsxParserBase",
"kind": "class",
"path": "omniread.xlsx.XlsxParserBase",
"signature": null,
"docstring": "Base xlsx parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces xlsx content-type compatibility and provides\n the extension point for implementing concrete xlsx parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.xlsx.XlsxParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (XLSX only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.XlsxParserBase.parse",
"signature": "parse()",
"docstring": "Parse xlsx content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
},
"XlsxScraper": {
"name": "XlsxScraper",
"kind": "class",
"path": "omniread.xlsx.XlsxScraper",
"signature": "XlsxScraper(*, client: BaseXlsxClient)",
"docstring": "Scraper for xlsx spreadsheet documents.\n\nNotes:\n **Responsibilities:**\n\n - Fetch raw xlsx bytes via the configured client.\n - Wrap the payload in a canonical `Content` instance with the\n XLSX content type and source identifier.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.XlsxScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None)",
"docstring": "Fetch an xlsx document from the given source.\n\nArgs:\n source (Any):\n Identifier of the spreadsheet source as understood by the\n configured client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw xlsx bytes, source\n identifier, XLSX content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the client."
}
}
},
"client": {
"name": "client",
"kind": "module",
"path": "omniread.xlsx.client",
"signature": null,
"docstring": "# Summary\n\nXLSX client abstractions for OmniRead.\n\nThis module defines the **client layer** responsible for retrieving raw\nOffice Open XML spreadsheet bytes from a concrete backing store.\n\nClients provide low-level access to xlsx binaries and are intentionally\ndecoupled from scraping and parsing logic. They do not perform validation,\ninterpretation, or content extraction.\n\nTypical backing stores include:\n\n- Local filesystems\n- Object storage (S3, GCS, etc.)\n- Network file systems",
"members": {
"BaseXlsxClient": {
"name": "BaseXlsxClient",
"kind": "class",
"path": "omniread.xlsx.client.BaseXlsxClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving spreadsheet bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full xlsx binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.client.BaseXlsxClient.fetch",
"signature": "fetch(source: Any) -> bytes",
"docstring": "Fetch raw xlsx bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the spreadsheet location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw xlsx bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"FileSystemXlsxClient": {
"name": "FileSystemXlsxClient",
"kind": "class",
"path": "omniread.xlsx.client.FileSystemXlsxClient",
"signature": null,
"docstring": "XLSX client that reads from the local filesystem.\n\nNotes:\n **Guarantees:**\n\n - This client reads spreadsheet files directly from the disk and\n returns their raw binary contents.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.client.FileSystemXlsxClient.fetch",
"signature": "fetch(path: Path) -> bytes",
"docstring": "Read an xlsx file from the local filesystem.\n\nArgs:\n path (Path):\n Filesystem path to the spreadsheet file.\n\nReturns:\n bytes:\n Raw xlsx bytes.\n\nRaises:\n FileNotFoundError:\n If the path does not exist.\n ValueError:\n If the path exists but is not a file."
}
}
}
}
},
"parser": {
"name": "parser",
"kind": "module",
"path": "omniread.xlsx.parser",
"signature": null,
"docstring": "# Summary\n\nXLSX parser implementations for OmniRead.\n\nThis module provides a concrete, generic parser for Office Open XML\nspreadsheets. It exposes workbook sheets as lists of string rows so\ndownstream consumers can interpret tabular content without depending on\nopenpyxl directly.\n\nThe parser is intentionally statement-agnostic: it performs no header\ndetection or column interpretation beyond basic cell normalization.",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.xlsx.parser.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.xlsx.parser.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.xlsx.parser.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.xlsx.parser.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.xlsx.parser.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"XlsxParserBase": {
"name": "XlsxParserBase",
"kind": "class",
"path": "omniread.xlsx.parser.XlsxParserBase",
"signature": null,
"docstring": "Base xlsx parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces xlsx content-type compatibility and provides\n the extension point for implementing concrete xlsx parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.xlsx.parser.XlsxParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (XLSX only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.parser.XlsxParserBase.parse",
"signature": "parse()",
"docstring": "Parse xlsx content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
},
"XlsxParser": {
"name": "XlsxParser",
"kind": "class",
"path": "omniread.xlsx.parser.XlsxParser",
"signature": "XlsxParser(content: Content, *, data_only: bool = True, read_only: bool = True)",
"docstring": "Generic xlsx parser producing string rows from a worksheet.\n\nNotes:\n **Responsibilities:**\n\n - Lazily load the workbook owned by the parser's content.\n - Normalize cells (including dates and numeric values) into\n deterministic string representations.\n - Expose sheet discovery and row extraction helpers.\n\n **Constraints:**\n\n - Cells are rendered with ``str(value)`` after trimming; date and\n datetime values are rendered in ISO format. Consumers requiring\n locale-specific formatting must convert on their side.",
"members": {
"workbook": {
"name": "workbook",
"kind": "attribute",
"path": "omniread.xlsx.parser.XlsxParser.workbook",
"signature": null,
"docstring": "The lazily loaded workbook backing this parser's content."
},
"sheet_names": {
"name": "sheet_names",
"kind": "attribute",
"path": "omniread.xlsx.parser.XlsxParser.sheet_names",
"signature": null,
"docstring": "Names of all worksheets contained in the workbook."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.parser.XlsxParser.parse",
"signature": "parse() -> list[list[str]]",
"docstring": "Parse the first worksheet into normalized string rows.\n\nReturns:\n list[list[str]]:\n Rows of the default (first) worksheet."
},
"rows": {
"name": "rows",
"kind": "function",
"path": "omniread.xlsx.parser.XlsxParser.rows",
"signature": "rows(sheet: int | str | None = None, *, skip_empty: bool = True) -> list[list[str]]",
"docstring": "Extract normalized string rows from a worksheet.\n\nArgs:\n sheet (int | str | None):\n Worksheet index or title; defaults to the first worksheet.\n skip_empty (bool):\n When True (default), rows whose cells are all blank are omitted.\n\nReturns:\n list[list[str]]:\n Normalized rows; trailing blank cells are trimmed per row.\n\nRaises:\n ValueError:\n If the requested sheet does not exist."
}
}
}
}
},
"parser_base": {
"name": "parser_base",
"kind": "module",
"path": "omniread.xlsx.parser_base",
"signature": null,
"docstring": "# Summary\n\nXLSX parser base implementation for OmniRead.\n\nThis module defines the **XLSX-specific parser contract**, extending the\nformat-agnostic `BaseParser` with constraints appropriate for Office Open\nXML spreadsheet content.",
"members": {
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.xlsx.parser_base.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.xlsx.parser_base.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.parser_base.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.xlsx.parser_base.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.T",
"signature": null,
"docstring": null
},
"XlsxParserBase": {
"name": "XlsxParserBase",
"kind": "class",
"path": "omniread.xlsx.parser_base.XlsxParserBase",
"signature": null,
"docstring": "Base xlsx parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces xlsx content-type compatibility and provides\n the extension point for implementing concrete xlsx parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.XlsxParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (XLSX only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.parser_base.XlsxParserBase.parse",
"signature": "parse() -> T",
"docstring": "Parse xlsx content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
}
}
},
"scraper": {
"name": "scraper",
"kind": "module",
"path": "omniread.xlsx.scraper",
"signature": null,
"docstring": "# Summary\n\nXLSX scraper for OmniRead.\n\nThis module defines the scraper responsible for acquiring raw Office Open\nXML spreadsheet content from a backing store via a configured client.\n\nThe scraper does not interpret or parse the acquired bytes; it wraps them in\nthe canonical `Content` model.",
"members": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.xlsx.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.xlsx.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.xlsx.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.xlsx.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.xlsx.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.xlsx.scraper.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseXlsxClient": {
"name": "BaseXlsxClient",
"kind": "class",
"path": "omniread.xlsx.scraper.BaseXlsxClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving spreadsheet bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full xlsx binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.scraper.BaseXlsxClient.fetch",
"signature": "fetch(source: Any)",
"docstring": "Fetch raw xlsx bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the spreadsheet location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw xlsx bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"XlsxScraper": {
"name": "XlsxScraper",
"kind": "class",
"path": "omniread.xlsx.scraper.XlsxScraper",
"signature": "XlsxScraper(*, client: BaseXlsxClient)",
"docstring": "Scraper for xlsx spreadsheet documents.\n\nNotes:\n **Responsibilities:**\n\n - Fetch raw xlsx bytes via the configured client.\n - Wrap the payload in a canonical `Content` instance with the\n XLSX content type and source identifier.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.scraper.XlsxScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch an xlsx document from the given source.\n\nArgs:\n source (Any):\n Identifier of the spreadsheet source as understood by the\n configured client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw xlsx bytes, source\n identifier, XLSX content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the client."
}
}
}
}
}
}
}
}

106
docs/mcp/modules/omniread.xlsx.parser.json Normal file
View File

@@ -0,0 +1,106 @@
{
"module": "omniread.xlsx.parser",
"content": {
"path": "omniread.xlsx.parser",
"docstring": "# Summary\n\nXLSX parser implementations for OmniRead.\n\nThis module provides a concrete, generic parser for Office Open XML\nspreadsheets. It exposes workbook sheets as lists of string rows so\ndownstream consumers can interpret tabular content without depending on\nopenpyxl directly.\n\nThe parser is intentionally statement-agnostic: it performs no header\ndetection or column interpretation beyond basic cell normalization.",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.xlsx.parser.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.xlsx.parser.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.xlsx.parser.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.xlsx.parser.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.xlsx.parser.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"XlsxParserBase": {
"name": "XlsxParserBase",
"kind": "class",
"path": "omniread.xlsx.parser.XlsxParserBase",
"signature": null,
"docstring": "Base xlsx parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces xlsx content-type compatibility and provides\n the extension point for implementing concrete xlsx parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.xlsx.parser.XlsxParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (XLSX only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.parser.XlsxParserBase.parse",
"signature": "parse()",
"docstring": "Parse xlsx content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
},
"XlsxParser": {
"name": "XlsxParser",
"kind": "class",
"path": "omniread.xlsx.parser.XlsxParser",
"signature": "XlsxParser(content: Content, *, data_only: bool = True, read_only: bool = True)",
"docstring": "Generic xlsx parser producing string rows from a worksheet.\n\nNotes:\n **Responsibilities:**\n\n - Lazily load the workbook owned by the parser's content.\n - Normalize cells (including dates and numeric values) into\n deterministic string representations.\n - Expose sheet discovery and row extraction helpers.\n\n **Constraints:**\n\n - Cells are rendered with ``str(value)`` after trimming; date and\n datetime values are rendered in ISO format. Consumers requiring\n locale-specific formatting must convert on their side.",
"members": {
"workbook": {
"name": "workbook",
"kind": "attribute",
"path": "omniread.xlsx.parser.XlsxParser.workbook",
"signature": null,
"docstring": "The lazily loaded workbook backing this parser's content."
},
"sheet_names": {
"name": "sheet_names",
"kind": "attribute",
"path": "omniread.xlsx.parser.XlsxParser.sheet_names",
"signature": null,
"docstring": "Names of all worksheets contained in the workbook."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.parser.XlsxParser.parse",
"signature": "parse() -> list[list[str]]",
"docstring": "Parse the first worksheet into normalized string rows.\n\nReturns:\n list[list[str]]:\n Rows of the default (first) worksheet."
},
"rows": {
"name": "rows",
"kind": "function",
"path": "omniread.xlsx.parser.XlsxParser.rows",
"signature": "rows(sheet: int | str | None = None, *, skip_empty: bool = True) -> list[list[str]]",
"docstring": "Extract normalized string rows from a worksheet.\n\nArgs:\n sheet (int | str | None):\n Worksheet index or title; defaults to the first worksheet.\n skip_empty (bool):\n When True (default), rows whose cells are all blank are omitted.\n\nReturns:\n list[list[str]]:\n Normalized rows; trailing blank cells are trimmed per row.\n\nRaises:\n ValueError:\n If the requested sheet does not exist."
}
}
}
}
}
}

127
docs/mcp/modules/omniread.xlsx.parser_base.json Normal file
View File

@@ -0,0 +1,127 @@
{
"module": "omniread.xlsx.parser_base",
"content": {
"path": "omniread.xlsx.parser_base",
"docstring": "# Summary\n\nXLSX parser base implementation for OmniRead.\n\nThis module defines the **XLSX-specific parser contract**, extending the\nformat-agnostic `BaseParser` with constraints appropriate for Office Open\nXML spreadsheet content.",
"objects": {
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.xlsx.parser_base.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseParser": {
"name": "BaseParser",
"kind": "class",
"path": "omniread.xlsx.parser_base.BaseParser",
"signature": "BaseParser(content: Content)",
"docstring": "Base interface for all parsers.\n\nNotes:\n **Guarantees:**\n\n - A parser is a self-contained object that owns the `Content` it is\n responsible for interpreting.\n - Consumers may rely on early validation of content compatibility\n and type-stable return values from `parse()`.\n\n **Responsibilities:**\n\n - Implementations must declare supported content types via `supported_types`.\n - Implementations must raise parsing-specific exceptions from `parse()`.\n - Implementations must remain deterministic for a given input.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.BaseParser.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic."
},
"content": {
"name": "content",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.BaseParser.content",
"signature": null,
"docstring": null
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.parser_base.BaseParser.parse",
"signature": "parse()",
"docstring": "Parse the owned content into structured output.\n\nReturns:\n T:\n Parsed, structured representation.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation.\n\nNotes:\n **Responsibilities:**\n\n - Implementations must fully consume the provided content and\n return a deterministic, structured output."
},
"supports": {
"name": "supports",
"kind": "function",
"path": "omniread.xlsx.parser_base.BaseParser.supports",
"signature": "supports()",
"docstring": "Check whether this parser supports the content's type.\n\nReturns:\n bool:\n True if the content type is supported; False otherwise."
}
}
},
"T": {
"name": "T",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.T",
"signature": null,
"docstring": null
},
"XlsxParserBase": {
"name": "XlsxParserBase",
"kind": "class",
"path": "omniread.xlsx.parser_base.XlsxParserBase",
"signature": null,
"docstring": "Base xlsx parser.\n\nNotes:\n **Responsibilities:**\n\n - This class enforces xlsx content-type compatibility and provides\n the extension point for implementing concrete xlsx parsing\n strategies.\n\n **Constraints:**\n\n - Concrete implementations must define the output type `T` and\n implement the `parse()` method.",
"members": {
"supported_types": {
"name": "supported_types",
"kind": "attribute",
"path": "omniread.xlsx.parser_base.XlsxParserBase.supported_types",
"signature": null,
"docstring": "Set of content types supported by this parser (XLSX only)."
},
"parse": {
"name": "parse",
"kind": "function",
"path": "omniread.xlsx.parser_base.XlsxParserBase.parse",
"signature": "parse() -> T",
"docstring": "Parse xlsx content into a structured output.\n\nReturns:\n T:\n Parsed representation of type `T`.\n\nRaises:\n Exception:\n Parsing-specific errors as defined by the implementation."
}
}
}
}
}
}

129
docs/mcp/modules/omniread.xlsx.scraper.json Normal file
View File

@@ -0,0 +1,129 @@
{
"module": "omniread.xlsx.scraper",
"content": {
"path": "omniread.xlsx.scraper",
"docstring": "# Summary\n\nXLSX scraper for OmniRead.\n\nThis module defines the scraper responsible for acquiring raw Office Open\nXML spreadsheet content from a backing store via a configured client.\n\nThe scraper does not interpret or parse the acquired bytes; it wraps them in\nthe canonical `Content` model.",
"objects": {
"Content": {
"name": "Content",
"kind": "class",
"path": "omniread.xlsx.scraper.Content",
"signature": "Content(raw: bytes, source: str, content_type: ContentType | None = ..., metadata: Mapping[str, Any] | None = ...)",
"docstring": "Normalized representation of extracted content.\n\nNotes:\n **Responsibilities:**\n\n - A `Content` instance represents a raw content payload along with\n minimal contextual metadata describing its origin and type.\n - This class is the primary exchange format between scrapers,\n parsers, and downstream consumers.",
"members": {
"raw": {
"name": "raw",
"kind": "attribute",
"path": "omniread.xlsx.scraper.Content.raw",
"signature": null,
"docstring": "Raw content bytes as retrieved from the source."
},
"source": {
"name": "source",
"kind": "attribute",
"path": "omniread.xlsx.scraper.Content.source",
"signature": null,
"docstring": "Identifier of the content origin (URL, file path, or logical name)."
},
"content_type": {
"name": "content_type",
"kind": "attribute",
"path": "omniread.xlsx.scraper.Content.content_type",
"signature": null,
"docstring": "Optional MIME type of the content, if known."
},
"metadata": {
"name": "metadata",
"kind": "attribute",
"path": "omniread.xlsx.scraper.Content.metadata",
"signature": null,
"docstring": "Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes)."
}
}
},
"ContentType": {
"name": "ContentType",
"kind": "class",
"path": "omniread.xlsx.scraper.ContentType",
"signature": null,
"docstring": "Supported MIME types for extracted content.\n\nNotes:\n **Guarantees:**\n\n - This enum represents the declared or inferred media type of the\n content source.\n - It is primarily used for routing content to the appropriate\n parser or downstream consumer.",
"members": {
"HTML": {
"name": "HTML",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.HTML",
"signature": null,
"docstring": "HTML document content."
},
"PDF": {
"name": "PDF",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.PDF",
"signature": null,
"docstring": "PDF document content."
},
"XLSX": {
"name": "XLSX",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.XLSX",
"signature": null,
"docstring": "Office Open XML spreadsheet (xlsx/xlsm) content."
},
"CSV": {
"name": "CSV",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.CSV",
"signature": null,
"docstring": "Comma-separated-value document content."
},
"JSON": {
"name": "JSON",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.JSON",
"signature": null,
"docstring": "JSON document content."
},
"XML": {
"name": "XML",
"kind": "attribute",
"path": "omniread.xlsx.scraper.ContentType.XML",
"signature": null,
"docstring": "XML document content."
}
}
},
"BaseXlsxClient": {
"name": "BaseXlsxClient",
"kind": "class",
"path": "omniread.xlsx.scraper.BaseXlsxClient",
"signature": null,
"docstring": "Abstract client responsible for retrieving spreadsheet bytes.\n\nRetrieves bytes from a specific backing store (filesystem, S3, FTP, etc.).\n\nNotes:\n **Responsibilities:**\n\n - Implementations must accept a source identifier appropriate to\n the backing store.\n - Return the full xlsx binary payload.\n - Raise retrieval-specific errors on failure.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.scraper.BaseXlsxClient.fetch",
"signature": "fetch(source: Any)",
"docstring": "Fetch raw xlsx bytes from the given source.\n\nArgs:\n source (Any):\n Identifier of the spreadsheet location, such as a file path,\n object storage key, or remote reference.\n\nReturns:\n bytes:\n Raw xlsx bytes.\n\nRaises:\n Exception:\n Retrieval-specific errors defined by the implementation."
}
}
},
"XlsxScraper": {
"name": "XlsxScraper",
"kind": "class",
"path": "omniread.xlsx.scraper.XlsxScraper",
"signature": "XlsxScraper(*, client: BaseXlsxClient)",
"docstring": "Scraper for xlsx spreadsheet documents.\n\nNotes:\n **Responsibilities:**\n\n - Fetch raw xlsx bytes via the configured client.\n - Wrap the payload in a canonical `Content` instance with the\n XLSX content type and source identifier.\n\n **Constraints:**\n\n - The scraper does not perform parsing or interpretation.\n - Does not assume a specific storage backend.",
"members": {
"fetch": {
"name": "fetch",
"kind": "function",
"path": "omniread.xlsx.scraper.XlsxScraper.fetch",
"signature": "fetch(source: Any, *, metadata: Mapping[str, Any] | None = None) -> Content",
"docstring": "Fetch an xlsx document from the given source.\n\nArgs:\n source (Any):\n Identifier of the spreadsheet source as understood by the\n configured client.\n metadata (Mapping[str, Any] | None, optional):\n Optional metadata to attach to the returned content.\n\nReturns:\n Content:\n A `Content` instance containing raw xlsx bytes, source\n identifier, XLSX content type, and optional metadata.\n\nRaises:\n Exception:\n Retrieval-specific errors raised by the client."
}
}
}
}
}
}

90
docs/mcp/nav.json Normal file
View File

@@ -0,0 +1,90 @@
[
{
"module": "omniread",
"resource": "docs://modules/omniread"
},
{
"module": "omniread.core",
"resource": "docs://modules/omniread.core"
},
{
"module": "omniread.core.content",
"resource": "docs://modules/omniread.core.content"
},
{
"module": "omniread.core.parser",
"resource": "docs://modules/omniread.core.parser"
},
{
"module": "omniread.core.scraper",
"resource": "docs://modules/omniread.core.scraper"
},
{
"module": "omniread.csv",
"resource": "docs://modules/omniread.csv"
},
{
"module": "omniread.csv.client",
"resource": "docs://modules/omniread.csv.client"
},
{
"module": "omniread.csv.parser",
"resource": "docs://modules/omniread.csv.parser"
},
{
"module": "omniread.csv.parser_base",
"resource": "docs://modules/omniread.csv.parser_base"
},
{
"module": "omniread.csv.scraper",
"resource": "docs://modules/omniread.csv.scraper"
},
{
"module": "omniread.html",
"resource": "docs://modules/omniread.html"
},
{
"module": "omniread.html.parser",
"resource": "docs://modules/omniread.html.parser"
},
{
"module": "omniread.html.scraper",
"resource": "docs://modules/omniread.html.scraper"
},
{
"module": "omniread.pdf",
"resource": "docs://modules/omniread.pdf"
},
{
"module": "omniread.pdf.client",
"resource": "docs://modules/omniread.pdf.client"
},
{
"module": "omniread.pdf.parser",
"resource": "docs://modules/omniread.pdf.parser"
},
{
"module": "omniread.pdf.scraper",
"resource": "docs://modules/omniread.pdf.scraper"
},
{
"module": "omniread.xlsx",
"resource": "docs://modules/omniread.xlsx"
},
{
"module": "omniread.xlsx.client",
"resource": "docs://modules/omniread.xlsx.client"
},
{
"module": "omniread.xlsx.parser",
"resource": "docs://modules/omniread.xlsx.parser"
},
{
"module": "omniread.xlsx.parser_base",
"resource": "docs://modules/omniread.xlsx.parser_base"
},
{
"module": "omniread.xlsx.scraper",
"resource": "docs://modules/omniread.xlsx.scraper"
}
]

99
docs/mkdocs.lib.yml Normal file
View File

@@ -0,0 +1,99 @@
theme:
name: material
palette:
- scheme: slate
primary: deep purple
accent: cyan
font:
text: Inter
code: JetBrains Mono
features:
- navigation.sections
- navigation.expand
- navigation.top
- navigation.instant
- navigation.tracking
- navigation.indexes
- content.code.copy
- content.code.annotate
- content.tabs.link
- content.action.edit
- search.highlight
- search.share
- search.suggest
icon:
logo: material/book-open-page-variant
repo: fontawesome/brands/github
markdown_extensions:
- pymdownx.superfences
- pymdownx.inlinehilite
- pymdownx.snippets
- admonition
- pymdownx.details
- pymdownx.highlight:
linenums: true
anchor_linenums: true
line_spans: __span
pygments_lang_class: true
- pymdownx.tabbed:
alternate_style: true
- pymdownx.tasklist:
custom_checkbox: true
- tables
- footnotes
- pymdownx.caret
- pymdownx.tilde
- pymdownx.mark
extra_css:
- https://unpkg.com/dracula-prism/dist/css/dracula-prism.css
plugins:
- search
- mkdocstrings:
handlers:
python:
paths:
- ..
options:
docstring_style: google
show_source: false
show_signature_annotations: true
separate_signature: true
merge_init_into_class: true
inherited_members: true
annotations_path: brief
show_root_heading: true
group_by_category: true
show_category_heading: true
show_object_full_path: false
show_symbol_type_heading: true
site_name: omniread
docs_dir: lib
site_dir: ../site/lib
nav:
- Home: index.md
- Core API:
- core/index.md
- core/content.md
- core/parser.md
- core/scraper.md
- HTML Handling:
- html/index.md
- html/parser.md
- html/scraper.md
- PDF Handling:
- pdf/index.md
- pdf/client.md
- pdf/parser.md
- pdf/scraper.md
- CSV Handling:
- csv/index.md
- csv/client.md
- csv/parser_base.md
- csv/parser.md
- csv/scraper.md
- XLSX Handling:
- xlsx/index.md
- xlsx/client.md
- xlsx/parser_base.md
- xlsx/parser.md
- xlsx/scraper.md

69
docs/mkdocs.wiki.yml Normal file
View File

@@ -0,0 +1,69 @@
site_name: OmniRead Documentation
docs_dir: wiki
site_dir: ../site/wiki
nav:
- Home: index.md
- Overview: 01_overview.md
- How to Use: 02_how_to_use.md
- Extending OmniRead: 03_extending.md
- Development: 04_development.md
theme:
name: material
palette:
scheme: slate
primary: blue grey
accent: teal
font:
text: Roboto
code: JetBrains Mono
features:
- navigation.sections
- navigation.expand
- navigation.top
- navigation.instant
- navigation.tracking
- navigation.indexes
- content.code.copy
- content.code.annotate
- content.tabs.link
- content.action.edit
- search.highlight
- search.share
- search.suggest
- navigation.tabs
- toc.integrate
- header.autohide
- announce.dismiss
- footer.social
- content.code.select
- content.code.line_numbers
- content.tooltips
icon:
logo: material/book-open-page-variant
repo: fontawesome/brands/github
markdown_extensions:
- pymdownx.superfences
- pymdownx.inlinehilite
- pymdownx.snippets
- admonition
- pymdownx.details
- pymdownx.highlight:
linenums: true
anchor_linenums: true
line_spans: __span
pygments_lang_class: true
- pymdownx.tabbed:
alternate_style: true
- pymdownx.tasklist:
custom_checkbox: true
- tables
- footnotes
- pymdownx.caret
- pymdownx.tilde
- pymdownx.mark
extra_css:
- https://unpkg.com/dracula-prism/dist/css/dracula-prism.css
plugins:
- search

86
docs/wiki/01_overview.md Normal file
View File

@@ -0,0 +1,86 @@
# 🧱 Overview
OmniRead is designed as a **decoupled content engine** with three distinct
layers. Understanding them is the key to using and extending the library.
---
## 🏗️ Architecture
```text
┌─────────────────────────────┐
│ Source (URL, file, storage) │
└────────────┬────────────────┘
│
┌─────────▼──────────┐
│ Scraper / Client │ fetches raw bytes
└─────────┬──────────┘
│ returns
┌─────────▼──────────┐
│ Content │ raw + source + type
└─────────┬──────────┘
│
┌─────────▼──────────┐
│ Parser │ parse() → structured T
└────────────────────┘
```
1. **Scraper** (`BaseScraper`) — fetches raw bytes from a source (HTTP URL,
filesystem path, object storage). Returns a `Content` instance.
Scrapers never interpret content.
2. **Content** (`Content`) — the canonical exchange model: raw bytes,
source identifier, and optional `ContentType` enum.
3. **Parser** (`BaseParser[T]`) — receives a `Content` and returns a
structured result of type `T` via `parse()`.
---
## 📦 The `Content` model
Defined in `omniread.core.content`:
```python
from dataclasses import dataclass
from omniread import Content, ContentType
@dataclass(slots=True)
class Content:
raw: bytes
source: str
content_type: ContentType | None = None
```
- `raw` — the raw bytes exactly as retrieved.
- `source` — URL, file path, or logical name identifying the origin.
- `content_type` — optional `ContentType` enum value.
---
## 🎭 The `ContentType` enum
| Value | MIME | Used by |
|---|---|---|
| `HTML` | `text/html` | HTMLScraper |
| `PDF` | `application/pdf` | PDFScraper |
| `XLSX` | `application/vnd.openxmlformats-...` | XlsxScraper |
| `CSV` | `text/csv` | CsvScraper |
| `JSON` | `application/json` | — |
| `XML` | `application/xml` | — |
---
## 🧩 Format modules at a glance
| Module | Scraper | Parser | Client | Notes |
|---|---|---|---|---|
| `omniread.html` | `HTMLScraper` | `HTMLParser` | — | httpx + BeautifulSoup |
| `omniread.pdf` | `PDFScraper` | `PDFParser` | `FileSystemPDFClient` | explicit client layer |
| `omniread.csv` | `CsvScraper` | `CsvParser` | `FileSystemCsvClient` | stdlib csv module |
| `omniread.xlsx` | `XlsxScraper` | `XlsxParser` | `FileSystemXlsxClient` | openpyxl-backed |
---
## 📚 Read Next
- [How to Use](02_how_to_use.md) — working examples per format.
- [Extending OmniRead](03_extending.md) — subclassing scrapers and parsers.

109
docs/wiki/02_how_to_use.md Normal file
View File

@@ -0,0 +1,109 @@
# 🖥️ How to Use
Every format follows the same pipeline: **scrape → `Content` → parse**.
This page shows each supported format with working patterns.
---
## 🌐 HTML
```python
from omniread import HTMLScraper, HTMLParser
class TitleParser(HTMLParser[str]):
def parse(self) -> str:
return self._soup.title.string
scraper = HTMLScraper() # httpx under the hood
content = scraper.fetch("https://example.com")
title = TitleParser(content).parse()
```
`HTMLScraper` accepts an optional `client` (an `httpx.Client`) for transport
control — the test suite wires one to a mock transport.
---
## 📕 PDF
PDFs need a **client** to supply raw bytes before parsing:
```python
from pathlib import Path
from omniread import FileSystemPDFClient, PDFScraper, PDFParser
class TextPDFParser(PDFParser[str]):
def parse(self) -> str:
# implement your extraction logic
return self.content.raw # bytes, decode as needed
client = FileSystemPDFClient()
scraper = PDFScraper(client=client)
content = scraper.fetch(Path("document.pdf"))
result = TextPDFParser(content).parse()
```
`PDFParser` subclasses receive `self.content` and implement `parse()`.
---
## 📊 CSV
```python
from omniread import FileSystemCsvClient, CsvScraper, CsvParser
scraper = CsvScraper(client=FileSystemCsvClient())
content = scraper.fetch("data.csv")
parser = CsvParser(content)
for row in parser.rows():
print(row)
```
`CsvParser.rows()` yields string rows trimmed of empties by default
(`skip_empty=True`).
---
## 📑 XLSX
```python
from omniread import FileSystemXlsxClient, XlsxScraper, XlsxParser
scraper = XlsxScraper(client=FileSystemXlsxClient())
content = scraper.fetch("statement.xlsx")
parser = XlsxParser(content)
print(parser.sheet_names) # e.g. ["Statement"]
rows = parser.rows(sheet="Statement") # by name or index
all_rows = parser.parse() # alias for rows()
```
Key behaviors:
- `rows(skip_empty=False)` keeps blank rows (off by default).
- Cells render as trimmed strings; date cells convert to ISO format
(`2026-06-01T00:00:00`).
- `rows(sheet="Missing")` raises for an unknown sheet.
---
## 🔀 End-to-end flow
```python
content = scraper.fetch(source) # 1. acquire → Content
assert isinstance(content.raw, bytes)
assert content.content_type is not None
parser = MyParser(content) # 2. interpret → T
result = parser.parse()
```
---
## 📚 Read Next
- [Overview](01_overview.md) — the full architecture.
- [Extending OmniRead](03_extending.md) — custom clients and parsers.

104
docs/wiki/03_extending.md Normal file
View File

@@ -0,0 +1,104 @@
# 🧩 Extending OmniRead
OmniRead is meant to be extended by subclassing. All public extension points
are generic over their result type, so your parser returns exactly the shape
you need.
---
## 🧬 Custom parsers
Subclass `BaseParser[T]` (or a format parser) and implement `parse()`:
```python
from pydantic import BaseModel
from omniread import HTMLParser
class Page(BaseModel):
title: str
content: str | None
class PageParser(HTMLParser[Page]):
def parse(self) -> Page:
soup = self._soup
div = soup.find("div", id="content")
return Page(
title=soup.title.string,
content=div.get_text() if div else None,
)
```
The parsed page is validated by Pydantic on construction — no manual
assertions required.
---
## 🧬 Custom PDF parsers
PDF binary layout is format-specific, so parsers return your own model:
```python
from typing import Literal
from pydantic import BaseModel
from omniread import PDFParser
class ParsedPDF(BaseModel):
size_bytes: int
magic: Literal[b"%PDF"]
class SimplePDFParser(PDFParser[ParsedPDF]):
def parse(self) -> ParsedPDF:
if not self.content.raw.startswith(b"%PDF"):
raise ValueError("Not a valid PDF")
return ParsedPDF(size_bytes=len(self.content.raw), magic=b"%PDF")
```
---
## 🧬 Custom clients
Clients supply raw bytes to a scraper. For PDFs, subclass
`BasePDFClient` (or `FileSystemPDFClient`) and implement
`fetch(source) -> bytes`:
```python
from omniread.pdf.client import BasePDFClient
class MockPDFClient(BasePDFClient):
def fetch(self, source):
return b"%PDF ..." # bytes for the logical identifier
```
The same pattern applies to `BaseCsvClient` and `BaseXlsxClient`.
---
## 🚀 Custom scrapers
Festch something that a built-in scraper does not cover by extending
`BaseScraper`:
```python
from omniread import BaseScraper, Content, ContentType
class StorageScraper(BaseScraper):
def fetch(self, source, *, metadata=None):
raw = my_object_storage.download(source) # your I/O
return Content(raw=raw, source=source, content_type=ContentType.JSON)
```
---
## ✅ Extension checklist
1. Keep **scraper** and **parser** separate — never mix I/O into `parse()`.
2. Return `Content` from any scraper/client so downstream stays uniform.
3. Return a *typed* result from your parser (Pydantic model, dataclass, str).
4. Test your custom layers with a mock client, not a live network.
---
## 📚 Read Next
- [How to Use](02_how_to_use.md) — built-in example flows.
- [Development](04_development.md) — running tests and docs.

87
docs/wiki/04_development.md Normal file
View File

@@ -0,0 +1,87 @@
# 🛠️ Development
Working on `omniread` itself.
---
## 📂 Repository layout
| Path | Purpose |
|---|---|
| `omniread/` | The library package (core + html, pdf, csv, xlsx modules) |
| `omniread/*.pyi` | Type stubs kept in sync with implementations |
| `tests/` | End-to-end and unit tests against mock transports/clients |
| `covers` | — |
| `docs/lib/` | Generated library reference (docforge, flat layout) |
| `docs/mcp/` | Machine-readable bundle served by the MCP server |
| `docs/wiki/` | This hand-written wiki |
---
## 🔧 Setup
```bash
python -m venv .venv
.venv/Scripts/pip install -e ".[dev]"
```
---
## 🧪 Tests
Run the suite (offline; a mock httpx transport and mock PDF client are used):
```bash
.venv/Scripts/pytest
```
Coverage spans end-to-end scrape → parse flows for HTML, PDF, and XLSX,
plus client validation and CSV/XLSX parsing edge cases.
---
## ✅ Quality gates
The CI quality gate runs, matching the Drone pipeline:
```bash
.venv/Scripts/black --check .
.venv/Scripts/ruff check .
.venv/Scripts/mypy
.venv/Scripts/pytest
```
---
## 📝 Building documentation (docforge)
The site is generated by [`docforge`](https://git.aetoskia.com/aetos/doc-forge)
and served per kind under `site/{kind}`:
```bash
doc-forge build \
--mkdocs --mcp \
--module-is-source --module omniread \
--site-name "OmniRead"
```
- `--module-is-source` renders the flat `docs/lib/` layout (no nesting under
`omniread/`), matching `docforge.nav.yml` and `docs/mkdocs.lib.yml`.
- `--mcp` regenerates the structured bundle in `docs/mcp/`.
- `--wiki` builds this wiki.
- `gen_api.py` (if present) regenerates the API docs.
Preview locally:
```bash
doc-forge serve --lib
doc-forge serve --wiki
doc-forge serve --mcp
```
---
## 📚 Read Next
- [Extending OmniRead](03_extending.md) — custom parsers, scrapers, clients.
- [Overview](01_overview.md) — the core architecture.

64
docs/wiki/index.md Normal file
View File

@@ -0,0 +1,64 @@
# 📄 OmniRead — Format-Agnostic Content Acquisition and Parsing
`OmniRead` is a composable content ingestion framework with pluggable scrapers
and parsers for HTML, PDF, CSV, and XLSX. It cleanly separates fetching
(I/O) from interpretation (parsing) through a canonical `Content` exchange
model, giving each format its own scraper and parser while sharing one
uniform pipeline.
> **Doc model:** this wiki is written for humans — how‑to guides, examples,
> and extension recipes. The authoritative API contracts live in the code
> (GSDFC docstrings) and the machine‑readable bundle under `docs/mcp/`.
---
## 🚀 Key Features
* 🧱 **Content/Scraper/Parser architecture** — fetch and interpret in
independent, testable layers
* 📄 **Canonical `Content` model** — raw bytes + metadata passed between all
layers via one dataclass
* 🌐 **HTML** — HTTP-based scraper with httpx; DOM parsing via BeautifulSoup
* 📕 **PDF** — client + scraper for raw PDF bytes; user-defined parsers
* 📊 **CSV** — standard-library CSV parser over `rows()` iterator
* 📑 **XLSX** — openpyxl-backed spreadsheet parser with sheet selection,
empty-row control, and ISO date conversion
---
## ⚡ Quick Start
```python
from omniread import HTMLScraper, HTMLParser
class TitleParser(HTMLParser[str]):
def parse(self) -> str:
return self._soup.title.string
content = HTMLScraper().fetch("https://example.com")
title = TitleParser(content).parse()
assert title == "Example Domain"
```
---
## 📁 Documentation Structure
| Section | What you'll find |
|---|---|
| [Overview](01_overview.md) | The three-layer architecture and the `Content` model |
| [How to Use](02_how_to_use.md) | Ready-to-run examples for HTML, PDF, CSV, and XLSX |
| [Extending OmniRead](03_extending.md) | Subclassing scrapers, parsers, and clients |
| [Development](04_development.md) | Setup, tests, and regenerating docs |
---
## 🔗 Related Resources
* **Source Code:** [Gitea Repository](https://git.aetoskia.com/aetos/omniread)
* **Internal PyPI:** [pip.aetoskia.com/simple/omniread](https://pip.aetoskia.com/simple/omniread)
* **CI:** Builds and publishes tagged releases, gated on black / ruff / mypy / pytest.
---
© Aetoskia Internal — `omniread` 0.0.1