Core

omniread.core

Summary

Core domain contracts for OmniRead.

This package defines the format-agnostic domain layer of OmniRead. It exposes canonical content models and abstract interfaces that are implemented by format-specific modules (HTML, PDF, etc.).

Public exports from this package are considered stable contracts and are safe for downstream consumers to depend on.

Submodules:

• content: Canonical content models and enums.
• parser: Abstract parsing contracts.
• scraper: Abstract scraping contracts.

Format-specific behavior must not be introduced at this layer.

---

Public API

• Content
• ContentType

---

Classes

BaseParser

BaseParser(content: Content)

Bases: ABC, Generic[T]

Base interface for all parsers.

Notes

Guarantees:

- A parser is a self-contained object that owns the `Content` it is
  responsible for interpreting.
- Consumers may rely on early validation of content compatibility
  and type-stable return values from `parse()`.

Responsibilities:

- Implementations must declare supported content types via `supported_types`.
- Implementations must raise parsing-specific exceptions from `parse()`.
- Implementations must remain deterministic for a given input.

Initialize the parser with content to be parsed.

Parameters:

Name	Type	Description	Default
content	Content	Content instance to be parsed.	required

Raises:

Type	Description
ValueError	If the content type is not supported by this parser.

Attributes

supported_types class-attribute instance-attribute

supported_types: set[ContentType] = set()

Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic.

Functions

parse abstractmethod

parse() -> T

Parse the owned content into structured output.

Returns:

Name	Type	Description
T	T	Parsed, structured representation.

Raises:

Type	Description
Exception	Parsing-specific errors as defined by the implementation.

Notes

Responsibilities:

- Implementations must fully consume the provided content and
  return a deterministic, structured output.

supports

supports() -> bool

Check whether this parser supports the content's type.

Returns:

Name	Type	Description
bool	bool	True if the content type is supported; False otherwise.

BaseScraper

Bases: ABC

Base interface for all scrapers.

Notes

Responsibilities:

- A scraper is responsible ONLY for fetching raw content (bytes)
  from a source. It must not interpret or parse it.
- A scraper is a stateless acquisition component that retrieves raw
  content from a source and returns it as a `Content` object.
- Scrapers define how content is obtained, not what the content means.
- Implementations may vary in transport mechanism, authentication
  strategy, retry and backoff behavior.

Constraints:

- Implementations must not parse content, modify content semantics,
  or couple scraping logic to a specific parser.

Functions

fetch abstractmethod

fetch(
    source: str,
    *,
    metadata: Mapping[str, Any] | None = None
) -> Content

Fetch raw content from the given source.

Parameters:

Name	Type	Description	Default
source	str	Location identifier (URL, file path, S3 URI, etc.).	required
metadata	Mapping[str, Any] | None	Optional hints for the scraper (headers, auth, etc.).	None

Returns:

Name	Type	Description
Content	Content	Content object containing raw bytes and metadata.

Raises:

Type	Description
Exception	Retrieval-specific errors as defined by the implementation.

Notes

Responsibilities:

- Implementations must retrieve the content referenced by `source`
  and return it as raw bytes wrapped in a `Content` object.

Content dataclass

Content(
    raw: bytes,
    source: str,
    content_type: ContentType | None = ...,
    metadata: Mapping[str, Any] | None = ...,
)

Normalized representation of extracted content.

Notes

Responsibilities:

- A `Content` instance represents a raw content payload along with
  minimal contextual metadata describing its origin and type.
- This class is the primary exchange format between scrapers,
  parsers, and downstream consumers.

Attributes

content_type class-attribute instance-attribute

content_type: ContentType | None = None

Optional MIME type of the content, if known.

metadata class-attribute instance-attribute

metadata: Mapping[str, Any] | None = None

Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes).

raw instance-attribute

raw: bytes

Raw content bytes as retrieved from the source.

source instance-attribute

source: str

Identifier of the content origin (URL, file path, or logical name).

ContentType

Bases: str, Enum

Supported MIME types for extracted content.

Notes

Guarantees:

- This enum represents the declared or inferred media type of the
  content source.
- It is primarily used for routing content to the appropriate
  parser or downstream consumer.

Attributes

CSV class-attribute instance-attribute

CSV = 'text/csv'

Comma-separated-value document content.

HTML class-attribute instance-attribute

HTML = 'text/html'

HTML document content.

JSON class-attribute instance-attribute

JSON = 'application/json'

JSON document content.

PDF class-attribute instance-attribute

PDF = 'application/pdf'

PDF document content.

XLSX class-attribute instance-attribute

XLSX = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"

Office Open XML spreadsheet (xlsx/xlsm) content.

XML class-attribute instance-attribute

XML = 'application/xml'

XML document content.