Skip to content

# Core

## `` omniread.core

### Summary

Core domain contracts for OmniRead.

This package defines the format-agnostic domain layer of OmniRead. It exposes canonical content models and abstract interfaces that are implemented by format-specific modules (HTML, PDF, etc.).

Public exports from this package are considered stable contracts and are safe for downstream consumers to depend on.

Submodules:

- `content`: Canonical content models and enums.
- `parser`: Abstract parsing contracts.
- `scraper`: Abstract scraping contracts.

Format-specific behavior must not be introduced at this layer.

### Public API

- `Content`
- `ContentType`

### Classes

#### `` BaseParser

    1

    BaseParser(content: Content)

Bases: `ABC`, `Generic[T]`

Base interface for all parsers.

Notes

Guarantees:

    1
    2
    3
    4

    - A parser is a self-contained object that owns the `Content` it is
      responsible for interpreting.
    - Consumers may rely on early validation of content compatibility
      and type-stable return values from `parse()`.

Responsibilities:

    1
    2
    3

    - Implementations must declare supported content types via `supported_types`.
    - Implementations must raise parsing-specific exceptions from `parse()`.
    - Implementations must remain deterministic for a given input.

Initialize the parser with content to be parsed.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`content` | `Content` | Content instance to be parsed. | required

Raises:

Type | Description
--- | ---
`ValueError` | If the content type is not supported by this parser.

##### Attributes

###### `` supported_types `class-attribute` `instance-attribute`

    1

    supported_types: Set[ContentType] = set()

Set of content types supported by this parser. An empty set indicates that the parser is content-type agnostic.

##### Functions

###### `` parse `abstractmethod`

    1

    parse() -> T

Parse the owned content into structured output.

Returns:

Name | Type | Description
--- | --- | ---
`T` | `T` | Parsed, structured representation.

Raises:

Type | Description
--- | ---
`Exception` | Parsing-specific errors as defined by the implementation.

Notes

Responsibilities:

    1
    2

    - Implementations must fully consume the provided content and
      return a deterministic, structured output.

###### `` supports

    1

    supports() -> bool

Check whether this parser supports the content's type.

Returns:

Name | Type | Description
--- | --- | ---
`bool` | `bool` | True if the content type is supported; False otherwise.

#### `` BaseScraper

Bases: `ABC`

Base interface for all scrapers.

Notes

Responsibilities:

    1
    2
    3
    4
    5
    6
    7

    - A scraper is responsible ONLY for fetching raw content (bytes)
      from a source. It must not interpret or parse it.
    - A scraper is a stateless acquisition component that retrieves raw
      content from a source and returns it as a `Content` object.
    - Scrapers define how content is obtained, not what the content means.
    - Implementations may vary in transport mechanism, authentication
      strategy, retry and backoff behavior.

Constraints:

    1
    2

    - Implementations must not parse content, modify content semantics,
      or couple scraping logic to a specific parser.

##### Functions

###### `` fetch `abstractmethod`

    1

    fetch(source: str, *, metadata: Optional[Mapping[str, Any]] = None) -> Content

Fetch raw content from the given source.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`source` | `str` | Location identifier (URL, file path, S3 URI, etc.). | required
`metadata` | `Optional[Mapping[str, Any]]` | Optional hints for the scraper (headers, auth, etc.). | `None`

Returns:

Name | Type | Description
--- | --- | ---
`Content` | `Content` | Content object containing raw bytes and metadata.

Raises:

Type | Description
--- | ---
`Exception` | Retrieval-specific errors as defined by the implementation.

Notes

Responsibilities:

    1
    2

    - Implementations must retrieve the content referenced by `source`
      and return it as raw bytes wrapped in a `Content` object.

#### `` Content `dataclass`

    1

    Content(raw: bytes, source: str, content_type: Optional[ContentType] = ..., metadata: Optional[Mapping[str, Any]] = ...)

Normalized representation of extracted content.

Notes

Responsibilities:

    1
    2
    3
    4

    - A `Content` instance represents a raw content payload along with
      minimal contextual metadata describing its origin and type.
    - This class is the primary exchange format between scrapers,
      parsers, and downstream consumers.

##### Attributes

###### `` content_type `class-attribute` `instance-attribute`

    1

    content_type: Optional[ContentType] = None

Optional MIME type of the content, if known.

###### `` metadata `class-attribute` `instance-attribute`

    1

    metadata: Optional[Mapping[str, Any]] = None

Optional, implementation-defined metadata associated with the content (e.g., headers, encoding hints, extraction notes).

###### `` raw `instance-attribute`

    1

    raw: bytes

Raw content bytes as retrieved from the source.

###### `` source `instance-attribute`

    1

    source: str

Identifier of the content origin (URL, file path, or logical name).

#### `` ContentType

Bases: `str`, `Enum`

Supported MIME types for extracted content.

Notes

Guarantees:

    1
    2
    3
    4

    - This enum represents the declared or inferred media type of the
      content source.
    - It is primarily used for routing content to the appropriate
      parser or downstream consumer.

##### Attributes

###### `` HTML `class-attribute` `instance-attribute`

    1

    HTML = 'text/html'

HTML document content.

###### `` JSON `class-attribute` `instance-attribute`

    1

    JSON = 'application/json'

JSON document content.

###### `` PDF `class-attribute` `instance-attribute`

    1

    PDF = 'application/pdf'

PDF document content.

###### `` XML `class-attribute` `instance-attribute`

    1

    XML = 'application/xml'

XML document content.