# 🧱 Overview

OmniRead is designed as a **decoupled content engine** with three distinct
layers. Understanding them is the key to using and extending the library.

---

## 🏗️ Architecture

```text
                 ┌─────────────────────────────┐
                 │  Source (URL, file, storage) │
                 └────────────┬────────────────┘
                              │
                    ┌─────────▼──────────┐
                    │   Scraper / Client  │  fetches raw bytes
                    └─────────┬──────────┘
                              │  returns
                    ┌─────────▼──────────┐
                    │     Content         │  raw + source + type
                    └─────────┬──────────┘
                              │
                    ┌─────────▼──────────┐
                    │      Parser         │  parse() → structured T
                    └────────────────────┘
```

1. **Scraper** (`BaseScraper`) — fetches raw bytes from a source (HTTP URL,
   filesystem path, object storage). Returns a `Content` instance.
   Scrapers never interpret content.
2. **Content** (`Content`) — the canonical exchange model: raw bytes,
   source identifier, and optional `ContentType` enum.
3. **Parser** (`BaseParser[T]`) — receives a `Content` and returns a
   structured result of type `T` via `parse()`.

---

## 📦 The `Content` model

Defined in `omniread.core.content`:

```python
from dataclasses import dataclass
from omniread import Content, ContentType

@dataclass(slots=True)
class Content:
    raw: bytes
    source: str
    content_type: ContentType | None = None
```

- `raw` — the raw bytes exactly as retrieved.
- `source` — URL, file path, or logical name identifying the origin.
- `content_type` — optional `ContentType` enum value.

---

## 🎭 The `ContentType` enum

| Value | MIME | Used by |
|---|---|---|
| `HTML` | `text/html` | HTMLScraper |
| `PDF` | `application/pdf` | PDFScraper |
| `XLSX` | `application/vnd.openxmlformats-...` | XlsxScraper |
| `CSV` | `text/csv` | CsvScraper |
| `JSON` | `application/json` | — |
| `XML` | `application/xml` | — |

---

## 🧩 Format modules at a glance

| Module | Scraper | Parser | Client | Notes |
|---|---|---|---|---|
| `omniread.html` | `HTMLScraper` | `HTMLParser` | — | httpx + BeautifulSoup |
| `omniread.pdf` | `PDFScraper` | `PDFParser` | `FileSystemPDFClient` | explicit client layer |
| `omniread.csv` | `CsvScraper` | `CsvParser` | `FileSystemCsvClient` | stdlib csv module |
| `omniread.xlsx` | `XlsxScraper` | `XlsxParser` | `FileSystemXlsxClient` | openpyxl-backed |

---

## 📚 Read Next

- [How to Use](02_how_to_use.md) — working examples per format.
- [Extending OmniRead](03_extending.md) — subclassing scrapers and parsers.