1 line
105 KiB
JSON
1 line
105 KiB
JSON
{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"\ud83e\udde9 mongo-ops \u2014 Async MongoDB Operations Layer for FastAPI","text":"<p><code>mongo-ops</code> is a modular, high-performance MongoDB operations library for FastAPI microservices. It standardizes connection lifecycle, Pydantic v2 document models, and repository-style async CRUD \u2014 with added layers for caching, reference population, and multi-document transactions. It is built on top of Motor.</p> <p>Doc model: this wiki is written for humans \u2014 how\u2011to guides, examples, and testing recipes. The authoritative API contracts live in the code (docstrings) and the machine\u2011readable bundle under <code>docs/mcp/</code>.</p>"},{"location":"#key-features","title":"\ud83d\ude80 Key Features","text":"<ul> <li>\ud83e\uddf1 Unified repository pattern \u2014 <code>BaseRepository</code> with generic CRUD</li> <li>\u26a1 Fully asynchronous (Motor-based)</li> <li>\ud83e\uddec Pydantic v2 data models with auto\u2011timestamps (<code>BaseDocument</code>)</li> <li>\ud83d\udd12 Model registry for multi-collection startup (<code>ModelRegistry</code>)</li> <li>\ud83d\uddc4\ufe0f ID-based caching \u2014 in-memory or Redis (<code>cache/</code>)</li> <li>\ud83d\udd17 Reference population with cycle detection (<code>populate/</code>)</li> <li>\ud83d\udd01 Transaction and session helpers (<code>transactions/</code>)</li> <li>\ud83e\uddea Mock-friendly architecture \u2014 the whole repo test suite runs without MongoDB</li> </ul>"},{"location":"#installation","title":"\ud83d\udce6 Installation","text":"<p>From your internal PyPI:</p> Bash<pre><code>pip install --extra-index-url https://$PYPI_USERNAME:$PYPI_PASSWORD@pip.aetoskia.com/simple mongo-ops\n</code></pre> <p>With the extra batteries your workload needs:</p> Bash<pre><code>pip install --extra-index-url https://$PYPI_USERNAME:$PYPI_PASSWORD@pip.aetoskia.com/simple \"mongo-ops[redis]\" # Redis cache backend\npip install --extra-index-url https://$PYPI_USERNAME:$PYPI_PASSWORD@pip.aetoskia.com/simple \"mongo-ops[all]\" # dev, redis, beanie, fastapi\n</code></pre> <p>From local source:</p> Bash<pre><code>pip install -e .\n</code></pre>"},{"location":"#documentation-structure","title":"\ud83d\udcc1 Documentation Structure","text":"Section Description Overview Architecture, lifecycle, and repository decision table Core Components Validated reference for the public API surface Use cases Step-by-step recipes \u00b7 01 \u2013 Basic CRUD API FastAPI user management app \u00b7 02 \u2013 Custom repository Product catalog with business logic \u00b7 03 \u2013 Transactions Atomic order + inventory updates \u00b7 04 \u2013 Pagination Filtering, sorting, page metadata \u00b7 05 \u2013 Soft deletes Recoverable delete pattern \u00b7 06 \u2013 Multi-model One service, many collections \u00b7 07 \u2013 Caching In-memory / Redis read cache \u00b7 08 \u2013 Population Resolve references on read \u00b7 09 \u2013 Advanced population Nested + circular references \u00b7 10 \u2013 Cache + population Reading through cache, populating on hit \u00b7 11 \u2013 Cache lifecycle Startup/shutdown hygiene \u00b7 12 \u2013 Transaction helper <code>execute_transaction</code> \u00b7 13 \u2013 Indexes Single, compound, and optioned indexes \u00b7 14 \u2013 Testing guide Mock Motor, engine, cache, transactions Best Practices Layering, lifecycle, and gotchas Common Patterns Service layer, aggregation, bulk ops Error Handling Library exceptions and FastAPI mapping Testing Example Mock-based quickstart"},{"location":"#related-resources","title":"\ud83d\udd17 Related Resources","text":"<ul> <li>Source Code: Gitea Repository</li> <li>Internal PyPI: pip.aetoskia.com/simple/mongo-ops</li> <li>Drone CI: Auto-builds and publishes tagged releases, gated on black / ruff / mypy / pytest.</li> </ul> <p>\u00a9 Aetoskia Internal \u2014 <code>mongo-ops</code> 0.1.5</p>"},{"location":"01_overview/","title":"Library Overview","text":"<p><code>mongo-ops</code> gives FastAPI/microservice teams a small, opinionated toolkit for talking to MongoDB asynchronously. This page builds the mental model: what the layers are, in what order they must be wired up, and which repository to reach for.</p>"},{"location":"01_overview/#architecture","title":"\ud83c\udfd7\ufe0f Architecture","text":"Text Only<pre><code>\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 Application (FastAPI) \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 CachedBase \u2502 Populating \u2502 BaseRepository\u2502 Raw \u2502 \u2502\n\u2502 \u2502 Repository \u2502 Repository \u2502 (CRUDMixin) \u2502 Motor \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u2502 \u2502 \u2502 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 Cache backend\u2502 \u2502 Population\u2502 \u2502 TransactionManager \u2502 \u2502\n\u2502 \u2502 memory/redis \u2502 \u2502 Engine \u2502 \u2502 (session helpers) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 ModelRegistry (models, indexes, cache lifecycle) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2502 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u2502\n\u2502 \u2502 MongoConnectionManager (Motor client + database) \u2502 \u2502\n\u2502 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u25bc\n MongoDB (Motor / PyMongo)\n</code></pre> <p>The layers build on each other:</p> <ol> <li>Connection \u2014 <code>MongoConnectionManager</code> owns the Motor client and active database.</li> <li>Models \u2014 <code>BaseDocument</code> (Pydantic v2) gives every document <code>id</code> (aliased <code>_id</code>) plus <code>created_at</code> / <code>updated_at</code>.</li> <li>CRUD \u2014 <code>CRUDMixin</code> / <code>BaseRepository</code> implement generic async CRUD against a collection.</li> <li>Startup \u2014 <code>ModelRegistry</code> centralizes model + index registration and cache lifecycle for multi-collection services.</li> <li>Advanced layers \u2014 caching and population compose around a repository:</li> <li><code>CachedBaseRepository</code> wraps <code>BaseRepository</code> with a <code>CacheBackend</code>.</li> <li><code>PopulatingRepository</code> wraps <code>BaseRepository</code> with a <code>PopulationEngine</code>.</li> <li>Transactions \u2014 <code>TransactionManager</code> runs a list of async operations atomically on a session.</li> </ol>"},{"location":"01_overview/#lifecycle-rules","title":"\ud83d\udd04 Lifecycle Rules","text":"<p><code>mongo-ops</code> has a strict startup order. Violating it raises fast, loud exceptions:</p> Step Call Why 1 <code>await MongoConnectionManager.connect(uri, db_name)</code> Without a connection, <code>get_database()</code>/<code>get_client()</code> raise <code>RuntimeError(\"Database not connected...\")</code>. 2 <code>await ModelRegistry.initialize_all()</code> Create registered indexes (idempotent via <code>create_index</code>). 3 <code>await ModelRegistry.set_cache_backend(...)</code> then <code>await ModelRegistry.initialize_cache()</code> Must happen after connection, before any cache-backed operation. Raises <code>RuntimeError(\"No cache backend registered...\")</code> if skipped. 4 Use repositories Constructed repos resolve the collection from the live database. 5 <code>await ModelRegistry.shutdown_cache()</code> + <code>await MongoConnectionManager.disconnect()</code> On shutdown (in-memory TTL task cancelled; Redis pub/sub closed). <p>The canonical wiring is the FastAPI lifespan context manager (see use case 01).</p>"},{"location":"01_overview/#which-repository-should-you-use","title":"\ud83e\udded Which Repository Should You Use?","text":"Repository Use when Adds <code>BaseRepository[T]</code> Plain CRUD \u2014 the default nothing extra <code>CachedBaseRepository[T]</code> Read\u2011heavy, low\u2011write fields (lookups by <code>_id</code>) cache\u2011first <code>get_by_id</code>, cache on <code>create</code>, invalidate on <code>update</code>/<code>delete</code>, <code>warm_cache</code> <code>PopulatingRepository[T]</code> You return related docs (FK references) denormalized <code>_populate</code> on read, <code>_depopulate</code> on write, FK\u2011guarded <code>patch</code> <code>CRUDMixin</code> Reusing CRUD inside an existing class raw CRUD against a collection you already have <p>There is no built\u2011in <code>CachedPopulatingRepository</code>. Use case 10 shows how to compose caching + population in a small subclass.</p>"},{"location":"01_overview/#read-next","title":"\u27a1\ufe0f Read Next","text":"<ul> <li>Core Components \u2014 the validated public API surface.</li> <li>Use case 01 \u2014 the minimal FastAPI app.</li> <li>Best Practices \u2014 team-wide conventions.</li> </ul>"},{"location":"02_components/","title":"Core Components","text":"<p>A validated reference to the public API. Signatures match the code in <code>mongo_ops/</code> exactly. See the use cases for runnable recipes.</p>"},{"location":"02_components/#1-mongoconnectionmanager","title":"1. MongoConnectionManager","text":"<p>Singleton-style owner of the Motor client and active database.</p> Method Signature Behavior <code>connect</code> <code>async (uri: str, db_name: str, **kwargs) -> AsyncIOMotorDatabase</code> Creates <code>AsyncIOMotorClient(uri, **kwargs)</code> and selects the database. Idempotent. <code>disconnect</code> <code>async () -> None</code> Closes the client and clears state. <code>get_database</code> <code>() -> AsyncIOMotorDatabase</code> Returns the active database; raises <code>RuntimeError(\"Database not connected. Call connect() first.\")</code>. <code>get_client</code> <code>() -> AsyncIOMotorClient</code> Returns the active client; raises <code>RuntimeError(\"Client not connected. Call connect() first.\")</code>. <code>lifespan</code> <code>async ctx manager (uri, db_name, **kwargs)</code> <code>connect()</code> on entry, yields the database, <code>disconnect()</code> on exit. Designed for FastAPI lifespan."},{"location":"02_components/#2-basedocument-pyobjectid","title":"2. BaseDocument & PyObjectId","text":"<p><code>BaseDocument(BaseModel)</code> \u2014 inherit for every Mongo entity.</p> <ul> <li><code>id: Optional[PyObjectId]</code> \u2014 aliased to <code>_id</code>, serialized to <code>str</code>.</li> <li><code>created_at: datetime</code> \u2014 defaults to <code>datetime.utcnow()</code>.</li> <li><code>updated_at: datetime</code> \u2014 defaults to <code>datetime.utcnow()</code>.</li> <li><code>Config.populate_by_name = True</code>, <code>arbitrary_types_allowed = True</code>, <code>json_encoders = {ObjectId: str}</code>.</li> </ul> <p><code>PyObjectId(ObjectId)</code> \u2014 Pydantic v2-compatible <code>ObjectId</code> that accepts <code>str</code> or <code>ObjectId</code> and validates with <code>ObjectId.is_valid</code>.</p>"},{"location":"02_components/#3-crudmixint","title":"3. CRUDMixin[T]","text":"<p>Generic CRUD over a Motor collection. The building block of all repositories.</p> Method Signature Notes <code>create</code> <code>async (data: T) -> T</code> Dumps model (excludes <code>id</code>, <code>None</code>), stamps <code>created_at</code>/<code>updated_at</code>, inserts, returns model with assigned <code>_id</code>. <code>get_by_id</code> <code>async (id: str \\| ObjectId) -> Optional[T]</code> <code>str</code> is accepted and converted to <code>ObjectId</code>. <code>get_many</code> <code>async (filter: dict \\| None = None, skip: int = 0, limit: int = 100, sort: list[tuple] \\| None = None) -> list[T]</code> Cursor <code>.skip().limit().sort(...)</code> then <code>to_list(limit)</code>. <code>limit=0</code> disables the limit clause. <code>update</code> <code>async (id, data: dict[str, Any]) -> Optional[T]</code> <code>$set</code> + refreshed <code>updated_at</code> via <code>find_one_and_update</code>. <code>patch</code> <code>async (id, data: dict[str, Any]) -> Optional[T]</code> Same as <code>update</code> but intended for REST PATCH semantics. <code>delete</code> <code>async (id) -> bool</code> True if a document was deleted. <code>count</code> <code>async (filter: dict \\| None = None) -> int</code> <code>count_documents</code>."},{"location":"02_components/#4-baserepositoryt","title":"4. BaseRepository[T]","text":"<p><code>BaseRepository(collection_name: str, model: type[T])</code> \u2014 resolves the collection from <code>MongoConnectionManager.get_database()[collection_name]</code>. Requires an active connection at construction time. Provides everything in <code>CRUDMixin</code> plus <code>collection_name</code>.</p>"},{"location":"02_components/#5-populatingrepositoryt","title":"5. PopulatingRepository[T]","text":"<p><code>PopulatingRepository(collection_name, model, population_engine: PopulationEngine | None = None, populate_rules: list[PopulateRule] | None = None)</code>.</p> <ul> <li><code>set_population_engine(engine)</code> / <code>set_populate_rules(rules)</code> \u2014 swap engine/rules at runtime.</li> <li><code>data_to_model</code> hooks <code>_populate</code> \u2014 <code>get_by_id</code>/<code>get_many</code> return fully populated models.</li> <li><code>_depopulate(document)</code> \u2014 collapses populated FK fields back to <code>ObjectId</code> before <code>create</code>/<code>update</code>.</li> <li><code>create</code> / <code>update</code> accept a model <code>T</code> (not a dict) so depopulation can run.</li> <li><code>patch</code> blocks FK fields \u2014 raises <code>ValueError(\"Cannot patch FK fields via patch(): ... Use update() to change FK fields.\")</code>.</li> </ul> <p>Populate semantics (important): a <code>PopulateRule</code> names a field that holds either an <code>ObjectId</code> or a <code>list[ObjectId]</code> and is the same field that gets replaced with the resolved document(s). There is no separate \"ref field\" vs \"target field\". See use case 08.</p>"},{"location":"02_components/#6-transactionmanager","title":"6. TransactionManager","text":"Method Signature Behavior <code>start_session</code> <code>async ctx manager (**kwargs) -> AsyncIOMotorClientSession</code> Yields a session with an active transaction. Pass <code>session=</code> to every collection call inside. <code>execute_transaction</code> <code>async (operations: list[Callable[[session], Awaitable[Any]]], **kwargs) -> list[Any]</code> Runs each op inside one transaction and returns results in order; any exception aborts the transaction and propagates."},{"location":"02_components/#7-modelregistry","title":"7. ModelRegistry","text":"<p>Centralized models, indexes, and cache lifecycle for multi-collection services.</p> Method Signature Behavior <code>register</code> <code>(collection_name: str, model: type[BaseDocument], indexes: list[Any] \\| None = None)</code> Records model + index specs. Index specs are passed as-is to pymongo <code>create_index</code> \u2014 single tuples, compound lists, or dicts with <code>keys</code>/<code>options</code>. <code>initialize_all</code> <code>async (db: AsyncIOMotorDatabase \\| None = None) -> None</code> <code>create_index</code> per registered spec (idempotent). Uses the manager database if <code>db</code> omitted. <code>get_model</code> <code>(collection_name) -> type[BaseDocument]</code> Raises <code>KeyError</code> if unregistered. <code>list_collections</code> <code>() -> list[str]</code> Registered collection names. <code>set_cache_backend</code> <code>(backend: CacheBackend) -> None</code> Register the single shared backend. <code>initialize_cache</code> <code>async () -> None</code> Starts the backend (background TTL cleanup) \u2014 raises <code>RuntimeError</code> if no backend registered. <code>shutdown_cache</code> <code>async () -> None</code> Stops the backend cleanly and clears it. <code>get_cache_backend</code> <code>() -> Optional[CacheBackend]</code> Current backend, if any."},{"location":"02_components/#8-cache-layer","title":"8. Cache Layer","text":""},{"location":"02_components/#81-cachebackend-abstract","title":"8.1 CacheBackend (abstract)","text":"<p>Async interface: <code>get(key) -> Optional[bytes]</code>, <code>set(key, value: bytes, ttl | None)</code>, <code>delete(key)</code>, <code>exists(key)</code>, <code>clear_pattern(pattern)</code>, <code>get_stats() -> CacheStats</code>, <code>initialize()</code>, <code>shutdown()</code>. Values are bytes (JSON-encoded).</p>"},{"location":"02_components/#82-cachestats","title":"8.2 CacheStats","text":"<p>Dataclass: <code>hits</code>, <code>misses</code>, <code>sets</code>, <code>deletes</code>, <code>current_size</code>, <code>max_size</code>.</p>"},{"location":"02_components/#83-cacheconfig","title":"8.3 CacheConfig","text":"<p>Dataclass: <code>enabled: bool = True</code>, <code>backend: Literal[\"memory\", \"redis\"] = \"memory\"</code>, <code>redis_client</code>, <code>default_ttl: int = 300</code>, <code>max_entries: int = 10000</code>, <code>key_prefix: str = \"\"</code>, <code>cleanup_interval: int = 60</code>. Raises <code>ValueError</code> if <code>backend=\"redis\"</code> without a client, and <code>ImportError</code> if <code>redis</code> is not installed.</p>"},{"location":"02_components/#84-inmemorycachebackend","title":"8.4 InMemoryCacheBackend","text":"<p><code>InMemoryCacheBackend(max_entries=10000, default_ttl=300, cleanup_interval=60)</code> \u2014 LRU <code>OrderedDict</code> + TTL heap; <code>initialize()</code> spawns the periodic cleanup task (TTL 0 expires immediately).</p>"},{"location":"02_components/#85-rediscachebackend","title":"8.5 RedisCacheBackend","text":"<p><code>RedisCacheBackend(redis_client, key_prefix=\"\", default_ttl=300)</code> \u2014 <code>setex</code> storage, SCAN-based <code>clear_pattern</code>, and <code>publish_invalidate(key)</code> for cross-service invalidation on <code>delete</code> via the <code>mongo_ops:cache:invalidate</code> channel.</p>"},{"location":"02_components/#86-cachedbaserepositoryt","title":"8.6 CachedBaseRepository[T]","text":"<p><code>CachedBaseRepository(collection_name, model, cache_backend: CacheBackend, config: CacheConfig | None = None)</code>.</p> <ul> <li>Cache keys are <code>\"{key_prefix}{id}\"</code> (prefix defaults to <code>\"{collection_name}:\"</code>).</li> <li><code>get_by_id</code> \u2014 cache-first; cache miss reads DB and stores <code>model_dump(by_alias=True)</code> (JSON-encoded) for <code>default_ttl</code>. Honors <code>config.enabled=False</code> (bypass).</li> <li><code>create</code> \u2014 inserts then caches the result.</li> <li><code>update</code>/<code>delete</code> \u2014 refresh or remove the cache entry.</li> <li><code>warm_cache(ids) -> int</code> \u2014 prefetch a list of IDs, returns count warmed.</li> <li><code>invalidate_cache(id)</code> \u2014 manual eviction.</li> </ul>"},{"location":"02_components/#9-population-layer","title":"9. Population Layer","text":""},{"location":"02_components/#91-populaterule","title":"9.1 PopulateRule","text":"<p>Dataclass:</p> Python<pre><code>@dataclass\nclass PopulateRule:\n field_name: str # field holding the ObjectId / list[ObjectId]; replaced in-place with the resolved doc(s)\n collection_name: str # collection the references point at\n nested_rules: list[PopulateRule] | None = None\n max_depth: int = 1\n filter: dict | None = None # DECLARED but NOT yet applied by the engine\n projection: dict | None = None # DECLARED but NOT yet applied by the engine\n</code></pre> <p>\u26a0\ufe0f <code>filter</code> and <code>projection</code> are accepted but currently ignored by <code>PopulationEngine</code> \u2014 do not rely on them yet.</p>"},{"location":"02_components/#92-populationengine","title":"9.2 PopulationEngine","text":"<p><code>PopulationEngine(repos: dict[str, Any], global_max_depth: int = 10)</code> \u2014 <code>repos</code> maps <code>collection_name</code> \u2192 repository.</p> <ul> <li><code>register_repo(collection_name, repo)</code> \u2014 add repositories at runtime.</li> <li><code>populate(document, rules, depth=0)</code> \u2014 resolves refs recursively, replacing <code>field_name</code> in place; raises <code>CircularReferenceError(collection, doc_id, path)</code> when a <code>(Class, id)</code> pair is revisited.</li> <li><code>depopulate(document, rules)</code> \u2014 collapse populated docs back to IDs (for storage).</li> <li><code>global_max_depth</code> caps recursion; per-rule <code>max_depth</code> bounds a rule's descent.</li> </ul>"},{"location":"02_components/#93-circularreferenceerrorvalueerror","title":"9.3 CircularReferenceError(ValueError)","text":"<p>Holds <code>collection</code>, <code>doc_id</code>, and the visited <code>path</code> for debugging cycle messages.</p>"},{"location":"02_components/#related","title":"Related","text":"<ul> <li>Use case 01 \u2013 Basic CRUD \u00b7 07 \u2013 Caching \u00b7 08 \u2013 Population</li> </ul>"},{"location":"04_best_practices/","title":"Best Practices","text":"<p>Team-wide conventions for building fast, testable MongoDB services with <code>mongo-ops</code>.</p>"},{"location":"04_best_practices/#layering","title":"\ud83c\udfd7\ufe0f Layering","text":"<ol> <li>One repository per collection. Encapsulate every query the domain needs behind repository methods; keep Mongo details (<code>$regex</code>, <code>$inc</code>, projections) inside the repository.</li> <li>Keep models thin. <code>BaseDocument</code> for the shape; use Pydantic <code>Field</code> constraints for validation; never put business rules in the model.</li> <li>Use services for cross-repository logic. A <code>Service</code> composes multiple repositories (and <code>TransactionManager</code>) \u2014 routes stay thin.</li> <li>Expose <code>get_many(filter=..., skip=..., limit=..., sort=...)</code> instead of raw <code>find</code> for list endpoints \u2014 you get controlled pagination for free.</li> </ol>"},{"location":"04_best_practices/#lifecycle","title":"\ud83d\udd04 Lifecycle","text":"<ol> <li>Connect once, in the lifespan. <code>MongoConnectionManager.lifespan(...)</code> (or explicit <code>connect</code>/<code>disconnect</code>) \u2014 never lazily per request.</li> <li>Construct repositories after <code>connect()</code>. Module-level <code>Repo()</code> before connection raises <code>RuntimeError(\"Database not connected...\")</code>. Use dependencies or construct inside the lifespan/request.</li> <li>Order the cache lifecycle strictly: <code>set_cache_backend(backend)</code> \u2192 <code>initialize_cache()</code> (after connect, before use) \u2192 <code>shutdown_cache()</code> on exit.</li> <li>Register all models up front via <code>ModelRegistry.register(...)</code> and let <code>initialize_all()</code> create indexes once at startup (idempotent).</li> </ol>"},{"location":"04_best_practices/#data-performance","title":"\ud83d\uddc4\ufe0f Data & Performance","text":"<ol> <li>Declare indexes for every hot query. Single-field, composite, and optioned (unique/TTL) specs all work via <code>ModelRegistry.register</code> \u2014 see use case 13.</li> <li>Cache only hot, low-write <code>_id</code> reads. Use <code>CachedBaseRepository</code> for lookups-by-id; invalidate (<code>update</code>/<code>delete</code> handle it) and pick a sensible <code>default_ttl</code>.</li> <li>Populate at the repository boundary. <code>PopulatingRepository</code> resolves refs on read and depopulates on write; do not hand-roll joins in endpoints.</li> <li>Respect the populate invariants: rules name the field that holds the reference and that is replaced; <code>patch()</code> cannot touch FK fields \u2014 use <code>update()</code>.</li> </ol>"},{"location":"04_best_practices/#transactions-errors","title":"\ud83d\udd01 Transactions & Errors","text":"<ol> <li>Use transactions for multi-document writes. <code>TransactionManager.start_session</code> (inline) or <code>execute_transaction</code> (list of ops) \u2014 and pass <code>session=</code> to every collection call inside.</li> <li>Handle the library's real exceptions at the edges: <code>DuplicateKeyError</code> \u2192 409, <code>InvalidId</code> \u2192 400, <code>CircularReferenceError</code> \u2192 409, <code>ValueError</code> guides \u2192 400/422 (see Error Handling).</li> </ol>"},{"location":"04_best_practices/#testing","title":"\ud83e\uddea Testing","text":"<ol> <li>Default to mock-based unit tests. Patch <code>MongoConnectionManager.get_database</code>, use <code>AsyncMock</code> collections and cursor chains \u2014 the whole suite runs without MongoDB (use case 14).</li> <li>Mirror the library tests. <code>tests/test_{repository,populating_repository,cache,registry,transactions}.py</code> are canonical examples of every pattern above.</li> <li>Use type hints end-to-end \u2014 mypy-gated CI (see pyproject) catches drift early.</li> </ol>"},{"location":"04_best_practices/#related","title":"Related","text":"<ul> <li>Overview \u00b7 Core Components \u00b7 Use Cases</li> </ul>"},{"location":"05_patterns/","title":"Common Patterns","text":"<p>Reusable code shapes built on <code>BaseRepository</code> and friends.</p>"},{"location":"05_patterns/#pattern-1-service-layer-with-repository","title":"Pattern 1: Service Layer with Repository","text":"<p>Keep orchestration out of routes. A service composes one or more repositories and owns the domain rules.</p> Python<pre><code>from mongo_ops import BaseRepository, BaseDocument\n\n\nclass User(BaseDocument):\n username: str = \"\"\n email: str = \"\"\n\n\nclass UserRepository(BaseRepository[User]):\n def __init__(self):\n super().__init__(\"users\", User)\n\n async def find_by_email(self, email: str) -> User | None:\n doc = await self.collection.find_one({\"email\": email})\n return self.model(**doc) if doc else None\n\n\nclass UserService:\n def __init__(self, user_repo: UserRepository):\n self.user_repo = user_repo\n\n async def register_user(self, username: str, email: str) -> User:\n if await self.user_repo.find_by_email(email):\n raise ValueError(\"Email already exists\")\n return await self.user_repo.create(User(username=username, email=email))\n</code></pre> <p>Register a unique index on <code>email</code> via <code>ModelRegistry</code> so the race is also caught by <code>DuplicateKeyError</code> \u2014 see use case 13.</p>"},{"location":"05_patterns/#pattern-2-aggregation-pipeline","title":"Pattern 2: Aggregation Pipeline","text":"<p>Aggregations hit the raw Motor collection \u2014 wrap them in a repository method so callers stay at the domain level.</p> Python<pre><code>from mongo_ops import BaseRepository, BaseDocument\n\n\nclass Post(BaseDocument):\n author_id: str = \"\"\n title: str = \"\"\n likes: int = 0\n\n\nclass PostRepository(BaseRepository[Post]):\n def __init__(self):\n super().__init__(\"posts\", Post)\n\n async def get_user_stats(self, user_id: str) -> dict:\n pipeline = [\n {\"$match\": {\"author_id\": user_id}},\n {\"$group\": {\"_id\": None, \"total_posts\": {\"$sum\": 1}, \"total_likes\": {\"$sum\": \"$likes\"}}},\n ]\n result = await self.collection.aggregate(pipeline).to_list(1)\n return result[0] if result else {\"total_posts\": 0, \"total_likes\": 0}\n</code></pre>"},{"location":"05_patterns/#pattern-3-bulk-operations","title":"Pattern 3: Bulk Operations","text":"<p>Multi-document mutation in one call \u2014 again wrapped at the repository boundary.</p> Python<pre><code>from datetime import datetime\n\nfrom bson import ObjectId\nfrom mongo_ops import BaseRepository, BaseDocument\n\n\nclass Task(BaseDocument):\n status: str = \"pending\"\n\n\nclass TaskRepository(BaseRepository[Task]):\n def __init__(self):\n super().__init__(\"tasks\", Task)\n\n async def bulk_update_status(self, ids: list[str], status: str) -> int:\n object_ids = [ObjectId(i) for i in ids]\n result = await self.collection.update_many(\n {\"_id\": {\"$in\": object_ids}},\n {\"$set\": {\"status\": status, \"updated_at\": datetime.utcnow()}},\n )\n return result.modified_count\n</code></pre> <p>Bulk/aggregation methods bypass the caching and population layers \u2014 keep them deliberate and documented.</p>"},{"location":"05_patterns/#pattern-4-transactional-multi-step-write","title":"Pattern 4: Transactional Multi-Step Write","text":"<p>Cross-collection atomicity with <code>execute_transaction</code> \u2014 see use case 12 for the full code.</p> Python<pre><code>results = await TransactionManager.execute_transaction([op1, op2, op3])\n# op1/op2/op3 each receive the shared session; one failure rolls back all.\n</code></pre>"},{"location":"05_patterns/#related","title":"Related","text":"<ul> <li>02 \u2013 Custom repository \u00b7 03 \u2013 Transactions \u00b7 Best Practices</li> </ul>"},{"location":"06_error_handling/","title":"Error Handling","text":"<p>What can raise, what it means, and how to map it in a FastAPI app.</p>"},{"location":"06_error_handling/#library-raised-exceptions","title":"\ud83d\udccb Library-Raised Exceptions","text":"Exception Source Meaning / fix <code>RuntimeError(\"Database not connected. Call connect() first.\")</code> <code>get_database()</code> / <code>get_client()</code> and any repository constructed first <code>MongoConnectionManager.connect()</code> hasn't run \u2014 wire the lifespan. <code>RuntimeError(\"No cache backend registered. Call set_cache_backend() first.\")</code> <code>ModelRegistry.initialize_cache()</code> Call <code>set_cache_backend(backend)</code> before <code>initialize_cache()</code>. <code>KeyError(\"Model for collection '...' not registered\")</code> <code>ModelRegistry.get_model()</code> Collection was never <code>register</code>ed (or typo). <code>ValueError(\"Cannot patch FK fields via patch(): ...\")</code> <code>PopulatingRepository.patch()</code> <code>patch</code> must not touch populated ref fields \u2014 use <code>update()</code> with a model. <code>ValueError(\"...contains embedded dict(s) \u2014 run repair script\")</code> <code>_populate</code> on read A FK field holds an embedded document instead of an <code>ObjectId</code> \u2014 migrate the data. <code>ValueError(\"...contains ObjectId \u2014 was populate skipped?\")</code> <code>_depopulate</code> on write A populate-ruled field is still an <code>ObjectId</code> at depopulate time \u2014 the read must have populated it first. <code>CircularReferenceError(collection, doc_id, path)</code> <code>PopulationEngine.populate</code> A <code>(Class, id)</code> pair was revisited \u2014 raise <code>max_depth</code> or fix the graph. <code>ImportError(\"redis package required ... mongo-ops[redis]\")</code> <code>CacheConfig</code> / <code>RedisCacheBackend</code> Missing <code>redis</code> extra. <code>ValueError(\"redis_client required when backend='redis'\")</code> <code>CacheConfig</code> <code>backend=\"redis\"</code> without a client. Pymongo <code>DuplicateKeyError</code> any insert/update Unique index violation (e.g., duplicate email). <code>bson.errors.InvalidId</code> <code>ObjectId(...)</code> on a bad string Wrapped by <code>PyObjectId</code> model validation on API inputs. <p><code>KeyError</code> for <code>ModelRegistry.get_model</code> and the <code>RuntimeError</code>/<code>ValueError</code> guards are by design \u2014 they fail loudly at startup or first call instead of misbehaving silently.</p>"},{"location":"06_error_handling/#fastapi-mapping-example","title":"\ud83d\ude80 FastAPI Mapping Example","text":"Python<pre><code>from fastapi import FastAPI, HTTPException, Request\nfrom fastapi.responses import JSONResponse\nfrom bson.errors import InvalidId\nfrom pymongo.errors import DuplicateKeyError\nfrom mongo_ops.cache import CircularReferenceError\n\n\napp = FastAPI()\n\n\n@app.exception_handler(DuplicateKeyError)\nasync def duplicate_key_handler(_: Request, __: DuplicateKeyError) -> JSONResponse:\n return JSONResponse(status_code=409, content={\"detail\": \"Resource already exists\"})\n\n\n@app.exception_handler(InvalidId)\nasync def invalid_id_handler(_: Request, __: InvalidId) -> JSONResponse:\n return JSONResponse(status_code=400, content={\"detail\": \"Invalid ID format\"})\n\n\n@app.exception_handler(CircularReferenceError)\nasync def circular_ref_handler(_: Request, exc: CircularReferenceError) -> JSONResponse:\n return JSONResponse(\n status_code=409,\n content={\"detail\": f\"Circular reference detected: {exc.path}\"},\n )\n\n\n# Or handle inline for route-specific responses:\n@app.post(\"/users/\")\nasync def create_user(user: User, repo=Depends(get_user_repository)):\n try:\n return await repo.create(user)\n except DuplicateKeyError:\n raise HTTPException(status_code=409, detail=\"User already exists\")\n except Exception as exc:\n raise HTTPException(status_code=500, detail=str(exc))\n\n\n@app.get(\"/users/{user_id}\")\nasync def get_user(user_id: str, repo=Depends(get_user_repository)):\n try:\n user = await repo.get_by_id(user_id)\n except InvalidId:\n raise HTTPException(status_code=400, detail=\"Invalid user ID\")\n if not user:\n raise HTTPException(status_code=404, detail=\"User not found\")\n return user\n</code></pre> <p>Prefer exception handlers for the library-level exceptions (409/400 above) and per-endpoint <code>try/except</code> for domain decisions (404).</p>"},{"location":"06_error_handling/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>The <code>ValueError</code> populate guards are your friends: they surface data-shape drift (embedded docs, skipped population) at the exact call site.</li> <li>In dev, log the <code>CircularReferenceError.path</code> \u2014 it prints the visited <code>Class:id</code> chain.</li> <li>Never swallow <code>RuntimeError</code>s at startup; let the app fail to load so the misconfiguration is obvious.</li> </ul>"},{"location":"06_error_handling/#related","title":"Related","text":"<ul> <li>03 \u2013 Transactions \u00b7 08 \u2013 Population \u00b7 11 \u2013 Cache lifecycle</li> </ul>"},{"location":"07_testing_example/","title":"Testing Example","text":"<p>A zero-MongoDB unit-test quickstart. The full guide is use case 14.</p>"},{"location":"07_testing_example/#mock-based-quickstart","title":"\ud83d\ude80 Mock-Based Quickstart","text":"<p>Patch the connection manager and stub the collection \u2014 no network, no docker.</p> Python<pre><code>import pytest\nfrom unittest.mock import AsyncMock, patch\n\nfrom bson import ObjectId\nfrom mongo_ops import BaseDocument, BaseRepository\nfrom mongo_ops.repository import MongoConnectionManager\n\n\nclass User(BaseDocument):\n username: str = \"\"\n email: str = \"\"\n\n\nclass UserRepository(BaseRepository[User]):\n def __init__(self):\n super().__init__(\"users\", User)\n\n\n@pytest.fixture\ndef repo():\n mock_collection = AsyncMock()\n with patch.object(MongoConnectionManager, \"get_database\") as mock_db:\n mock_db.return_value.__getitem__.return_value = mock_collection\n r = UserRepository()\n r.collection = mock_collection\n return r\n\n\n@pytest.mark.asyncio\nasync def test_create_user(repo):\n mock_collection = repo.collection\n oid = ObjectId()\n mock_collection.insert_one.return_value.inserted_id = oid\n\n user = await repo.create(User(username=\"test\", email=\"test@example.com\"))\n\n assert user.id == oid\n assert user.username == \"test\"\n\n\n@pytest.mark.asyncio\nasync def test_get_by_id(repo):\n oid = ObjectId()\n repo.collection.find_one.return_value = {\n \"_id\": oid,\n \"username\": \"test\",\n \"email\": \"test@example.com\",\n \"created_at\": \"2024-01-01T00:00:00\",\n \"updated_at\": \"2024-01-01T00:00:00\",\n }\n\n fetched = await repo.get_by_id(str(oid))\n\n assert fetched is not None\n assert fetched.username == \"test\"\n</code></pre>"},{"location":"07_testing_example/#notes","title":"\ud83d\udca1 Notes","text":"<ul> <li><code>from mongo_ops.repository import MongoConnectionManager</code> \u2014 patch where it is used (<code>mongo_ops.repository.MongoConnectionManager</code>), matching the library's own tests.</li> <li>pytest-asyncio runs as <code>auto</code> mode per <code>pyproject.toml</code>, so <code>@pytest.mark.asyncio</code> works without extra config.</li> <li>For population, cache, registry, and transaction mockups \u2014 see the full testing guide.</li> </ul>"},{"location":"07_testing_example/#related","title":"Related","text":"<ul> <li>14 \u2013 Testing guide \u00b7 Best Practices</li> </ul>"},{"location":"03_use_cases/01_basic_crud/","title":"Use Case 1: Basic FastAPI CRUD API","text":"<p>Scenario: A simple user management API with CRUD endpoints, index registration, and a correct connection lifecycle.</p>"},{"location":"03_use_cases/01_basic_crud/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>MongoConnectionManager.lifespan</code> Connect on startup, disconnect on shutdown. <code>BaseRepository[User]</code> Generic CRUD \u2014 no endpoint-level Mongo code. <code>ModelRegistry.register</code> Declares the <code>email</code> index; <code>initialize_all</code> creates it at startup."},{"location":"03_use_cases/01_basic_crud/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>import os\nfrom contextlib import asynccontextmanager\n\nfrom fastapi import Depends, FastAPI, HTTPException\nfrom pydantic import Field\n\nfrom mongo_ops import BaseDocument, BaseRepository, ModelRegistry, MongoConnectionManager\n\n# ---------------------------\n# Model\n# ---------------------------\nclass User(BaseDocument):\n username: str = Field(..., min_length=3, max_length=50)\n email: str = Field(...)\n is_active: bool = True\n\n\n# ---------------------------\n# Repository\n# ---------------------------\nclass UserRepository(BaseRepository[User]):\n def __init__(self):\n super().__init__(\"users\", User)\n\n\n# ---------------------------\n# Registration + index\n# ---------------------------\nModelRegistry.register(\"users\", User, indexes=[(\"email\", 1)])\n\n\n# ---------------------------\n# Lifecycle\n# ---------------------------\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=os.getenv(\"MONGO_URI\", \"mongodb://localhost:27017\"),\n db_name=\"mydb\",\n ):\n await ModelRegistry.initialize_all()\n yield\n\n\napp = FastAPI(lifespan=lifespan)\n\n\n# ---------------------------\n# Dependency \u2014 created per request, after connect()\n# ---------------------------\ndef get_user_repository() -> UserRepository:\n return UserRepository()\n\n\n# ---------------------------\n# Routes\n# ---------------------------\n@app.post(\"/users/\", response_model=User)\nasync def create_user(user: User, repo: UserRepository = Depends(get_user_repository)):\n return await repo.create(user)\n\n\n@app.get(\"/users/{user_id}\", response_model=User)\nasync def get_user(user_id: str, repo: UserRepository = Depends(get_user_repository)):\n user = await repo.get_by_id(user_id)\n if not user:\n raise HTTPException(status_code=404, detail=\"User not found\")\n return user\n\n\n@app.get(\"/users/\", response_model=list[User])\nasync def list_users(\n skip: int = 0,\n limit: int = 10,\n repo: UserRepository = Depends(get_user_repository),\n):\n return await repo.get_many(skip=skip, limit=limit)\n\n\n@app.put(\"/users/{user_id}\", response_model=User)\nasync def update_user(\n user_id: str,\n email: str,\n repo: UserRepository = Depends(get_user_repository),\n):\n user = await repo.update(user_id, {\"email\": email})\n if not user:\n raise HTTPException(status_code=404, detail=\"User not found\")\n return user\n\n\n@app.delete(\"/users/{user_id}\")\nasync def delete_user(user_id: str, repo: UserRepository = Depends(get_user_repository)):\n deleted = await repo.delete(user_id)\n if not deleted:\n raise HTTPException(status_code=404, detail=\"User not found\")\n return {\"message\": \"User deleted successfully\"}\n</code></pre>"},{"location":"03_use_cases/01_basic_crud/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li><code>UserRepository()</code> is created inside the dependency, i.e., only after the lifespan has connected. Instantiating a repository at module import time raises <code>RuntimeError</code> because the database isn't connected yet.</li> <li><code>BaseRepository.update(id, {...})</code> takes a dict; only <code>PopulatingRepository.update</code> takes a model.</li> <li>Use <code>patch()</code> for REST PATCH semantics \u2014 it accepts a partial dict like <code>update</code>, but <code>PopulatingRepository</code> rejects FK fields.</li> </ul>"},{"location":"03_use_cases/01_basic_crud/#related","title":"Related","text":"<ul> <li>02 \u2013 Custom repository \u00b7 04 \u2013 Pagination \u00b7 Overview</li> </ul>"},{"location":"03_use_cases/02_custom_repo/","title":"Use Case 2: Custom Repository with Business Logic","text":"<p>Scenario: An e-commerce product catalog needs search, filtering, and stock updates without Mongo leaking into routes.</p>"},{"location":"03_use_cases/02_custom_repo/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description Repository methods Encapsulate queries (<code>$regex</code>, filters, <code>$inc</code>) behind domain methods. <code>get_many</code> Filtering + default pagination via the base repository. Direct <code>collection</code> access For operations with no base-repo helper (regex search, atomic <code>$inc</code>)."},{"location":"03_use_cases/02_custom_repo/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from fastapi import FastAPI, HTTPException, Query\nfrom mongo_ops import BaseDocument, BaseRepository\n\n# ---------------------------\n# Model\n# ---------------------------\nclass Product(BaseDocument):\n name: str\n description: str = \"\"\n price: float = 0.0\n category: str = \"\"\n in_stock: bool = True\n quantity: int = 0\n tags: list[str] = []\n\n\n# ---------------------------\n# Repository\n# ---------------------------\nclass ProductRepository(BaseRepository[Product]):\n def __init__(self):\n super().__init__(\"products\", Product)\n\n async def search_by_name(self, query: str) -> list[Product]:\n \"\"\"Case-insensitive name search.\"\"\"\n docs = await self.collection.find(\n {\"name\": {\"$regex\": query, \"$options\": \"i\"}}\n ).to_list(length=100)\n return [self.model(**doc) for doc in docs]\n\n async def get_by_category(self, category: str, in_stock_only: bool = True) -> list[Product]:\n filter_query = {\"category\": category}\n if in_stock_only:\n filter_query[\"in_stock\"] = True\n return await self.get_many(filter=filter_query)\n\n async def get_low_stock(self, threshold: int = 10) -> list[Product]:\n return await self.get_many(filter={\"quantity\": {\"$lt\": threshold}, \"in_stock\": True})\n\n async def update_stock(self, product_id: str, quantity_delta: int) -> Product | None:\n \"\"\"Atomically increment/decrement stock.\"\"\"\n from bson import ObjectId\n from datetime import datetime\n\n result = await self.collection.find_one_and_update(\n {\"_id\": ObjectId(product_id)},\n {\"$inc\": {\"quantity\": quantity_delta}, \"$set\": {\"updated_at\": datetime.utcnow()}},\n return_document=True,\n )\n return self.model(**result) if result else None\n\n\napp = FastAPI()\nproduct_repo = ProductRepository()\n\n\n@app.get(\"/products/search\", response_model=list[Product])\nasync def search_products(q: str = Query(..., min_length=1)):\n return await product_repo.search_by_name(q)\n\n\n@app.get(\"/products/category/{category}\", response_model=list[Product])\nasync def products_by_category(category: str, in_stock: bool = True):\n return await product_repo.get_by_category(category, in_stock)\n\n\n@app.get(\"/products/low-stock\", response_model=list[Product])\nasync def low_stock_products(threshold: int = 10):\n return await product_repo.get_low_stock(threshold)\n\n\n@app.patch(\"/products/{product_id}/stock\")\nasync def update_product_stock(product_id: str, quantity_delta: int):\n product = await product_repo.update_stock(product_id, quantity_delta)\n if not product:\n raise HTTPException(status_code=404, detail=\"Product not found\")\n return product\n</code></pre> <p>Note: this snippet omits the FastAPI <code>lifespan</code> connection wiring for brevity \u2014 copy it from use case 01 so <code>ProductRepository()</code> is created only after <code>MongoConnectionManager.connect()</code>.</p>"},{"location":"03_use_cases/02_custom_repo/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>Methods that hit <code>self.collection</code> directly (regex search, <code>$inc</code>) bypass the caching and population layers. If a feature composes them \u2014 extend <code>CachedBaseRepository</code> or <code>PopulatingRepository</code> instead and add the domain methods there.</li> <li>Prefer <code>get_many(filter=...)</code> over raw <code>find</code> when you want pagination/sort defaults for free.</li> <li>Reuse <code>self.model(**doc)</code> to convert raw dicts to model instances consistently.</li> </ul>"},{"location":"03_use_cases/02_custom_repo/#related","title":"Related","text":"<ul> <li>01 \u2013 Basic CRUD \u00b7 04 \u2013 Pagination \u00b7 05 \u2013 Soft deletes</li> </ul>"},{"location":"03_use_cases/03_transactions/","title":"Use Case 3: Transaction Support for Multi-Document Operations","text":"<p>Scenario: Order processing must update inventory and create an order atomically. Any failure rolls both back.</p>"},{"location":"03_use_cases/03_transactions/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>TransactionManager.start_session</code> Async context manager yielding a session with an active transaction. <code>session=</code> kwarg Pass to every <code>insert_one</code> / <code>update_one</code> / <code>find_one</code> inside the block."},{"location":"03_use_cases/03_transactions/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\nfrom datetime import datetime\n\nfrom bson import ObjectId\nfrom fastapi import FastAPI, HTTPException\nfrom mongo_ops import BaseDocument, BaseRepository, MongoConnectionManager, TransactionManager\n\n\nclass Order(BaseDocument):\n user_id: str\n items: list[dict] # [{\"product_id\": \"...\", \"quantity\": 2}]\n total_amount: float = 0.0\n status: str = \"pending\"\n\n\nclass Inventory(BaseDocument):\n product_id: ObjectId = None\n quantity: int = 0\n\n\nclass OrderRepository(BaseRepository[Order]):\n def __init__(self):\n super().__init__(\"orders\", Order)\n\n\nclass InventoryRepository(BaseRepository[Inventory]):\n def __init__(self):\n super().__init__(\"inventory\", Inventory)\n\n\nasync def create_order_with_inventory_update(\n order: Order,\n order_repo: OrderRepository,\n inv_repo: InventoryRepository,\n) -> Order:\n async def insert_order(session):\n doc = order.model_dump(exclude={\"id\"}, exclude_none=True)\n doc[\"created_at\"] = doc[\"updated_at\"] = datetime.utcnow()\n result = await order_repo.collection.insert_one(doc, session=session)\n return await order_repo.collection.find_one({\"_id\": result.inserted_id}, session=session)\n\n async def update_inventory(session):\n for item in order.items:\n await inv_repo.collection.update_one(\n {\"product_id\": ObjectId(item[\"product_id\"])},\n {\n \"$inc\": {\"quantity\": -item[\"quantity\"]},\n \"$set\": {\"updated_at\": datetime.utcnow()},\n },\n session=session,\n )\n return None\n\n async with TransactionManager.start_session() as session:\n created = await insert_order(session)\n await update_inventory(session)\n return Order(**created)\n\n\n# ---------------------------\n# FastAPI wiring\n# ---------------------------\napp = FastAPI()\n\norder_repo = OrderRepository()\ninv_repo = InventoryRepository()\n\n\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\", db_name=\"shop\"\n ):\n yield\n\n\napp.lifespan = lifespan\n\n\n@app.post(\"/orders/\", response_model=Order)\nasync def create_order(order: Order):\n try:\n return await create_order_with_inventory_update(order, order_repo, inv_repo)\n except Exception as exc:\n raise HTTPException(status_code=400, detail=str(exc))\n</code></pre>"},{"location":"03_use_cases/03_transactions/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>Repositories are created inside the lifespan (or a dependency) so the database is connected \u2014 see the note in use case 01.</li> <li>Every operation inside the transaction context must receive <code>session=<session></code> \u2014 a missing session silently runs outside the transaction.</li> <li>MongoDB transactions require a replica set (or the local standalone test server that emulates one).</li> <li>For a list-of-operations style, see <code>TransactionManager.execute_transaction</code> in use case 12.</li> </ul>"},{"location":"03_use_cases/03_transactions/#related","title":"Related","text":"<ul> <li>06 \u2013 Multi-model \u00b7 12 \u2013 Transaction helper \u00b7 Error Handling</li> </ul>"},{"location":"03_use_cases/04_pagination/","title":"Use Case 4: Pagination & Filtering","text":"<p>Scenario: A blog post API lists posts with page metadata, filtering, and sorting.</p>"},{"location":"03_use_cases/04_pagination/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>BaseRepository.get_many</code> <code>filter</code>, <code>skip</code>, <code>limit</code>, <code>sort</code> in one call. <code>BaseRepository.count</code> Total matching documents for page metadata. <code>PaginatedResponse</code> Reusable generic response wrapper."},{"location":"03_use_cases/04_pagination/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from typing import Generic, TypeVar\n\nfrom fastapi import FastAPI, Query\nfrom pydantic import BaseModel\nfrom mongo_ops import BaseDocument, BaseRepository\n\nT = TypeVar(\"T\")\n\n\nclass PaginatedResponse(BaseModel, Generic[T]):\n items: list[T]\n total: int\n page: int\n page_size: int\n total_pages: int\n has_next: bool\n has_prev: bool\n\n\nclass BlogPost(BaseDocument):\n title: str = \"\"\n content: str = \"\"\n author_id: str = \"\"\n published: bool = False\n tags: list[str] = []\n views: int = 0\n\n\nclass BlogPostRepository(BaseRepository[BlogPost]):\n def __init__(self):\n super().__init__(\"blog_posts\", BlogPost)\n\n async def paginate(\n self,\n page: int = 1,\n page_size: int = 10,\n filter_dict: dict | None = None,\n sort_by: str = \"created_at\",\n sort_order: int = -1,\n ) -> PaginatedResponse[BlogPost]:\n filter_dict = filter_dict or {}\n skip = (page - 1) * page_size\n total = await self.count(filter_dict)\n items = await self.get_many(\n filter=filter_dict,\n skip=skip,\n limit=page_size,\n sort=[(sort_by, sort_order)],\n )\n total_pages = (total + page_size - 1) // page_size\n return PaginatedResponse(\n items=items,\n total=total,\n page=page,\n page_size=page_size,\n total_pages=total_pages,\n has_next=page < total_pages,\n has_prev=page > 1,\n )\n\n async def get_by_author(self, author_id: str, published_only: bool = True) -> list[BlogPost]:\n filter_dict = {\"author_id\": author_id}\n if published_only:\n filter_dict[\"published\"] = True\n return await self.get_many(filter=filter_dict, sort=[(\"created_at\", -1)])\n\n async def search_by_tags(self, tags: list[str]) -> list[BlogPost]:\n return await self.get_many(filter={\"tags\": {\"$in\": tags}, \"published\": True})\n\n\napp = FastAPI()\nblog_repo = BlogPostRepository()\n\n\n@app.get(\"/posts/\", response_model=PaginatedResponse[BlogPost])\nasync def list_posts(\n page: int = 1,\n page_size: int = 10,\n published: bool | None = None,\n author_id: str | None = None,\n):\n filter_dict = {}\n if published is not None:\n filter_dict[\"published\"] = published\n if author_id:\n filter_dict[\"author_id\"] = author_id\n return await blog_repo.paginate(page, page_size, filter_dict)\n\n\n@app.get(\"/posts/author/{author_id}\", response_model=list[BlogPost])\nasync def posts_by_author(author_id: str, published: bool = True):\n return await blog_repo.get_by_author(author_id, published)\n\n\n@app.get(\"/posts/tags\", response_model=list[BlogPost])\nasync def posts_by_tags(tags: list[str] = Query(...)):\n return await blog_repo.search_by_tags(tags)\n</code></pre> <p>Note: add the <code>lifespan</code> wiring from use case 01 so <code>blog_repo</code> is created after connection.</p>"},{"location":"03_use_cases/04_pagination/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li><code>get_many</code> calls <code>cursor.to_list(limit)</code>; pass <code>limit=0</code> to skip the limit entirely, otherwise a large explicit limit is safer than unbounded reads.</li> <li>Always <code>count</code> first for stable metadata \u2014 it uses the same filter as the page query.</li> <li>Combine with <code>sort=[(field, -1|1)]</code> for deterministic ordering; create matching indexes to avoid full collection scans (see use case 13).</li> </ul>"},{"location":"03_use_cases/04_pagination/#related","title":"Related","text":"<ul> <li>02 \u2013 Custom repository \u00b7 13 \u2013 Indexes \u00b7 05 \u2013 Soft deletes</li> </ul>"},{"location":"03_use_cases/05_soft_deletes/","title":"Use Case 5: Soft Deletes Pattern","text":"<p>Scenario: Deleting a task should be recoverable. Instead of removing the document, set a tombstone flag and filter it from normal queries.</p>"},{"location":"03_use_cases/05_soft_deletes/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>SoftDeleteDocument</code> Base model carrying <code>is_deleted</code>, <code>deleted_at</code>, <code>deleted_by</code>. <code>SoftDeleteRepository[T]</code> Repository-level soft delete / restore / filtering using <code>update</code> + <code>get_many</code>. <code>permanent_delete</code> Escapes to the real <code>delete</code>."},{"location":"03_use_cases/05_soft_deletes/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from datetime import datetime\n\nfrom fastapi import FastAPI, HTTPException\nfrom mongo_ops import BaseDocument, BaseRepository\n\n\nclass SoftDeleteDocument(BaseDocument):\n is_deleted: bool = False\n deleted_at: datetime | None = None\n deleted_by: str | None = None\n\n\nclass Task(SoftDeleteDocument):\n title: str = \"\"\n description: str = \"\"\n assignee_id: str = \"\"\n status: str = \"pending\"\n priority: str = \"medium\"\n\n\nclass SoftDeleteRepository(BaseRepository[T]):\n async def soft_delete(self, id: str, deleted_by: str | None = None):\n return await self.update(\n id,\n {\"is_deleted\": True, \"deleted_at\": datetime.utcnow(), \"deleted_by\": deleted_by},\n )\n\n async def restore(self, id: str):\n return await self.update(\n id, {\"is_deleted\": False, \"deleted_at\": None, \"deleted_by\": None}\n )\n\n async def get_active(self, skip: int = 0, limit: int = 100):\n return await self.get_many(filter={\"is_deleted\": False}, skip=skip, limit=limit)\n\n async def get_deleted(self, skip: int = 0, limit: int = 100):\n return await self.get_many(filter={\"is_deleted\": True}, skip=skip, limit=limit)\n\n async def permanent_delete(self, id: str) -> bool:\n return await self.delete(id)\n\n\nclass TaskRepository(SoftDeleteRepository[Task]):\n def __init__(self):\n super().__init__(\"tasks\", Task)\n\n\napp = FastAPI()\ntask_repo = TaskRepository()\n\n\n@app.delete(\"/tasks/{task_id}\")\nasync def soft_delete_task(task_id: str, user_id: str):\n task = await task_repo.soft_delete(task_id, deleted_by=user_id)\n if not task:\n raise HTTPException(status_code=404, detail=\"Task not found\")\n return {\"message\": \"Task deleted\", \"task\": task}\n\n\n@app.post(\"/tasks/{task_id}/restore\")\nasync def restore_task(task_id: str):\n task = await task_repo.restore(task_id)\n if not task:\n raise HTTPException(status_code=404, detail=\"Task not found\")\n return {\"message\": \"Task restored\", \"task\": task}\n\n\n@app.get(\"/tasks/\", response_model=list[Task])\nasync def list_active_tasks(skip: int = 0, limit: int = 10):\n return await task_repo.get_active(skip, limit)\n\n\n@app.get(\"/tasks/deleted\", response_model=list[Task])\nasync def list_deleted_tasks(skip: int = 0, limit: int = 10):\n return await task_repo.get_deleted(skip, limit)\n</code></pre> <p>Note: add the <code>lifespan</code> wiring from use case 01 so <code>task_repo</code> is created after connection.</p>"},{"location":"03_use_cases/05_soft_deletes/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li><code>BaseRepository[T]</code> is generic \u2014 subclasses must parameterize it (see <code>SoftDeleteRepository[T]</code> above). <code>from mongo_ops import BaseRepository; BaseRepository[T]</code> works out of the box.</li> <li>Keep a compound index on <code>{\"is_deleted\": 1, \"status\": 1}</code> for active-list queries (see use case 13).</li> <li>Soft-deleted documents should be excluded at the repository boundary, never re-filtered ad hoc in endpoints.</li> </ul>"},{"location":"03_use_cases/05_soft_deletes/#related","title":"Related","text":"<ul> <li>01 \u2013 Basic CRUD \u00b7 13 \u2013 Indexes \u00b7 04 \u2013 Pagination</li> </ul>"},{"location":"03_use_cases/06_multi_model/","title":"Use Case 6: Multi-Model Service with Registration","text":"<p>Scenario: A social app manages <code>users</code>, <code>posts</code>, and <code>comments</code>. Each has its own model, repository, and indexes \u2014 registered centrally and initialized at startup.</p>"},{"location":"03_use_cases/06_multi_model/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>ModelRegistry.register</code> One call per collection \u2014 model + indexes together. <code>ModelRegistry.initialize_all</code> Creates every registered index at startup (idempotent). Repositories One repository class per collection, all sharing the same connection."},{"location":"03_use_cases/06_multi_model/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\n\nfrom fastapi import FastAPI\nfrom mongo_ops import BaseDocument, BaseRepository, ModelRegistry, MongoConnectionManager\n\n\nclass User(BaseDocument):\n username: str = \"\"\n email: str = \"\"\n role: str = \"user\"\n\n\nclass Post(BaseDocument):\n title: str = \"\"\n content: str = \"\"\n author_id: str = \"\"\n likes: int = 0\n\n\nclass Comment(BaseDocument):\n post_id: str = \"\"\n user_id: str = \"\"\n content: str = \"\"\n\n\nclass UserRepository(BaseRepository[User]):\n def __init__(self):\n super().__init__(\"users\", User)\n\n\nclass PostRepository(BaseRepository[Post]):\n def __init__(self):\n super().__init__(\"posts\", Post)\n\n\nclass CommentRepository(BaseRepository[Comment]):\n def __init__(self):\n super().__init__(\"comments\", Comment)\n\n\n# ---------------------------\n# Central registration + indexes\n# ---------------------------\nModelRegistry.register(\"users\", User, indexes=[(\"email\", 1), (\"username\", 1)])\nModelRegistry.register(\"posts\", Post, indexes=[(\"author_id\", 1), (\"created_at\", -1)])\nModelRegistry.register(\"comments\", Comment, indexes=[(\"post_id\", 1), (\"user_id\", 1)])\n\n\n# ---------------------------\n# Lifecycle\n# ---------------------------\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\", db_name=\"social_app\"\n ):\n await ModelRegistry.initialize_all()\n yield\n\n\napp = FastAPI(lifespan=lifespan)\n\n\n# Created after connect() \u2014 inside the lifespan body is fine, or use a dependency.\nuser_repo = UserRepository()\npost_repo = PostRepository()\ncomment_repo = CommentRepository()\n\n\n@app.post(\"/users/\", response_model=User)\nasync def create_user(user: User):\n return await user_repo.create(user)\n\n\n@app.post(\"/posts/\", response_model=Post)\nasync def create_post(post: Post):\n return await post_repo.create(post)\n\n\n@app.post(\"/comments/\", response_model=Comment)\nasync def create_comment(comment: Comment):\n return await comment_repo.create(comment)\n</code></pre>"},{"location":"03_use_cases/06_multi_model/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>Place <code>user_repo = ...</code> inside the lifespan/after connect. Module-level instantiation before <code>connect()</code> raises <code>RuntimeError(\"Database not connected...\")</code>.</li> <li>Model relationships here are plain <code>ObjectId</code> strings stored on the child docs. To resolve them on read, see use case 08 \u2013 Population.</li> <li>Registering indexes on <code>created_at</code>/<code>author_id</code>/<code>post_id</code> keeps the common queries indexed (see use case 13).</li> </ul>"},{"location":"03_use_cases/06_multi_model/#related","title":"Related","text":"<ul> <li>01 \u2013 Basic CRUD \u00b7 08 \u2013 Population \u00b7 13 \u2013 Indexes</li> </ul>"},{"location":"03_use_cases/07_caching/","title":"Use Case 7: Caching for High-Performance Reads","text":"<p>Scenario: A read-heavy API (product catalog) reduces DB load by caching documents by <code>_id</code> \u2014 in-memory locally, or shared via Redis.</p>"},{"location":"03_use_cases/07_caching/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>InMemoryCacheBackend</code> TTL + LRU cache with a background cleanup task. <code>RedisCacheBackend</code> Distributed cache on <code>redis.asyncio</code> with pub/sub invalidation. <code>CachedBaseRepository[T]</code> Extends <code>BaseRepository</code> \u2014 cache-first <code>get_by_id</code>, cache on <code>create</code>, invalidate on <code>update</code>/<code>delete</code>, <code>warm_cache(ids)</code>, <code>invalidate_cache(id)</code>. Backend lifecycle The same backend instance must be both passed to the repository AND registered via <code>ModelRegistry.set_cache_backend</code> so <code>initialize_cache()</code> starts its task."},{"location":"03_use_cases/07_caching/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\n\nfrom fastapi import FastAPI, HTTPException\nfrom mongo_ops import BaseDocument, ModelRegistry, MongoConnectionManager\nfrom mongo_ops.cache import CacheConfig, CachedBaseRepository, InMemoryCacheBackend\n\n\nclass Product(BaseDocument):\n name: str = \"\"\n price: float = 0.0\n\n\n# One shared backend \u2014 used by both the repository and the registry lifecycle.\ncache = InMemoryCacheBackend(max_entries=10_000, default_ttl=300)\nModelRegistry.set_cache_backend(cache)\n\n\nclass ProductRepo(CachedBaseRepository[Product]):\n def __init__(self):\n super().__init__(\n collection_name=\"products\",\n model=Product,\n cache_backend=cache,\n config=CacheConfig(enabled=True, backend=\"memory\"),\n )\n\n\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\", db_name=\"shop\"\n ):\n await ModelRegistry.initialize_all()\n await ModelRegistry.initialize_cache() # starts the TTL cleanup task\n yield\n await ModelRegistry.shutdown_cache() # cancels it on exit\n\n\napp = FastAPI(lifespan=lifespan)\n\n\n@app.post(\"/products/\", response_model=Product)\nasync def create_product(product: Product):\n return await ProductRepo().create(product) # created after connect()\n\n\n@app.get(\"/products/{product_id}\", response_model=Product)\nasync def get_product(product_id: str):\n product = await ProductRepo().get_by_id(product_id)\n if not product:\n raise HTTPException(status_code=404, detail=\"Product not found\")\n return product\n\n\n@app.put(\"/products/{product_id}\", response_model=Product)\nasync def update_product(product_id: str, name: str | None = None, price: float | None = None):\n data = {}\n if name is not None:\n data[\"name\"] = name\n if price is not None:\n data[\"price\"] = price\n return await ProductRepo().update(product_id, data)\n\n\n# Outside request handlers:\n# await ProductRepo().warm_cache([object_id_1, object_id_2]) # pre-load n ids -> int\n# await ProductRepo().invalidate_cache(object_id_3) # manual eviction\n</code></pre>"},{"location":"03_use_cases/07_caching/#redis-backend","title":"\ud83d\udd04 Redis Backend","text":"<p>Swap the backend \u2014 the repository code stays identical:</p> Python<pre><code>from mongo_ops.cache import RedisCacheBackend\nfrom redis.asyncio import Redis\n\nredis_client = Redis(host=\"localhost\", port=6379)\nredis_backend = RedisCacheBackend(redis_client, key_prefix=\"prod:\")\n\nModelRegistry.set_cache_backend(redis_backend) # for the lifecycle in lifespan()\n\nclass ProductRepo(CachedBaseRepository[Product]):\n def __init__(self):\n super().__init__(\n collection_name=\"products\",\n model=Product,\n cache_backend=redis_backend,\n config=CacheConfig(enabled=True, backend=\"redis\"),\n )\n</code></pre>"},{"location":"03_use_cases/07_caching/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>Cache keys are <code>\"{key_prefix}{id}\"</code> (default prefix <code>\"products:\"</code>). <code>clear_pattern(\"products:*\")</code> wipes a whole collection's entries.</li> <li>Values are JSON-encoded (<code>json.dumps(obj, default=str)</code>) \u2014 nested models inside a cached document are stored as dicts, not objects.</li> <li>Enable/disable per repository with <code>CacheConfig(enabled=False)</code>; a disabled repo bypasses the cache entirely.</li> <li>Distinguish the two <code>initialize*</code> calls: <code>initialize_cache()</code> starts the backend task; <code>initialize_all()</code> creates indexes. Both belong in the lifespan, after <code>connect()</code>.</li> </ul>"},{"location":"03_use_cases/07_caching/#related","title":"Related","text":"<ul> <li>11 \u2013 Cache lifecycle \u00b7 10 \u2013 Cache + population \u00b7 Components: cache</li> </ul>"},{"location":"03_use_cases/08_population/","title":"Use Case 8: Document Population","text":"<p>Scenario: An API returns a <code>User</code> with its related <code>Profile</code> embedded in one JSON payload \u2014 no second round-trip from the client, no joins.</p>"},{"location":"03_use_cases/08_population/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>PopulateRule</code> Declares which field to resolve. <code>field_name</code> holds the <code>ObjectId</code> (or <code>list[ObjectId]</code>) and is the same field the resolved document replaces in place. <code>collection_name</code> says where the referenced documents live. <code>PopulationEngine</code> Holds repositories by collection name and resolves rules recursively, detecting cycles (<code>CircularReferenceError</code>). <code>PopulatingRepository[T]</code> <code>get_by_id</code>/<code>get_many</code> populate on read; <code>create</code>/<code>update</code> depopulate on write; <code>patch</code> rejects FK fields. <p>\u26a0\ufe0f <code>PopulateRule</code> does not have a separate \"ref field\" vs \"target field\" \u2014 the ref field is the populated field. <code>filter</code>/<code>projection</code> on <code>PopulateRule</code> are declared but not yet applied by the engine.</p>"},{"location":"03_use_cases/08_population/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\n\nfrom fastapi import FastAPI, HTTPException\nfrom mongo_ops import BaseDocument, MongoConnectionManager, ModelRegistry, PopulatingRepository\nfrom mongo_ops.populate import PopulateRule, PopulationEngine\n\n\n# 1. Models \u2014 `profile` holds an ObjectId in the DB and becomes a Profile on read.\nclass Profile(BaseDocument):\n avatar_url: str = \"\"\n bio: str = \"\"\n\n\nclass User(BaseDocument):\n username: str = \"\"\n email: str = \"\"\n profile: Profile | None = None # ObjectId in MongoDB, Profile in memory\n\n\n# 2. Engine + rule \u2014 repositories are registered later (after connect()).\nengine = PopulationEngine({})\nprofile_rule = PopulateRule(\n field_name=\"profile\",\n collection_name=\"profiles\",\n)\n\n\n# 3. The repository the app uses.\nclass UserRepository(PopulatingRepository[User]):\n def __init__(self):\n super().__init__(\n collection_name=\"users\",\n model=User,\n population_engine=engine,\n populate_rules=[profile_rule],\n )\n\n\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\", db_name=\"mydb\"\n ):\n # Tells the engine which repository resolves \"profiles\" refs.\n engine.register_repo(\"profiles\", PopulatingRepository[Profile](\"profiles\", Profile))\n await ModelRegistry.initialize_all()\n yield\n\n\napp = FastAPI(lifespan=lifespan)\n\n\n# 4. Writing \u2014 pass the model; the repository depopulates to an ObjectId.\n@app.post(\"/users/\", response_model=User)\nasync def create_user():\n user = User(\n username=\"alice\",\n email=\"alice@example.com\",\n profile=Profile(avatar_url=\"alice.png\", bio=\"hi\"),\n )\n return await UserRepository().create(user)\n # ^ stored as {\"profile\": <ObjectId>}, returned with profile populated\n\n\n# 5. Reading \u2014 get_by_id resolves the reference on the way back.\n@app.get(\"/users/{user_id}\", response_model=User)\nasync def get_user(user_id: str):\n user = await UserRepository().get_by_id(user_id)\n if not user:\n raise HTTPException(status_code=404, detail=\"User not found\")\n return user # user.profile is a Profile instance\n</code></pre>"},{"location":"03_use_cases/08_population/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li><code>Patch</code>ing an FK field raises <code>ValueError</code> \u2014 switch FK changes to <code>update(user_id, model)</code> instead. See use case 05 for the reasoning with soft deletes.</li> <li>Missing references resolve to <code>None</code>, not an error.</li> <li>A field holding an embedded dict (instead of an <code>ObjectId</code>) raises a <code>ValueError</code> (\"run repair script\") \u2014 migrate embedded docs to a separate collection first.</li> <li>If <code>create</code>/<code>update</code> receives a field that is already an <code>ObjectId</code> under a populate rule, <code>_depopulate</code> raises <code>ValueError(\"...was populate skipped?\")</code> \u2014 populate-then-depopulate pairs must be balanced.</li> </ul>"},{"location":"03_use_cases/08_population/#related","title":"Related","text":"<ul> <li>09 \u2013 Advanced population \u00b7 10 \u2013 Cache + population \u00b7 14 \u2013 Testing guide</li> </ul>"},{"location":"03_use_cases/09_advanced_population/","title":"Use Case 9: Nested Document Population & Circular-Ref Handling","text":"<p>Scenario: An <code>Author</code> has <code>books</code> (a list of references), each <code>Book</code> references a <code>publisher</code>, and an <code>Author</code> may reference a <code>mentor</code> \u2014 which is another <code>Author</code> (a potential cycle).</p>"},{"location":"03_use_cases/09_advanced_population/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>nested_rules</code> Populates deeper levels: resolve <code>books</code> on <code>Author</code>, then <code>publisher</code> inside each <code>Book</code>. <code>max_depth</code> Per-rule recursion bound \u2014 the safety net for cyclic graphs. <code>global_max_depth</code> <code>PopulationEngine(repos, global_max_depth=10)</code> global cap. <code>CircularReferenceError</code> Raised when a <code>(Class, id)</code> pair is revisited; carries the visited path."},{"location":"03_use_cases/09_advanced_population/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\n\nfrom bson import ObjectId\nfrom fastapi import FastAPI, HTTPException\nfrom mongo_ops import BaseDocument, ModelRegistry, MongoConnectionManager, PopulatingRepository\nfrom mongo_ops.cache import CircularReferenceError\nfrom mongo_ops.populate import PopulateRule, PopulationEngine\n\n\n# 1. Models \u2014 each ref field holds ObjectId(s) in the DB and becomes model(s) in memory.\nclass Publisher(BaseDocument):\n name: str = \"\"\n country: str = \"\"\n\n\nclass Book(BaseDocument):\n title: str = \"\"\n publisher: Publisher | None = None # ObjectId in DB, Publisher in memory\n\n\nclass Author(BaseDocument):\n name: str = \"\"\n books: list[Book] | None = None # list[ObjectId] in DB, list[Book] in memory\n mentor: \"Author\" | None = None # self-reference \u2014 potential cycle\n\n\n# 2. Engine + nested rules.\nengine = PopulationEngine({})\n\npublisher_rule = PopulateRule(\n field_name=\"publisher\",\n collection_name=\"publishers\",\n)\n\nbook_rule = PopulateRule(\n field_name=\"books\",\n collection_name=\"books\",\n nested_rules=[publisher_rule], # fetch each book, then its publisher\n max_depth=3,\n)\n\nmentor_rule = PopulateRule(\n field_name=\"mentor\",\n collection_name=\"authors\",\n max_depth=2, # stops mentor chains early \u2014 also avoids unbounded cycles\n)\n\n\nclass AuthorRepository(PopulatingRepository[Author]):\n def __init__(self):\n super().__init__(\n collection_name=\"authors\",\n model=Author,\n population_engine=engine,\n populate_rules=[book_rule, mentor_rule],\n )\n\n\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\", db_name=\"library\"\n ):\n engine.register_repo(\"publishers\", PopulatingRepository[Publisher](\"publishers\", Publisher))\n engine.register_repo(\"books\", PopulatingRepository[Book](\"books\", Book))\n engine.register_repo(\"authors\", AuthorRepository())\n await ModelRegistry.initialize_all()\n yield\n\n\napp = FastAPI()\n\n\n@app.get(\"/authors/{author_id}\")\nasync def get_author(author_id: str):\n try:\n author = await AuthorRepository().get_by_id(author_id)\n except CircularReferenceError as exc:\n raise HTTPException(status_code=409, detail=f\"Circular reference: {exc}\")\n if not author:\n raise HTTPException(status_code=404, detail=\"Author not found\")\n return author\n</code></pre>"},{"location":"03_use_cases/09_advanced_population/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li><code>CircularReferenceError</code> is a <code>ValueError</code> subtype exposing <code>.collection</code>, <code>.doc_id</code>, and the visited <code>.path</code> \u2014 use it in error responses and logging.</li> <li>Keep <code>max_depth</code> conservative (2\u20133) for most graphs; combine with the global <code>global_max_depth=10</code> default.</li> <li>List refs depopulate back to <code>list[ObjectId]</code> on <code>create</code>/<code>update</code>, mirroring the scalar case.</li> </ul>"},{"location":"03_use_cases/09_advanced_population/#related","title":"Related","text":"<ul> <li>08 \u2013 Population \u00b7 10 \u2013 Cache + population \u00b7 Error Handling</li> </ul>"},{"location":"03_use_cases/10_cache_and_population/","title":"Use Case 10: Caching + Population (Read-Through, Populated on Read)","text":"<p>Scenario: A service needs fast reads of a <code>User</code> and its related <code>Profile</code>. We want the cache-hit path to skip the DB entirely while still returning a populated document.</p>"},{"location":"03_use_cases/10_cache_and_population/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>CachedBaseRepository</code> Provides the cache layer (<code>cache-first get_by_id</code>, invalidation on <code>update</code>/<code>delete</code>). <code>PopulationEngine</code> Provides the reference resolution. Composed subclass A small <code>CachedBaseRepository</code> subclass adds two <code>_populate</code> helpers so cache hits return populated models. <p>\u26a0\ufe0f Known limitation: there is no built-in <code>CachedPopulatingRepository</code> in the library. This use case documents the composition. We cache the raw (depopulated) document \u2014 references stay <code>ObjectId</code>s until read time, so <code>Profile</code> changes are reflected on the next fetch (within TTL) and <code>update</code>/<code>delete</code> invalidation remains correct.</p>"},{"location":"03_use_cases/10_cache_and_population/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\nfrom typing import Any\n\nfrom bson import ObjectId\nfrom fastapi import FastAPI\nfrom mongo_ops import BaseDocument, ModelRegistry, MongoConnectionManager\nfrom mongo_ops.cache import CacheConfig, CachedBaseRepository, InMemoryCacheBackend, decode_value\nfrom mongo_ops.populate import PopulateRule, PopulationEngine\n\n\n# 1. Models \u2014 `profile` holds an ObjectId in DB, a Profile in memory.\nclass Profile(BaseDocument):\n avatar_url: str = \"\"\n bio: str = \"\"\n\n\nclass User(BaseDocument):\n username: str = \"\"\n email: str = \"\"\n profile: Profile | None = None\n\n\n# 2. Engine + rule.\nengine = PopulationEngine({})\nprofile_rule = PopulateRule(field_name=\"profile\", collection_name=\"profiles\")\n\n\n# 3. Composed repository.\nclass CachedUserRepository(CachedBaseRepository[User]):\n def __init__(\n self,\n cache_backend: InMemoryCacheBackend,\n population_engine: PopulationEngine,\n populate_rules: list[PopulateRule],\n config: CacheConfig | None = None,\n ):\n super().__init__(\"users\", User, cache_backend, config)\n self.population_engine = population_engine\n self._populate_rules = populate_rules\n\n async def _populate(self, data: dict[str, Any]) -> dict[str, Any]:\n \"\"\"Dict-level resolution (mirrors PopulatingRepository._populate).\"\"\"\n for rule in self._populate_rules:\n ref = data.get(rule.field_name)\n if ref is None:\n continue\n if isinstance(ref, str): # JSON round-trip turns ObjectId into hex str\n ref = ObjectId(ref)\n if isinstance(ref, list):\n resolved = []\n for item in ref:\n item = ObjectId(item) if isinstance(item, str) else item\n repo = self.population_engine._repos.get(rule.collection_name)\n doc = await repo.get_by_id(item) if repo else None\n resolved.append(doc)\n data[rule.field_name] = resolved\n elif isinstance(ref, ObjectId):\n repo = self.population_engine._repos.get(rule.collection_name)\n data[rule.field_name] = await repo.get_by_id(ref) if repo else None\n return data\n\n async def get_by_id(self, id):\n if not self._cache_config.enabled:\n return await super().get_by_id(id) # no cache -> raw, unpopulated\n\n cached = await self._cache.get(self._cache_key(id))\n if cached is not None:\n # Cache hit: decode the RAW doc, then populate before returning.\n data = await self._populate(decode_value(cached))\n return self.model(**data)\n\n # Cache miss: single DB read; the base class caches the raw (depopulated) doc.\n result = await super().get_by_id(id)\n if result is None:\n return None\n return await self.model(**await self._populate(result.model_dump(by_alias=True)))\n\n\n# 4. Wire-up \u2014 one backend for both the repo and the registry lifecycle.\ncache = InMemoryCacheBackend(max_entries=20_000, default_ttl=600)\nModelRegistry.set_cache_backend(cache)\n\n\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\", db_name=\"app_db\"\n ):\n # Register the repositories the engine resolves refs against:\n # engine.register_repo(\"profiles\", ProfileRepo())\n engine.register_repo(\"users\", CachedUserRepository(cache, engine, [profile_rule]))\n await ModelRegistry.initialize_all()\n await ModelRegistry.initialize_cache()\n yield\n await ModelRegistry.shutdown_cache()\n\n\napp = FastAPI(lifespan=lifespan)\n</code></pre> <p>Note: replace the placeholder <code>engine.register_repo(\"profiles\", ...)</code> line with a repository for <code>Profile</code>. A matching <code>CachedBaseRepository[Profile](\"profiles\", Profile, cache)</code> keeps profile reads cached too \u2014 e.g. <code>engine.register_repo(\"profiles\", CachedBaseRepository[Profile](\"profiles\", Profile, cache))</code>.</p>"},{"location":"03_use_cases/10_cache_and_population/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>Cache the raw doc, populate at read time. Ref fields stay <code>ObjectId</code>s in the cache; JSON round-trips them to hex strings, so <code>_populate</code> must normalize <code>str \u2192 ObjectId</code> (shown above).</li> <li><code>update</code>/<code>delete</code> from the base class still invalidate the right key because we never repopulate in the cache.</li> <li>Set a sensible <code>default_ttl</code> \u2014 cached <code>User</code> entries resolve <code>Profile</code> on each read, so profile edits show up within the TTL (or call <code>invalidate_cache(user_id)</code> explicitly).</li> <li>If you want one repo class used app-wide, keep the populate helpers in a mixin shared with regular <code>PopulatingRepository</code>.</li> </ul>"},{"location":"03_use_cases/10_cache_and_population/#related","title":"Related","text":"<ul> <li>07 \u2013 Caching \u00b7 08 \u2013 Population \u00b7 11 \u2013 Cache lifecycle</li> </ul>"},{"location":"03_use_cases/11_cache_lifecycle/","title":"Use Case 11: Proper Cache Lifecycle in FastAPI","text":"<p>Scenario: A microservice uses an in-memory (or Redis) cache backend and must start the cleanup task on app start, then shut it down cleanly on termination.</p>"},{"location":"03_use_cases/11_cache_lifecycle/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>ModelRegistry.set_cache_backend(backend)</code> Registers the single shared backend. <code>ModelRegistry.initialize_cache()</code> Starts the backend (spawns the in-memory TTL cleanup task). Raises <code>RuntimeError</code> if no backend was registered. <code>ModelRegistry.shutdown_cache()</code> Cancels the cleanup task, closes pub/sub, clears the backend. If a repo uses a backend directly Same instance must be registered so <code>initialize_cache</code> starts its task."},{"location":"03_use_cases/11_cache_lifecycle/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\n\nfrom fastapi import FastAPI\nfrom mongo_ops import ModelRegistry, MongoConnectionManager\nfrom mongo_ops.cache import InMemoryCacheBackend\n\ncache = InMemoryCacheBackend(max_entries=10_000, default_ttl=300)\nModelRegistry.set_cache_backend(cache) # before any cache-backed repo is used\n\n\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\", db_name=\"mydb\"\n ):\n await ModelRegistry.initialize_all() # create indexes (idempotent)\n await ModelRegistry.initialize_cache() # start the TTL cleanup task\n yield\n await ModelRegistry.shutdown_cache() # cancel task + close cleanly\n\n\napp = FastAPI(lifespan=lifespan)\n</code></pre>"},{"location":"03_use_cases/11_cache_lifecycle/#what-actually-happens","title":"\ud83d\udd01 What Actually Happens","text":"<ul> <li><code>InMemoryCacheBackend.initialize()</code> spawns an <code>asyncio.Task</code> that evicts expired entries every <code>cleanup_interval</code> seconds. Without a shutdown, the event loop flags the dangling task on exit \u2014 <code>shutdown_cache()</code> cancels it and awaits it.</li> <li><code>shutdown_cache()</code> also NULs the registry cache backend and closes the Redis pub/sub handle (if Redis).</li> <li>If <code>initialize_cache()</code> is called before <code>set_cache_backend()</code>, it raises:</li> </ul> Text Only<pre><code>RuntimeError: No cache backend registered. Call set_cache_backend() first.\n</code></pre>"},{"location":"03_use_cases/11_cache_lifecycle/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>Per-repository toggling is independent of the lifecycle: <code>CacheConfig(enabled=False)</code> bypasses the cache for that repo even after <code>initialize_cache()</code>.</li> <li>The cleanup task uses <code>cleanup_interval</code> seconds for scans; entries also expire on access via the TTL heap (<code>default_ttl=0</code> expires immediately).</li> <li>Register the backend before constructing any <code>CachedBaseRepository</code> that references it \u2014 otherwise a repo may hold an uninitialized backend (no cleanup task, no Redis pub/sub).</li> </ul>"},{"location":"03_use_cases/11_cache_lifecycle/#related","title":"Related","text":"<ul> <li>07 \u2013 Caching \u00b7 Components: ModelRegistry \u00b7 10 \u2013 Cache + population</li> </ul>"},{"location":"03_use_cases/12_transaction_helper/","title":"Use Case 12: Using <code>TransactionManager.execute_transaction</code>","text":"<p>Scenario: Perform several writes across different collections atomically \u2014 e.g., create an <code>Order</code> and decrement <code>Inventory</code>. The low-level <code>start_session</code> context works, but <code>execute_transaction</code> collects results from a list of async operations.</p>"},{"location":"03_use_cases/12_transaction_helper/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>TransactionManager.execute_transaction</code> Runs a list of async callables (each receives a session) inside one transaction; returns the result of each callable, in order. Automatic rollback Any raised exception aborts the transaction and propagates to the caller. <code>start_session</code> The underlying async context manager <code>async with TransactionManager.start_session() as session:</code>."},{"location":"03_use_cases/12_transaction_helper/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\nfrom datetime import datetime\n\nfrom bson import ObjectId\nfrom fastapi import FastAPI, HTTPException\nfrom mongo_ops import BaseDocument, ModelRegistry, MongoConnectionManager\nfrom mongo_ops.repository import BaseRepository\nfrom mongo_ops.transactions import TransactionManager\n\n\n# 1. Models\nclass Order(BaseDocument):\n user_id: str = \"\"\n items: list[dict] = [] # [{\"product_id\": ObjectId, \"qty\": int}]\n total: float = 0.0\n\n\nclass Inventory(BaseDocument):\n product_id: ObjectId = None\n quantity: int = 0\n\n\n# 2. Repositories (constructed inside the lifespan \u2014 after connect()).\nclass OrderRepo(BaseRepository[Order]):\n def __init__(self):\n super().__init__(\"orders\", Order)\n\n\nclass InventoryRepo(BaseRepository[Inventory]):\n def __init__(self):\n super().__init__(\"inventory\", Inventory)\n\n\n# 3. The atomic operation.\nasync def create_order_with_inventory(order: Order, order_repo: OrderRepo, inv_repo: InventoryRepo):\n async def insert_order(session):\n doc = order.model_dump(exclude={\"id\"}, exclude_none=True)\n doc[\"created_at\"] = doc[\"updated_at\"] = datetime.utcnow()\n result = await order_repo.collection.insert_one(doc, session=session)\n return await order_repo.collection.find_one({\"_id\": result.inserted_id}, session=session)\n\n async def update_inventory(session):\n for item in order.items:\n await inv_repo.collection.update_one(\n {\"product_id\": ObjectId(item[\"product_id\"])},\n {\"$inc\": {\"quantity\": -item[\"qty\"]}},\n session=session,\n )\n return \"inventory-updated\"\n\n results = await TransactionManager.execute_transaction(\n [insert_order, update_inventory]\n )\n return results[0] # created order document; results[1] == \"inventory-updated\"\n\n\n# 4. FastAPI wiring.\napp = FastAPI()\n\n\n@asynccontextmanager\nasync def lifespan(_app: FastAPI):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\", db_name=\"shop\"\n ):\n await ModelRegistry.initialize_all()\n yield\n\n\napp.lifespan = lifespan\n\n\n@app.post(\"/orders/\", response_model=Order)\nasync def create_order(order: Order):\n created = await create_order_with_inventory(order, OrderRepo(), InventoryRepo())\n if created is None:\n raise HTTPException(status_code=400, detail=\"Transaction failed\")\n return created\n</code></pre>"},{"location":"03_use_cases/12_transaction_helper/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>Return values: each callable can return whatever you need; results are collected in the same order.</li> <li>Errors: raise inside any callable \u2192 the whole transaction aborts (session rolls back) and the exception propagates.</li> <li>Reads inside a transaction: pass <code>session=session</code> to <code>find_one</code>/<code>find</code> too.</li> <li>Testing (no Mongo): <code>monkeypatch</code> a fake client on <code>mongo_ops.transactions.MongoConnectionManager.get_client</code> and stub <code>start_session</code> \u2014 see <code>tests/test_transactions.py</code>.</li> <li>Repo models still carry <code>created_at</code>/<code>updated_at</code>; for raw collection inserts inside the transaction you set them manually (as shown).</li> </ul>"},{"location":"03_use_cases/12_transaction_helper/#related","title":"Related","text":"<ul> <li>03 \u2013 Transactions \u00b7 14 \u2013 Testing guide \u00b7 Error Handling</li> </ul>"},{"location":"03_use_cases/13_index_creation/","title":"Use Case 13: Declaring Indexes (Single-Field, Composite, Unique)","text":"<p>Scenario: Ensure each collection has the right indexes for fast queries and data integrity \u2014 declared in one place and created at startup.</p>"},{"location":"03_use_cases/13_index_creation/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>ModelRegistry.register(indexes=...)</code> Each spec is passed as-is to pymongo <code>create_index</code>. Supported forms: tuple <code>(\"field\", direction)</code>, compound list <code>[(\"a\", 1), (\"b\", -1)]</code>, or a dict with <code>keys</code> + <code>options</code>. <code>ModelRegistry.initialize_all</code> Creates every registered index during startup (idempotent \u2014 <code>create_index</code> skips existing indexes)."},{"location":"03_use_cases/13_index_creation/#example","title":"\ud83d\ude80 Example","text":"Python<pre><code>from contextlib import asynccontextmanager\n\nfrom pymongo import ASCENDING, DESCENDING\nfrom mongo_ops import BaseDocument, ModelRegistry, MongoConnectionManager\n\n\nclass User(BaseDocument):\n username: str = \"\"\n email: str = \"\"\n\n\nclass BlogPost(BaseDocument):\n author_id: str = \"\"\n created_at: str = \"\"\n title: str = \"\"\n\n\nclass Passenger(BaseDocument):\n email: str = \"\"\n seat: str = \"\"\n\n\n# 1\ufe0f\u20e3 Single-field index \u2014 email lookups.\nModelRegistry.register(\n collection_name=\"users\",\n model=User,\n indexes=[(\"email\", ASCENDING)],\n)\n\n# 2\ufe0f\u20e3 Composite index \u2014 queries filtering by author + creation date.\nModelRegistry.register(\n collection_name=\"posts\",\n model=BlogPost,\n indexes=[[(\"author_id\", ASCENDING), (\"created_at\", DESCENDING)]],\n)\n\n# 3\ufe0f\u20e3 Unique index with a custom name \u2014 enforce unique emails.\nModelRegistry.register(\n collection_name=\"passengers\",\n model=Passenger,\n indexes=[\n {\n \"keys\": [(\"email\", ASCENDING)],\n \"options\": {\"unique\": True, \"name\": \"uq_passenger_email\"},\n }\n ],\n)\n\n\n# 4\ufe0f\u20e3 Everything is created on startup.\n@asynccontextmanager\nasync def lifespan(_app):\n async with MongoConnectionManager.lifespan(\n uri=\"mongodb://localhost:27017\",\n db_name=\"mydb\",\n ):\n await ModelRegistry.initialize_all()\n yield\n</code></pre>"},{"location":"03_use_cases/13_index_creation/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>Every spec passes through to <code>collection.create_index(spec)</code> \u2014 so MongoDB options like <code>unique</code>, <code>sparse</code>, and <code>expireAfterSeconds</code> (TTL) belong in the <code>options</code> dict.</li> <li>Idempotent by construction: <code>create_index</code> is a no-op when a same-shape index already exists.</li> <li>Verify with the Mongo shell:</li> </ul> JavaScript<pre><code>db.<collection>.getIndexes()\n</code></pre> <ul> <li>A unique index on an already-duplicated field will fail with <code>DuplicateKeyError</code> on startup \u2014 clean the data first.</li> </ul>"},{"location":"03_use_cases/13_index_creation/#related","title":"Related","text":"<ul> <li>01 \u2013 Basic CRUD \u00b7 06 \u2013 Multi-model \u00b7 Components: ModelRegistry</li> </ul>"},{"location":"03_use_cases/14_testing_guide/","title":"Use Case 14: Testing Guide \u2014 Mocking Motor, Cache, Registry & Transactions","text":"<p>Scenario: Write unit tests that never touch a real MongoDB \u2014 mock the collection, the cache backend, and the client, exactly like the library's own test suite (<code>tests/</code>).</p>"},{"location":"03_use_cases/14_testing_guide/#whats-new","title":"\ud83d\udce6 What's New?","text":"Component Description <code>AsyncMock</code> collections Stub <code>find_one</code>, <code>insert_one</code>, <code>find_one_and_update</code>, cursor chains. Patching <code>MongoConnectionManager.get_database</code> Gives repositories a mock collection without a live connection. <code>monkeypatch</code> on <code>get_client</code> Fakes sessions for <code>TransactionManager</code> tests. Reference tests <code>tests/test_populating_repository.py</code>, <code>tests/test_cache.py</code>, <code>tests/test_registry.py</code>, <code>tests/test_transactions.py</code>."},{"location":"03_use_cases/14_testing_guide/#example-boilerplate","title":"\ud83d\ude80 Example Boilerplate","text":"Python<pre><code>import pytest\nfrom unittest.mock import AsyncMock, MagicMock, patch\n\nfrom bson import ObjectId\nfrom mongo_ops.cache import CacheConfig, InMemoryCacheBackend, CachedBaseRepository\nfrom mongo_ops.models import BaseDocument\nfrom mongo_ops.registry import ModelRegistry\nfrom mongo_ops.populate import PopulateRule, PopulationEngine\nfrom mongo_ops.repository import PopulatingRepository\n\n\n# ----------------------------------------------------------------------\n# 1. Models (same shape as the library tests)\n# ----------------------------------------------------------------------\nclass Profile(BaseDocument):\n avatar_url: str = \"\"\n\n\nclass User(BaseDocument):\n name: str = \"\"\n profile: Profile | None = None # ObjectId in DB, Profile in memory\n\n\n# ----------------------------------------------------------------------\n# 2. Fixtures\n# ----------------------------------------------------------------------\n@pytest.fixture\ndef mock_collection():\n return AsyncMock()\n\n\n@pytest.fixture\ndef engine():\n profile_repo = AsyncMock()\n return PopulationEngine({\"profiles\": profile_repo})\n\n\n@pytest.fixture\ndef repo(mock_collection, engine):\n with patch(\"mongo_ops.repository.MongoConnectionManager.get_database\") as mock_db:\n mock_db.return_value.__getitem__.return_value = mock_collection\n r = PopulatingRepository(\n \"users\",\n User,\n population_engine=engine,\n populate_rules=[PopulateRule(field_name=\"profile\", collection_name=\"profiles\")],\n )\n r.collection = mock_collection\n return r\n\n\n# ----------------------------------------------------------------------\n# 3. Population \u2014 get_by_id resolves the reference\n# ----------------------------------------------------------------------\n@pytest.mark.asyncio\nasync def test_get_by_id_populates(repo, mock_collection, engine):\n uid, pid = ObjectId(), ObjectId()\n mock_collection.find_one.return_value = {\n \"_id\": uid,\n \"name\": \"Alice\",\n \"profile\": pid, # ObjectId stored in DB\n \"created_at\": \"2024-01-01T00:00:00\",\n \"updated_at\": \"2024-01-01T00:00:00\",\n }\n engine._repos[\"profiles\"].get_by_id.return_value = Profile(id=pid, avatar_url=\"pic.png\")\n\n result = await repo.get_by_id(uid)\n\n assert result is not None\n assert result.name == \"Alice\"\n assert isinstance(result.profile, Profile)\n assert result.profile.avatar_url == \"pic.png\"\n\n\n# ----------------------------------------------------------------------\n# 4. Patch FK guard\n# ----------------------------------------------------------------------\n@pytest.mark.asyncio\nasync def test_patch_rejects_fk_field(repo):\n with pytest.raises(ValueError, match=\"Cannot patch FK fields\"):\n await repo.patch(ObjectId(), {\"profile\": ObjectId()})\n\n\n# ----------------------------------------------------------------------\n# 5. Registry \u2014 index specs pass through to create_index\n# ----------------------------------------------------------------------\n@pytest.mark.asyncio\nasync def test_initialize_all_creates_indexes():\n ModelRegistry.register(\"users\", User, indexes=[(\"email\", 1)])\n\n fake_collection = AsyncMock()\n await ModelRegistry.initialize_all(db={\"users\": fake_collection})\n\n fake_collection.create_index.assert_awaited_once_with((\"email\", 1))\n\n\n# ----------------------------------------------------------------------\n# 6. Cached repository \u2014 cache-first reads + invalidation\n# ----------------------------------------------------------------------\n@pytest.mark.asyncio\nasync def test_cached_get_by_id_populates_cache():\n backend = InMemoryCacheBackend(\n max_entries=100, default_ttl=300, cleanup_interval=9999\n )\n await backend.initialize()\n try:\n with patch(\"mongo_ops.repository.MongoConnectionManager.get_database\") as mock_db:\n mock_collection = AsyncMock()\n mock_db.return_value.__getitem__.return_value = mock_collection\n repo = CachedBaseRepository(\n \"users\", User, backend, CacheConfig(enabled=True)\n )\n repo.collection = mock_collection\n\n oid = ObjectId()\n mock_collection.find_one.return_value = {\n \"_id\": oid,\n \"name\": \"cached\",\n \"created_at\": \"2024-01-01T00:00:00\",\n \"updated_at\": \"2024-01-01T00:00:00\",\n }\n\n first = await repo.get_by_id(oid)\n assert first is not None\n\n mock_collection.find_one.return_value = None # DB now \"empty\"\n second = await repo.get_by_id(oid) # served from cache\n\n assert second is not None\n assert second.name == \"cached\"\n mock_collection.find_one.assert_awaited_once() # only one DB read\n finally:\n await backend.shutdown()\n\n\n# ----------------------------------------------------------------------\n# 7. Transactions \u2014 fake the client's start_session\n# ----------------------------------------------------------------------\nfrom mongo_ops.transactions import TransactionManager\n\n\n@pytest.mark.asyncio\nasync def test_execute_transaction(monkeypatch):\n session_ctx = AsyncMock()\n session_ctx.__aenter__.return_value = AsyncMock()\n session_ctx.__aexit__.return_value = None\n\n client = MagicMock()\n client.start_session = AsyncMock(return_value=session_ctx)\n\n # NOTE: start_transaction must return a context manager, not a coroutine.\n async_session = session_ctx.__aenter__.return_value\n async_session.start_transaction = lambda **_: session_ctx\n\n monkeypatch.setattr(\n \"mongo_ops.transactions.MongoConnectionManager.get_client\",\n lambda: client,\n )\n\n async def fake_op(session):\n return \"ok\"\n\n results = await TransactionManager.execute_transaction([fake_op])\n assert results == [\"ok\"]\n</code></pre>"},{"location":"03_use_cases/14_testing_guide/#tips","title":"\ud83d\udca1 Tips","text":"<ul> <li>pytest-asyncio is already configured in <code>pyproject.toml</code> (<code>asyncio_mode = \"auto\"</code>), so <code>@pytest.mark.asyncio</code> tests work out of the box. Run with <code>pytest</code> (coverage reports are enabled there too).</li> <li>Never hit the network. Keep the patches in fixtures (or a <code>conftest.py</code>) and reuse them.</li> <li>Mock <code>cursor</code> chains with <code>MagicMock()</code> + <code>.to_list = AsyncMock(...)</code>, exactly like <code>tests/test_repository.py</code>.</li> <li>For an optional integration check (real Mongo), use <code>MongoConnectionManager.lifespan</code> against a local replica set and drop the test database in teardown \u2014 keep it separate from the unit suite.</li> <li>The pattern works symmetrically for Redis: mock the <code>RedisCacheBackend</code> methods (<code>get</code>, <code>set</code>, <code>delete</code>) \u2014 no Redis process required.</li> </ul>"},{"location":"03_use_cases/14_testing_guide/#related","title":"Related","text":"<ul> <li>08 \u2013 Population \u00b7 07 \u2013 Caching \u00b7 12 \u2013 Transaction helper</li> </ul>"}]} |