aetos/docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: collect refreshed mongo-ops wiki site (API-correct overhaul, v0.1.5)

Browse Source
This commit is contained in:
Vishesh 'ironeagle' Bangotra
2026-09-14 13:46:11 +05:30
parent bbd9863d23
commit f406aa2f21
24 changed files with 2880 additions and 1525 deletions
Download Patch File Download Diff File Expand all files Collapse all files

229
mongo-ops/03_use_cases/10_cache_and_population/index.html
View File

@@ -74,7 +74,7 @@
<div data-md-component="skip">
<a href="#use-case-10-caching-population-readthrough-populated-docs" class="md-skip">
<a href="#use-case-10-caching-population-read-through-populated-on-read" class="md-skip">
Skip to content
</a>
@@ -457,7 +457,7 @@
<span class="md-ellipsis">
02 components
Core Components
</span>
@@ -782,7 +782,7 @@
<li class="md-nav__item">
<a href="#whats-new" class="md-nav__link">
<span class="md-ellipsis">
📦 What’s New?
📦 What's New?
</span>
</a>
@@ -804,6 +804,15 @@
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#related" class="md-nav__link">
<span class="md-ellipsis">
Related
</span>
</a>
</li>
</ul>
@@ -1026,11 +1035,10 @@
<h1 id="use-case-10-caching-population-readthrough-populated-docs">Use Case 10: Caching + Population (Read‑Through + Populated Docs)</h1>
<p><strong>Scenario:</strong>
A service needs fast reads of a <code>User</code> document <strong>and</strong> its related <code>Profile</code>. The repository should cache the final populated result so subsequent calls hit the cache directly.</p>
<h1 id="use-case-10-caching-population-read-through-populated-on-read">Use Case 10: Caching + Population (Read-Through, Populated on Read)</h1>
<p><strong>Scenario:</strong> A service needs fast reads of a <code>User</code> <strong>and</strong> its related <code>Profile</code>. We want the cache-hit path to skip the DB entirely while still returning a populated document.</p>
<hr />
<h2 id="whats-new">📦 What’s New?</h2>
<h2 id="whats-new">📦 What's New?</h2>
<table>
<thead>
<tr>
@@ -1040,116 +1048,139 @@ A service needs fast reads of a <code>User</code> document <strong>and</strong>
</thead>
<tbody>
<tr>
<td><strong><code>CachedBaseRepository</code></strong></td>
<td>Provides transparent ID‑based caching for CRUD operations.</td>
<td><code>CachedBaseRepository</code></td>
<td>Provides the cache layer (<code>cache-first get_by_id</code>, invalidation on <code>update</code>/<code>delete</code>).</td>
</tr>
<tr>
<td><strong><code>PopulatingRepository</code></strong></td>
<td>Adds automatic population of referenced documents.</td>
<td><code>PopulationEngine</code></td>
<td>Provides the reference resolution.</td>
</tr>
<tr>
<td><strong>Combined usage</strong></td>
<td>By inheriting from <code>CachedBaseRepository</code> <em>and</em> wiring a <code>PopulationEngine</code>, the repository caches the <strong>populated</strong> document, eliminating both DB and population overhead on cache hits.</td>
<td><strong>Composed subclass</strong></td>
<td>A small <code>CachedBaseRepository</code> subclass adds two <code>_populate</code> helpers so cache hits return populated models.</td>
</tr>
</tbody>
</table>
<blockquote>
<p>⚠️ <strong>Known limitation:</strong> there is <strong>no built-in <code>CachedPopulatingRepository</code></strong> in the library. This use case documents the composition. We cache the <strong>raw (depopulated)</strong> document — references stay <code>ObjectId</code>s until read time, so <code>Profile</code> changes are reflected on the next fetch (within TTL) and <code>update</code>/<code>delete</code> invalidation remains correct.</p>
</blockquote>
<hr />
<h2 id="example">🚀 Example</h2>
<div class="language-python highlight"><span class="filename">Python</span><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">bson</span><span class="w"> </span><span class="kn">import</span> <span class="n">ObjectId</span>
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops</span><span class="w"> </span><span class="kn">import</span> <span class="n">BaseDocument</span><span class="p">,</span> <span class="n">CachedBaseRepository</span><span class="p">,</span> <span class="n">ModelRegistry</span>
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops.populate</span><span class="w"> </span><span class="kn">import</span> <span class="n">PopulateRule</span><span class="p">,</span> <span class="n">PopulationEngine</span>
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops.cache</span><span class="w"> </span><span class="kn">import</span> <span class="n">InMemoryCacheBackend</span>
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a>
<a id="__codelineno-0-6" name="__codelineno-0-6" href="#__codelineno-0-6"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-7" name="__codelineno-0-7" href="#__codelineno-0-7"></a><span class="c1"># 1️⃣ Define models</span>
<a id="__codelineno-0-8" name="__codelineno-0-8" href="#__codelineno-0-8"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-9" name="__codelineno-0-9" href="#__codelineno-0-9"></a><span class="k">class</span><span class="w"> </span><span class="nc">Profile</span><span class="p">(</span><span class="n">BaseDocument</span><span class="p">):</span>
<a id="__codelineno-0-10" name="__codelineno-0-10" href="#__codelineno-0-10"></a> <span class="n">avatar_url</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>
<a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a> <span class="n">bio</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>
<a id="__codelineno-0-12" name="__codelineno-0-12" href="#__codelineno-0-12"></a>
<a id="__codelineno-0-13" name="__codelineno-0-13" href="#__codelineno-0-13"></a><span class="k">class</span><span class="w"> </span><span class="nc">User</span><span class="p">(</span><span class="n">BaseDocument</span><span class="p">):</span>
<a id="__codelineno-0-14" name="__codelineno-0-14" href="#__codelineno-0-14"></a> <span class="n">username</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>
<a id="__codelineno-0-15" name="__codelineno-0-15" href="#__codelineno-0-15"></a> <span class="n">email</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>
<a id="__codelineno-0-16" name="__codelineno-0-16" href="#__codelineno-0-16"></a> <span class="n">profile_id</span><span class="p">:</span> <span class="n">ObjectId</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># Reference to Profile</span>
<a id="__codelineno-0-17" name="__codelineno-0-17" href="#__codelineno-0-17"></a> <span class="n">profile</span><span class="p">:</span> <span class="n">Profile</span> <span class="o">|</span> <span class="kc">None</span> <span class="o">=</span> <span class="kc">None</span> <span class="c1"># Populated field</span>
<a id="__codelineno-0-18" name="__codelineno-0-18" href="#__codelineno-0-18"></a>
<a id="__codelineno-0-19" name="__codelineno-0-19" href="#__codelineno-0-19"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-20" name="__codelineno-0-20" href="#__codelineno-0-20"></a><span class="c1"># 2️⃣ Initialise cache backend (in‑memory example)</span>
<a id="__codelineno-0-21" name="__codelineno-0-21" href="#__codelineno-0-21"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-22" name="__codelineno-0-22" href="#__codelineno-0-22"></a><span class="n">cache</span> <span class="o">=</span> <span class="n">InMemoryCacheBackend</span><span class="p">(</span><span class="n">max_entries</span><span class="o">=</span><span class="mi">20_000</span><span class="p">,</span> <span class="n">default_ttl</span><span class="o">=</span><span class="mi">600</span><span class="p">)</span>
<a id="__codelineno-0-23" name="__codelineno-0-23" href="#__codelineno-0-23"></a><span class="n">ModelRegistry</span><span class="o">.</span><span class="n">set_cache_backend</span><span class="p">(</span><span class="n">cache</span><span class="p">)</span>
<a id="__codelineno-0-24" name="__codelineno-0-24" href="#__codelineno-0-24"></a>
<a id="__codelineno-0-25" name="__codelineno-0-25" href="#__codelineno-0-25"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-26" name="__codelineno-0-26" href="#__codelineno-0-26"></a><span class="c1"># 3️⃣ Set up a PopulationEngine (repositories will be registered later)</span>
<a id="__codelineno-0-27" name="__codelineno-0-27" href="#__codelineno-0-27"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-28" name="__codelineno-0-28" href="#__codelineno-0-28"></a><span class="n">engine</span> <span class="o">=</span> <span class="n">PopulationEngine</span><span class="p">({})</span>
<a id="__codelineno-0-29" name="__codelineno-0-29" href="#__codelineno-0-29"></a><span class="n">profile_rule</span> <span class="o">=</span> <span class="n">PopulateRule</span><span class="p">(</span>
<a id="__codelineno-0-30" name="__codelineno-0-30" href="#__codelineno-0-30"></a> <span class="n">field_name</span><span class="o">=</span><span class="s2">&quot;profile&quot;</span><span class="p">,</span>
<a id="__codelineno-0-31" name="__codelineno-0-31" href="#__codelineno-0-31"></a> <span class="n">collection_name</span><span class="o">=</span><span class="s2">&quot;profiles&quot;</span><span class="p">,</span>
<a id="__codelineno-0-32" name="__codelineno-0-32" href="#__codelineno-0-32"></a> <span class="n">ref_field</span><span class="o">=</span><span class="s2">&quot;profile_id&quot;</span><span class="p">,</span>
<a id="__codelineno-0-33" name="__codelineno-0-33" href="#__codelineno-0-33"></a><span class="p">)</span>
<a id="__codelineno-0-34" name="__codelineno-0-34" href="#__codelineno-0-34"></a>
<a id="__codelineno-0-35" name="__codelineno-0-35" href="#__codelineno-0-35"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-36" name="__codelineno-0-36" href="#__codelineno-0-36"></a><span class="c1"># 4️⃣ Cached + Populating repository</span>
<a id="__codelineno-0-37" name="__codelineno-0-37" href="#__codelineno-0-37"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-38" name="__codelineno-0-38" href="#__codelineno-0-38"></a><span class="k">class</span><span class="w"> </span><span class="nc">UserRepository</span><span class="p">(</span><span class="n">CachedBaseRepository</span><span class="p">[</span><span class="n">User</span><span class="p">]):</span>
<a id="__codelineno-0-39" name="__codelineno-0-39" href="#__codelineno-0-39"></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<a id="__codelineno-0-40" name="__codelineno-0-40" href="#__codelineno-0-40"></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span>
<a id="__codelineno-0-41" name="__codelineno-0-41" href="#__codelineno-0-41"></a> <span class="n">collection_name</span><span class="o">=</span><span class="s2">&quot;users&quot;</span><span class="p">,</span>
<a id="__codelineno-0-42" name="__codelineno-0-42" href="#__codelineno-0-42"></a> <span class="n">model</span><span class="o">=</span><span class="n">User</span><span class="p">,</span>
<a id="__codelineno-0-43" name="__codelineno-0-43" href="#__codelineno-0-43"></a> <span class="n">cache_backend</span><span class="o">=</span><span class="n">cache</span><span class="p">,</span>
<a id="__codelineno-0-44" name="__codelineno-0-44" href="#__codelineno-0-44"></a> <span class="n">config</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="c1"># defaults to enabled=True, backend=&quot;memory&quot;</span>
<a id="__codelineno-0-45" name="__codelineno-0-45" href="#__codelineno-0-45"></a> <span class="p">)</span>
<a id="__codelineno-0-46" name="__codelineno-0-46" href="#__codelineno-0-46"></a> <span class="c1"># Attach the population engine after the base repo is ready</span>
<a id="__codelineno-0-47" name="__codelineno-0-47" href="#__codelineno-0-47"></a> <span class="bp">self</span><span class="o">.</span><span class="n">population_engine</span> <span class="o">=</span> <span class="n">engine</span>
<a id="__codelineno-0-48" name="__codelineno-0-48" href="#__codelineno-0-48"></a> <span class="bp">self</span><span class="o">.</span><span class="n">_populate_rules</span> <span class="o">=</span> <span class="p">[</span><span class="n">profile_rule</span><span class="p">]</span>
<a id="__codelineno-0-49" name="__codelineno-0-49" href="#__codelineno-0-49"></a>
<a id="__codelineno-0-50" name="__codelineno-0-50" href="#__codelineno-0-50"></a> <span class="c1"># Override get_by_id to include population before caching</span>
<a id="__codelineno-0-51" name="__codelineno-0-51" href="#__codelineno-0-51"></a> <span class="k">async</span> <span class="k">def</span><span class="w"> </span><span class="nf">get_by_id</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">id</span><span class="p">):</span>
<a id="__codelineno-0-52" name="__codelineno-0-52" href="#__codelineno-0-52"></a> <span class="c1"># First attempt cache lookup (as in CachedBaseRepository)</span>
<a id="__codelineno-0-53" name="__codelineno-0-53" href="#__codelineno-0-53"></a> <span class="n">cached</span> <span class="o">=</span> <span class="k">await</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">get_by_id</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
<a id="__codelineno-0-54" name="__codelineno-0-54" href="#__codelineno-0-54"></a> <span class="k">if</span> <span class="n">cached</span><span class="p">:</span>
<a id="__codelineno-0-55" name="__codelineno-0-55" href="#__codelineno-0-55"></a> <span class="k">return</span> <span class="n">cached</span>
<a id="__codelineno-0-56" name="__codelineno-0-56" href="#__codelineno-0-56"></a> <span class="c1"># Not in cache – fetch from DB and populate</span>
<a id="__codelineno-0-57" name="__codelineno-0-57" href="#__codelineno-0-57"></a> <span class="n">doc</span> <span class="o">=</span> <span class="k">await</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">get_by_id</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span> <span class="c1"># this will hit the DB (no cache hit)</span>
<a id="__codelineno-0-58" name="__codelineno-0-58" href="#__codelineno-0-58"></a> <span class="k">if</span> <span class="n">doc</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<a id="__codelineno-0-59" name="__codelineno-0-59" href="#__codelineno-0-59"></a> <span class="k">return</span> <span class="kc">None</span>
<a id="__codelineno-0-60" name="__codelineno-0-60" href="#__codelineno-0-60"></a> <span class="c1"># Populate related docs</span>
<a id="__codelineno-0-61" name="__codelineno-0-61" href="#__codelineno-0-61"></a> <span class="n">populated</span> <span class="o">=</span> <span class="k">await</span> <span class="n">engine</span><span class="o">.</span><span class="n">populate</span><span class="p">(</span><span class="n">doc</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">_populate_rules</span><span class="p">)</span>
<a id="__codelineno-0-62" name="__codelineno-0-62" href="#__codelineno-0-62"></a> <span class="c1"># Store the fully populated result in cache for next call</span>
<a id="__codelineno-0-63" name="__codelineno-0-63" href="#__codelineno-0-63"></a> <span class="k">await</span> <span class="bp">self</span><span class="o">.</span><span class="n">_cache</span><span class="o">.</span><span class="n">set</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_cache_key</span><span class="p">(</span><span class="nb">id</span><span class="p">),</span> <span class="n">populated</span><span class="p">)</span>
<a id="__codelineno-0-64" name="__codelineno-0-64" href="#__codelineno-0-64"></a> <span class="k">return</span> <span class="n">populated</span>
<div class="language-python highlight"><span class="filename">Python</span><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">contextlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">asynccontextmanager</span>
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="kn">from</span><span class="w"> </span><span class="nn">typing</span><span class="w"> </span><span class="kn">import</span> <span class="n">Any</span>
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a>
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a><span class="kn">from</span><span class="w"> </span><span class="nn">bson</span><span class="w"> </span><span class="kn">import</span> <span class="n">ObjectId</span>
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a><span class="kn">from</span><span class="w"> </span><span class="nn">fastapi</span><span class="w"> </span><span class="kn">import</span> <span class="n">FastAPI</span>
<a id="__codelineno-0-6" name="__codelineno-0-6" href="#__codelineno-0-6"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops</span><span class="w"> </span><span class="kn">import</span> <span class="n">BaseDocument</span><span class="p">,</span> <span class="n">ModelRegistry</span><span class="p">,</span> <span class="n">MongoConnectionManager</span>
<a id="__codelineno-0-7" name="__codelineno-0-7" href="#__codelineno-0-7"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops.cache</span><span class="w"> </span><span class="kn">import</span> <span class="n">CacheConfig</span><span class="p">,</span> <span class="n">CachedBaseRepository</span><span class="p">,</span> <span class="n">InMemoryCacheBackend</span><span class="p">,</span> <span class="n">decode_value</span>
<a id="__codelineno-0-8" name="__codelineno-0-8" href="#__codelineno-0-8"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops.populate</span><span class="w"> </span><span class="kn">import</span> <span class="n">PopulateRule</span><span class="p">,</span> <span class="n">PopulationEngine</span>
<a id="__codelineno-0-9" name="__codelineno-0-9" href="#__codelineno-0-9"></a>
<a id="__codelineno-0-10" name="__codelineno-0-10" href="#__codelineno-0-10"></a>
<a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a><span class="c1"># 1. Models — `profile` holds an ObjectId in DB, a Profile in memory.</span>
<a id="__codelineno-0-12" name="__codelineno-0-12" href="#__codelineno-0-12"></a><span class="k">class</span><span class="w"> </span><span class="nc">Profile</span><span class="p">(</span><span class="n">BaseDocument</span><span class="p">):</span>
<a id="__codelineno-0-13" name="__codelineno-0-13" href="#__codelineno-0-13"></a> <span class="n">avatar_url</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>
<a id="__codelineno-0-14" name="__codelineno-0-14" href="#__codelineno-0-14"></a> <span class="n">bio</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>
<a id="__codelineno-0-15" name="__codelineno-0-15" href="#__codelineno-0-15"></a>
<a id="__codelineno-0-16" name="__codelineno-0-16" href="#__codelineno-0-16"></a>
<a id="__codelineno-0-17" name="__codelineno-0-17" href="#__codelineno-0-17"></a><span class="k">class</span><span class="w"> </span><span class="nc">User</span><span class="p">(</span><span class="n">BaseDocument</span><span class="p">):</span>
<a id="__codelineno-0-18" name="__codelineno-0-18" href="#__codelineno-0-18"></a> <span class="n">username</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>
<a id="__codelineno-0-19" name="__codelineno-0-19" href="#__codelineno-0-19"></a> <span class="n">email</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>
<a id="__codelineno-0-20" name="__codelineno-0-20" href="#__codelineno-0-20"></a> <span class="n">profile</span><span class="p">:</span> <span class="n">Profile</span> <span class="o">|</span> <span class="kc">None</span> <span class="o">=</span> <span class="kc">None</span>
<a id="__codelineno-0-21" name="__codelineno-0-21" href="#__codelineno-0-21"></a>
<a id="__codelineno-0-22" name="__codelineno-0-22" href="#__codelineno-0-22"></a>
<a id="__codelineno-0-23" name="__codelineno-0-23" href="#__codelineno-0-23"></a><span class="c1"># 2. Engine + rule.</span>
<a id="__codelineno-0-24" name="__codelineno-0-24" href="#__codelineno-0-24"></a><span class="n">engine</span> <span class="o">=</span> <span class="n">PopulationEngine</span><span class="p">({})</span>
<a id="__codelineno-0-25" name="__codelineno-0-25" href="#__codelineno-0-25"></a><span class="n">profile_rule</span> <span class="o">=</span> <span class="n">PopulateRule</span><span class="p">(</span><span class="n">field_name</span><span class="o">=</span><span class="s2">&quot;profile&quot;</span><span class="p">,</span> <span class="n">collection_name</span><span class="o">=</span><span class="s2">&quot;profiles&quot;</span><span class="p">)</span>
<a id="__codelineno-0-26" name="__codelineno-0-26" href="#__codelineno-0-26"></a>
<a id="__codelineno-0-27" name="__codelineno-0-27" href="#__codelineno-0-27"></a>
<a id="__codelineno-0-28" name="__codelineno-0-28" href="#__codelineno-0-28"></a><span class="c1"># 3. Composed repository.</span>
<a id="__codelineno-0-29" name="__codelineno-0-29" href="#__codelineno-0-29"></a><span class="k">class</span><span class="w"> </span><span class="nc">CachedUserRepository</span><span class="p">(</span><span class="n">CachedBaseRepository</span><span class="p">[</span><span class="n">User</span><span class="p">]):</span>
<a id="__codelineno-0-30" name="__codelineno-0-30" href="#__codelineno-0-30"></a> <span class="k">def</span><span class="w"> </span><span class="fm">__init__</span><span class="p">(</span>
<a id="__codelineno-0-31" name="__codelineno-0-31" href="#__codelineno-0-31"></a> <span class="bp">self</span><span class="p">,</span>
<a id="__codelineno-0-32" name="__codelineno-0-32" href="#__codelineno-0-32"></a> <span class="n">cache_backend</span><span class="p">:</span> <span class="n">InMemoryCacheBackend</span><span class="p">,</span>
<a id="__codelineno-0-33" name="__codelineno-0-33" href="#__codelineno-0-33"></a> <span class="n">population_engine</span><span class="p">:</span> <span class="n">PopulationEngine</span><span class="p">,</span>
<a id="__codelineno-0-34" name="__codelineno-0-34" href="#__codelineno-0-34"></a> <span class="n">populate_rules</span><span class="p">:</span> <span class="nb">list</span><span class="p">[</span><span class="n">PopulateRule</span><span class="p">],</span>
<a id="__codelineno-0-35" name="__codelineno-0-35" href="#__codelineno-0-35"></a> <span class="n">config</span><span class="p">:</span> <span class="n">CacheConfig</span> <span class="o">|</span> <span class="kc">None</span> <span class="o">=</span> <span class="kc">None</span><span class="p">,</span>
<a id="__codelineno-0-36" name="__codelineno-0-36" href="#__codelineno-0-36"></a> <span class="p">):</span>
<a id="__codelineno-0-37" name="__codelineno-0-37" href="#__codelineno-0-37"></a> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="s2">&quot;users&quot;</span><span class="p">,</span> <span class="n">User</span><span class="p">,</span> <span class="n">cache_backend</span><span class="p">,</span> <span class="n">config</span><span class="p">)</span>
<a id="__codelineno-0-38" name="__codelineno-0-38" href="#__codelineno-0-38"></a> <span class="bp">self</span><span class="o">.</span><span class="n">population_engine</span> <span class="o">=</span> <span class="n">population_engine</span>
<a id="__codelineno-0-39" name="__codelineno-0-39" href="#__codelineno-0-39"></a> <span class="bp">self</span><span class="o">.</span><span class="n">_populate_rules</span> <span class="o">=</span> <span class="n">populate_rules</span>
<a id="__codelineno-0-40" name="__codelineno-0-40" href="#__codelineno-0-40"></a>
<a id="__codelineno-0-41" name="__codelineno-0-41" href="#__codelineno-0-41"></a> <span class="k">async</span> <span class="k">def</span><span class="w"> </span><span class="nf">_populate</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">data</span><span class="p">:</span> <span class="nb">dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="nb">dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">]:</span>
<a id="__codelineno-0-42" name="__codelineno-0-42" href="#__codelineno-0-42"></a><span class="w"> </span><span class="sd">&quot;&quot;&quot;Dict-level resolution (mirrors PopulatingRepository._populate).&quot;&quot;&quot;</span>
<a id="__codelineno-0-43" name="__codelineno-0-43" href="#__codelineno-0-43"></a> <span class="k">for</span> <span class="n">rule</span> <span class="ow">in</span> <span class="bp">self</span><span class="o">.</span><span class="n">_populate_rules</span><span class="p">:</span>
<a id="__codelineno-0-44" name="__codelineno-0-44" href="#__codelineno-0-44"></a> <span class="n">ref</span> <span class="o">=</span> <span class="n">data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">rule</span><span class="o">.</span><span class="n">field_name</span><span class="p">)</span>
<a id="__codelineno-0-45" name="__codelineno-0-45" href="#__codelineno-0-45"></a> <span class="k">if</span> <span class="n">ref</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<a id="__codelineno-0-46" name="__codelineno-0-46" href="#__codelineno-0-46"></a> <span class="k">continue</span>
<a id="__codelineno-0-47" name="__codelineno-0-47" href="#__codelineno-0-47"></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ref</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span> <span class="c1"># JSON round-trip turns ObjectId into hex str</span>
<a id="__codelineno-0-48" name="__codelineno-0-48" href="#__codelineno-0-48"></a> <span class="n">ref</span> <span class="o">=</span> <span class="n">ObjectId</span><span class="p">(</span><span class="n">ref</span><span class="p">)</span>
<a id="__codelineno-0-49" name="__codelineno-0-49" href="#__codelineno-0-49"></a> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ref</span><span class="p">,</span> <span class="nb">list</span><span class="p">):</span>
<a id="__codelineno-0-50" name="__codelineno-0-50" href="#__codelineno-0-50"></a> <span class="n">resolved</span> <span class="o">=</span> <span class="p">[]</span>
<a id="__codelineno-0-51" name="__codelineno-0-51" href="#__codelineno-0-51"></a> <span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">ref</span><span class="p">:</span>
<a id="__codelineno-0-52" name="__codelineno-0-52" href="#__codelineno-0-52"></a> <span class="n">item</span> <span class="o">=</span> <span class="n">ObjectId</span><span class="p">(</span><span class="n">item</span><span class="p">)</span> <span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">item</span><span class="p">,</span> <span class="nb">str</span><span class="p">)</span> <span class="k">else</span> <span class="n">item</span>
<a id="__codelineno-0-53" name="__codelineno-0-53" href="#__codelineno-0-53"></a> <span class="n">repo</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">population_engine</span><span class="o">.</span><span class="n">_repos</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">rule</span><span class="o">.</span><span class="n">collection_name</span><span class="p">)</span>
<a id="__codelineno-0-54" name="__codelineno-0-54" href="#__codelineno-0-54"></a> <span class="n">doc</span> <span class="o">=</span> <span class="k">await</span> <span class="n">repo</span><span class="o">.</span><span class="n">get_by_id</span><span class="p">(</span><span class="n">item</span><span class="p">)</span> <span class="k">if</span> <span class="n">repo</span> <span class="k">else</span> <span class="kc">None</span>
<a id="__codelineno-0-55" name="__codelineno-0-55" href="#__codelineno-0-55"></a> <span class="n">resolved</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">doc</span><span class="p">)</span>
<a id="__codelineno-0-56" name="__codelineno-0-56" href="#__codelineno-0-56"></a> <span class="n">data</span><span class="p">[</span><span class="n">rule</span><span class="o">.</span><span class="n">field_name</span><span class="p">]</span> <span class="o">=</span> <span class="n">resolved</span>
<a id="__codelineno-0-57" name="__codelineno-0-57" href="#__codelineno-0-57"></a> <span class="k">elif</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">ref</span><span class="p">,</span> <span class="n">ObjectId</span><span class="p">):</span>
<a id="__codelineno-0-58" name="__codelineno-0-58" href="#__codelineno-0-58"></a> <span class="n">repo</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">population_engine</span><span class="o">.</span><span class="n">_repos</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">rule</span><span class="o">.</span><span class="n">collection_name</span><span class="p">)</span>
<a id="__codelineno-0-59" name="__codelineno-0-59" href="#__codelineno-0-59"></a> <span class="n">data</span><span class="p">[</span><span class="n">rule</span><span class="o">.</span><span class="n">field_name</span><span class="p">]</span> <span class="o">=</span> <span class="k">await</span> <span class="n">repo</span><span class="o">.</span><span class="n">get_by_id</span><span class="p">(</span><span class="n">ref</span><span class="p">)</span> <span class="k">if</span> <span class="n">repo</span> <span class="k">else</span> <span class="kc">None</span>
<a id="__codelineno-0-60" name="__codelineno-0-60" href="#__codelineno-0-60"></a> <span class="k">return</span> <span class="n">data</span>
<a id="__codelineno-0-61" name="__codelineno-0-61" href="#__codelineno-0-61"></a>
<a id="__codelineno-0-62" name="__codelineno-0-62" href="#__codelineno-0-62"></a> <span class="k">async</span> <span class="k">def</span><span class="w"> </span><span class="nf">get_by_id</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">id</span><span class="p">):</span>
<a id="__codelineno-0-63" name="__codelineno-0-63" href="#__codelineno-0-63"></a> <span class="k">if</span> <span class="ow">not</span> <span class="bp">self</span><span class="o">.</span><span class="n">_cache_config</span><span class="o">.</span><span class="n">enabled</span><span class="p">:</span>
<a id="__codelineno-0-64" name="__codelineno-0-64" href="#__codelineno-0-64"></a> <span class="k">return</span> <span class="k">await</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">get_by_id</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span> <span class="c1"># no cache -&gt; raw, unpopulated</span>
<a id="__codelineno-0-65" name="__codelineno-0-65" href="#__codelineno-0-65"></a>
<a id="__codelineno-0-66" name="__codelineno-0-66" href="#__codelineno-0-66"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-67" name="__codelineno-0-67" href="#__codelineno-0-67"></a><span class="c1"># 5️⃣ Register repositories with the engine for population resolution</span>
<a id="__codelineno-0-68" name="__codelineno-0-68" href="#__codelineno-0-68"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-69" name="__codelineno-0-69" href="#__codelineno-0-69"></a><span class="n">engine</span><span class="o">.</span><span class="n">register_repo</span><span class="p">(</span><span class="s2">&quot;profiles&quot;</span><span class="p">,</span> <span class="n">PopulatingRepository</span><span class="p">[</span><span class="n">Profile</span><span class="p">](</span><span class="s2">&quot;profiles&quot;</span><span class="p">,</span> <span class="n">Profile</span><span class="p">))</span>
<a id="__codelineno-0-70" name="__codelineno-0-70" href="#__codelineno-0-70"></a><span class="n">engine</span><span class="o">.</span><span class="n">register_repo</span><span class="p">(</span><span class="s2">&quot;users&quot;</span><span class="p">,</span> <span class="n">UserRepository</span><span class="p">())</span>
<a id="__codelineno-0-66" name="__codelineno-0-66" href="#__codelineno-0-66"></a> <span class="n">cached</span> <span class="o">=</span> <span class="k">await</span> <span class="bp">self</span><span class="o">.</span><span class="n">_cache</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_cache_key</span><span class="p">(</span><span class="nb">id</span><span class="p">))</span>
<a id="__codelineno-0-67" name="__codelineno-0-67" href="#__codelineno-0-67"></a> <span class="k">if</span> <span class="n">cached</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<a id="__codelineno-0-68" name="__codelineno-0-68" href="#__codelineno-0-68"></a> <span class="c1"># Cache hit: decode the RAW doc, then populate before returning.</span>
<a id="__codelineno-0-69" name="__codelineno-0-69" href="#__codelineno-0-69"></a> <span class="n">data</span> <span class="o">=</span> <span class="k">await</span> <span class="bp">self</span><span class="o">.</span><span class="n">_populate</span><span class="p">(</span><span class="n">decode_value</span><span class="p">(</span><span class="n">cached</span><span class="p">))</span>
<a id="__codelineno-0-70" name="__codelineno-0-70" href="#__codelineno-0-70"></a> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">model</span><span class="p">(</span><span class="o">**</span><span class="n">data</span><span class="p">)</span>
<a id="__codelineno-0-71" name="__codelineno-0-71" href="#__codelineno-0-71"></a>
<a id="__codelineno-0-72" name="__codelineno-0-72" href="#__codelineno-0-72"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-73" name="__codelineno-0-73" href="#__codelineno-0-73"></a><span class="c1"># 6️⃣ FastAPI lifespan – initialise DB, create indexes, and start cache cleanup</span>
<a id="__codelineno-0-74" name="__codelineno-0-74" href="#__codelineno-0-74"></a><span class="c1"># ----------------------------------------------------------------------</span>
<a id="__codelineno-0-75" name="__codelineno-0-75" href="#__codelineno-0-75"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops</span><span class="w"> </span><span class="kn">import</span> <span class="n">MongoConnectionManager</span>
<a id="__codelineno-0-76" name="__codelineno-0-76" href="#__codelineno-0-76"></a><span class="kn">from</span><span class="w"> </span><span class="nn">contextlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">asynccontextmanager</span>
<a id="__codelineno-0-72" name="__codelineno-0-72" href="#__codelineno-0-72"></a> <span class="c1"># Cache miss: single DB read; the base class caches the raw (depopulated) doc.</span>
<a id="__codelineno-0-73" name="__codelineno-0-73" href="#__codelineno-0-73"></a> <span class="n">result</span> <span class="o">=</span> <span class="k">await</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">get_by_id</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
<a id="__codelineno-0-74" name="__codelineno-0-74" href="#__codelineno-0-74"></a> <span class="k">if</span> <span class="n">result</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<a id="__codelineno-0-75" name="__codelineno-0-75" href="#__codelineno-0-75"></a> <span class="k">return</span> <span class="kc">None</span>
<a id="__codelineno-0-76" name="__codelineno-0-76" href="#__codelineno-0-76"></a> <span class="k">return</span> <span class="k">await</span> <span class="bp">self</span><span class="o">.</span><span class="n">model</span><span class="p">(</span><span class="o">**</span><span class="k">await</span> <span class="bp">self</span><span class="o">.</span><span class="n">_populate</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">model_dump</span><span class="p">(</span><span class="n">by_alias</span><span class="o">=</span><span class="kc">True</span><span class="p">)))</span>
<a id="__codelineno-0-77" name="__codelineno-0-77" href="#__codelineno-0-77"></a>
<a id="__codelineno-0-78" name="__codelineno-0-78" href="#__codelineno-0-78"></a><span class="nd">@asynccontextmanager</span>
<a id="__codelineno-0-79" name="__codelineno-0-79" href="#__codelineno-0-79"></a><span class="k">async</span> <span class="k">def</span><span class="w"> </span><span class="nf">lifespan</span><span class="p">(</span><span class="n">app</span><span class="p">):</span>
<a id="__codelineno-0-80" name="__codelineno-0-80" href="#__codelineno-0-80"></a> <span class="k">async</span> <span class="k">with</span> <span class="n">MongoConnectionManager</span><span class="o">.</span><span class="n">lifespan</span><span class="p">(</span>
<a id="__codelineno-0-81" name="__codelineno-0-81" href="#__codelineno-0-81"></a> <span class="n">uri</span><span class="o">=</span><span class="s2">&quot;mongodb://localhost:27017&quot;</span><span class="p">,</span>
<a id="__codelineno-0-82" name="__codelineno-0-82" href="#__codelineno-0-82"></a> <span class="n">db_name</span><span class="o">=</span><span class="s2">&quot;app_db&quot;</span><span class="p">,</span>
<a id="__codelineno-0-83" name="__codelineno-0-83" href="#__codelineno-0-83"></a> <span class="p">):</span>
<a id="__codelineno-0-84" name="__codelineno-0-84" href="#__codelineno-0-84"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">initialize_all</span><span class="p">()</span>
<a id="__codelineno-0-85" name="__codelineno-0-85" href="#__codelineno-0-85"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">initialize_cache</span><span class="p">()</span>
<a id="__codelineno-0-86" name="__codelineno-0-86" href="#__codelineno-0-86"></a> <span class="k">yield</span>
<a id="__codelineno-0-78" name="__codelineno-0-78" href="#__codelineno-0-78"></a>
<a id="__codelineno-0-79" name="__codelineno-0-79" href="#__codelineno-0-79"></a><span class="c1"># 4. Wire-up — one backend for both the repo and the registry lifecycle.</span>
<a id="__codelineno-0-80" name="__codelineno-0-80" href="#__codelineno-0-80"></a><span class="n">cache</span> <span class="o">=</span> <span class="n">InMemoryCacheBackend</span><span class="p">(</span><span class="n">max_entries</span><span class="o">=</span><span class="mi">20_000</span><span class="p">,</span> <span class="n">default_ttl</span><span class="o">=</span><span class="mi">600</span><span class="p">)</span>
<a id="__codelineno-0-81" name="__codelineno-0-81" href="#__codelineno-0-81"></a><span class="n">ModelRegistry</span><span class="o">.</span><span class="n">set_cache_backend</span><span class="p">(</span><span class="n">cache</span><span class="p">)</span>
<a id="__codelineno-0-82" name="__codelineno-0-82" href="#__codelineno-0-82"></a>
<a id="__codelineno-0-83" name="__codelineno-0-83" href="#__codelineno-0-83"></a>
<a id="__codelineno-0-84" name="__codelineno-0-84" href="#__codelineno-0-84"></a><span class="nd">@asynccontextmanager</span>
<a id="__codelineno-0-85" name="__codelineno-0-85" href="#__codelineno-0-85"></a><span class="k">async</span> <span class="k">def</span><span class="w"> </span><span class="nf">lifespan</span><span class="p">(</span><span class="n">_app</span><span class="p">:</span> <span class="n">FastAPI</span><span class="p">):</span>
<a id="__codelineno-0-86" name="__codelineno-0-86" href="#__codelineno-0-86"></a> <span class="k">async</span> <span class="k">with</span> <span class="n">MongoConnectionManager</span><span class="o">.</span><span class="n">lifespan</span><span class="p">(</span>
<a id="__codelineno-0-87" name="__codelineno-0-87" href="#__codelineno-0-87"></a> <span class="n">uri</span><span class="o">=</span><span class="s2">&quot;mongodb://localhost:27017&quot;</span><span class="p">,</span> <span class="n">db_name</span><span class="o">=</span><span class="s2">&quot;app_db&quot;</span>
<a id="__codelineno-0-88" name="__codelineno-0-88" href="#__codelineno-0-88"></a> <span class="p">):</span>
<a id="__codelineno-0-89" name="__codelineno-0-89" href="#__codelineno-0-89"></a> <span class="c1"># Register the repositories the engine resolves refs against:</span>
<a id="__codelineno-0-90" name="__codelineno-0-90" href="#__codelineno-0-90"></a> <span class="c1"># engine.register_repo(&quot;profiles&quot;, ProfileRepo())</span>
<a id="__codelineno-0-91" name="__codelineno-0-91" href="#__codelineno-0-91"></a> <span class="n">engine</span><span class="o">.</span><span class="n">register_repo</span><span class="p">(</span><span class="s2">&quot;users&quot;</span><span class="p">,</span> <span class="n">CachedUserRepository</span><span class="p">(</span><span class="n">cache</span><span class="p">,</span> <span class="n">engine</span><span class="p">,</span> <span class="p">[</span><span class="n">profile_rule</span><span class="p">]))</span>
<a id="__codelineno-0-92" name="__codelineno-0-92" href="#__codelineno-0-92"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">initialize_all</span><span class="p">()</span>
<a id="__codelineno-0-93" name="__codelineno-0-93" href="#__codelineno-0-93"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">initialize_cache</span><span class="p">()</span>
<a id="__codelineno-0-94" name="__codelineno-0-94" href="#__codelineno-0-94"></a> <span class="k">yield</span>
<a id="__codelineno-0-95" name="__codelineno-0-95" href="#__codelineno-0-95"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">shutdown_cache</span><span class="p">()</span>
<a id="__codelineno-0-96" name="__codelineno-0-96" href="#__codelineno-0-96"></a>
<a id="__codelineno-0-97" name="__codelineno-0-97" href="#__codelineno-0-97"></a>
<a id="__codelineno-0-98" name="__codelineno-0-98" href="#__codelineno-0-98"></a><span class="n">app</span> <span class="o">=</span> <span class="n">FastAPI</span><span class="p">(</span><span class="n">lifespan</span><span class="o">=</span><span class="n">lifespan</span><span class="p">)</span>
</code></pre></div>
<blockquote>
<p><strong>Note:</strong> replace the placeholder <code>engine.register_repo("profiles", ...)</code> line with a repository for <code>Profile</code>. A matching <code>CachedBaseRepository[Profile]("profiles", Profile, cache)</code> keeps profile reads cached too — e.g. <code>engine.register_repo("profiles", CachedBaseRepository[Profile]("profiles", Profile, cache))</code>.</p>
</blockquote>
<hr />
<h2 id="tips">💡 Tips</h2>
<ul>
<li>The example overrides <code>get_by_id</code> to keep the caching logic simple; you could also create a mixin that composes <code>CachedBaseRepository</code> and <code>PopulatingRepository</code>.</li>
<li>Remember to call <code>await ModelRegistry.initialize_cache()</code> <strong>after</strong> the DB connection is ready; otherwise the background cleanup task will never start.</li>
<li>Cache keys are generated by <code>CachedBaseRepository._cache_key(id)</code>. If you change the repository’s <code>config.key_prefix</code>, the same prefix will be used for populated results.</li>
<li><strong>Cache the raw doc, populate at read time.</strong> Ref fields stay <code>ObjectId</code>s in the cache; JSON round-trips them to hex strings, so <code>_populate</code> must normalize <code>str → ObjectId</code> (shown above).</li>
<li><code>update</code>/<code>delete</code> from the base class still invalidate the right key because we never repopulate <em>in</em> the cache.</li>
<li>Set a sensible <code>default_ttl</code> — cached <code>User</code> entries resolve <code>Profile</code> on each read, so profile edits show up within the TTL (or call <code>invalidate_cache(user_id)</code> explicitly).</li>
<li>If you want one repo class used app-wide, keep the populate helpers in a mixin shared with regular <code>PopulatingRepository</code>.</li>
</ul>
<hr />
<h2 id="related">Related</h2>
<ul>
<li><a href="../07_caching/">07 – Caching</a> · <a href="../08_population/">08 – Population</a> · <a href="../11_cache_lifecycle/">11 – Cache lifecycle</a></li>
</ul>