docs: collect refreshed mongo-ops wiki site (API-correct overhaul, v0.1.5)
This commit is contained in:
@@ -457,7 +457,7 @@
|
||||
|
||||
|
||||
<span class="md-ellipsis">
|
||||
02 components
|
||||
Core Components
|
||||
|
||||
</span>
|
||||
|
||||
@@ -805,7 +805,7 @@
|
||||
<li class="md-nav__item">
|
||||
<a href="#whats-new" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
📦 What’s New?
|
||||
📦 What's New?
|
||||
</span>
|
||||
</a>
|
||||
|
||||
@@ -818,21 +818,15 @@
|
||||
</span>
|
||||
</a>
|
||||
|
||||
<nav class="md-nav" aria-label="🚀 Example">
|
||||
<ul class="md-nav__list">
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#why-a-separate-shutdown-step" class="md-nav__link">
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#what-actually-happens" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
Why a separate shutdown step?
|
||||
🔁 What Actually Happens
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
</nav>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
@@ -842,6 +836,15 @@
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
<li class="md-nav__item">
|
||||
<a href="#related" class="md-nav__link">
|
||||
<span class="md-ellipsis">
|
||||
Related
|
||||
</span>
|
||||
</a>
|
||||
|
||||
</li>
|
||||
|
||||
</ul>
|
||||
@@ -1042,10 +1045,9 @@
|
||||
|
||||
|
||||
<h1 id="use-case-11-proper-cache-lifecycle-in-fastapi">Use Case 11: Proper Cache Lifecycle in FastAPI</h1>
|
||||
<p><strong>Scenario:</strong>
|
||||
A microservice uses <code>InMemoryCacheBackend</code> (or Redis) and needs to start the cache cleanup task when the app starts, then shut it down cleanly on termination.</p>
|
||||
<p><strong>Scenario:</strong> A microservice uses an in-memory (or Redis) cache backend and must start the cleanup task on app start, then shut it down cleanly on termination.</p>
|
||||
<hr />
|
||||
<h2 id="whats-new">📦 What’s New?</h2>
|
||||
<h2 id="whats-new">📦 What's New?</h2>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
@@ -1055,55 +1057,69 @@ A microservice uses <code>InMemoryCacheBackend</code> (or Redis) and needs to st
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>ModelRegistry.set_cache_backend(backend)</code></td>
|
||||
<td>Registers the single shared backend.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ModelRegistry.initialize_cache()</code></td>
|
||||
<td>Starts the async background task for the registered cache backend.</td>
|
||||
<td>Starts the backend (spawns the in-memory TTL cleanup task). Raises <code>RuntimeError</code> if no backend was registered.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ModelRegistry.shutdown_cache()</code></td>
|
||||
<td>Gracefully stops the background task and releases resources.</td>
|
||||
<td>Cancels the cleanup task, closes pub/sub, clears the backend.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>FastAPI lifespan integration</td>
|
||||
<td>Demonstrates where to call both init and shutdown methods.</td>
|
||||
<td>If a repo uses a backend directly</td>
|
||||
<td>Same instance must be registered so <code>initialize_cache</code> starts <em>its</em> task.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<hr />
|
||||
<h2 id="example">🚀 Example</h2>
|
||||
<div class="language-python highlight"><span class="filename">Python</span><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops</span><span class="w"> </span><span class="kn">import</span> <span class="n">MongoConnectionManager</span><span class="p">,</span> <span class="n">ModelRegistry</span>
|
||||
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops.cache</span><span class="w"> </span><span class="kn">import</span> <span class="n">InMemoryCacheBackend</span>
|
||||
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a><span class="kn">from</span><span class="w"> </span><span class="nn">contextlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">asynccontextmanager</span>
|
||||
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a>
|
||||
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a><span class="c1"># Initialise a cache backend (in‑memory example)</span>
|
||||
<a id="__codelineno-0-6" name="__codelineno-0-6" href="#__codelineno-0-6"></a><span class="n">cache</span> <span class="o">=</span> <span class="n">InMemoryCacheBackend</span><span class="p">(</span><span class="n">max_entries</span><span class="o">=</span><span class="mi">10_000</span><span class="p">,</span> <span class="n">default_ttl</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-7" name="__codelineno-0-7" href="#__codelineno-0-7"></a><span class="n">ModelRegistry</span><span class="o">.</span><span class="n">set_cache_backend</span><span class="p">(</span><span class="n">cache</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-8" name="__codelineno-0-8" href="#__codelineno-0-8"></a>
|
||||
<a id="__codelineno-0-9" name="__codelineno-0-9" href="#__codelineno-0-9"></a><span class="nd">@asynccontextmanager</span>
|
||||
<a id="__codelineno-0-10" name="__codelineno-0-10" href="#__codelineno-0-10"></a><span class="k">async</span> <span class="k">def</span><span class="w"> </span><span class="nf">lifespan</span><span class="p">(</span><span class="n">app</span><span class="p">):</span>
|
||||
<a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a> <span class="c1"># Start MongoDB connection and cache background task</span>
|
||||
<a id="__codelineno-0-12" name="__codelineno-0-12" href="#__codelineno-0-12"></a> <span class="k">async</span> <span class="k">with</span> <span class="n">MongoConnectionManager</span><span class="o">.</span><span class="n">lifespan</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-13" name="__codelineno-0-13" href="#__codelineno-0-13"></a> <span class="n">uri</span><span class="o">=</span><span class="s2">"mongodb://localhost:27017"</span><span class="p">,</span>
|
||||
<a id="__codelineno-0-14" name="__codelineno-0-14" href="#__codelineno-0-14"></a> <span class="n">db_name</span><span class="o">=</span><span class="s2">"mydb"</span><span class="p">,</span>
|
||||
<div class="language-python highlight"><span class="filename">Python</span><pre><span></span><code><a id="__codelineno-0-1" name="__codelineno-0-1" href="#__codelineno-0-1"></a><span class="kn">from</span><span class="w"> </span><span class="nn">contextlib</span><span class="w"> </span><span class="kn">import</span> <span class="n">asynccontextmanager</span>
|
||||
<a id="__codelineno-0-2" name="__codelineno-0-2" href="#__codelineno-0-2"></a>
|
||||
<a id="__codelineno-0-3" name="__codelineno-0-3" href="#__codelineno-0-3"></a><span class="kn">from</span><span class="w"> </span><span class="nn">fastapi</span><span class="w"> </span><span class="kn">import</span> <span class="n">FastAPI</span>
|
||||
<a id="__codelineno-0-4" name="__codelineno-0-4" href="#__codelineno-0-4"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops</span><span class="w"> </span><span class="kn">import</span> <span class="n">ModelRegistry</span><span class="p">,</span> <span class="n">MongoConnectionManager</span>
|
||||
<a id="__codelineno-0-5" name="__codelineno-0-5" href="#__codelineno-0-5"></a><span class="kn">from</span><span class="w"> </span><span class="nn">mongo_ops.cache</span><span class="w"> </span><span class="kn">import</span> <span class="n">InMemoryCacheBackend</span>
|
||||
<a id="__codelineno-0-6" name="__codelineno-0-6" href="#__codelineno-0-6"></a>
|
||||
<a id="__codelineno-0-7" name="__codelineno-0-7" href="#__codelineno-0-7"></a><span class="n">cache</span> <span class="o">=</span> <span class="n">InMemoryCacheBackend</span><span class="p">(</span><span class="n">max_entries</span><span class="o">=</span><span class="mi">10_000</span><span class="p">,</span> <span class="n">default_ttl</span><span class="o">=</span><span class="mi">300</span><span class="p">)</span>
|
||||
<a id="__codelineno-0-8" name="__codelineno-0-8" href="#__codelineno-0-8"></a><span class="n">ModelRegistry</span><span class="o">.</span><span class="n">set_cache_backend</span><span class="p">(</span><span class="n">cache</span><span class="p">)</span> <span class="c1"># before any cache-backed repo is used</span>
|
||||
<a id="__codelineno-0-9" name="__codelineno-0-9" href="#__codelineno-0-9"></a>
|
||||
<a id="__codelineno-0-10" name="__codelineno-0-10" href="#__codelineno-0-10"></a>
|
||||
<a id="__codelineno-0-11" name="__codelineno-0-11" href="#__codelineno-0-11"></a><span class="nd">@asynccontextmanager</span>
|
||||
<a id="__codelineno-0-12" name="__codelineno-0-12" href="#__codelineno-0-12"></a><span class="k">async</span> <span class="k">def</span><span class="w"> </span><span class="nf">lifespan</span><span class="p">(</span><span class="n">_app</span><span class="p">:</span> <span class="n">FastAPI</span><span class="p">):</span>
|
||||
<a id="__codelineno-0-13" name="__codelineno-0-13" href="#__codelineno-0-13"></a> <span class="k">async</span> <span class="k">with</span> <span class="n">MongoConnectionManager</span><span class="o">.</span><span class="n">lifespan</span><span class="p">(</span>
|
||||
<a id="__codelineno-0-14" name="__codelineno-0-14" href="#__codelineno-0-14"></a> <span class="n">uri</span><span class="o">=</span><span class="s2">"mongodb://localhost:27017"</span><span class="p">,</span> <span class="n">db_name</span><span class="o">=</span><span class="s2">"mydb"</span>
|
||||
<a id="__codelineno-0-15" name="__codelineno-0-15" href="#__codelineno-0-15"></a> <span class="p">):</span>
|
||||
<a id="__codelineno-0-16" name="__codelineno-0-16" href="#__codelineno-0-16"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">initialize_all</span><span class="p">()</span> <span class="c1"># create indexes</span>
|
||||
<a id="__codelineno-0-17" name="__codelineno-0-17" href="#__codelineno-0-17"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">initialize_cache</span><span class="p">()</span> <span class="c1"># start cache cleanup</span>
|
||||
<a id="__codelineno-0-16" name="__codelineno-0-16" href="#__codelineno-0-16"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">initialize_all</span><span class="p">()</span> <span class="c1"># create indexes (idempotent)</span>
|
||||
<a id="__codelineno-0-17" name="__codelineno-0-17" href="#__codelineno-0-17"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">initialize_cache</span><span class="p">()</span> <span class="c1"># start the TTL cleanup task</span>
|
||||
<a id="__codelineno-0-18" name="__codelineno-0-18" href="#__codelineno-0-18"></a> <span class="k">yield</span>
|
||||
<a id="__codelineno-0-19" name="__codelineno-0-19" href="#__codelineno-0-19"></a> <span class="c1"># FastAPI will exit the `with` block here – clean up cache</span>
|
||||
<a id="__codelineno-0-20" name="__codelineno-0-20" href="#__codelineno-0-20"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">shutdown_cache</span><span class="p">()</span>
|
||||
<a id="__codelineno-0-19" name="__codelineno-0-19" href="#__codelineno-0-19"></a> <span class="k">await</span> <span class="n">ModelRegistry</span><span class="o">.</span><span class="n">shutdown_cache</span><span class="p">()</span> <span class="c1"># cancel task + close cleanly</span>
|
||||
<a id="__codelineno-0-20" name="__codelineno-0-20" href="#__codelineno-0-20"></a>
|
||||
<a id="__codelineno-0-21" name="__codelineno-0-21" href="#__codelineno-0-21"></a>
|
||||
<a id="__codelineno-0-22" name="__codelineno-0-22" href="#__codelineno-0-22"></a><span class="n">app</span> <span class="o">=</span> <span class="n">FastAPI</span><span class="p">(</span><span class="n">lifespan</span><span class="o">=</span><span class="n">lifespan</span><span class="p">)</span>
|
||||
</code></pre></div>
|
||||
<h3 id="why-a-separate-shutdown-step">Why a separate shutdown step?</h3>
|
||||
<hr />
|
||||
<h2 id="what-actually-happens">🔁 What Actually Happens</h2>
|
||||
<ul>
|
||||
<li>The cache backend may spawn an <code>asyncio.Task</code> that periodically removes expired entries. If the task is left dangling, the event loop may complain about pending tasks on shutdown.</li>
|
||||
<li><code>shutdown_cache()</code> cancels the internal task and waits for it to finish, ensuring a clean exit.</li>
|
||||
<li><code>InMemoryCacheBackend.initialize()</code> spawns an <code>asyncio.Task</code> that evicts expired entries every <code>cleanup_interval</code> seconds. Without a shutdown, the event loop flags the dangling task on exit — <code>shutdown_cache()</code> cancels it and awaits it.</li>
|
||||
<li><code>shutdown_cache()</code> also NULs the registry cache backend and closes the Redis pub/sub handle (if Redis).</li>
|
||||
<li>If <code>initialize_cache()</code> is called before <code>set_cache_backend()</code>, it raises:</li>
|
||||
</ul>
|
||||
<div class="language-text highlight"><span class="filename">Text Only</span><pre><span></span><code><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a>RuntimeError: No cache backend registered. Call set_cache_backend() first.
|
||||
</code></pre></div>
|
||||
<hr />
|
||||
<h2 id="tips">💡 Tips</h2>
|
||||
<ul>
|
||||
<li>Register the cache <strong>before</strong> calling <code>initialize_cache()</code>; otherwise the registry won’t know which backend to start.</li>
|
||||
<li>For Redis backends, the shutdown step also closes the underlying <code>redis.asyncio.Redis</code> client connection.</li>
|
||||
<li>You can also hook the shutdown into a FastAPI <code>@app.on_event("shutdown")</code> handler if you prefer not to use the lifespan context manager.</li>
|
||||
<li>Per-repository toggling is independent of the lifecycle: <code>CacheConfig(enabled=False)</code> bypasses the cache for that repo even after <code>initialize_cache()</code>.</li>
|
||||
<li>The cleanup task uses <code>cleanup_interval</code> seconds for scans; entries also expire on access via the TTL heap (<code>default_ttl=0</code> expires immediately).</li>
|
||||
<li>Register the backend <strong>before</strong> constructing any <code>CachedBaseRepository</code> that references it — otherwise a repo may hold an uninitialized backend (no cleanup task, no Redis pub/sub).</li>
|
||||
</ul>
|
||||
<hr />
|
||||
<h2 id="related">Related</h2>
|
||||
<ul>
|
||||
<li><a href="../07_caching/">07 – Caching</a> · <a href="../../02_components/">Components: ModelRegistry</a> · <a href="../10_cache_and_population/">10 – Cache + population</a></li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user