Use Case 8: Document Population

Scenario:
An API needs to return a single, denormalised JSON payload that includes related documents (e.g., a User with an embedded Profile, or an Order with its LineItems) without the caller having to issue multiple requests.

---

📦 What’s New?

Component	Description
PopulateRule	Declarative rule that tells the engine how to populate a field. Key attributes: • field_name – Target attribute on the source model. • collection_name – Collection that stores the referenced documents. • ref_field – Name of the reference field (normally an ObjectId). • Optional nested_rules for deep population, max_depth to bound recursion, and filter / projection for fine‑grained queries.
PopulationEngine	Core engine that resolves a list of PopulateRules recursively. It: • Detects circular references and raises CircularReferenceError. • Honors per‑rule max_depth and a global global_max_depth (default 10). • Supports MongoDB filters and projections for each populated relation.
PopulatingRepository	Extends BaseRepository with automatic population support. Stores a reference to a PopulationEngine and a list of PopulateRules, exposing two helpers: • _populate(document) – Returns the same document with the requested relations populated. • _depopulate(document) – Strips populated fields (useful before serialisation).
Cache‑aware population (optional) – The engine works with any CacheBackend. When used together with CachedBaseRepository, populated documents benefit from caching in the same way as regular CRUD results.

---

🚀 Quick Example

Python

from bson import ObjectId
from mongo_ops import BaseDocument, PopulatingRepository, ModelRegistry
from mongo_ops.populate import PopulateRule, PopulationEngine
from mongo_ops.cache import InMemoryCacheBackend

# ----------------------------------------------------------------------
# 1️⃣ Define the data models
# ----------------------------------------------------------------------
class Profile(BaseDocument):
    avatar_url: str = ""
    bio: str = ""

class User(BaseDocument):
    username: str = ""
    email: str = ""
    profile_id: ObjectId = None          # Reference to Profile
    profile: Profile | None = None       # Populated field (filled by engine)

# ----------------------------------------------------------------------
# 2️⃣ (Optional) Set up a cache; helpful for deep graphs
# ----------------------------------------------------------------------
cache = InMemoryCacheBackend(max_entries=10_000, default_ttl=300)
ModelRegistry.set_cache_backend(cache)

# ----------------------------------------------------------------------
# 3️⃣ Create a PopulationEngine – repositories will be registered later
# ----------------------------------------------------------------------
engine = PopulationEngine({})

# ----------------------------------------------------------------------
# 4️⃣ Declare how the `profile` field should be populated
# ----------------------------------------------------------------------
profile_rule = PopulateRule(
    field_name="profile",
    collection_name="profiles",
    ref_field="profile_id",
    # No further nesting in this simple example
)

# ----------------------------------------------------------------------
# 5️⃣ Repository that uses the engine + rule
# ----------------------------------------------------------------------
class UserRepository(PopulatingRepository[User]):
    def __init__(self):
        super().__init__(
            collection_name="users",
            model=User,
            population_engine=engine,
            populate_rules=[profile_rule],
        )

# ----------------------------------------------------------------------
# 6️⃣ Register the repository with the engine so it can resolve refs
# ----------------------------------------------------------------------
engine.register_repo("profiles", PopulatingRepository[Profile]("profiles", Profile))
engine.register_repo("users", UserRepository())

# ----------------------------------------------------------------------
# 7️⃣ FastAPI lifespan – initialise DB, indexes, and the cache
# ----------------------------------------------------------------------
from mongo_ops import MongoConnectionManager
from contextlib import asynccontextmanager

@asynccontextmanager
async def lifespan(app):
    async with MongoConnectionManager.lifespan(
        uri="mongodb://localhost:27017",
        db_name="mydb"
    ):
        await ModelRegistry.initialize_all()
        # If you set up a cache backend (see step 2), initialize it here
        await ModelRegistry.initialize_cache()
        yield