Cosmos DB Agent Kit: Auditing My Own Sample Repo

Posted on by

A bonus post, series-adjacent, running the Cosmos DB Agent Kit against cosmos-agent-memory-lab to see what six posts of hand-checked rules missed.

Every rule in this series hierarchical partition keys, TTL modes, vector index types, the composite ID trick came from reading Microsoft Learn docs and debugging real errors one at a time. The Cosmos DB Agent Kit packages that same category of knowledge differently: 100+ best-practice rules across 12 categories that AI coding agents apply while writing or reviewing Cosmos DB code. This post checks the rules against the actual sample repo behind posts 2 through 4, rule by rule, instead of taking the kit’s word for it.

What the Cosmos DB Agent Kit Actually Ships

It’s not a linter or a static-analysis tool. It’s a set of Markdown rule files, one per practice, each with an incorrect example, a corrected example, and a short explanation of why, following the Agent Skills format, so tools like Claude Code, GitHub Copilot, and Gemini CLI can load them and apply them while generating or reviewing code. npx skills add AzureCosmosDB/cosmosdb-agent-kit installs it. The categories track the same ground this series covered: data modeling and partition keys rank CRITICAL, queries and SDK usage rank HIGH, vector search and full-text search get their own dedicated categories.

Nine Things the Repo Already Gets Right

Vector, full-text, and hybrid search behave differently against the same container. A practical guide to Cosmos DB agent memory search — quantizedFlat vs. DiskANN, sharded vector indexes, BM25, and RRF — with four SQL patterns run against the same question. Post 3 of 6.

Cross-checking models.py, schema.py, and checkpointer.py against the kit’s rule files turned up a solid list of matches. The hierarchical partition key orders tenantId before threadId, broad to narrow, exactly as partition-hierarchical recommends. The composite turn ID uses: as a separator — the kit’s model-id-constraints rule calls out #, ?, /, and \ as characters that break Cosmos DB’s REST auth signing, and: sits on its explicit safe list. TTL follows the kit’s own “correct” pattern precisely: container default -1, item-level ttl overriding it per turn. The vector embedding policy and DiskANN index both match vector-embedding-policy and vector-index-type field for field, and the indexing policy excludes the embedding path from the regular range index to avoid double-indexing cost.

The full-text policy uses en-US, case-sensitive, as the kit’s fts-define-policy rule insists, and the content field sits in fullTextIndexes without also cluttering excludedPaths incorrectly. Turns live in their own container, separate from checkpoints, precisely the pattern-langgraph-chat-history-separate rule, which exists because checkpoint blobs make poor chat history. And checkpointer.py reaches for a point read wherever it already has both the id and the partition key, rather than running a query that costs roughly 2.5x more.

Three Things It Would Flag

Not everything cleared. seed.py never normalizes its mock embeddings to unit length vector-normalize-embeddings flags exactly this, and it’s a genuine miss, not a style preference: unnormalized vectors produce inconsistent cosine-similarity scores, and the fix is one line (v / ||v||₂) that never made it into the SHA256-based fake_embedding() function. Second, the checkpointer is a hand-rolled BaseCheckpointSaver implementation against the synchronous azure-cosmos SDK, not the official async CosmosDBSaver from langchain-azure-cosmosdb that sdk-langchain-cosmosdb-saver recommends. That one’s a deliberate tradeoff rather than an oversight — building it by hand is what made post 4’s checkpointing section possible to explain from the inside. Still, a production app should almost certainly reach for the maintained package instead. Third, turn upserts carry no ETag check so that sdk-etag-concurrency would flag the read-modify-write path as vulnerable to lost updates under concurrent writes. The demo’s single-writer pattern never triggers it, but two agents racing to update the same turn would silently drop one of them.

Where a Rulebook Like This Actually Helps

A tool that catches “your mock embeddings aren’t normalized” in seconds, instead of after a confusing test failure, earns its place in a workflow. What it doesn’t replace is the reason each rule exists — Cosmos DB Agent Kit tells you: is a safe ID separator, but this series spent a paragraph on why # breaks HMAC signing specifically in Gateway mode. Both matter: the rulebook for speed, the explanation for judgment calls like the checkpointer tradeoff above, which no automated check can make for you. Six posts of hand-checked Cosmos DB rules and a 100-rule kit landed on the same answers almost everywhere — that convergence is worth more than either source alone.

Sources

Leave a Reply