Daniel sent us this one, and it's actually three questions folded into one. He's asking about the operational nightmare of embedding model deprecation in RAG applications — what happens when the model you built your entire vector index on gets retired. Then he wants to know whether the Model Context Protocol with dynamic database querying could sidestep the whole problem by keeping data in regular databases and exposing it through an API. And finally, he's curious about caching embeddings client-side as a middle ground. There's a lot to unpack here, and honestly this hits something I've been chewing on for a while.
Oh, this is such a rich topic. And before we dive in — quick note, today's script is being generated by DeepSeek V four Pro. Which feels appropriate, given we're about to talk about models getting deprecated.
Alright, so let's start with the core problem. Embedding model deprecation. Herman, you've been following this space closely. How bad does it actually get?
It gets bad in ways most teams don't anticipate. There was a piece by Ricardo Ferreira last November where he documented a real case — a company that built their RAG pipeline on OpenAI's ada-002 embedding model. When that model got deprecated, they faced a forty thousand dollar bill just to re-embed all their data. Not forty thousand in API costs — that part was actually manageable. The real hit was two full weeks of developer time for the migration, plus all the management friction of justifying the expense. And here's the kicker: the new model had different embedding dimensions and produced different similarity scores, which meant their entire index was broken. It wasn't a drop-in replacement.
It's not just like swapping out a library version. The whole retrieval layer breaks.
And it's worse than most people realize because the failure is silent. There was a detailed analysis on dbi services earlier this year — February twenty twenty-six — that called stale embeddings the silent killer of RAG in production. Their point was that almost nobody measures retrieval quality. No precision at K, no normalized discounted cumulative gain, no confidence scoring. Nobody complains about the embedding pipeline — they complain that the chatbot gives wrong answers. And by the time you trace it back to stale embeddings, the trust is already gone. The users have already decided the system is unreliable.
That's the part that makes me twitchy. You don't get a crash. You don't get an error log. You just get subtly worse results, and nobody notices until it's too late.
And the LlamaIndex team has this RAG failure mode checklist that specifically calls out config drift — what they call embedding mismatch — as one of the primary ways deployments fail silently. They recommend pinning model versions explicitly. So instead of saying "use the latest embedding model," you specify text-embedding-ada-002 or whatever version you built on. And you store that model name in your metadata so you can detect drift. But here's the thing — most teams don't do that. They just call the default endpoint and assume it'll work forever.
What's the actual recommended fix? Because batch re-embedding everything periodically sounds like a brute-force approach that's going to have its own problems.
And the dbi services post tore into this. For a corpus of fifty thousand documents — roughly five hundred thousand chunks — a full re-embed costs about five dollars in API calls. That's trivial. But the operational complexity is not trivial. During the backfill, your index is in a partially stale state. Some chunks have new embeddings, some have old ones. Your retrieval quality actually gets worse during the migration, not better. And batch treats every document the same way — a document that hasn't changed in six months gets re-embedded alongside one that was updated yesterday. There's no prioritization.
What's the alternative?
Event-driven architecture. Instead of periodic batch jobs, you trigger re-embedding when the source data actually changes. The dbi services recommendation is PostgreSQL triggers with a queue pattern — using SELECT FOR UPDATE SKIP LOCKED for safe concurrent processing. Their schema design includes is_current flags, model_version columns, and source_hash values so you can do idempotent re-embedding. You only re-embed what actually changed, and you always know which model version produced which embedding.
That's elegant, but it's also a lot of infrastructure to maintain. And it still doesn't solve the fundamental lock-in problem. You're still dependent on a specific embedding model, and when it gets deprecated, you're still doing a migration — even if it's a smarter migration.
Which is exactly why Daniel's second question is so interesting. Can MCP with dynamic querying just sidestep the whole thing?
So let's talk about the Model Context Protocol. My understanding is it's essentially a standardized way for language models to discover and call external tools at runtime. JSON-RPC two point zero. But I've seen some pretty breathless takes suggesting it makes RAG obsolete.
I've seen those too, and I think they're mostly wrong. Or at least, they're confusing two different things. MCP standardizes tool invocation — how a model discovers that a database query tool exists, what parameters it takes, how to call it. It doesn't do retrieval in the semantic sense. AWS, Google Cloud, Oracle, Databricks — they've all published guidance on this, and they're unanimous. MCP is not a RAG substitute. It complements RAG.
The appeal is obvious, right? If you can just query your database directly through MCP, you don't need to maintain a parallel vector index. Your data stays in its original form. You get authoritative, real-time answers with full provenance. No embedding model to deprecate because there's no embedding happening.
For certain use cases, that's absolutely the right call. Microsoft's SQL MCP server is a great example — it generates deterministic T-SQL from natural language queries. There's no semantic fuzziness. You ask "what were our sales last quarter," it generates a SQL query, you get an exact number. That's better than any vector search could give you. But that only works for structured data with clear schemas. The moment you're dealing with unstructured prose — legal documents, support tickets, research papers — you need semantic search. SQL isn't going to find you the paragraph that's conceptually similar to your query but uses completely different words.
The question isn't "does MCP replace RAG." It's "when does each approach make sense.
And there's a really useful framework for this. Ricardo Ferreira — same person who documented the forty thousand dollar re-embedding horror story — created something called the VICE scoring model. It stands for Value, Impact, Confidence, and Effort. You score both traditional search and vector search across those four dimensions. If one approach scores more than two times the other, it's a clear winner. If they're within a factor of two, you probably want a hybrid approach.
Give me an example of how that plays out.
He applied it to e-commerce product search. Traditional keyword search scored one hundred eighty nine, vector search scored eighty four. That's a two point three times ratio, so hybrid wins — use both. For legal document discovery, vectors scored one hundred eighty versus sixty seven for traditional. That's a two point seven ratio, so pure vector search was the clear winner. The structure of your data dictates the retrieval method.
This connects to something interesting I read recently. There was a MindStudio analysis from March of this year that made a pretty provocative argument about coding agents specifically. They found that top AI coding tools — Claude Code, Cursor, Devin — have largely abandoned traditional vector RAG. Instead, they use grep, file tree inspection, AST-based code navigation, and just stuffing more context into those two hundred thousand to one million token windows.
That makes perfect sense for code. Code has structure. It has import graphs and call chains and explicit references. Flattening that into embedding space actually destroys information. An AST-based retrieval that follows the import graph is going to find relevant code more reliably than any vector similarity search. The MindStudio piece put it bluntly: RAG was designed to solve a context window problem that has largely been solved differently. When you can fit an entire codebase into context, why would you bother with chunking and embedding?
That framing only works for code, right? You can't grep a million support tickets for "customer is frustrated about billing." You need semantic search for that.
And this is where I think the real principle emerges. The retrieval method should match the data structure. If your data is structured and queryable — relational databases, APIs, anything with a schema — MCP-style dynamic querying is probably superior. If your data is unstructured prose, vector search still wins. And if it's somewhere in between, hybrid approaches that combine keyword retrieval like BM twenty five with vector similarity, then re-rank with a cross-encoder — that's the current best practice.
MCP isn't a replacement for RAG. It's an orchestration layer that can route different query types to different backends. And for the right kind of data, it does avoid the embedding deprecation problem entirely because there are no embeddings.
But here's where Daniel's third question gets interesting. What about caching embeddings client-side as a middle ground? Because that doesn't avoid embeddings — it just changes where they live and how they get refreshed.
Let's dig into that. The idea, as I understand it, is that instead of hitting a server-side vector database for every query, you store pre-computed embeddings locally — in the browser, say, using IndexedDB. You check the local cache first, and only fall back to the server when you need to.
The latency implications are significant. There was a piece on the Agentic Thinking blog earlier this year that benchmarked this. Cold starts — when the model weights need to be loaded — take three to ten seconds. But warm queries, once everything's cached, come in at two hundred to five hundred fifty milliseconds. That's genuinely usable for interactive applications.
The privacy angle is interesting too. If the embeddings never leave the client, you're not shipping sensitive data to a third-party API. SitePoint had a writeup on this — browser-based RAG where documents are chunked and embedded server-side initially, but then the vectors and metadata get sent to the client for local storage. After that, everything happens in the browser.
The hybrid pattern is what I find most compelling. You cache embeddings for frequent or static chunks client-side, but keep a server-side vector database as the source of truth. The client checks local storage first, and if the embedding isn't there or has expired, it fetches from the server and caches the result with a time-to-live. This is essentially what CDNs do for web content, applied to embeddings.
This is where it gets interesting for the deprecation problem. If your embeddings are cached with a TTL, model deprecation becomes a gradual, non-breaking event. Old embeddings continue to work locally until their TTL expires. New embeddings get fetched on refresh. You don't have a big bang migration. You have a rolling transition.
Which eliminates the two-week developer crunch and the partially-stale-index problem during backfill. The trade-off, of course, is that you're now managing cache invalidation. And as the famous saying goes, there are only two hard problems in computer science: naming things, cache invalidation, and off-by-one errors.
I was waiting for that joke. But seriously, cache invalidation is hard. How do you know when a cached embedding is stale because the source document changed, versus stale because the embedding model changed?
That's where the event-driven architecture from the dbi services approach comes back in. You need source_hash values to detect content changes, and model_version columns to detect model changes. If the source hash is the same but the model version is newer, you know you need to re-embed due to model deprecation. If the source hash changed but the model version is the same, it's a content update. And you can handle these differently — content updates might justify an immediate cache invalidation, while model updates might be fine with a slower TTL-based rollout.
There's a limit to how much you can cache client-side though, right? We're talking about browser storage caps in the gigabyte range.
This pattern works for small to medium corpora. If you're dealing with millions of documents, you're not fitting all those embeddings in IndexedDB. You need server-side infrastructure. And there are other limitations — KV caches don't persist between browser sessions, shader recompilation happens every time. The Agentic Thinking post was clear about this. It's not a universal solution.
Where does this leave us? We've got three approaches, each with different trade-offs. Traditional RAG with vector embeddings gives you powerful semantic search but locks you into an embedding model and creates a deprecation tax. MCP with dynamic querying avoids embeddings entirely but only works well for structured, queryable data. Client-side caching reduces the blast radius of deprecation but adds cache management complexity and doesn't scale to large corpora.
I think the synthesis is that you don't pick one. You pick based on your data and your scale. And you probably end up with some combination. Use MCP-style dynamic queries for structured data where SQL or API calls give you exact answers. Use vector RAG for unstructured prose where semantic search is needed — but pin your model versions, store metadata about which model produced which embedding, and use event-driven re-embedding rather than batch jobs. And if you're building a client-facing application, consider caching embeddings locally with TTLs to smooth out deprecation transitions.
The thing that keeps nagging at me though is the observability gap. Multiple sources we've referenced agree that most organizations deploy RAG without measuring retrieval quality at all. And if you can't measure it, you can't manage it. You don't know if your embeddings are drifting, if your model's been deprecated, if your chunking strategy has degraded.
That might actually be the most important point in this whole discussion. Before you worry about whether to use MCP or RAG or client-side caching, you need to answer a more basic question: how do you know if your retrieval is working at all? Are you measuring precision at K? Are you tracking normalized discounted cumulative gain? Do you have confidence scores on your retrieved chunks? If the answer is no to all of those, you're flying blind. And it doesn't matter which architecture you pick — you won't know when it breaks.
The dbi services quote really lands here. Nobody complains about the embedding pipeline. They complain that the chatbot gives wrong answers. By the time you trace it back, the damage is done.
There's a subtler lock-in effect here too that I don't think gets enough attention. The re-embedding tax isn't just about money and developer time. It creates a perverse incentive to stick with outdated models. If you know that switching embedding models means a multi-week migration and a big bill, you're going to be extremely reluctant to upgrade — even if the new model is significantly better. You're locked in not by contract, but by operational inertia.
Which is exactly the kind of lock-in that MCP with dynamic querying breaks, at least for the use cases where it applies. If your data lives in plain SQL databases and you're querying it through an API, the embedding model is a thin, swappable layer rather than the foundation of your entire retrieval system. You can switch models trivially because there's nothing to re-embed.
For the use cases where you do need embeddings, client-side caching at least reduces the lock-in. If your embeddings are distributed across clients with TTLs, a model change is a gradual rollout rather than a flag day. You can A/B test the new model on a subset of traffic. You can roll back if something goes wrong. It's not zero-cost, but it's dramatically cheaper than the big bang migration.
Let me play devil's advocate for a second. Is the embedding deprecation problem actually as widespread as we're making it sound? Or are we describing a worst-case scenario that most teams never hit?
That's a fair question. I think it depends on your scale and your timeline. If you're building a prototype or an internal tool with a few hundred documents, you probably won't feel this pain. You can re-embed everything in an afternoon. But if you're building a production system that's supposed to last years — and especially if you're embedding customer data or legal documents where retrieval quality actually matters — the deprecation cycle is real. Embedding models are improving fast, and the old ones are getting retired. The pace of deprecation is accelerating, not slowing down.
The MindStudio piece makes a related point that I think is worth pulling out. For certain domains — coding being the obvious one — the entire premise of vector RAG might have been a detour. We spent years building sophisticated chunking and embedding pipelines for codebases, and it turns out grep plus AST navigation plus large context windows works better. The retrieval method should have matched the data structure from the start.
Which is humbling, honestly. It's a reminder that just because a technology is exciting doesn't mean it's the right tool for every job. Vector embeddings are incredible for semantic search over unstructured text. They're mediocre for structured data and actively harmful for code, where they destroy structural relationships that are more informative than semantic similarity.
If we're giving practical guidance here — and Daniel did ask for concrete analysis — what's the decision framework?
I'd say it's three questions. First, what is your data's structure? If it's highly structured with clear schemas, start with MCP-style dynamic querying. If it's unstructured prose, you probably need embeddings. If it's somewhere in between, plan for hybrid. Second, what's your scale? If you're dealing with millions of documents, client-side caching alone won't cut it — you need server-side infrastructure with event-driven re-embedding. If you're dealing with thousands, client-side caching becomes viable and gives you a smoother deprecation path. Third, how are you measuring success? If you can't answer that question with specific metrics, stop and fix that before you do anything else.
The third question might be the hardest one for most teams. Setting up precision-at-K tracking and confidence scoring is not trivial. It requires labeled evaluation data, which most organizations don't have.
But the alternative is deploying a system where you have no idea if it's working until users start complaining. And by then, as we've established, the trust is gone. You can't A/B test retrieval quality after the fact.
Alright, one more angle before we move to practical takeaways. There's something about the client-side caching approach that I think has implications beyond just the deprecation problem. It's part of a broader shift toward local-first architectures. KCDC this year has sessions on using IndexedDB as the client-side source of truth with server-side change data capture. GraphRAG-rs supports client-side deployments for privacy-first analytics. We're seeing a pendulum swing back toward keeping data close to the user.
There are good reasons for that beyond just latency and offline capability. If your RAG system works entirely in the browser, there's no server to go down, no API to deprecate, no third party to trust with your documents. The trade-off is that you're limited by browser storage and compute. But for a surprising number of use cases, that trade-off is worth it.
Especially if you're dealing with sensitive documents. Legal contracts, medical records, internal strategy documents — the kinds of things you really don't want shipping off to a third-party embedding API.
And the hybrid approach gives you a path there. Do the initial chunking and embedding server-side — where you have more compute and you're not constrained by browser storage — then ship the vectors and metadata to the client. After that, everything happens locally. The server never sees the user's queries.
Which brings us back to Daniel's original framing. He was asking whether MCP with dynamic querying could replace traditional RAG and avoid the embedding deprecation headache. I think the answer is: partially, for the right kinds of data, and not as a complete replacement. But the real insight is that we've been treating RAG as a monolith when it should be a set of composable patterns. Sometimes you want live database queries. Sometimes you want semantic search. Sometimes you want client-side caching with TTLs. The orchestration layer — whether it's MCP or something else — should route to the right backend based on the query and the data.
The deprecation problem doesn't go away entirely, but it becomes manageable. If your embeddings are a thin layer rather than the foundation, you can swap them without a crisis. If they're cached with TTLs, you get gradual rollouts instead of flag days. If you're measuring retrieval quality, you catch problems before your users do.
The common thread across all of this is that the operational practices matter more than the specific technology choice. Event-driven re-embedding. Model version pinning. Cache invalidation strategies. These are the things that determine whether your RAG system is maintainable over time, regardless of which embedding model or query protocol you're using.
Now: Hilbert's daily fun fact.
The collective noun for a group of sloths is a "bed." As in, a bed of sloths. Which is either the most accurate or the most misleading collective noun in the animal kingdom, depending on whether you're observing them at two in the afternoon or two in the morning.
What should listeners actually do with all this? First, audit your current RAG setup. Do you know which embedding model you're using, and is it pinned to a specific version? If you're calling a default endpoint, fix that today — it's a one-line change that could save you weeks of migration pain later. Second, add model_version and source_hash columns to your embedding metadata if you haven't already. You can't manage what you can't measure, and you can't detect drift without metadata. Third, if you're building something new, ask the data structure question before you reach for embeddings. Not everything needs to be a vector.
If you're already deep into a RAG deployment and worried about deprecation risk, start planning an event-driven re-embedding pipeline. It doesn't have to be complicated — PostgreSQL triggers and a simple queue table will get you surprisingly far. The dbi services post has a solid reference architecture. The key is to stop treating re-embedding as something you do in a panic when the old model gets deprecated, and start treating it as a normal operational workflow.
For teams considering MCP, I'd say start with the low-hanging fruit. If you have structured data in a SQL database, expose it through MCP and see how much of your retrieval workload it can handle. You might be surprised. A lot of "semantic search" use cases are actually just "I want to ask questions about my data in natural language," and SQL generated from natural language handles that surprisingly well for structured data.
If you're building a client-facing application — especially one dealing with sensitive documents — seriously evaluate client-side embedding caching. The latency benefits are real, the privacy benefits are real, and the deprecation resilience is a nice bonus. Just be clear about the scale limitations and plan your cache invalidation strategy upfront.
The one thing I'd leave listeners with is this: the deprecation of embedding models is not a bug in the RAG ecosystem. It's a feature of a rapidly improving technology. The models are getting better, which means the old ones are getting retired. That's good news in the long run. The question is whether your architecture can absorb that churn gracefully, or whether every model update is an emergency.
If every model update is an emergency right now, you're not alone. But you have options. MCP for the structured stuff, event-driven re-embedding for the unstructured stuff, client-side caching to smooth the transitions, and observability across all of it so you know when things are going sideways before your users do.
Thanks to our producer Hilbert Flumingtop for keeping this show running. This has been My Weird Prompts — find us at myweirdprompts.com or on Spotify.
Until next time.
