The Index Card

AI used to forget what you told it. An entire industry sold the workaround but it just stopped working.

You have met this problem even if you never learned its name. You paste a long document into a chatbot, a contract, a report, a chapter of something, and you ask a question, and the answer comes back as though the machine read the first page and skimmed the rest. Or you are ten messages into a conversation and it has quietly forgotten the thing you told it at the very start. It feels like talking to a brilliant colleague who has no short-term memory.

That forgetting was never a personality flaw. It was a hard technical limit with a dull name: the context window, the amount of text a model can hold in mind at one time. For most of the last three years that window was small, a few dozen pages, and everything an AI could reason about had to fit inside it or be left out.

Around that one limitation, an entire industry quietly grew. The logic was simple. If the model can only hold a few pages, then the trick is to feed it the right few pages at the right instant: search across your millions of documents, find the handful of passages that answer this exact question, and slip only those into the window before the model reads. The technique carries an unlovely name, retrieval-augmented generation, RAG for short, and beneath it sits a second product, the vector database, the specialised engine that does the finding. The plain-English version is this: it is the machinery for handing a forgetful genius one index card at a time.

That machinery became a real market. Analysts put the retrieval-augmented-generation software category at roughly 2.3 billion dollars in 2025, with the vector databases underneath it worth a similar amount, both compounding near forty percent a year. It is one of the fastest-growing corners of enterprise software. And it exists, top to bottom, to route around a single limitation.

Here is the uncomfortable thing about a workaround: it dies the day the limitation it was built to dodge goes away.

The limitation is going away. The window a model can hold at once has jumped from a few dozen pages to something closer to a small library. Anthropic’s Claude Sonnet now accepts a million tokens, roughly fifteen hundred pages, in a single request. Google’s Gemini 3 takes the same, and its top Ultra tier reportedly stretches to two million. The Chinese open models are there as well: Z.ai’s GLM-5.2 and Meituan’s LongCat-2.0, both released in June, both holding a native million-token window, and LongCat built on an architecture designed so that a window that large no longer costs what a window that large used to cost.

For an enormous share of ordinary work, this rewrites the arithmetic. If the model can simply keep the entire contract set, the whole employee handbook, the full codebase in mind at once, then you do not need to build and babysit a pipeline that hunts for the right index card. You hand it the book.

The person who has to act on that is not a strategist in a corner office. She is an engineer at a large company, and for two years part of her job has been a retrieval pipeline she built and now maintains: forty thousand documents chopped into fragments, embedded, indexed, tuned, and re-tuned every time the results came back subtly wrong. It works. It also breaks in small ways more or less constantly, and keeping it alive is close to a third of her week. This quarter, for the first time, she ran the same questions by loading whole documents straight into a long-context model, no pipeline in the middle. For most of them it simply worked. She has started asking something that did not occur to her a year ago: what is she still maintaining all of this for?

This is where the obituary writers get ahead of the story, and where the real shape of the thing finally shows. The workaround is not simply dead, because the enormous window hides a secret the launch posts skip past: the models cannot actually use all of it well.

Independent testing through 2026 has been unusually blunt about this. A model that advertises a million-token window does not reliably reason across a million tokens. Feed it enough and its attention frays. It misses facts buried in the middle, it loses instructions you gave it early, and it does not fade gently as the text grows. It falls off cliffs. For most of the frontier models, the amount of context you can genuinely trust for careful work is a fraction of the number printed on the box. So the index card was never only a crutch. For some jobs it was the thing keeping the answer correct.

Which is why the honest reading is not “an industry just died.” It is something narrower and far more useful. A fast-growing piece of infrastructure is being repriced in real time, from the default way to build AI systems into a specialist tool for the cases that still truly need it. The everyday middle, the small and medium and mostly-static piles of documents that make up the bulk of real work, is being absorbed into the model itself. The edges hold: the genuinely vast archives, the data that changes by the minute, the jobs where you must be able to prove exactly which document an answer came from. The question worth billions is not whether the workaround lives or dies. It is where the line between absorbed and essential finally settles, and how fast that line is moving toward the edges.

The Deep Dive

What follows is the arithmetic that fixes that line: how cheap “just hand it the book” actually is once the meter is running, why the million-token window lies and what that lie is worth to the workaround, which technology graveyard this most resembles and the one place the resemblance breaks, and which of four futures the next year most plausibly delivers for a category that was, until this month, simply the default.

Start with the meter, because this is a cost question wearing a capability costume. Retrieval did not exist only because old windows were small. It existed because reading is billed by the word. A model charges for every token it takes in, on every request, and a long document read in full, thousands of times a day, is an expensive habit that no amount of capability makes free.

The gap is not subtle. A well-built retrieval pipeline pulls only the fragments that matter, perhaps three thousand tokens for a given question, and answers it for roughly three cents. Hand the same model a two-million-token archive to read from scratch and the same question can cost around twenty dollars, because the meter runs across the entire pile whether the answer sat on page four or page four thousand. At the volumes real products run, half a million queries in a month, that is the distance between a bill near fifteen thousand dollars and one near ten million. Put that starkly and retrieval does not sound like a dying workaround at all. It sounds like the only sane way to operate a business.

Except the arithmetic has a second term the obituaries omit, and it is the term quietly dragging the line toward the edges: caching. When the documents you are reading are stable, the same handbook, the same contract set, the same codebase, queried again and again, you do not have to pay full freight to read them every time. The model labs will now hold that context in a cache and charge a steep discount to reuse it. Google prices cached context at roughly a quarter of the standard rate; Anthropic has moved to flat-rate long-context pricing that changes the shape of the bill entirely. Loading the whole book stops being a twenty-dollar event on every question and becomes a load-once, read-cheaply-forever arrangement.

That single mechanism is what turns a clean obituary into a moving boundary. Where a corpus is small, static, and reused, caching collapses the cost advantage retrieval used to own, and the pipeline becomes maintenance you are paying for with nothing to show. Where a corpus is genuinely enormous, or changes by the minute, or gets asked one unique question and never queried again, the meter still favours pulling three thousand tokens over reading two million. The crossover is not a philosophical debate about architecture. It is a spreadsheet, and the inputs are corpus size, how often the content changes, and how many times you reread it.

The spreadsheet, though, assumes the big window works, and the second forensic layer is that it often does not. This is the part the capability marketing is quietest about, and it is the reason the boundary does not simply race to the edges and delete the whole category on the way.

Through 2026, independent evaluations pulled the advertised numbers apart. One widely cited study ran eighteen frontier models across their full windows and found every one of them degraded as the input grew. The cross-checks were harsher still: measured against a controlled benchmark, the context a model can actually use tends to land somewhere between half and two-thirds of what it advertises, and the caps teams quietly adopt in production are tighter than that, often a quarter to a third of the window. A million-token box, in practice, is frequently a two-to-four-hundred-thousand-token tool wearing a bigger label.

The failure is not random, which is what makes it dangerous. Models show positional bias, paying more attention to the beginning and end of a long input than the middle, so a fact stranded in the centre of a huge document can simply be missed. They suffer what researchers call attention-sink collapse, where the machinery that spreads focus across a long text quietly breaks down past a certain length. And the degradation is not linear. A system that is perfect at fifty thousand tokens and fine at a hundred thousand can fall apart at two hundred thousand with no warning in between. You do not get a gentle slope you can plan around. You get a cliff you find by falling off it.

There is one telling exception, and it points at where this ends up. The reasoning-heavy tier of Google’s Gemini line holds retrieval and reasoning quality across the full window where its rivals fray, and even the rivals are near-perfect at the narrow job of finding a specific fact. What they lose over distance is not memory but judgement: the ability to hold instructions and constraints steady while reasoning over the whole pile. That distinction matters enormously for the boundary. It means the simplest retrieval jobs, find the clause, quote the number, are exactly the ones the big window now does well and cheaply, while the hardest reasoning jobs are the ones where cramming everything in still breaks, and where feeding the model a curated, ordered, trustworthy set of index cards remains not a workaround but a genuine improvement.

To understand where a category goes when the platform absorbs its reason for existing, it helps to look at the ones that went before, because this exact story has run many times. In the 1990s a thriving little industry sold software that squeezed more space out of a hard drive; the operating system absorbed disk compression into itself and the products vanished. Utilities that accelerated dial-up modems died when broadband made the acceleration pointless. Standalone satellite-navigation units were a booming consumer category until the phone in every pocket swallowed the map. The pattern is identical each time. A tool exists to patch a limitation in a platform; the platform removes the limitation; the tool does not get out-competed so much as rendered unnecessary, and it disappears fastest in exactly the mainstream cases that made it a big market in the first place.

The parallel is close enough to be uncomfortable for anyone selling the pipeline as the default. But there is one place it breaks, and the break is the whole reason this is a repricing rather than an extinction. Disk compression did nothing except patch a limitation, so when the limitation went, it had no residual purpose. Retrieval does other things. It enforces who is allowed to see which document before a single token reaches the model, which a giant undifferentiated context cannot do. It lets you prove exactly which source produced an answer, which regulated industries require and a blurred million-token read cannot provide. It reaches data that is changing continuously, and it holds the cost line at genuinely vast scale. Those are not workarounds for a missing capability. They are functions the big window does not perform at all. Unlike the disk compressor, retrieval has somewhere to stand once the crutch role is gone. The question is how much ground that leaves it, and it is a good deal less than the ground it holds today.

Follow the money out of the middle and you can see where it goes, because value in this shift does not evaporate. It moves. The dollars that used to fund a retrieval pipeline, the engineering time, the vector-database licence, the constant tuning, migrate into two places. The first is the model call itself, now doing work the pipeline used to do, which quietly hands the model labs a larger slice of each workflow. The second is the caching layer, which turns out to be the real lever: whoever prices the reuse of long context controls the crossover, and the labs price it. A cheaper cache pushes the boundary toward the edges and eats more of the category; an expensive one holds the boundary back and keeps the pipeline alive. The workaround’s fate is set, to a striking degree, by a pricing decision made by the very companies whose windows made it obsolete.

The engineer with the forty-thousand-document pipeline sits precisely on that line, and so does the vendor who sold her the vector database. For new projects, especially the small and static ones, the rational default is flipping from “build the pipeline” to “try the window first,” and that flip is a procurement decision happening in thousands of companies at once this year. The tooling vendors know it, which is why the sharper ones are already repositioning, away from “we help the model remember” and toward “we manage what an agent pays attention to across long tasks,” a job that grows rather than shrinks as models take on more autonomous work. The category is not standing still to be deleted. It is trying to move up the stack faster than the window can chase it. Worth noting, too, is that the cheapening of the window is not gated by any single Western vendor’s price list: LongCat-2.0 delivers its native million-token window through a sparse-attention design that keeps the cost of a long read from exploding, and it is open, downloadable, and Chinese, which means the pressure on the crossover comes from more than one direction at once.

So the line moves, but how far, and how fast? The next year resolves the boundary in one of four directions, and they are worth pricing directly, because the difference between them is the difference between a category that compresses and one that is quietly deleted.

The most likely path, and it is genuinely the leading one rather than a comfortable middle, is that the boundary settles with the everyday middle absorbed and retrieval retreating to the edges. In this world caching does its work, the small and static corpora move into the window, and the pipeline stops being the default for new builds, while the truly vast archives, the fresh-data feeds, and the audit-bound workflows keep it alive as a specialist tool. The category keeps growing in raw dollars, because AI adoption is growing faster than the boundary is eroding it, but it loses the thing that made it valuable, its status as the assumed architecture, and with that goes its pricing power and its strategic centre. If you are running or buying this kind of tooling, this is the scenario to plan against, and it carries something like a forty-four percent weight, held below certainty only because the two forces that could push it either way, caching prices and the reliability of the big window, are both still moving.

Push those forces one way and you get the second path, in which retrieval holds the backbone and the substitution stalls. The cost gap at scale is real and the context cliffs are real, and if the labs keep long-context reads expensive and the reliability of huge windows improves only slowly, then for serious production systems the pipeline stays the sane default, evolved into hybrid and agentic forms but very much alive. This is the world the current cost arithmetic, taken at face value, actually points to, and it deserves real weight, twenty-four percent, because a 333-times cost difference does not surrender to a capability trend overnight.

Then there is the faster, harder version of the first path, where native context routs the category within the year. If caching gets aggressively cheap, flat-rate pricing spreads, and the reliability problem is largely solved the way the strongest reasoning models are already solving it, the crossover lurches toward the edges and most new enterprise builds simply skip the pipeline, leaving retrieval a legacy technology maintained on old systems rather than chosen for new ones. This is the true extinction scenario, and it sits below the odds of a coin-flip on the specific ground that the cost and reliability evidence today argues against it, but it is far from remote: call it twenty percent, one in five, and rising every time a lab cuts its caching price.

The last path is the one the vendors are betting their survival on, in which the category does not so much hold its ground as change what its ground is, surviving by becoming the memory and attention layer for autonomous agents rather than the snippet-feeder for forgetful chatbots. Here the retrieval machinery persists and even thrives, but as something meaningfully different from what it is today, which is why it is a distinct outcome rather than a version of the first. It is the least probable of the four at twelve percent, not because the pivot is impossible but because platform absorptions rarely leave the incumbent tooling enough time to reinvent itself before the window has already taken the volume.

What moves these numbers is not opinion, it is a short list of dated, watchable events. Watch the caching price sheets: any move by a major lab to cut cached long-context rates sharply, plausibly at the next round of model launches this autumn, drags the boundary toward the edges and lifts the extinction odds directly. Watch the next independent long-context evaluations: if a wave of frontier models start holding reasoning quality across their full advertised windows the way only the top reasoning tier does today, the reliability objection that keeps retrieval alive begins to dissolve, and the timing to watch is the benchmark refreshes that follow each major release. Watch enterprise procurement language through the second half of 2026: when large buyers start writing “long-context-first, retrieval where required” into their architecture standards rather than the reverse, the default has already flipped in practice. Watch the vector-database vendors themselves: a decisive repositioning toward agent memory and context orchestration, announced in product rather than in blog posts, tells you the smart money inside the category has concluded the snippet-feeding business is finished. And watch the open models: each new open release that ships a genuinely usable long window at low cost, LongCat and GLM will not be the last, removes another vendor’s ability to keep the crossover artificially back by keeping long context expensive.

Step back far enough and the specific fight over retrieval is a small instance of the largest pattern in this industry, the one worth carrying out of here even after the vector-database market has been forgotten. The most valuable businesses in AI infrastructure are the ones solving a problem the models genuinely cannot solve yet. The most precarious are the ones solving a problem the models are about to solve for themselves. Every capability the frontier absorbs turns some thriving category into a legacy line, and the forgetting that made AI feel dim, the thing that sent you hunting for the right index card in the first place, was not only a flaw in the machine. It was a business model. The engineer who spent a third of her week keeping the pipeline alive will spend next quarter deciding how much of it to switch off. She will keep the parts that guard who sees what, and prove where answers came from, and reach the data that never stops changing. The rest she will let the model remember, because it finally can, and because the thing she was really being paid to work around just stopped being in the way.

Sources:

Artificial Analysis, “GLM-5.2 is the New Leading Open Weights Model on the Artificial Analysis Intelligence Index,” June 2026.

VentureBeat, “Meituan Open-Sources LongCat-2.0, the 1.6T Near-Frontier Agentic Coding Model Trained Entirely on Chinese Chips,” 30 June 2026.

MarkTechPost, “Meituan Releases LongCat-2.0: A 1.6T-Parameter Open MoE Model With Native 1M Context and LongCat Sparse Attention,” 5 July 2026.

Google AI for Developers, “Long Context” and “Gemini 3 Developer Guide” (context window and context caching documentation), 2026.

Anthropic, Claude Platform Docs, “Models and Pricing” (Claude Sonnet one-million-token context; long-context pricing), 2026.

Chroma Research, “Context Rot: How Increasing Input Tokens Impacts LLM Performance” (eighteen-model long-context degradation study), 2026.

Acing AI, “Effective Context Length: Why 1M-Token Windows Fall Short, and When RAG Still Wins” (RULER cross-check; effective-context estimates), 2026.

Redis, “RAG vs Large Context Window: Real Trade-offs for AI Apps” (per-query cost comparison; three-thousand-token retrieval versus full-context reads), 2026.

MindStudio, “What Is Flat-Rate Long-Context Pricing? How Anthropic Changed the Economics of RAG,” 2026.

Grand View Research, “Retrieval Augmented Generation Market Size Report” (RAG market valued near 2.3 billion dollars in 2025), 2026.

Ailog, “Vector Databases 2026: Trends and New Players” (vector-database market size and growth), 2026.

Disclaimer: This report is published by Scenarica Intelligence for informational purposes only. It does not constitute investment advice, a solicitation to buy or sell any financial instrument, or a recommendation regarding any particular investment strategy. Scenarica Intelligence is not a registered investment adviser or broker-dealer. All scenario probabilities and assessments represent the analytical judgment of Scenarica Intelligence and are subject to change without notice. Past performance of any asset or strategy discussed does not guarantee future results. Readers should conduct their own due diligence and consult with qualified financial advisers before making investment decisions.

Scenarica Premium: The full Scenarica suite includes Geopolitics, Economy, Bitcoin, AI, and Sunday Edition.

Scenarica Intelligence

We don’t predict the future. We price it.