The RAG Frontier: When the Model Computes and the Corpus Standardizes

Everything up to here was the proven core. The seven families, the ingestion pipeline, the model-driven loop, the discipline of choosing and measuring — all of it is battle-tested, in production, boring in the good way. Parts one through seven are what you build with today.

This part is different. This is the edge — the ideas that are new enough that nobody has ten thousand hours in them yet, including me. So the register changes with it: less "here's how," more "here's what this might become, and where it might break."

One honest label before we start. This is a snapshot as of mid-2026, and it will age faster than anything else in the series. The fundamentals in the first seven parts are durable — the five dimensions, the question shapes, the golden set. These two bets are not fundamentals. They're wagers on which of those dimensions gets redefined next.

Two wagers, specifically. One says relevance shouldn't mean similarity — it should mean computation. The other says you shouldn't retrieve from raw documents — you should retrieve from a standardized, designed substrate. One attacks the selection dial. One attacks the substrate dial. Neither, notice, is a better index.

Bet one: stop retrieving, start computing

Every family in this series measured relevance. Even structured queries, which return exact rows, still answer which records match. The shared assumption underneath: retrieval means selecting a subset of text that already exists.

The first bet abandons it. Don't select a subset. Compute over the whole thing.

Give the model the corpus as data in an execution environment rather than as a pile of retrieved passages, and the operations available change completely. It can grep, filter, count, sort, sample, partition — and run those over the entire corpus, not over the k fragments a retriever guessed at.

Recursive Language Models formalize this. The context lives in a Python REPL as a variable instead of in the prompt. The model writes code to peek at it, split it, search it, and recursively call itself on the pieces — handling inputs far larger than any context window, because the context never has to fit in the window at all. It sits in memory, and the model reaches into it programmatically. (InfoQ's summary is a gentler entry point than the paper.)

Watch which failure this dissolves. The aggregation question from part one — how many runbooks mention the deprecated auth service? — is structurally unanswerable by top-k retrieval, because retrieval returns k and the answer needs all n. As code, it's a count over a filter. Two lines. The question stops being hard the instant it stops being a retrieval problem and becomes a computation.

And the wildest part is that this already exists, in production, in your editor. A coding agent working across a repository doesn't embed your codebase and pray. It greps, opens files, follows references, re-reads, runs things. That's retrieval-by-computation over a corpus, at scale, right now. The provocation of RLMs is just: why did we accept something weaker for document corpora than we already use for code?

The potential. If this matures, a whole class of questions the seven families struggle with — aggregations, exhaustive scans, cross-corpus computation, anything where the answer is a function of the data rather than a passage in it — stops needing bespoke architecture. You don't build a structured store for the counting questions and a graph for the relational ones; you hand the model the corpus and the ability to compute. It's the most general answer to "the retrieval strategy is a property of the question," because the model picks the computation per question. In the limit, it could subsume much of what parts two through five describe.

Where it breaks, and what's unresolved. The costs are not small, and they're the interesting part.

• Executing model-written code against your data is a security posture, not a feature flag. You need a real sandbox, and "the model wrote some code and we ran it on the corpus" is a sentence that should make a security reviewer sit up.
• Cost and latency are unpredictable in a way fixed pipelines aren't. A question might resolve in one cheap pass or spiral into a dozen recursive calls. Budgeting for that is an open problem.
• The model has to write correct code. A different, and often louder, failure surface than bad retrieval — which is a mercy, because loud failures get caught, but it's still a new class of bug: the confidently-wrong query.
• When does it actually beat retrieval? For a lookup, spinning up a REPL to answer "what's the warranty period" is absurd overkill. The boundary between "retrieve this" and "compute this" is itself an unsolved routing question — arguably the same agentic routing problem from part five, one level up.

This is the newest idea in the series and the least settled. Production patterns are still forming. Watch it closely; don't bet a roadmap on it yet.

Bet two: standardize the substrate

The second bet is quieter, and it comes at the problem from the opposite end.

Part three and part four both landed on the same move: the best answer is often to build a derived layer above the raw documents — a graph of resolved entities, a tree of summaries, tables extracted into structure. Manufacture the substrate the questions actually need, once, offline.

The second bet generalizes that. If you're going to build a curated layer for machines to read, why does everyone invent their own shape for it? Standardize it.

In June 2026, Google Cloud published the Open Knowledge Format — OKF v0.1. An open spec that represents knowledge as a directory of markdown files with YAML frontmatter. Vendor-neutral, no proprietary SDK to read or write. And because it's just files, the whole bundle lives in git — so you diff a knowledge change, review it in a pull request, and roll back a bad edit like any other commit.

I'll declare an interest, because it's relevant rather than flattering. This is close to what the copilot on this site already does: markdown with YAML frontmatter in a directory, deterministic retrieval instead of a cosine-distance lottery, git diff as the review workflow. I didn't arrive there from foresight — I arrived from constraints. Solo maintenance, no database, content as the product. It's a small validation that the shape falls out of those constraints naturally, and migrating to conform to the actual spec is a sensible later step. I'd do it for interoperability, not capability.

The potential. The interesting promise isn't the format itself — markdown with frontmatter is not an invention. It's interoperability. A knowledge bundle authored once, in a standard shape, that any compliant agent can consume without a custom ingestion pipeline. Knowledge that travels between tools and organizations the way a CSV travels between spreadsheets. Review-as-code for organizational knowledge, so a change to what the machines believe goes through the same gate as a change to what they run. If it catches on, the messy, bespoke ingestion work from part four gets a common target to build toward, instead of every team reinventing the curated layer from scratch.

Where it breaks, and the caution that matters more than the endorsement.

OKF is a format, not an architecture. It improves the substrate dimension and does nothing about the other four. A beautifully curated OKF bundle still can't tell you how many documents mention something if you query it by similarity. It doesn't route. It doesn't traverse relationships. It doesn't decide when to retrieve, or compute, or refuse. It makes the thing you retrieve from better organized — genuinely valuable, and strictly, narrowly bounded.

It's also v0.1, published weeks before I wrote this, with essentially no production track record and no guarantee of adoption. Standards win by ecosystem, not by merit, and most proposed standards quietly don't. So the honest posture is: promising, aligned with where curated substrates are already heading, worth tracking — and absolutely not something to treat as the answer to enterprise knowledge. Doing that would repeat the exact mistake this series opened with: taking one good idea and mistaking it for the whole problem. "Use OKF for everything" is just "use a vector database for everything" wearing newer clothes.

What the two bets share, and where the series lands

Step back and the pattern is the same one that's run through all eight parts: neither of these is a better index.

One changes what the operation is — computation instead of similarity. One changes what you retrieve from — a designed standard instead of raw documents. The field spent years treating retrieval quality as an index-tuning exercise, and the genuinely interesting work has walked away from the index entirely, out toward the dimensions around it. Which is exactly what you'd predict if part one was right: when four of your five dials were set by a diagram nobody chose on purpose, the leverage was never in the one dial everybody kept turning.

That's the note to end eight parts on.

The specifics here will date. RLMs might become how we do half of retrieval, or a footnote. OKF might become the substrate everyone targets, or a spec three people remember. I genuinely don't know, and anyone who tells you they do is selling something.

But the frame outlasts the specifics. Whatever the frontier ships next, it will be an answer to one of five questions — what you retrieve from, how you select, when you decide, at what granularity, how you verify. New answers, same questions. The dials will keep changing.

Which dial your question needs won't. That was true before any of this, it's true at the frontier, and it'll be true for whatever comes after the frontier.

So the last line of this series is the same as the first real advice in it. Before you reach for any of this — the proven families or the frontier bets — do the one thing that tells you which you actually need.

Write down the questions first.