When the Model Should Read Whole Documents

This is the last module. Add it only if users ask about documents as a whole, such as what changed between two revisions.

Atlas Bikes revised the T-7 battery warranty at the start of 2026. A service manager wants to brief the team:

What changed between revision 1 and revision 2 of the T-7 battery warranty?

With Part 12's code, the search route does well at first. It retrieves all four sections of both revisions, the warranty term and the exclusions of each, all with the same score of 0.575. Then it answers:

Answer (clarify):Which purchase date is this about?

Part 6's rule kicked in: the evidence is date-scoped, no purchase date was given, so ask for one. That rule is right for "what warranty applies to my bike?", a question about one customer's situation. This question is about the documents themselves. No purchase date will help.

Even without the rule, the model would receive four pieces in score order, tied, with nothing saying which old section matches which new one. It would have to compare the texts by eye, and to say "the exclusions didn't change", it would need to have read the exclusions of both revisions in full and confirmed they're identical. Search is built to return what's relevant to a question. A comparison needs everything, including what stayed the same.

A question about documents

The router gets a sixth path, compare, for questions that ask what changed or differs between two revisions. It proposes the kind of document and the two revision numbers; the product comes from the extractor, as always:

route     path=compare, document=battery warranty, revisions=['1', '2']resolve   found=['WARR-T7-R1', 'WARR-T7-R2']

The resolve step finds the documents by title ("T-7 battery warranty, revision 1") and checks that the user may read both. If one doesn't exist or isn't readable, the answer says so: "I can't find revision 3 of the T-7 battery warranty that you can read." There's no clarification about purchase dates on this path: the question names its documents, so there's no situation to pin down.

Let code find the differences

Comparing two texts reliably is an old, solved problem, and it's solved in code. The comparison works in two steps:

1. Align the sections of the two revisions by number and title: section 2 "Warranty term" with section 2 "Warranty term", and so on. A section present in only one revision is reported as added or removed.
2. Diff each aligned pair word by word, with Python's standard difflib, marking removed words [-like this-] and added words {+like this+}, the convention of git diff --word-diff.

For the T-7 warranty:

diff   changed=['2'], unchanged=['3']
Section 2 (Warranty term): changed: T-7 batteries purchased [-during this policy's eligibility period-]  {+on or after 1 January 2026+} carry a warranty of [-2-] {+3+} years from the purchase  [-date. Later policy revisions do not change the term of these purchases.-]  {+date, or 1,000 full charge cycles, whichever comes first.+}Section 3 (Exclusions): unchanged

Two things here are hard for a model reading by eye. The term went from 2 to 3 years, which anyone would notice, but the same sentence now ends with "or 1,000 full charge cycles, whichever comes first", a new limit that matters for every heavily used battery. Small additions inside a long, mostly unchanged sentence are exactly what comparison by reading misses. And "unchanged" is a finding too: the code compared the exclusions character for character, so the answer can say so with confidence.

Part 10 made the same split for arithmetic: the model explains, and code computes. A difference between two texts is a computation.

Whole documents, within a budget

The model receives the computed changes. Should it also receive the documents? Usually yes: the changes say what differs, and the surrounding text says what it means. The cost is length. So the compare path uses a budget:

• If both revisions together fit whole_budget (2,000 tokens in the companion code), the model receives them whole, every section of both.
• If not, it receives only the changed sections of each revision. The diff has already found them, and an unchanged section is reported as unchanged without being supplied.

The T-7 warranties are short, about 170 tokens together, so they're supplied whole:

assemble   mode=whole documents, document_tokens=172, budget=2000, prompt_tokens=398

With the budget set artificially low, the same question switches to changed sections only:

assemble   mode=changed sections, document_tokens=172, budget=50, prompt_tokens=303

Real revisions are longer: a 40-page service manual is tens of thousands of tokens, and two of them can fill a large share of a context window at real cost per question. The diff keeps comparisons affordable at any length, because it's usually small even when the documents aren't.

The prompt

The prompt for this path puts the long, stable parts first and the question last:

Compare the two document revisions below.Code computed the changes, section by section. Explain each change, and say which sections did not change.Don't report changes that aren't listed. Cite the source label, for example [S1], beside each claim.
[S1] WARR-T7-R1, section 22 Warranty termT-7 batteries purchased during this policy's eligibility period carry a warranty of 2 years ...
[S2] WARR-T7-R1, section 3   ...[S3] WARR-T7-R2, section 2   ...[S4] WARR-T7-R2, section 3   ...
Changes (computed by code):Section 2 (Warranty term): changed: ...Section 3 (Exclusions): unchanged
Question:What changed between revision 1 and revision 2 of the T-7 battery warranty?

Every other route puts the question near the top. Here the question comes last, after the long, stable part. Model providers can reuse the processed beginning of a prompt when the next prompt starts the same way (prompt caching), so several questions about the same two revisions pay full price for the documents only once. Speed and cost, caching included, are the subject of a companion article planned alongside a real-implementation version of the code.

The answer, with our stand-in model, which repeats the computed changes and quotes each source:

- Section 2 (Warranty term): changed: T-7 batteries purchased [-during this policy's eligibility period-] {+on or after 1 January 2026+} carry a warranty of [-2-] {+3+} years from the purchase [-date. ...-] {+date, or 1,000 full charge cycles, whichever comes first.+}- Section 3 (Exclusions): unchanged- T-7 batteries purchased during this policy's eligibility period carry a warranty of 2 years from the purchase date. [WARR-T7-R1, section 2]- The warranty does not cover damage from water immersion, ... [WARR-T7-R1, section 3]- T-7 batteries purchased on or after 1 January 2026 carry a warranty of 3 years from the purchase date, or 1,000 full charge cycles, whichever comes first. [WARR-T7-R2, section 2]- The warranty does not cover damage from water immersion, ... [WARR-T7-R2, section 3]

When a document doesn't fit

A comparison stays small thanks to the diff. Other whole-document questions don't have that shortcut: "summarize the risks in this 300-page supplier contract" needs every page, and the pages may not fit.

The common answer is to work in parts, and there are three usual shapes:

• Map-reduce. Ask the same question of each section separately (map), then combine the partial answers (reduce). Sections run in parallel, so it's fast. It loses whatever depends on reading two distant parts together: a liability cap in section 4 that an exception in section 19 lifts.
• Refine. Read the sections in order, carrying a running summary or a running answer from one to the next. It keeps more of the connections, but it's sequential and therefore slow, and an early mistake travels to the end.
• Extract, then answer. If the question is about specific things (every deadline, every penalty clause), first extract them from each section as records with fields, then answer from the records. This is Part 9 applied to one document at question time, and it's often the most reliable of the three, because the final answer reads a short, complete list instead of a long text.

All three keep the rules of this series: the parts and the partial results are traced, every claim cites the section it came from, and arithmetic over the extracted values happens in code.

If the same large documents are asked about again and again, do this work once, at ingestion, instead of per question. That's the intermediate layer from Part 9's side note (derived summaries and records), or a summary tree like RAPTOR from Part 11's side note. Here we only look at the per-question case.

A newer approach changes where the document lives. Recursive language models (Zhang, Kraska, and Khattab, 2025) don't put the long input in the prompt at all. It sits in a programming environment as a variable, and the model writes code to inspect it: search it, slice it, and call a model (itself, recursively) on the slices it chooses. The authors report handling inputs about a hundred times longer than the context window, at comparable cost. If that sounds familiar, it's close to what coding agents already do with a repository: search, open files, follow references, rather than reading everything at once. It's young as a technique for document questions, and it's an agent loop (Part 12) with all its costs and guards, but it's the direction to watch when documents outgrow context windows.

What changed

Q25 guards the router: "What is the warranty term in revision 2 of the T-7 battery warranty?" names a revision but doesn't ask for a comparison, so it stays on search.

The held-out set found a gap. "How does revision 2 of the C-3 battery warranty differ from revision 1?" isn't recognized by the stand-in router's rule, which expects "revision 1 and 2". It goes to search and gets asked for a purchase date, and held-out recall drops from 0.90 to 0.86. A model-based router would very likely recognize it. We leave it as found: adjusting the rule to this question would turn it into a development question, which is the mistake Part 7 warned about.

The system now

flowchart LR
  Q[Question] --> R{Router}
  R -- "search / list / calculate / follow / agent" --> M[Answer]
  R -- "compare" --> RV[Resolve both revisions]:::new
  RV --> D["Align sections, diff in code"]:::new
  D --> B{"Fit the budget?"}:::new
  B -- "yes" --> W[Whole documents + changes]
  B -- "no" --> CS[Changed sections + changes]
  W --> M
  CS --> M
  classDef new stroke-width:3px

routes: search, list, calculate, follow, agent, comparewhole-document budget: 2,000 tokens (whole documents), else changed sections only28 passing, 0 known gaps, 0 regressions; held-out recall 0.86 (H23 missed)

Stop here if

Your whole-document questions are comparisons of versioned documents, or questions about documents that fit your model's context at a cost you accept. Each new kind of document comparison is a small addition: titles that identify revisions, sections that align, and test cases with known changes, including sections that didn't change.

This is the last module. The series has built one system, one gap at a time, and every checkpoint was complete for the questions it answered. Part 14 steps back: given your users' questions, which of these parts do you need, and which can you skip?