From Documents to Evidence

After Part 3, the text in our index is clean. Three known gaps remain, all about how that text is cut:

Take the first one. Asked "Do I need to fit a rear mudguard on a C-3?", the model receives these passages, in this order:

Source: ASSEMBLY-C3-R3, section 4.2Fit the rear mudguard before the first ride. Use the two bolts supplied, tightenthem to 4 Nm, and check that the mudguard does not touch the tyre when the wheel turns.
Source: SPEC-C3-R5, section 5The C-3 frame is aluminum with an internal battery tube. ...
Source: ASSEMBLY-C3-R3, section 4.2Exception: on the cargo version of the C-3, the rear mudguard is not requiredbecause the cargo rack covers the wheel. Keep the rack drainage holes clear.

The rule and its exception both arrived, but as two separate passages, with something unrelated between them. Nothing tells the model they belong together. Here the exception happened to rank third. In a collection of thousands of documents, it could just as easily rank twentieth and never arrive.

The battery check question has the same shape. Asked "Which battery check applies when the battery casing has been opened?", the model receives check R07, but the line saying which bike and battery it applies to ("Scope: C-3, 36 V battery pack...") sits in a different chunk. If Atlas Bikes had a second table for another battery pack, the model couldn't tell which R07 this is.

A chunk has two jobs

Until now, each chunk does two things at once:

1. Be found. Search compares the question with each chunk. A short, focused chunk matches well, because nothing unrelated dilutes it.
2. Be read. The model answers from what it receives. It needs enough context to interpret the text correctly: the exception next to the rule, the scope above the table.

These pull in opposite directions. Make chunks small and they are easy to find but hard to interpret. Make them large and they read well but match poorly, because one large chunk covers many topics. No single chunk size solves both.

The way out is to stop asking one piece of text to do both jobs. Give each piece of content two forms: one that is searched, and one that is supplied to the model when the search finds it.

Two forms for each kind of content

Part 3 kept the type of every element: paragraph, table, figure, slide. Each type gets its own pair of forms:

Each searched piece points to the block it belongs to. Here is the piece for check R07, as the code stores it:

text        = "R07: Replace the battery seal wherever the casing has been opened."search_text = ("Battery safety checks before delivery\n"               "Scope: C-3, 36 V battery pack, frame-mounted, standard charger.\n"               "R07: Replace the battery seal wherever the casing has been opened.")parent_id   = "SPEC-C3-R5-6-table0"

The row is searched together with its section title and scope, so a question about battery checks on the C-3 matches it even though the row itself mentions neither. When it matches, the model receives the table it points to.

The code that builds table rows is short (evidence.py):

for record_id, *cells in d["rows"]:    row = f"{record_id}: {' '.join(cells)}"    searchable.append(Chunk(        f"{table.id}-{record_id}", row, doc.source_id, section.number,        {**meta, "record_id": record_id},        parent_id=table.id, search_text=f"{section.title}\n{d['scope']}\n{row}",    ))

Paragraphs work the same way: each paragraph is searched with its section title and points to the whole section. Figures are searched by their caption and description and point to a block that also names the image file and page. Slides are searched and supplied whole, because a slide is already a small, self-contained unit.

Replace each hit with its block

At question time, one step is new. After search returns the top five pieces, each is swapped for the block it belongs to, and each block is supplied only once:

def supplied_blocks(self, hits):    context, seen = [], set()    for chunk in hits:        block = self.blocks[chunk.parent_id] if chunk.parent_id else chunk        if block.id not in seen:            seen.add(block.id)            context.append(block)    return context

Now the mudguard question receives section 4.2 whole, rule and exception together:

Source: ASSEMBLY-C3-R3, section 4.24.2 MudguardsFit the rear mudguard before the first ride. Use the two bolts supplied, tightenthem to 4 Nm, and check that the mudguard does not touch the tyre when the wheel turns.Exception: on the cargo version of the C-3, the rear mudguard is not requiredbecause the cargo rack covers the wheel. Keep the rack drainage holes clear.

The battery check question is more interesting. Search returns five pieces, all rows of the same table:

0.716  SPEC-C3-R5-6-table0-R070.615  SPEC-C3-R5-6-table0-R020.535  SPEC-C3-R5-6-table0-R060.535  SPEC-C3-R5-6-table0-R100.520  SPEC-C3-R5-6-table0-R01

All five point to the same block, so the model receives one thing: the full table, with its scope line and header. R07 is now unambiguous.

That also changes what "top five" means. We still search for five pieces, but the model may receive one block or five, and a block can be much longer than a piece. What we control is how many pieces search returns; the size of what the model reads now depends on the blocks behind them. Part 6 returns to fitting evidence into the model's limits.

The figure question now receives the figure itself, not only its description:

Source: TEST-BAT-R1, section 3Figure 2. Remaining battery capacity after 500 charge cyclesFigure description: Horizontal bar chart. Pack A keeps 82 percent of its originalcapacity after 500 charge cycles. Pack B keeps 91 percent ...Image: figures/TEST-BAT-R1-fig2.svg (page 8)

A model that reads images can now check the description against the chart, and the answer can point a person to page 8.

Something worth noticing

The case asking for every battery safety check now reports something new:

All eleven checks reach the model, because one row matched and the whole table came with it. That looks like a fix, but it depends on luck: the table happens to be small and in one place. A register with three hundred checks, or checks spread across several documents, would not arrive whole, and nothing in the system knows whether the list it supplied is complete. The case stays open, and Part 9 deals with it properly.

The system now

flowchart LR
  subgraph prepare["Prepare documents"]
    P[Parser output] --> C{{Convert}} --> E{{"Searched + supplied forms"}}:::new
    E --> I[(Index of pieces)]
    E --> B[(Blocks)]
  end
  subgraph answer["Answer a question"]
    Q[Question] --> S[Search pieces] --> X{{"Swap each piece for its block"}}:::new --> PR[Prompt] --> M[Model]
  end
  I -.-> S
  B -.-> X
  classDef new stroke-width:3px

The index holds 32 searchable pieces pointing to 21 blocks. The checks:

Stop here if

Stop here if your questions are answered by finding one passage, table, figure, or slide, and plain wording is enough to find it.

One question still fails at search itself. Asked "What does error E-417 mean, and what should I check?", the top result is the battery safety checks table. It matches the word "check", and our search can't see the error code at all:

Part 5 looks at why search misses evidence that is right there in the index, and what to add so it doesn't.