Peak Phase 7 — The Bill of Quantities We Never Looked At

Every phase before this one pointed a vision model at a drawing and scored what came back. Phase 7 asked a different question: what are we actually scoring it against? The bill of quantities — the spreadsheet or PDF that prices the job and supplies every "ground truth" number this whole research programme has been comparing the AI to — had never once been examined for correctness. It turned out to have a bug of its own, and on Peak's own P&D sheet, that bug was severe enough to make some SOR lines mathematically impossible to score correctly.

A bill line says "150 mm dia." twice under the same heading — once for 513 metres, once for 54. No extractor, however good, can tell those apart. That is not a vision problem. It is a ground-truth problem, and we had been scoring against it for six phases.

Why the material was missing

Bill of quantities lines are usually bare dimensions — "225 mm dia.", "100 mm diameter" — and mean nothing without the heading above them, which is where the material lives: "i. C.I. pipe to BS 437", "ii. Precast concrete pipe", "iii. Ductile iron pipe". Peak's own parser recognised section headings with a pattern matching a., b., c. — letters "a" through "g" only. Roman numerals — i., ii., iii. — fell outside that pattern and were never recognised as headings at all.

The consequence, measured directly in the cached extraction artefacts: 118 of Peak P&D's 146 priced rows — 81% — were bare dimensions whose material had been silently dropped, all of them inheriting the same truncated wrapped heading. Worse, 18 rows across 9 groups shared an identical (parent heading, description, unit) combination — the same "150 mm dia." appearing twice with different quantities, or "100 mm dia." appearing twice. Those pairs are not merely hard to match; they are structurally indistinguishable. FS and MVAC, Peak's other two sheets, had zero such collisions. P&D — the sheet every phase from 3 onward struggled hardest with — was carrying a data-integrity bug the whole time.

One misplaced keyword, most of a language

The Chinese-language bills exposed a second, smaller-looking bug with an outsized effect. Our column detector treated the header word 項目 as a synonym for "description". In these bills it is not — it is the item reference column ("1.1", "2.1.1"), with the actual description one column over under 描述. Column 0 therefore beat the real description column on every Chinese bill in the corpus, and classification collapsed to whatever text happened to land in the reference column.

N23's 100% is not a fluke worth distrusting — after the fix, our general parser produces exactly the same 14 priced rows as the bespoke Python converter Phase 4 had to write by hand for that one project. One keyword, misfiled, was worth most of the Chinese half of the corpus.

One parser instead of five

Before this phase, every bill format had grown its own bespoke, throwaway parser: the production TypeScript parser for xlsx, a Python script for N23's legacy .xls, another Python script for Kwai On that read the workbook's raw XML directly, a third parser buried in the S33 benchmark script, and a fourth ad-hoc BQ-text reader for good measure. None of them shared code. The one that got the heading logic right — a heading STACK rather than a single nearest heading, which alone was worth 51.6 points of classification accuracy on a held-out project in Phase 5 — lived only in the Kwai On throwaway script and had never reached production.

Phase 7 replaced that with one parser split cleanly into readers and structure. Readers do nothing but turn a file — xlsx, legacy xls, or a text-layer PDF — into rows of plain strings. A single shared structure pass then does everything else: finding the header row, mapping columns, inferring heading depth per sheet rather than assuming a fixed hierarchy (because different bills nest "a." under "1." and others do the reverse), and rejoining descriptions that wrap across rows. A fix made once now applies to every format, instead of being rediscovered five times.

Two details mattered enough to note. First, PDF bills and spreadsheets wrap descriptions in opposite directions: a spreadsheet wraps the text before the quantity row, while a PDF continues the description on lines after it — the parser now handles both. Second, no new dependency was added anywhere in this work. A JavaScript library for legacy .xls files, or a proper test framework, would each have changed the project's lockfile — and AGENTS.md records that an out-of-sync lockfile is a known Vercel deployment failure, a risk not worth taking while another phase was actively working the same tree. Legacy files are read by shelling out to Python's existing xlrd library, and the new tests run on Node's built-in test runner.

Results across the corpus

Four metrics, chosen so none of them can be improved by narrowing what counts. Ingestion coverage: how many of the 21 training projects can be read at all. Classification: of the lines that were read, how many get assigned a real construction family rather than the catch-all "other" — the hard ceiling on everything downstream, since a line filed as "other" can never be reached by the matching passes no matter how good the vision extraction is. Attribution: of the bare-dimension lines, how many now carry a real material in their parent heading. Collisions: the structurally-unpairable duplicate rows.

Eight projects remain unreachable, and we are explicit about which and why rather than guessing at them: five are scanned PDFs with no text layer at all (optical character recognition is deliberately out of scope for this phase), two are text-layer PDFs whose table layout our line-matcher does not yet recognise, and one project has no bill of quantities in the corpus at all. A plausible-looking wrong answer is worse than an honest "unsupported", and every one of those eight is reported as the latter.

The honest result

Here is where the phase earns its title. We took the exact same AI extraction Phase 5 had already scored — nothing re-run, nothing re-tuned — and swapped only the ground truth it was compared against, from the old parser's output to the corrected one. If the corrected ground truth were simply "more correct", Peak's score should rise, or at worst hold. It did neither.

Two fewer correct answers, on a slightly larger set of questions. Our first guess was that the corrected ground truth's longer heading paths — up to 120 characters of section titles prepended before the actual dimension — were diluting the text-similarity matching. We tested that directly, feeding the matcher progressively shorter slices of the heading stack instead of the full path. The result stayed flat: 74, 74, 75, 73 correct across four different amounts of heading context, against a baseline of 76. The dilution theory does not hold.

The likelier, more uncomfortable explanation: the matcher was tuned against the broken ground truth. Every matching pass, every quality gate, every family pattern built across Phases 2 through 5 was developed by looking at failures produced under the old, buggy parsing. Correcting the label after the fact invalidates part of that fitting. Peak's published headline was, to some degree, measuring agreement between two artefacts of the same pipeline — not agreement with the actual bill.

This does not make the correction wrong. A ground truth where "150 mm dia." appears twice with different quantities under an identical key is indefensible whatever number it happens to produce downstream. What it means is that the benefit of fixing it shows up where it should — on projects nothing was ever tuned against, like Kwai On and the Macau bills above — rather than on Peak, where six phases of matcher development had already learned to work around the very bug we just removed. We report this as the headline finding of the phase rather than quietly setting it aside, because a research programme that only publishes the numbers that went up is not one that can be trusted on the numbers that matter.

Running alongside another phase without colliding

Phase 7 executed in full while Phase 6 — the live tiling and detection sprint published separately — was working the same codebase at the same time. That was only possible because the two phases were scoped to touch disjoint files, and because every new parsing behaviour here sits behind a mode flag defaulting to the old, unchanged behaviour. Concurrent work kept getting byte-for-byte the same ground truth it started with; flipping the default to the corrected parser is a separate, deliberate decision for after both phases have landed. We verified this rather than assumed it: the exact same rescoring command produced identical output, line for line, before and after this phase's changes.

What is still unresolved

• Peak shows no measured gain, and we did not chase one — re-tuning the matcher against the freshly corrected labels is how the original bug went unnoticed for six phases.
• The corrected parser is not yet the default anywhere in production or in the live extraction pipeline.
• Kwai On Factory Estate still shows a 48.7% collision rate — these look like genuine repeated items across different building blocks that a block-level heading is not yet capturing, not a parsing bug.
• Two text-layer PDF bills (HKUST Medicine, MCC) use a table layout our line-matching pattern does not yet recognise.
• Five scanned bills need OCR, which was deliberately kept out of scope for this phase.

Six phases in, the pattern holds: every time we look closely enough at a number we are proud of, we find the measurement was easier to fix than the thing it was supposed to measure. That is not a comfortable finding to publish, and it is exactly why it belongs in the record.