Peak Phase 15 — The Quantity-Blind Investigation That Refused a Breakthrough

Phase 15 began with an uncomfortable premise: after fourteen phases, the next apparent gain could still be an artefact of answer leakage, a changed denominator, or a full-page model inventing a plausible total. We therefore treated this phase as an investigation, not a demo. The model had to infer before any ground-truth quantity was joined. Every count had to reconcile with localized instances; every measured length ultimately had to reconcile with unique geometric edges, scale, diameter and drawing scope. If the evidence was absent, the answer had to remain absent.

The honest result is not 80–90%. The honest result is that the first full-page quantity-blind arm failed, the localized arm is incomplete, and the evidence foundation is finally strong enough to tell the difference.

Investigation question

The target was not classification accuracy. Peak full-SOR classification had already reached 90.2%. The unresolved problem was quantity fidelity: can the platform count Schedule of Rates items and measure linear work from drawings, preserve the correct identity and diameter, scale page evidence to building scope only when the drawings justify it, and keep every label and line fixed to the correct geometry under zoom, rotation and pan?

Phase 14 had located the opportunity. On 11,421 ground-truth metres, length detection was already 95.4%, but only 36.4% of metre-mass landed within ±20%. A diagnostic counting oracle on already-paired Peak P&D rows was worth +50.7 percentage points of QA@20. The objects or metre-mass were often present; quantity, identity and scope were not yet trustworthy. Phase 15 tested whether a quantity-blind full-page route could close that gap, while building the evidence machinery required if it could not.

The quantity-blind protocol

For every model quantity task, the permissible SOR context was limited to description, unit, heading, material, diameter and service. Ground-truth quantity, expected metres, comparison verdicts and answer-like corrections were prohibited from prompts, caches and few-shot context. Inference outputs were sealed first; only then could the evaluator join the Schedule C quantity. A runtime guard rejected suspicious quantity fields and phrases, and a source audit found zero product leakage hits.

• One denominator: missed and failed rows stay in the score; difficult pages are not removed.
• No unlocated total receives count credit; an accepted count must reconcile with accepted instance locations.
• No edge may be allocated twice; length must eventually reconcile with accepted unique edges multiplied only by evidenced scope.
• Development, validation and blind projects are split at project level; development results cannot be relabelled as blind.

A0: the full-page live baseline

A0 used Peak P&D as a development-burned set. It requested 22 model runs and completed 21. The count request for P drawing page 2 failed three times when the provider closed the transport socket. That failure remained in the ledger and its rows remained in the denominator. Count used localized extraction locations plus global one-to-one correspondence. Length used pipeLengthScan, which returns estimated runs rather than measured polylines; it is therefore an answer-free estimation baseline, not a geometry result.

The count arm generated 67 result candidates and assigned 23 to the 92 count recipes. Forty-four candidates remained unused, with zero duplicate evidence allocations. The low score was not caused by an evaluator silently discarding duplicates; the extraction and identity evidence simply did not support most required rows. Only two rows landed within ±20%, and none was exact.

The length result reveals a different failure. Thirty-eight of 43 rows were detected, yet only ten were within ±20%, WAPE was 76.4%, and just 11.8% of ground-truth metre-mass belonged to passing rows. The model frequently recognised that a pipe family existed, but did not measure the correct network, separate diameters, or resolve whether the page represented one plan, several houses, or the complete building bill.

A high detection rate for length did not mean the drawing had been measured. It meant the system often named the right kind of work while assigning the wrong amount to it.

What A0 falsified

A0 falsified the idea that a large full-page image, a complete list of SOR identities and a request for a total are sufficient for 80–90% quantity accuracy. The count arm lost local symbols and confused notes, legends and real instances. The length arm named many pipe rows but lacked edge-level geometry. Making that prompt longer would increase apparent explanation without creating the missing measurement evidence.

• Count bottleneck: native-resolution localized evidence, identity-specific search and cross-page scope.
• Length bottleneck: short measured edges, junction and seam reconciliation, scale, material and diameter attribution.
• Building-quantity bottleneck: evidence-backed floor and drawing scope, not a multiplier inferred from the expected SOR total.

A1: localized tiles, interrupted rather than defeated

The next treatment divided each page into a native-resolution 4×3 grid with 15% overlap. Every detected instance had to carry a tile-local location that was transformed into page-global display_normalized_v2 coordinates. A 0.75% seam rule removed repeated evidence at tile borders. Quantity without a location received no credit, and P and D drawing families were routed through separate quantity-blind recipe identities.

The stored partial output could mechanically produce a score, but doing so would be misleading: 15% tile coverage is not the same treatment as a complete tiled page set. The journal therefore records completion and failure counts, preserves all cached responses for resumption, and makes no accuracy claim. Phase 16 starts by completing this exact arm rather than inventing a replacement baseline.

The platform work that survived the failed arm

A failed accuracy arm did not mean the phase produced nothing. Phase 15 moved several pieces from informal research convention into enforceable product and evaluation contracts. These changes do not raise the score by themselves; they make later gains auditable and prevent the viewer from displaying evidence at the wrong place.

1. All 21 training projects are visible

The training_data corpus is now discovered dynamically: 21 projects, 259 assets, 221 PDFs, 10 spreadsheets, 15 DWGs and 12 images. The /projects surface can search the complete corpus and opens a deterministic detail route for every project. Thirteen projects have ingestible SOR or BQ data. The remaining eight display their real blocker instead of pretending that an unevaluated project achieved 0% or silently omitting it. Adding a new training folder now extends the manifest rather than requiring a hand-edited showcase card.

2. Overlay and measurement coordinates were normalised

Evidence is stored in display_normalized_v2: after PDF rotation, top-left origin, coordinates from 0 to 1. Manual measurements no longer preserve the content pixels that happened to exist when the user drew them. At every zoom, the viewer reprojects normalized page geometry into the current viewport, while measured length remains derived from page-point distance. Selected pins, labels and polylines no longer disappear merely because bulk items are hidden at an overview level of detail.

The numeric fixture matrix covered rotations 0, 90, 180 and 270 degrees; zoom 0.25× through 8×; device pixel ratios 1 and 2; and three pan positions. Normalized anchor drift remained within numerical precision and all coordinate domain tests passed. A real authenticated browser visual matrix was not completed in Phase 15, so the journal does not substitute build success for a visual pass.

3. Measurement recipes and evidence engines became explicit

Across 13 readable bills, 6,068 rows now compile into a measurement recipe or an explicit manual-only status. After bilingual unit repairs, count recipes increased from 3,843 to 4,273, network-length recipes from 779 to 823, and manual-only rows fell from 759 to 611. The product length routes consume quantity-blind recipe hints rather than answer-bearing quantities.

• Project scene graph: every page, asset, evidence location and identity edge keeps provenance.
• Closed-world count: only localized plan instances or localized schedule rows are accepted; legends, details and notes are excluded.
• Attributed network length: accepted polylines require scale evidence and compatible material and diameter before the edge ledger can rebuild final metres.
• Global correspondence: unit, measurement basis, family, diameter, material and tag constrain a one-to-one assignment with exclusive evidence allocation.

What was verified — and what was not

The blind evaluator deliberately refused to score. The five frozen blind projects do not yet have a sealed, human-confirmed evidence ledger with at least 300 count rows and 120 length rows. Old Phase 14 outputs were correctly rejected as contaminated for this purpose. A missing blind result is not a zero and not a pass; it is a blocked claim.

Failure ledger

• The full-page count route missed most required localized instances and could not separate many SOR identities by diameter and context.
• The length route returned estimated runs without geometry, so high detection concealed major measurement error.
• Long full-page structured responses repeatedly ended with closed provider sockets.
• Only Peak has seeded live CAD evidence; the other 20 training projects must not display fabricated labels or lines.
• Legacy pixel measurements with insufficient coordinate provenance are not guessed; they must be redrawn.

Decision

NO PROMOTE. Phase 15 did not achieve the 80–90% quantity target.

The full-page A0 estimator remains only as a falsified baseline. The quantity-blind boundary, 21-project manifest, display-normalized coordinates, measurement recipes, evidence engines and strict evaluator remain valuable and should be retained. The A1 tile responses are resumable evidence, not a result. No new accuracy route becomes the product default from this phase.

Phase 16: the arm worth finishing

Phase 16 will not restart from a new prompt. It will resume the exact localized tile cache, complete page coverage including the repeatedly failing page, and preserve the same denominator. Once localized instances are complete, the length treatment will ask each tile for short polylines rather than a page total. A server-side edge ledger will merge compatible junctions and seams, prevent duplicate length, attribute each edge to material and diameter from local evidence, and apply a typical-floor or building multiplier only when the title block, floor schedule, drawing title or another explicit drawing source proves the scope.

• Stage 1 — complete localized tiles and publish coverage, retry and failure ledgers before scoring.
• Stage 2 — add short page-global polyline edges with source tile, endpoint, tangent, scale region and seam fingerprint.
• Stage 3 — add evidence-scored diameter attribution; unknown diameter stays unknown instead of inheriting across reducers or uncertain branches.
• Stage 4 — add evidence-backed scope multipliers; without explicit scope evidence the multiplier is exactly 1.
• Stage 5 — run one-variable-at-a-time ablations, then open the sealed multi-project blind set only after the development gates pass.

The intermediate stop/go gates are count QA@20 of at least 75% and length-weighted QA@20 of at least 70% on the development set, with evidence-position integrity intact. The final claim still requires multi-project blind coverage and 80–90% performance gates; a promising Peak-only result is not enough. The next journal will report the full arm, every failed page and every multiplier source — or it will report another honest falsification.