Accuracy Research Phase 2: Five Studies That Pushed MEP Classification from 22% to 31% — Distribution Boards, Architectural Finishes, Junction Detection, and Chinese Drawing Verification

In our previous post, we documented the cross-market audit of our AI takeoff system against the 701-item Macau University S33 BQ. That audit revealed a 22.3% MEP classification rate — meaning nearly 4 in 5 MEP items fell into the "other" bucket, invisible to pipe-family matching and quantity verification. This paper presents five follow-up studies that systematically addressed the largest classification gaps, raising MEP coverage to 31.4% and establishing infrastructure for the next phase of accuracy improvements.

Study 1: Distribution Board Classification

The S33 electrical section (C 強電系統) contains 29 distribution board items — panel boards, switchboards, floor distribution boxes — that were all classified as "other". These items follow a distinctive pattern: item codes like "Q.D", "Q.SC", "Q.E" with Chinese descriptions referencing 配電箱, 開關箱, 分線箱, or 配電開關設備.

Result: 29 items moved from "other" to "distribution_board", representing 100 套 (sets) of electrical distribution equipment. Additionally, we added socket_outlet (16 items), switch_isolator (14 items), junction_box (5 items), and luminaire (6 items) families, together capturing 70 previously unclassified electrical items.

Study 2: Architectural Finishes as Standalone Domain

The S33 "B 建築及室內裝修" sheet contains 177 items including 37 area-based items totalling 54,554 m². Previously, only linear items (skirting, handrails) were classified. We added six new area-based families driven by Chinese + English keywords with m² unit validation.

The waterproofing family required careful ordering — it must match before generic floor/wall tiling to avoid mis-classifying "防水水泥砂漿平整地臺" (waterproof cement mortar levelling) as floor tiling. Similarly, plastering items like "水泥砂漿平整地臺" (cement mortar levelling) without the 防水 prefix are correctly separated.

Study 3: Branching Pipe Detection (T-Junction Awareness)

Drainage and plumbing networks inherently branch — a single soil stack connects to multiple waste branches via T-junctions, elbows, and reducers. Our original lengthTakeoff prompt treated each pipe run as independent, potentially under-counting total network length when branches were not traced as separate polylines.

We extended the Gemini vision prompt to explicitly detect and report junctions with type classification (tee/elbow/reducer/cross), location coordinates, and connected run keys. The geometric-length-takeoff.ts parser now processes the optional junctions array and records branchCount in the audit snapshot. This data enables future network-aware length summation where connected polylines sharing junction points can be merged for accurate total-network measurement.

Study 4: Chinese Drawing Text Extraction Verification

We created a dedicated verification script (scripts/verify-chinese-extraction.ts) that renders S33 drawing pages via pdftoppm, calls Gemini vision with both textExtract and lengthTakeoff modes, and measures detection rates against a curated ground truth of 19 Chinese room labels (客廳, 衛生間, 廚房, etc.), 20 equipment names (排風機, 配電箱, 花灑, etc.), and pipe/scale annotation patterns.

The script runs in offline mode without GEMINI_API_KEY (reporting ground truth analysis only) or in live mode with API access. This infrastructure enables continuous regression testing of Chinese drawing support as the extraction pipeline evolves.

Study 5: Live API Benchmark Framework

We extended the benchmark-length-takeoff.ts script with a Phase 2 section that, when GEMINI_API_KEY is available, renders Peak P&D pages via pdftoppm, calls Gemini's lengthTakeoff with SOR hints, classifies each detected run into pipe families, and computes per-family precision, recall, and mean absolute error (MAE) against the 44-row SOR ground truth.

Combined Impact: S33 Before vs After

Implementation Details

• takeoff-structure.ts: Added 11 new TakeoffFamily types (distribution_board, socket_outlet, switch_isolator, junction_box, luminaire, hydrant, fire_pump, wall_tiling, floor_tiling, ceiling_finish, painting, waterproofing, plastering) with Chinese + English regex patterns.
• prompt.ts: Extended lengthTakeoff prompt with junction detection schema (type, location, connectedRunKeys). System prompt updated to instruct "trace EACH branch as a separate polyline segment".
• geometric-length-takeoff.ts: Added Junction interface, VisionJunction parsing, branchCount tracking in audit snapshots, and junctions array in PageLengthTakeoffResult.
• verify-chinese-extraction.ts: New benchmark script with 19 room labels, 20 equipment names, 4 pipe annotation patterns, 3 scale patterns as ground truth. Supports offline and live (GEMINI_API_KEY) modes.
• benchmark-length-takeoff.ts: Phase 2 added with live Gemini calls, per-family precision/recall/MAE computation, and SOR hint injection for grounded extraction.

Follow-Up Research Roadmap

• Remaining 281 unclassified MEP items (59.2%): Many are item-specific electrical accessories (valve boxes, test points, cable markers) that need a "generic_electrical" catch-all or more specialized families.
• Live API benchmark with actual P&D drawings: Run Phase 2 benchmark with GEMINI_API_KEY to measure real precision/recall against the 44-row SOR ground truth.
• Chinese drawing verification with S33 actual pages: Run verify-chinese-extraction.ts on S33 to measure Gemini's Chinese text detection accuracy on real Macau drawings.
• Network-aware length summation: Use junction data to connect polyline segments sharing junction points, computing true network length vs independent-run sum.
• Fire services dedicated classification: Hydrant, hose reel, fire pump, sprinkler system as integrated families rather than isolated detection patterns.

Conclusion

This phase demonstrates that classification accuracy follows a long-tail distribution: the first 22% came from pipe families targeting Hong Kong P&D conventions; the next 9 percentage points required understanding Macau BQ patterns (Chinese item codes, 分線箱 as distribution boards, FRC cable designations), architectural finishes as a first-class domain, and infrastructure for measuring extraction quality on Chinese drawings. The marginal cost of each percentage point increases — but each point represents real items that a QS reviewer would otherwise need to manually identify. The benchmark scripts and verification tools created in this phase ensure that future improvements can be measured, not merely claimed.