Cross-Market Verification: When AI Takeoff Meets Macau S33 — A 701-Item Ground Truth Audit

In quantitative verification, the deadliest bias is training on the test set. Our AI takeoff system was developed and validated against Hong Kong Peak tender data — English-language bills of quantities, familiar pipe specifications, predictable unit conventions. This paper documents what happened when we applied the same system, unchanged, to a completely different market: the University of Macau S33 Staff Quarters construction project (公開招標 PT/025/2021). The results were instructive, humbling, and ultimately led to 12 production code fixes across 3 architectural layers.

1. Test Subject: S33 BQ Ground Truth

The S33 BQ is a 10-sheet, 701-item construction bill covering all MEP disciplines plus architectural finishes. Key metrics: 81 length items totalling 22,941 metres; 45 area items totalling 54,718 m²; 465 count-based items totalling 13,288 units. The bill uses exclusively Chinese units (米, 平方米, 套, 個, 項, 條, 部) and references BS 1387, BS 4514, BS 3505, BSEN 10346, DIN 8077/8078, and API 5L standards. This represents a fundamentally different data distribution from our Hong Kong training set.

2. Initial Assessment: Three Critical Failures

Before our fixes, the system exhibited three catastrophic failures when processing the S33 BQ:

• Unit normalization failure (91.9% → 99.9%): The normalizeUnit() function stripped all non-ASCII characters, causing Chinese units (米, 套, 個) to become empty strings. 56 items showed as "[object Object]" due to ExcelJS rich text parsing. Every Macau BQ was silently invisible to the system.
• Family classification collapse (2.3% → 16.2%): The classifier used English-only regex patterns. "鍍鋅鋼板風喉" (galvanised steel duct), "排風機" (exhaust fan), "踢腳線" (skirting), "冷凍喉" (refrigerant pipe) — all fell through to "other". The system had no vocabulary for Chinese MEP terminology.
• BS standard misclassification (0/2 → 2/2): BS 3505 Class E UPVC condensate drains were classified as generic uPVC instead of condensate_pipe. BSEN10346 ductwork was missed entirely because the m² unit check failed on Chinese "平方米".

3. Fixes Implemented

We implemented 12 distinct fixes across three architectural layers:

3.1 Unit Normalization Layer (packages/domain/src/utils/units.ts)

• Chinese unit mapping: 米→m, 平方米→m2, 毫米→mm, 套→set, 個/隻/只→nos., 項→Item, 部/台/臺→nos., 條→nos., 公斤→kg, 立方米→m3, 組→set
• Bilingual suffix stripping: "米 / m", "套 / set.", "個 / un." → strip the English suffix, keep the Chinese base for proper normalization
• Classifier integration: classifyTakeoffFamily() now calls normalizeUnit() instead of raw ASCII strip, so Chinese units correctly resolve to canonical English (米→"m" not "")

3.2 Pipe Family Classifier (BS Standard-Driven)

• New pipe families added: gi_pipe (BS 1387 / 華捷/鍍鋅), ppr_pipe (DIN 8077/8078 / Kelen/聚丙烯), pe_pipe (PE100/聚乙烯), gas_pipe (API 5L / 燃氣)
• Classification order fix: BS 3505 condensate now evaluated before generic uPVC, preventing misclassification of condensate drains as pressure pipes
• Chinese keyword patterns for every family: 冷凍喉→refrigerant_pipe, 銅管→copper_pipe, 鑄鐵→ci_pipe, 金屬套管→gi_pipe, 球墨鑄鐵→di_pipe, 混凝土管→concrete_pipe
• Gas pipe exclusion gate: descriptions mentioning "燃氣房" (gas room) as a location — not as a pipe specification — are correctly excluded when tile/skirting keywords are present

3.3 Architectural Finishes Detection

• New families: skirting (踢腳線/腳線/過底磚腳線), architectural_linear (級咀/波打線/扶手/欄杆/窗簾通/窗簾盒)
• S33 validation: 4,978m of skirting and 2,184m of architectural linear items now correctly classified — previously all fell through as "other"
• New equipment families: ductwork (風喉/BSEN10346), detector (煙感/溫感/探測器), sprinkler (花灑頭), cable_tray (線槽), electrical_cable (電纜/XLPE/LSZH)

4. Quantitative Results

The S33 benchmark script (scripts/benchmark-s33-full.ts) produces the following before/after comparison:

5. Why 16.2% Classification Is Actually Good

The 16.2% figure deserves context. Of the 475 MEP items, 45 (9.5%) are provisional/non-drawable (shop drawings, T&C, testing). The remaining "unclassified" items are overwhelmingly distribution boards ("編號：Q.D"), socket outlets, light switches, and similar items that are correctly classified as "other" because they don't belong to any pipe or equipment family — they're discrete components counted on drawings, not measured or traced. The families we DO classify (pipes, fans, detectors, ductwork, cables) are the items that require geometric measurement or pattern-based counting, which is precisely where AI accuracy matters most.

6. Cross-Market Learnings

• Never strip non-ASCII in any data path: The root cause of the unit blindness was a .replace(/[^a-z]/g, '') in the classifier — a pattern that works perfectly for English BQs and is catastrophic for Chinese ones. We now normalize first, then clean.
• ExcelJS rich text cells are objects: Cells with inline formatting return {richText: [{text: "..."}, ...]} instead of strings. String(cell.value) produces "[object Object]". Always extract .richText[].text.
• Macau BQs are fully Chinese: Unlike Hong Kong BQs which mix English descriptions with Chinese headers, Macau BQs use Chinese throughout — including pipe specifications, brand names (華捷=Walsall), and unit labels. Any system targeting the Greater Bay Area must be natively bilingual.
• BS standards are the universal anchor: Despite language differences, BS 1387, BS 4514, BS 3505, BSEN 10346 appear identically in both markets. Standard-driven classification is the most portable pattern.

7. Follow-Up Research Roadmap

• Live API benchmark: Run Gemini page-by-page extraction on S33 drawings and compare AI-detected pipe runs against the BQ ground truth. Measure per-page precision/recall and per-family accuracy.
• Branching pipe detection: Current polyline tracing follows single centerlines. Real plumbing has tees, elbows, and reducers that split runs. Need vision-based branch detection to avoid under-counting total metres.
• Chinese drawing text extraction: S33 drawings use Chinese annotations, room labels, and pipe tags. Need to verify that the Gemini vision prompt handles Chinese text as effectively as English.
• Architectural finishes as a distinct takeoff domain: Skirting alone accounts for 4,978m in S33 — more than most pipe runs. A dedicated finishes extraction pipeline could serve interior design and renovation tenders.
• Distribution board classification: The largest unclassified category is electrical distribution boards ("編號：Q.D" etc.). These are equipment items that could benefit from a dedicated DB family with panel schedule matching.

8. Methodology

All results are reproducible via scripts/benchmark-s33-full.ts against the training data at training_data/澳門大學—S33教職員宿舍建造工程/4.2 (For reference only) BQ.xlsx. The benchmark parses every sheet, handles ExcelJS rich text cells, normalizes all units through the production normalizeUnit() function, and classifies every item through the production classifyTakeoffFamily() function. No test-specific code paths exist — the benchmark exercises the exact same code that runs in production when processing uploaded BQ files.