Peak Phase 10 — Promote Readiness: Earning Default-On (and Why We Did Not)
Phase 9 asked whether the evidence path works. Phase 10 asked whether we may turn it on by default. Working is not the same as ready.
We previously froze a control vs tiled-evidence ablation on Peak P&D (Phase 9): oneshot detection 33.8% / QA@20 14.1% versus tiles 71.8% / 21.1%. Pooled evidence held Phase 6 at 78.2% / 38.0%. Both experiment flags stayed opt-in — cost, offline-only FS/MVAC, and no fresh product-path re-run blocked default-on. Phase 10 (Tasks CA–CH) attacked those promote blockers and the residuals that keep P&D quantity accuracy stuck. Numbers below trace to .tmp/phase10/ — not re-invented for the write-up.
Headline: promote not earned; flags stay off
| Decision | Result |
|---|---|
| CA product-path on BA freeze | Ran — det 72.5% / QA@20 20.4% on 142 drawable |
| vs oneshot | Det +38.7 pp; QA@20 +6.3 pp — product wins |
| vs Phase 9 tiles | Det +0.7 pp; QA@20 −0.7 pp — hold, not a big new win |
| Cost envelope | Fail — ~$4.53 / ~98 min API vs ~$0.79 / ~25 min |
| CB live FS/MVAC | REGRESS — FS QA@20 −49.2 pp; MVAC −27.3 pp |
| Promote either flag default-on? | No |
| Regression / CI floors | PASS on continuity; floors remain Phase 6 |
Headline KPIs remain detection and QA@20 on every drawable ground-truth line — the honest Phase 5 denominator. We do not revive legacy soft-among-eligible as the public score.
Product-path P&D: oneshot vs Phase 9 tiles vs CA
Task CA ran a fresh product-path tiled extract on the Phase 9 BA frozen 142 drawable P&D rows — not a silent reuse of Phase 6 tile inventory. Code path: drawing-extract runTiledEvidencePass defaults (maxEdge 2048, maxTiles 12, overlap 0.15) under FORCE_XAI_VISION. Production defaults stayed off outside the measured arm.
| Arm | Detection | QA@20 | QA@0 | Paired / 142 |
|---|---|---|---|---|
| Oneshot (BA freeze) | 33.8% | 14.1% | 8.5% | 48 |
| Phase 9 tiles (P6 inventory) | 71.8% | 21.1% | 14.1% | 102 |
| Product path (CA fresh) | 72.5% | 20.4% | 12.7% | 103 |
| Δ product − oneshot | +38.7 pp | +6.3 pp | +4.2 | +55 |
| Δ product − P9 tiles | +0.7 pp | −0.7 pp | −1.4 | +1 |
Fresh product path wins the primary comparison against oneshot and holds Phase 9 tiles within −2 pp. It is not a large new accuracy win over the inventory freeze. Stretch P&D QA@20 ≥25% was not earned (20.4%). Fittings and pumps remain hard; detection still outruns usable quantity.
Cost and latency: the hard promote gate
| Arm | Tiles / API | Latency | Est. cost (USD) |
|---|---|---|---|
| Phase 9 inventory (heavy cache resume) | 132 planned / ~27 live | ~25 min | ~$0.79 |
| CA product path | 132 planned / 108 tile API | ~98 min API / ~142 min wall | $4.53 |
Phase 9’s ~$0.79 reflected heavy Phase 6 cache resume, not a fair cold product budget. Even so, a multi-dollar multi-hour cold path is not acceptable as silent default-on without a documented product-accepted budget. Beating oneshot accuracy does not authorize that spend. CF marks costEnvelope.beatsPhase9 = false — a hard promote fail independent of the det/QA@20 hold.
Live FS / MVAC tiles: REGRESS
Phase 9 only offline-held FS and MVAC. Phase 10 Task CB ran live EVIDENCE_TILED_EXTRACT on Peak FS (5 pages) and AC/MVAC (2 pages). Native renders are 7021 px on the long edge — well above the 2400 min-edge no-op skip — so product default-on would tile these sheets.
| Sheet | Metric | Offline (P9) | Live CB | Δ (pp) | Hold (−2pp) |
|---|---|---|---|---|---|
| FS | Detection | 81.4% | 79.7% | −1.7 | yes (det only) |
| FS | QA@20 | 64.4% | 15.3% | −49.2 | no |
| MVAC | Detection | 100% | 72.7% | −27.3 | no |
| MVAC | QA@20 | 63.6% | 36.4% | −27.3 | no |
Verdict: REGRESS. Combined cost ~$2.01 / ~55.5 min does not rescue the hold failure. Default-on tiles would move FS/MVAC quantity accuracy the wrong way. This alone blocks EVIDENCE_TILED_EXTRACT promotion.
Pooled compositions: label the arms
| Composition | Pooled det | Pooled QA@20 | Role |
|---|---|---|---|
| P9 evidence (P&D tiles + FS/MVAC offline) | 78.2% | 38.0% | Phase 9 / Phase 6 baseline |
| CA product P&D + offline FS/MVAC | 78.6% | 37.6% | Headline continuity (−2pp hold) |
| CA product P&D + live FS/MVAC | 74.4% | 21.4% | Default-on risk — regresses |
Report 78.6% / 37.6% only with the continuity label. Do not headline the full live composition as promote-ready. Stretch pooled QA@20 ≥42% remains unearned (37.6% continuity; 21.4% live).
Residuals: legend clustering and Cov@P90
| Residual | Control | After Phase 10 | Δ / verdict |
|---|---|---|---|
| Legend-seeded QA@20 (P&D) | 21.1% | 21.8% | +0.7 pp — stretch ≥25% not earned |
| Legend detection | 71.8% | 71.8% | 0.0 pp |
| P&D Cov@P90 | 0% | 10.6% | NONZERO at ~93% precision |
| FS Cov@P90 (sanity) | ~66.1% | 67.8% | must not collapse |
Legend work is aggregation-only template matching on Phase 6 tiled AI raw (111 seeds; plan pins preferred on legend–plan conflicts). Cov@P90 uses structural confidence floors without leaking ground-truth quantities. Both inform residual narrative; neither alone promotes tiles.
Cross-project INFO
| Project / bank | Result | CI floor? |
|---|---|---|
| Kwai On | Classification 84.5% reconfirmed; det/QA@20 null (no scorable qty path) | No |
| CPS | Still INFO (det 61.5% / QA@20 38.5% freeze); graduateToCi=false; GT delta stub only | No |
| Corrections seeds | v1→v2 rebalance (15/15); live after deferred; seedHelped false (n=1 before) | No |
Regression and promote path
Task CG reports overall PASS against Phase 6 continuity floors on product-path P&D plus offline FS/MVAC (pooled det ≥76.2%, QA@20 ≥36.0%, P&D QA@20 ≥19.1%). Floors are not raised: promote requires CF yes plus clean CA+CB. Cost failed; live FS/MVAC regressed. Both flags stay opt-in. Targeted recount had no redesigned arm this phase — Phase 9 already showed −0.7 pp QA@20 at extra cost.
Phase 11 backlog
- Cost cuts for product tiles — adaptive skip, fewer tiles, or a documented product-accepted default-on budget (cold path ~$4.53 / multi-hour is the main P&D promote blocker after accuracy holds).
- Recover FS/MVAC live tile hold within −2 pp of offline floors — default-on must not collapse quantity accuracy on those sheets.
- P&D QA@20 residual — product 20.4% / legend 21.8% still short of stretch ≥25%; fittings, valves, pumps remain hard.
- Full Kwai On quantity path — classification 84.5% reconfirmed; det/QA@20 still null until estate-vs-block GT is designed.
- CPS CI graduation after a frozen GT delta table, parent-heading contract tests, and fresh vision under the graduated parser.
- Redesigned targeted recount only if it beats tiled QA@20/det at justified cost (Phase 9 BD failed).
- Network-aware length summation and remaining S33 unclassified MEP families (older backlog).
Phase 10 closes the promote-readiness gate Phase 9 opened. Fresh product tiles still beat oneshot on Peak P&D when you pay for them; cold cost and live FS/MVAC regression mean they are not yet the right silent default. Residuals moved a little (legend +0.7 pp, Cov@P90 10.6%); they did not buy promote. The next claims will need the same discipline: freeze rows, label every arm, and refuse default-on without measurement.
Related articles
Phase 28 — Coverage Ladder 5/21 Held; Count CP Still 3/63, Not 80/80
Phase 28 is the execute-close of the same qty_3d coverage + S33 count-CP families: inner COVERAGE_LADDER_PASS, five training_data graphs kept (S33, N23, Kwai On, S27, N22), fabricatedGeometry false. Extra S33 plan pages 12/14/16/19 raised world elements to 41 on the same slug — not a sixth coverage credit. Count CP stayed evaluated at 3/63 on S33 BQ sheet E (detection 5/63; N=63 held; reasonCode null). Frozen loop RSI_PLATEAU. Live 圖則.pdf is a residual, not a coverage credit. Not Peak P&D 3D 80/80. Not corpus 80–90%.
14 min readPhase 27 — Coverage Ladder 5/21; Count CP 3/63, Still Not 80/80
Phase 27 is the execute-close of the coverage-ladder 3→5 + S33 count-CP pack: COVERAGE_LADDER_PASS, five training_data graphs (S33, N23, and Kwai On preserved; S27 and N22 new), fabricatedGeometry false. Count CP is evaluated at 3/63 on S33 BQ sheet E — unblocked, not IDENTITY_ONLY_NO_GT, and not an accuracy win. S27 geometric takeoff was empty (NO_PLAN_POLYLINES); fittings-only still qualified. Live 圖則.pdf is a residual, not a coverage credit. Not Peak P&D 3D 80/80. Not corpus 80–90%.
12 min read