# OpenAuditor as a gate in front of 30 models

This analysis replays the completed 30-model routing experiment with the same
local confidence threshold selected before the original test. It sends no new
API requests and does not tune a threshold against test labels.

Run from the repository root:

```sh
python tools/analyze_routing_gate.py
```

The tool first verifies every frozen source file and reproduces the saved
threshold from the 200 validation cases. For each of 500 test cases, it accepts
the local answer when its maximum candidate probability reaches the threshold;
otherwise it uses that API's recorded answer. The threshold is 0.8502035140991211.

The gate accepts 490 cases and escalates 10. Accepted cases contain eight errors:
1.63%, above the original validation-selection target of 1%. Every case was
actually sent to every API in the source benchmark. The 98% reduction in calls
describes the replay, not actual calls avoided during the experiment.

`summary.json` contains all 30 before/after comparisons, paired gains and losses,
95% Wilson accuracy intervals, median and p95 replay durations, actual charges
used in the replay, and unverified-cost indicators. `comparison.csv` is a compact
export. The scenario charges local warm compute at $0.50 per allocated hour and
100% utilization. Latency adds recorded local and hosted durations for escalated
cases; load, queueing, and cold starts require a live experiment.

This is an exploratory system comparison on familiar eight-way intent routing,
not a new held-out result, an audit-workflow benchmark, a matched-quality proof,
or a universal cost win. Cheap API baselines remain in the comparison. The
separate audit-gate protocol describes the next workload and has no results yet.