q2 2026 reports open
Report 047 / Q2 2026

Choose a modelthe way you'd choosea long-term partner.

Compass is disciplined model evaluation. Sample your traffic, score against your rubric, produce a sealed report. Read it, decide, sleep. Repeat next quarter.

Begin a reportRead the method
Reproducible·Sealed reports·Region-pinned·Audit-ready
report · 047 / customer-support
SEALED
recommended default
claude-sonnet-4.6
+0.91
candidate
qual
cost
latency
claude-sonnet-4.6
0.91
$0.012
612ms
gpt-5
0.93
$0.071
1.6s
claude-opus-4.7
0.94
$0.084
1.4s
gemini-3-pro
0.88
$0.018
812ms
llama-4-70b
0.86
$0.0002
240ms
1,200 examples · κ 0.84 · seed 8412↓ report.pdf
Helvetia Cartographic▪Verge Editions▪Field Notes Co.▪Mercury OS▪Northwind Labs▪Foundry 47▪Substrate▪Plover Studio▪Linnea Bakery▪Marlow Workwear▪Helvetia Cartographic▪Verge Editions▪Field Notes Co.▪Mercury OS▪Northwind Labs▪Foundry 47▪Substrate▪Plover Studio▪Linnea Bakery▪Marlow Workwear▪
02 / Method

Four instruments. One reading.

Sampling
billing412
refund218
feature ask184
complaint96
praise47

Stratified samples of live traffic. The bench reflects what your customers actually ask.

Rubric
v 3.4 · 4 dimensions
accuracy1.0×
tone0.6×
brevity0.4×
safety1.2×
κ inter-rater 0.84
Compass
N
S
E
W
true north · 047°
Sealed report
cover
Survey 047 — Q2 2026
customer-support · 1,200 ex.
§Method & sampling
§Rubric & agreement
§Results table
§Cost & latency analysis
§Recommendation
deterministic re-run
$ compass run \
  --report 047 \
  --seed 8412 \
  --region eu-west-1
↓ report-047.pdf
Quarterly
Q1Q2Q3Q4
03 / In production

Three jobs. One discipline.

Choose, review, audit. Three answers a sober team needs from its model stack — and the one report that carries them all.

01
Selection

Choose a model with evidence, not affidavits.

Compass runs your candidates against your own evaluation set, on your data, in your jurisdiction. The leaderboard reflects your problem — not the press release.

0
% of teams change defaults after first report
02
Quarterly review

Models drift. So should the choice of model.

Compass repeats the same survey every quarter, against fresh samples from your traffic. The numbers move only when reality moves. Replace your model when, and only when, the report says so.

0
% reduction in unnecessary churn
03
Audit

A report your regulator can read in twenty minutes.

Sealed PDF, deterministic re-run command, full provenance — model versions, prompts, seeds, scores, agreement metrics. Hand it over with confidence.

0
× faster audit close time
04 / Vs leaderboard

The leaderboard is theatre. Your report is the play.

Capability
Compass
Public leaderboard
Evaluation set
Your sampled traffic
MMLU, vibes
Reproducibility
Sealed report + re-run cmd
Spreadsheet you lost
Quality measure
Rubric + agreement score
Single accuracy number
Drift detection
Quarterly automatic
When complaints arrive
Cost / latency axis
First-class in the report
Mentioned in the appendix
field reports

“Compass replaced four arguments and a planning meeting with one report. We picked our model in an afternoon.”

Maja Nilsen
Head of Engineering, Helvetia Cartographic
0
reports issued
0%
reproducibility
0%
cut from model bills
0×
audit speed-up
05 / Subscriptions

Pay for reports. Not for opinions.

Cardinal
$0free for solo

For independent practitioners.

  • 1,000 evaluations / mo
  • 3 candidate models
  • Public reports
  • Community Discord
Begin a survey
True north
$148per seat / month

For teams choosing models with discipline.

  • 120k evaluations / mo
  • Unlimited candidates
  • Quarterly auto-reruns
  • Sealed PDF reports
  • Slack + Linear alerts
  • Email + chat support
Start 14-day survey
Atlas
—annual contract

For institutions with audit obligations.

  • Unlimited evaluations
  • Air-gap runner
  • Region-locked storage
  • Regulator export
  • Methodologist on retainer
  • 99.95% SLA
Talk to founders
06 / Method notes

Plain answers to recurring questions.

Public benchmarks measure how a model does on questions it might have seen during training. Compass measures how it does on questions your customers actually ask. We never publish your evaluation set, and we encourage you to keep it confidential.

Recent updates

all entries
Apr 25
method
Inter-rater agreement now reported alongside every quality score by default.
Apr 14
report
Sealed PDFs include a deterministic re-run command and full prompt provenance.
Apr 03
atlas
Air-gapped survey runner ships as a single Helm chart with no external dependencies.
Mar 21
drift
Quarterly auto-reruns ship to inbox by Monday morning, with deltas pre-computed.

Begin a report. Find true north.

One thousand evaluations on the house. Three candidate models. A sealed report you can hand to anyone.

Begin a report Speak with methodologist