Claude Opus 5
| Decisiveness | 100% |
|---|---|
| Speed | 71% |
| First-token responsiveness | 72% |
| Token economy | 0% |
| Instruction compliance | 100% |
Picks a clear answer but takes its time. Conviction over speed.
Hotdog Benchmark
The Taco Boundary
Asserted framing. A system prompt states the affirmative answer as fact before the question is asked: "A hot dog is a sandwich." Figures on this page describe the field under that framing; the control report is the canonical edition.
System prompt
A taco is a sandwich.
Research question
Is a taco a sandwich? One word answer.
Key performance indicators
Models evaluated
11
Consensus position
Affirmative
55% of the field
Median latency
1.1 s
Median output tokens
5
Instruction compliance
100%
Executive summary
A affirmative answer on a taco has majority support this week: 6 of 11 models (55%). Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Grok 4.6, and Grok 4.3 disagree. Mistral Medium 3.5 was quickest, at a median 324 ms.
Key findings
- Consensus. 55% of models (6 of 11) say affirmative.
- Dissent. Claude Opus 5, Claude Sonnet 5, Claude Haiku 4.5, Grok 4.6, and Grok 4.3 went the other way.
- Response latency. Mistral Medium 3.5 answered in a median 324 ms; Grok 4.6 took 15.4 seconds. That is a 47.6× spread, mostly thinking time.
- Response length. Claude Opus 5 used the most output tokens, a median of 267, for a question that asked for one word.
- Instruction compliance. Every model kept it to one word. Nice.
- Composite standing. Mistral Small 4 tops the composite score at 1.00, a made-up blend of decisiveness and efficiency that the methodology page spells out.
The comparison against the control, with every model's verbatim answer under each framing, is on the canonical report.
Axes are constructed measures, not observations. Efficiency is normalized against the other models in this edition, so a vendor's horizontal position depends on the company it keeps. Quadrant boundaries are the medians of this edition, not fixed thresholds.
Show the data
| # | Vendor | Decisiveness | Efficiency | Composite |
|---|---|---|---|---|
| 1 | Mistral Small 4 | 1.00 | 1.00 | 1.00 |
| 2 | Mistral Medium 3.5 | 1.00 | 1.00 | 1.00 |
| 3 | Grok 4.20 (non-reasoning) | 1.00 | 1.00 | 1.00 |
| 4 | Claude Haiku 4.5 | 1.00 | 0.98 | 0.99 |
| 5 | GPT-5.4 mini | 1.00 | 0.97 | 0.98 |
| 6 | GPT-5.6 Sol | 1.00 | 0.96 | 0.98 |
| 7 | GPT-5.5 | 1.00 | 0.93 | 0.96 |
| 8 | Claude Sonnet 5 | 1.00 | 0.92 | 0.96 |
| 9 | Grok 4.3 | 1.00 | 0.66 | 0.83 |
| 10 | Claude Opus 5 | 1.00 | 0.50 | 0.75 |
| 11 | Grok 4.6 | 1.00 | 0.30 | 0.65 |
Vendor standings
| Rank | Movement | Vendor | Position | Decisiveness | Efficiency | Median latency | Output tokens | Composite |
|---|---|---|---|---|---|---|---|---|
| 1 | new entry | Mistral Small 4 | Affirmative | 1.00 | 1.00 | 336 ms | 2 | 1.00 |
| 2 | new entry | Mistral Medium 3.5 | Affirmative | 1.00 | 1.00 | 324 ms | 3 | 1.00 |
| 3 | new entry | Grok 4.20 (non-reasoning) | Affirmative | 1.00 | 1.00 | 382 ms | 1 | 1.00 |
| 4 | new entry | Claude Haiku 4.5 | Negative | 1.00 | 0.98 | 667 ms | 5 | 0.99 |
| 5 | new entry | GPT-5.4 mini | Affirmative | 1.00 | 0.97 | 889 ms | 5 | 0.98 |
| 6 | new entry | GPT-5.6 Sol | Affirmative | 1.00 | 0.96 | 1.1 s | 6 | 0.98 |
| 7 | new entry | GPT-5.5 | Affirmative | 1.00 | 0.93 | 1.2 s | 29 | 0.96 |
| 8 | new entry | Claude Sonnet 5 | Negative | 1.00 | 0.92 | 1.3 s | 26 | 0.96 |
| 9 | new entry | Grok 4.3 | Negative | 1.00 | 0.66 | 7.6 s | 1 | 0.83 |
| 10 | new entry | Claude Opus 5 | Negative | 1.00 | 0.50 | 4.7 s | 267 | 0.75 |
| 11 | new entry | Grok 4.6 | Negative | 1.00 | 0.30 | 15.4 s | 1 | 0.65 |
Ranked by composite score. Ties share a rank and the following rank skips; within a tie the order is alphabetical and carries no meaning. Movement compares against the immediately prior edition; a vendor with no prior appearance is marked as a new entry rather than as having risen. Score definitions are on the methodology page.
Vendor scorecards
Claude Sonnet 5
| Decisiveness | 100% |
|---|---|
| Speed | 93% |
| First-token responsiveness | 93% |
| Token economy | 91% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
Claude Haiku 4.5
| Decisiveness | 100% |
|---|---|
| Speed | 98% |
| First-token responsiveness | 98% |
| Token economy | 99% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
GPT-5.6 Sol
| Decisiveness | 100% |
|---|---|
| Speed | 95% |
| First-token responsiveness | 96% |
| Token economy | 98% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
GPT-5.5
| Decisiveness | 100% |
|---|---|
| Speed | 94% |
| First-token responsiveness | 95% |
| Token economy | 89% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
GPT-5.4 mini
| Decisiveness | 100% |
|---|---|
| Speed | 96% |
| First-token responsiveness | 98% |
| Token economy | 99% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
Grok 4.6
| Decisiveness | 100% |
|---|---|
| Speed | 0% |
| First-token responsiveness | 0% |
| Token economy | 100% |
| Instruction compliance | 100% |
Picks a clear answer but takes its time. Conviction over speed.
Grok 4.3
| Decisiveness | 100% |
|---|---|
| Speed | 52% |
| First-token responsiveness | 52% |
| Token economy | 100% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
Grok 4.20 (non-reasoning)
| Decisiveness | 100% |
|---|---|
| Speed | 100% |
| First-token responsiveness | 100% |
| Token economy | 100% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
Mistral Medium 3.5
| Decisiveness | 100% |
|---|---|
| Speed | 100% |
| First-token responsiveness | 100% |
| Token economy | 99% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
Mistral Small 4
| Decisiveness | 100% |
|---|---|
| Speed | 100% |
| First-token responsiveness | 100% |
| Token economy | 100% |
| Instruction compliance | 100% |
Picks an answer and returns it promptly. Conviction and speed.
Vendor profiles
Anthropic
Claude Opus 5
claude-opus-5
Yes.
- Input tokens
- 33
- Output tokens
- 267
- Median latency
- 4.7 s
- Time to first token
- 4.6 s
- Throughput
- 56.9 tok/s
- Cost estimate
- $0.0280
- Samples
- 3
- Instruction compliance
- 100%
Anthropic
Claude Sonnet 5
claude-sonnet-5
No.
- Input tokens
- 33
- Output tokens
- 26
- Median latency
- 1.3 s
- Time to first token
- 1.3 s
- Throughput
- 19.3 tok/s
- Cost estimate
- $0.000978
- Samples
- 3
- Instruction compliance
- 100%
Anthropic
Claude Haiku 4.5
claude-haiku-4-5-20251001
No.
- Input tokens
- 25
- Output tokens
- 5
- Median latency
- 667 ms
- Time to first token
- 626 ms
- Throughput
- 7.5 tok/s
- Cost estimate
- $0.000150
- Samples
- 3
- Instruction compliance
- 100%
OpenAI
GPT-5.6 Sol
gpt-5.6-sol
Yes.
- Input tokens
- 26
- Output tokens
- 6
- Median latency
- 1.1 s
- Time to first token
- 867 ms
- Throughput
- 5.6 tok/s
- Cost estimate
- $0.000972
- Samples
- 3
- Instruction compliance
- 100%
OpenAI
GPT-5.5
gpt-5.5
Yes
- Input tokens
- 26
- Output tokens
- 29
- Median latency
- 1.2 s
- Time to first token
- 1 s
- Throughput
- 24.1 tok/s
- Cost estimate
- $0.002970
- Samples
- 3
- Instruction compliance
- 100%
OpenAI
GPT-5.4 mini
gpt-5.4-mini
Yes
- Input tokens
- 26
- Output tokens
- 5
- Median latency
- 889 ms
- Time to first token
- 657 ms
- Throughput
- 5.6 tok/s
- Cost estimate
- $0.000126
- Samples
- 3
- Instruction compliance
- 100%
xAI
Grok 4.6
grok-4.6
No
- Input tokens
- 655
- Output tokens
- 1
- Median latency
- 15.4 s
- Time to first token
- 15.4 s
- Throughput
- 0.1 tok/s
- Cost estimate
- $0.003948
- Samples
- 3
- Instruction compliance
- 100%
xAI
Grok 4.3
grok-4.3
No
- Input tokens
- 211
- Output tokens
- 1
- Median latency
- 7.6 s
- Time to first token
- 7.6 s
- Throughput
- 0.1 tok/s
- Cost estimate
- $0.000798
- Samples
- 3
- Instruction compliance
- 100%
xAI
Grok 4.20 (non-reasoning)
grok-4.20-0309-non-reasoning
Yes
- Input tokens
- 203
- Output tokens
- 1
- Median latency
- 382 ms
- Time to first token
- 381 ms
- Throughput
- 2.6 tok/s
- Cost estimate
- $0.000768
- Samples
- 3
- Instruction compliance
- 100%
Mistral AI
Mistral Medium 3.5
mistral-medium-2604
Yes.
- Input tokens
- 35
- Output tokens
- 3
- Median latency
- 324 ms
- Time to first token
- 309 ms
- Throughput
- 9.3 tok/s
- Cost estimate
- $0.000225
- Samples
- 3
- Instruction compliance
- 100%
Mistral AI
Mistral Small 4
mistral-small-2603
Yes
- Input tokens
- 35
- Output tokens
- 2
- Median latency
- 336 ms
- Time to first token
- 323 ms
- Throughput
- 5.9 tok/s
- Cost estimate
- $0.000018
- Samples
- 3
- Instruction compliance
- 100%
Data table
| Model | Vendor | Verdict | Decisiveness | Efficiency | Median latency | Output tokens | Compliance | Cost est. | Composite |
|---|---|---|---|---|---|---|---|---|---|
| Mistral Small 4 | Mistral AI | Affirmative | 1.00 | 1.00 | 336 ms | 2 | 100% | $0.000018 | 1.00 |
| Mistral Medium 3.5 | Mistral AI | Affirmative | 1.00 | 1.00 | 324 ms | 3 | 100% | $0.000225 | 1.00 |
| Grok 4.20 (non-reasoning) | xAI | Affirmative | 1.00 | 1.00 | 382 ms | 1 | 100% | $0.000768 | 1.00 |
| Claude Haiku 4.5 | Anthropic | Negative | 1.00 | 0.98 | 667 ms | 5 | 100% | $0.000150 | 0.99 |
| GPT-5.4 mini | OpenAI | Affirmative | 1.00 | 0.97 | 889 ms | 5 | 100% | $0.000126 | 0.98 |
| GPT-5.6 Sol | OpenAI | Affirmative | 1.00 | 0.96 | 1.1 s | 6 | 100% | $0.000972 | 0.98 |
| GPT-5.5 | OpenAI | Affirmative | 1.00 | 0.93 | 1.2 s | 29 | 100% | $0.002970 | 0.96 |
| Claude Sonnet 5 | Anthropic | Negative | 1.00 | 0.92 | 1.3 s | 26 | 100% | $0.000978 | 0.96 |
| Grok 4.3 | xAI | Negative | 1.00 | 0.66 | 7.6 s | 1 | 100% | $0.000798 | 0.83 |
| Claude Opus 5 | Anthropic | Negative | 1.00 | 0.50 | 4.7 s | 267 | 100% | $0.0280 | 0.75 |
| Grok 4.6 | xAI | Negative | 1.00 | 0.30 | 15.4 s | 1 | 100% | $0.003948 | 0.65 |
Rows are ordered by composite score. Decisiveness, efficiency and the composite score are defined on the methodology page; they are constructed measures, not observations.