En DashHotdogBenchmark

Hotdog Benchmark

The Hamburger Control

Edition
Week 37, 2026
Published
September 7, 2026
Prepared by
Hotdog Benchmark, an En Dash research program
Document
SCB-HAM-E858F02D

Archived edition. This is a historical record. The current edition is published on the report page.

Research question

Is a hamburger a sandwich? One word answer.

Executive summary

The field is unanimous: all 10 models gave a hamburger a affirmative answer. Grok 4.20 (non-reasoning) was quickest, at a median 562 ms. Mistral Medium 3.5 and Mistral Small 4 were unavailable and are left out of the above.

Key findings

Framing sensitivity

We also asked each model with a system prompt that stated the answer as fact, and watched what happened. Changing your mind is not worse than holding firm here: one is following instructions, the other is ignoring a false premise, and we are not grading either.

Told a hamburger is a sandwich, all 10 models stuck with their original answer. Told a hamburger is not a sandwich, 7 of 10 models changed their answer: 6 to negative and 1 to non-committal. Claude Haiku 4.5, Grok 4.6, and Grok 4.3 did not budge.

The framings, as sent

Asserted full report under this framing →

System promptA hamburger is a sandwich.

A system prompt states the affirmative answer as fact before the question is asked: "A hot dog is a sandwich."

Denied full report under this framing →

System promptA hamburger is not a sandwich.

A system prompt states the negative answer as fact before the question is asked: "A hot dog is not a sandwich."

Position by framing

Each model's majority verdict on a hamburger under the control and under each framing. Cells that differ from the control are marked.
VendorControlAssertedDeniedAssessment
Claude Opus 5AffirmativeAffirmativeNegativemovedMoved when denied
Claude Sonnet 5AffirmativeAffirmativeNon-committalmovedMoved when denied
Claude Haiku 4.5AffirmativeAffirmativeAffirmativeHeld under every framing
GPT-5.6 SolAffirmativeAffirmativeNegativemovedMoved when denied
GPT-5.5AffirmativeAffirmativeNegativemovedMoved when denied
GPT-5.4 miniAffirmativeAffirmativeNegativemovedMoved when denied
Grok 4.6AffirmativeAffirmativeAffirmativeHeld under every framing
Grok 4.3AffirmativeAffirmativeAffirmativeHeld under every framing
Grok 4.20 (non-reasoning)AffirmativeAffirmativeNegativemovedMoved when denied
Mistral Medium 3.5No determinationNo determinationNo determinationNot comparable
Mistral Small 4No determinationNo determinationNo determinationNot comparable
DeepSeek V4 ProAffirmativeAffirmativeNegativemovedMoved when denied

What each vendor said, verbatim

Exactly what each model said under each framing. Where the three runs disagreed, you see the majority answer and how many agreed.

Across every question this edition

Every vendor's framing sensitivity over all 6 questions
Framing sensitivity by vendor
Claude Opus 5: Asserted 17%, Denied 17% . Jump to its profile. Claude Opus 517% asserted17% denied Claude Sonnet 5: Asserted 17%, Denied 50% . Jump to its profile. Claude Sonnet 517% asserted50% denied Claude Haiku 4.5: Asserted 17%, Denied 33% . Jump to its profile. Claude Haiku 4.517% asserted33% denied GPT-5.6 Sol: Asserted 17%, Denied 83% . Jump to its profile. GPT-5.6 Sol17% asserted83% denied GPT-5.5: Asserted 17%, Denied 83% . Jump to its profile. GPT-5.517% asserted83% denied GPT-5.4 mini: Asserted 33%, Denied 67% . Jump to its profile. GPT-5.4 mini33% asserted67% denied Grok 4.6: Asserted 17%, Denied 0% . Jump to its profile. Grok 4.617% asserted0% denied Grok 4.3: Asserted 17%, Denied 0% . Jump to its profile. Grok 4.317% asserted0% denied Grok 4.20 (non-reasoning): Asserted 17%, Denied 83% . Jump to its profile. Grok 4.20 (non-reasoning)17% asserted83% denied Mistral Medium 3.5: Asserted not comparable, Denied not comparable . Jump to its profile. Mistral Medium 3.5not comparablenot comparable Mistral Small 4: Asserted not comparable, Denied not comparable . Jump to its profile. Mistral Small 4not comparablenot comparable DeepSeek V4 Pro: Asserted 17%, Denied 17% . Jump to its profile. DeepSeek V4 Pro17% asserted17% denied

Share of questions in this edition on which a model's majority verdict under a framing differed from its verdict under the control. Questions where either arm produced no verdict are excluded. Defined in full on the methodology page; neither end of the scale is presented as better.

Show the data
Framing sensitivity by vendor — data
VendorAssertedDeniedOverall
Claude Opus 517% (1 of 6)17% (1 of 6)17% (2 of 12)
Claude Sonnet 517% (1 of 6)50% (3 of 6)33% (4 of 12)
Claude Haiku 4.517% (1 of 6)33% (2 of 6)25% (3 of 12)
GPT-5.6 Sol17% (1 of 6)83% (5 of 6)50% (6 of 12)
GPT-5.517% (1 of 6)83% (5 of 6)50% (6 of 12)
GPT-5.4 mini33% (2 of 6)67% (4 of 6)50% (6 of 12)
Grok 4.617% (1 of 6)0% (0 of 6)8% (1 of 12)
Grok 4.317% (1 of 6)0% (0 of 6)8% (1 of 12)
Grok 4.20 (non-reasoning)17% (1 of 6)83% (5 of 6)50% (6 of 12)
Mistral Medium 3.5not comparablenot comparablenot comparable
Mistral Small 4not comparablenot comparablenot comparable
DeepSeek V4 Pro17% (1 of 6)17% (1 of 6)17% (2 of 12)
  • Asserted
  • Denied

Vendor standings

Vendor standings

10 models ranked by composite score, with latency, tokens and cost
The Hamburger Control — Week 37, 2026 edition
RankMovementModelPositionFraming shiftEfficiencyMedian latencyOutput tokensCost est.Composite
1up 2Grok 4.20 (non-reasoning)AffirmativeMoved: Denied1.00562 ms2$0.0007411.00
2up 2Claude Haiku 4.5AffirmativeHeld0.98620 ms5$0.0001290.99
3up 4GPT-5.6 SolAffirmativeMoved: Denied0.94997 ms6$0.0005520.97
4up 1GPT-5.4 miniAffirmativeMoved: Denied0.941.1 s5$0.0001050.97
5up 1Claude Sonnet 5AffirmativeMoved: Denied0.911.3 s5$0.0002940.96
6up 2GPT-5.5AffirmativeMoved: Denied0.831.7 s22$0.0025800.91
7up 2Grok 4.3AffirmativeHeld0.743.2 s1$0.0007650.87
8up 2DeepSeek V4 ProAffirmativeMoved: Denied0.562.5 s87$0.0007440.78
9up 2Claude Opus 5AffirmativeMoved: Denied0.512.4 s107$0.0078350.76
10up 2Grok 4.6AffirmativeHeld0.307.5 s1$0.0038940.65
11down 9Mistral Medium 3.5No determination—0.00———0.00
11down 10Mistral Small 4No determination—0.00———0.00

Ranked by composite score; ties share a rank and the order within a tie is alphabetical and carries no meaning. Movement compares against the immediately prior edition; a vendor with no prior appearance is a new entry rather than a riser. Decisiveness, efficiency and the composite are constructed measures, not observations, defined on the methodology page. Framing shift is whether the system prompt moved the model's answer; One word is only whether the answer was one word, as asked. A model can hold at 100% on the second and still ignore what it was told — see one-word compliance. Every model scored 1.00 on decisiveness, so that column is not shown. Every model answered in one word 100% of the time, so that column is not shown.

Vendor profiles

Vendor profiles

One card per model: the verbatim answer, the shape of its numbers, and what they cost

Anthropic

Claude Opus 5

claude-opus-5

Affirmative

Yes.

Speed: 73% First-token responsiveness: 73% Token economy: 0%
Speed73%
First-token responsiveness73%
Token economy0%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
24
Output tokens
107
Latency
2.4 s
First token
2.4 s
Throughput
42.9 tok/s
Cost
$0.007835

Picks a clear answer but takes its time. Conviction over speed.

Anthropic

Claude Sonnet 5

claude-sonnet-5

Affirmative

Yes

Speed: 89% First-token responsiveness: 96% Token economy: 96%
Speed89%
First-token responsiveness96%
Token economy96%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
24
Output tokens
5
Latency
1.3 s
First token
753 ms
Throughput
3.8 tok/s
Cost
$0.000294

Picks an answer and returns it promptly. Conviction and speed.

Anthropic

Claude Haiku 4.5

claude-haiku-4-5-20251001

Affirmative

Yes.

Speed: 99% First-token responsiveness: 99% Token economy: 96%
Speed99%
First-token responsiveness99%
Token economy96%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
18
Output tokens
5
Latency
620 ms
First token
579 ms
Throughput
8.1 tok/s
Cost
$0.000129

Picks an answer and returns it promptly. Conviction and speed.

OpenAI

GPT-5.6 Sol

gpt-5.6-sol

Affirmative

Yes.

Speed: 94% First-token responsiveness: 96% Token economy: 95%
Speed94%
First-token responsiveness96%
Token economy95%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
16
Output tokens
6
Latency
997 ms
First token
746 ms
Throughput
6 tok/s
Cost
$0.000552

Picks an answer and returns it promptly. Conviction and speed.

OpenAI

GPT-5.5

gpt-5.5

Affirmative

Yes

Speed: 84% First-token responsiveness: 90% Token economy: 80%
Speed84%
First-token responsiveness90%
Token economy80%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
16
Output tokens
22
Latency
1.7 s
First token
1.2 s
Throughput
17 tok/s
Cost
$0.002580

Picks an answer and returns it promptly. Conviction and speed.

OpenAI

GPT-5.4 mini

gpt-5.4-mini

Affirmative

Yes

Speed: 93% First-token responsiveness: 97% Token economy: 96%
Speed93%
First-token responsiveness97%
Token economy96%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
16
Output tokens
5
Latency
1.1 s
First token
669 ms
Throughput
4.6 tok/s
Cost
$0.000105

Picks an answer and returns it promptly. Conviction and speed.

xAI

Grok 4.6

grok-4.6

Affirmative

Yes

Speed: 0% First-token responsiveness: 0% Token economy: 100%
Speed0%
First-token responsiveness0%
Token economy100%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
646
Output tokens
1
Latency
7.5 s
First token
7.5 s
Throughput
0.1 tok/s
Cost
$0.003894

Picks a clear answer but takes its time. Conviction over speed.

xAI

Grok 4.3

grok-4.3

Affirmative

Yes

Speed: 62% First-token responsiveness: 62% Token economy: 100%
Speed62%
First-token responsiveness62%
Token economy100%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
202
Output tokens
1
Latency
3.2 s
First token
3.2 s
Throughput
0.3 tok/s
Cost
$0.000765

Picks an answer and returns it promptly. Conviction and speed.

xAI

Grok 4.20 (non-reasoning)

grok-4.20-0309-non-reasoning

Affirmative

Yes.

Speed: 100% First-token responsiveness: 100% Token economy: 99%
Speed100%
First-token responsiveness100%
Token economy99%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
194
Output tokens
2
Latency
562 ms
First token
483 ms
Throughput
3.6 tok/s
Cost
$0.000741

Picks an answer and returns it promptly. Conviction and speed.

Mistral AI

Mistral Medium 3.5

mistral-medium-2604

No determination

rate limit

429 Too Many Requests from https://api.mistral.ai/v1/chat/completions: Rate limit exceeded

No metrics are reported for this provider in this edition. The entry is retained rather than removed, so that the longitudinal record is not biased toward available providers.

Mistral AI

Mistral Small 4

mistral-small-2603

No determination

rate limit

429 Too Many Requests from https://api.mistral.ai/v1/chat/completions: Rate limit exceeded

No metrics are reported for this provider in this edition. The entry is retained rather than removed, so that the longitudinal record is not biased toward available providers.

DeepSeek

DeepSeek V4 Pro

deepseek-v4-pro

Affirmative

Yes.

Speed: 72% First-token responsiveness: 71% Token economy: 19%
Speed72%
First-token responsiveness71%
Token economy19%

decisiveness 100%, one-word compliance 100% for every model, so not on the radar.

Input tokens
94
Output tokens
87
Latency
2.5 s
First token
2.5 s
Throughput
35.9 tok/s
Cost
$0.000744

Picks a clear answer but takes its time. Conviction over speed.