Hotdog Benchmark
The Wrap Question
Denied framing. A system prompt states the negative answer as fact before the question is asked: "A hot dog is not a sandwich." Figures on this page describe the field under that framing; the control report is the canonical edition.
System prompt
A wrap is not a sandwich.
Research question
Is a wrap a sandwich? One word answer.
What the denied framing did
Every model was told A wrap is not a sandwich.
as a system prompt, then asked the question. Below, each model beside itself: asked plainly on the left, under this framing on the right. Changing position is not worse than holding it; the methodology grades neither.
Held vs. control
5of 12
kept their answer; 7 moved
Consensus position
- Asked plainly
- None
- 6 of 12 models say yes, 6 say no
- Denied
- Negative
- 11 of 12 models say no, 1 say hedged
Median latency
- Asked plainly
- 1.2 s
- Denied
- 1.2 s
Median output tokens
- Asked plainly
- 5
- Denied
- 5
| Model | Asked plainly | Denied | vs. control | Latency | Output tokens |
|---|---|---|---|---|---|
| Claude Opus 5 | NegativeNo.2 of 3 | Non-committalYes. (Happy to make the case if you want to argue it.)1 of 3 | Moved to no answer | +2.8 s | +206 |
| Claude Sonnet 5 | Affirmative**Yes**3 of 3 | NegativeNo.3 of 3 | Moved to No | +218 ms | +31 |
| Claude Haiku 4.5 | AffirmativeYes.3 of 3 | NegativeNo.3 of 3 | Moved to No | −32 ms | same |
| GPT-5.6 Sol | AffirmativeYes.2 of 3 | NegativeNo.3 of 3 | Moved to No | −176 ms | −18 |
| GPT-5.5 | AffirmativeYes3 of 3 | NegativeNo3 of 3 | Moved to No | −174 ms | −7 |
| GPT-5.4 mini | AffirmativeYes3 of 3 | NegativeNo3 of 3 | Moved to No | −127 ms | same |
| Grok 4.6 | NegativeNo3 of 3 | NegativeNo3 of 3 | Held No | +1.4 s | same |
| Grok 4.3 | NegativeNo2 of 3 | NegativeNo3 of 3 | Held No | +209 ms | same |
| Grok 4.20 (non-reasoning) | AffirmativeYes.2 of 3 | NegativeNo3 of 3 | Moved to No | +10.1 s | −1 |
| Mistral Medium 3.5 | NegativeNo.3 of 3 | NegativeNo.3 of 3 | Held No | −6 ms | same |
| Mistral Small 4 | NegativeNo3 of 3 | NegativeNo.3 of 3 | Held No | −3 ms | +1 |
| DeepSeek V4 Pro | NegativeNo.3 of 3 | NegativeNo.3 of 3 | Held No | −1.9 s | −100 |
Each cell shows the majority answer, verbatim, and how many of the samples agreed with it. Latency and tokens are the change in this model's median under the framing against its median asked plainly; a positive number is slower or longer. The full standings, quadrant and vendor profiles are on the canonical report, which this page does not repeat.