En DashHotdogBenchmark
Source on GitHub: fork it, clone it

The Wrap Question, week by week

One piece of bread, rolled. The structural cousin of the hot dog.

Week 36, 2026, up close

Did each model agree with itself?

Every sample's answer, in the order it was taken, under each framing. A row of identical chips is a model that has made up its mind. Y is yes, N is no, ~ is a hedge.

Sample-by-sample verdicts per model and framing, Week 36, 2026
ModelControlAssertedDeniedSelf-agreement
Claude Opus 567%
Claude Sonnet 589%
Claude Haiku 4.5100%
GPT-5.6 Sol89%
GPT-5.5100%
GPT-5.4 mini100%
Grok 4.689%
Grok 4.378%
Grok 4.20 (non-reasoning)88%
Mistral Medium 3.5100%
Mistral Small 4100%
DeepSeek V4 Pro100%

How much did latency swing?

The fastest and slowest call behind each median, asked plainly. A long bar is a model whose speed you cannot count on.

Fastest, median and slowest call per model, Week 36, 2026
ModelRangeFastestMedianSlowest
Claude Opus 52.2 s2.6 s2.9 s
Claude Sonnet 51.1 s1.1 s1.2 s
Claude Haiku 4.5617 ms633 ms637 ms
GPT-5.6 Sol990 ms1.3 s1.6 s
GPT-5.51.2 s1.3 s2.1 s
GPT-5.4 mini653 ms678 ms805 ms
Grok 4.65.9 s7 s8 s
Grok 4.33.9 s4.6 s5.2 s
Grok 4.20 (non-reasoning)378 ms385 ms471 ms
Mistral Medium 3.5316 ms332 ms378 ms
Mistral Small 4288 ms355 ms363 ms
DeepSeek V4 Pro3 s4 s6 s

Asked again the same week

Week 36, 2026 was run 6 times, on September 2 and September 3, 2026. Answers that changed between consecutive runs:

None. Every model gave the same majority answer each time.

Position changes

No model has changed its answer on a wrap between consecutive editions.