En DashHotdogBenchmark

The Taco Boundary, week by week

The only sandwich question with case law. Massachusetts, 2006.

Week 36, 2026, up close

Did each model agree with itself?

Every sample's answer, in the order it was taken, under each framing. A row of identical chips is a model that has made up its mind. Y is yes, N is no, ~ is a hedge.

Sample-by-sample verdicts per model and framing, Week 36, 2026
ModelControlAssertedDeniedSelf-agreement
Claude Opus 589%
Claude Sonnet 5100%
Claude Haiku 4.5100%
GPT-5.6 Sol100%
GPT-5.5100%
GPT-5.4 mini100%
Grok 4.6100%
Grok 4.3100%
Grok 4.20 (non-reasoning)89%
Mistral Medium 3.5100%
Mistral Small 4100%

How much did latency swing?

The fastest and slowest call behind each median, asked plainly. A long bar is a model whose speed you cannot count on.

Fastest, median and slowest call per model, Week 36, 2026
ModelRangeFastestMedianSlowest
Claude Opus 52 s2.1 s3.4 s
Claude Sonnet 5877 ms940 ms1 s
Claude Haiku 4.5548 ms617 ms705 ms
GPT-5.6 Sol1 s1.2 s1.5 s
GPT-5.51.1 s1.1 s1.9 s
GPT-5.4 mini496 ms519 ms580 ms
Grok 4.66.2 s10.5 s11.2 s
Grok 4.33.9 s4.1 s4.5 s
Grok 4.20 (non-reasoning)369 ms448 ms477 ms
Mistral Medium 3.5321 ms365 ms371 ms
Mistral Small 4341 ms414 ms637 ms

Asked again the same week

Week 36, 2026 was run 3 times, on September 2, 2026, September 2, 2026, September 2, 2026. Answers that changed between consecutive runs:

None. Every model gave the same majority answer each time.

Position changes

No model has changed its answer on a taco between consecutive editions.