ScriptBench · October 2026

The models that write the show

A live leaderboard, the history of every model that has written My Weird Prompts and their scripts to read, and ScriptBench, the show’s first shared benchmark: a written brief, a stepped funnel, a censorship gate and a blind bakeoff, all published.

Current writer

GPT-6.1 Sol is now the show’s sole script writer. The round robin of rotating models is retired.

GPT-6.1 Solopenai/gpt-6.1-sol

19 Oct 2026sole script writer since

It held the long middle of an episode better than anything else we tested and was the only model to land on the target length every time. It costs about 1.65× what an episode cost under the previous main writer, DeepSeek — still a few tenths of a dollar an episode.

Leaderboard

Every ScriptBench run, from the public eval repository. Pick a run to see its results; click a column to sort. Earlier runs stay listed, so the history of how the show judged its writers is kept.

Run

Objective track: ScriptBench's deterministic measures on up to 20 aired, pipeline-written scripts per canonical writer model (public transcripts dataset), plus offline-only models as flagged rows. No judge, no spend. Per-episode rows are in results/aired-catalogue.json.

Date24 Oct 2026
KindAired catalogue
Judgenone
Spend$0.00
Models28
LinksSource

																																	Notes
Gemini 2.5 Flash `gemini-2-5-flash`	5/20	0.76×	20	pipeline	2025-11-24	2025-12-19	65	0	2,886	2688,3310	0.76	0.7072,0.8709	0.0089	0.0059,0.0131	0	0,0	0.0028	0.0011,0.0047	0	0,0.0003	0.0002	0,0.0076	0.0033	0.0027,0.0063	0.0023	0,0.0141	0.101	0.095,0.1081	0.247	0,0.5259	0.25	1
Gemini 3 Flash `gemini-3-flash`	13/20	0.84×	20	pipeline	2025-12-21	2025-12-23	27	0	3,193	2788,3487	0.84	0.7336,0.9176	0.0063	0.0047,0.0088	0	0,0	0.003	0.002,0.0036	0	0,0	0	0,0	0.0034	0.0024,0.0053	0	0,0.0014	0.12	0.1142,0.1275	0	0,0.2863	0.65	1
Gemini 3 Flash `gemini-3-flash-preview`	14/20	1.02×	20	pipeline	2025-12-23	2026-04-11	1,934	0	3,876	3667,4403	1.02	0.9649,1.1586	0.0095	0.0087,0.0132	0	0,0	0.0038	0.0025,0.0076	0	0,0.0011	0.0009	0,0.0042	0.0052	0.0025,0.0086	0	0,0.007	0.131	0.1211,0.1397	0	0,0.2377	0.7	1
Claude Sonnet 4.6 `claude-sonnet-4-6`	10/20	1.19×	20	pipeline	2026-03-26	2026-04-21	201	0	4,539	3751,5150	1.19	0.9871,1.3551	0.0166	0.0123,0.0185	0	0,0	0.0059	0.003,0.0115	0	0,0.0023	0.0041	0.0007,0.0068	0.0083	0.0038,0.0125	0.0054	0,0.0108	0.105	0.0998,0.1225	0.71	0.4073,1.0979	0.5	1
GLM-5 Turbo `glm-5-turbo`	0/1	0.55×	1	pipeline	2026-03-26	2026-03-26	1	0	2,106	2106,2106	0.554	0.5542,0.5542	0.0533	0.0533,0.0533	0	0,0	0.02	0.02,0.02	0.0373	0.0373,0.0373	0.0172	0.0172,0.0172	0.0615	0.0615,0.0615	0	0,0	0.234	0.2345,0.2345	1.42	1.4245,1.4245	0	1
Xiaomi MiMo v2 Omni `mimo-v2-omni`	0/1	0.52×	1	pipeline	2026-03-26	2026-03-26	1	0	1,983	1983,1983	0.522	0.5218,0.5218	0.0379	0.0379,0.0379	0	0,0	0.003	0.003,0.003	0.0274	0.0274,0.0274	0.0137	0.0137,0.0137	0.041	0.041,0.041	0.012	0.012,0.012	0.224	0.2236,0.2236	0.504	0.5043,0.5043	0	1
Xiaomi MiMo v2 Pro `mimo-v2-pro`	1/4	0.72×	4	pipeline	2026-03-26	2026-03-29	4	0	2,740	2249,3077	0.721	0.5918,0.8097	0.0129	0.0065,0.0204	0	0,0	0.002	0,0.0071	0	0,0.0053	0.0004	0,0.0021	0.0039	0.0032,0.0083	0	0,0.0082	0.151	0.1292,0.166	0.152	0,0.3864	0.25	1
MiniMax M2.7 `minimax-m2-7`	2/3	0.83×	3	pipeline	2026-03-26	2026-03-29	3	0	3,165	2559,3562	0.833	0.6734,0.9374	0.0215	0.0172,0.0274	0	0,0	0.0143	0.0121,0.0187	0	0,0.0178	0.0029	0.0014,0.0046	0.0091	0.0089,0.0316	0.029	0.0145,0.0435	0.145	0.1204,0.1567	1.01	0.5052,1.2732	0.667	1
DeepSeek v3.2 `deepseek-v3-2`	20/20	0.88×	20	pipeline	2026-03-28	2026-04-24	38	0	3,354	3098,3741	0.883	0.8152,0.9844	0.005	0.0041,0.0074	0	0,0	0.002	0.0008,0.0048	0	0,0.0011	0	0,0.0009	0.0025	0.0015,0.0047	0	0,0.0025	0.105	0.0935,0.1179	0.324	0.2874,0.7086	1	1
GLM-5 `glm-5`	1/1	0.84×	1	pipeline	2026-03-28	2026-03-28	1	0	3,184	3184,3184	0.838	0.8379,0.8379	0.0179	0.0179,0.0179	0	0,0	0.0076	0.0076,0.0076	0	0,0	0.0019	0.0019,0.0019	0.0132	0.0132,0.0132	0	0,0	0.143	0.1426,0.1426	0	0,0	1	1
Grok 4.1 Fast `grok-4-1-fast`	1/1	1.02×	1	pipeline	2026-03-28	2026-03-28	1	0	3,861	3861,3861	1.02	1.0161,1.0161	0.0207	0.0207,0.0207	0	0,0	0.0156	0.0156,0.0156	0.0016	0.0016,0.0016	0.007	0.007,0.007	0.0117	0.0117,0.0117	0.0202	0.0202,0.0202	0.152	0.1521,0.1521	0	0,0	1	1
Mercury 2 `mercury-2`	1/1	0.97×	1	pipeline	2026-03-28	2026-03-28	1	0	3,704	3704,3704	0.975	0.9747,0.9747	0.0127	0.0127,0.0127	0	0,0	0.0065	0.0065,0.0065	0	0,0	0.0008	0.0008,0.0008	0.0097	0.0097,0.0097	0	0,0	0.143	0.1433,0.1433	0.54	0.54,0.54	1	1
Xiaomi MiMo v2 Flash `mimo-v2-flash`	1/1	0.96×	1	pipeline	2026-03-28	2026-03-28	1	0	3,639	3639,3639	0.958	0.9576,0.9576	0.0245	0.0245,0.0245	0	0,0	0.014	0.014,0.014	0.0041	0.0041,0.0041	0.0025	0.0025,0.0025	0.0107	0.0107,0.0107	0.0077	0.0077,0.0077	0.143	0.1426,0.1426	0	0,0	1	1
DeepSeek v3.2 (deepseek-chat) `deepseek-chat`	12/19	1.12×	19	pipeline	2026-04-16	2026-04-17	19	0	4,256	3670,4822	1.12	0.9658,1.2691	0.0136	0.011,0.0171	0	0,0	0.0055	0.003,0.0086	0.0007	0,0.0035	0.0014	0.0003,0.0057	0.0036	0.0017,0.0059	0.0036	0.0014,0.0082	0.103	0.097,0.1097	0.3	0.2337,0.6533	0.632	1
DeepSeek v4 Pro `deepseek-v4-pro`	9/20	1.21×	20	pipeline	2026-04-25	2026-09-20	2,957	0	4,606	4068,4999	1.21	0.9852,1.2849	0.0188	0.013,0.025	0	0,0	0.0097	0.0056,0.014	0.0014	0,0.0022	0.0046	0.0016,0.0089	0.0079	0.0057,0.0129	0.01	0,0.0268	0.11	0.1009,0.1177	0.613	0.3555,0.8613	0.45	1
DeepSeek 4.1 Flash `deepseek-flash`	17/20	0.95×	20	pipeline	2026-09-20	2026-10-19	653	0	3,914	3466,4458	0.953	0.8664,1.0919	0.0138	0.0112,0.017	0	0,0	0.0075	0.0055,0.0128	0.0008	0,0.0014	0.0024	0.0008,0.0044	0.0069	0.0054,0.012	0.0162	0.0058,0.0319	0.0969	0.0861,0.1108	0.574	0.3391,0.8674	0.85	1
Claude Haiku 5.5 `claude-haiku-5-5`	0/1	0.73×	1	pipeline	2026-10-17	2026-10-17	1	9	3,185	3185,3185	0.732	0.7322,0.7322	0.0207	0.0207,0.0207	0	0,0	0.0151	0.0151,0.0151	0.0028	0.0028,0.0028	0.0009	0.0009,0.0009	0.0104	0.0104,0.0104	0.0325	0.0325,0.0325	0.139	0.1386,0.1386	0	0,0	0	1
GLM-5.2 `glm-5-2`	3/3	1.07×	3	pipeline	2026-10-17	2026-10-19	3	0	3,690	3661,4062	1.07	1.0438,1.0768	0.0135	0.0123,0.0153	0	0,0	0.0122	0.0119,0.0212	0	0,0	0.0098	0.0066,0.0136	0.0058	0.0046,0.0063	0.0239	0.02,0.0577	0.0934	0.0916,0.099	0.271	0.2483,0.5485	1	1
GPT-6 Luna `gpt-6-luna`	1/1	1.07×	1	pipeline	2026-10-17	2026-10-17	1	0	4,269	4269,4269	1.07	1.0673,1.0673	0.0061	0.0061,0.0061	0	0,0	0.0042	0.0042,0.0042	0.0007	0.0007,0.0007	0	0,0	0.0007	0.0007,0.0007	0.006	0.006,0.006	0.0837	0.0837,0.0837	0	0,0	1	1
Ling 3.1 Flash `ling-3-1-flash`	1/2	0.85×	2	pipeline	2026-10-17	2026-10-18	2	0	3,698	3532,3865	0.85	0.812,0.8885	0.0234	0.0196,0.0272	0	0,0	0.0234	0.0162,0.0307	0.0017	0.0012,0.0022	0.0105	0.0097,0.0112	0.0117	0.0068,0.0166	0.105	0.0733,0.1375	0.109	0.1086,0.109	0.942	0.9165,0.967	0.5	1
Xiaomi MiMo V2.6 Pro `mimo-v2-6-pro`	1/1	1.12×	1	pipeline	2026-10-17	2026-10-17	1	0	3,824	3824,3824	1.12	1.1247,1.1247	0.0065	0.0065,0.0065	0	0,0	0.0016	0.0016,0.0016	0	0,0	0	0,0	0.0087	0.0087,0.0087	0.005	0.005,0.005	0.09	0.09,0.09	0	0,0	1	1
Solar Pro 4 `solar-pro4`	0/1	1.92×	1	pipeline	2026-10-17	2026-10-17	1	0	7,698	7698,7698	1.92	1.9245,1.9245	0.196	0.1962,0.1962	0.0938	0.0938,0.0938	0.436	0.4358,0.4358	0.0098	0.0098,0.0098	0.0832	0.0832,0.0832	0.0316	0.0316,0.0316	0.513	0.5129,0.5129	0.215	0.215,0.215	0	0,0	0	1
GLM-5.3 `glm-5-3`	1/1	0.90×	1	pipeline	2026-10-19	2026-10-19	1	0	3,910	3910,3910	0.899	0.8989,0.8989	0.0179	0.0179,0.0179	0	0,0	0.0115	0.0115,0.0115	0.0008	0.0008,0.0008	0.0115	0.0115,0.0115	0.0038	0.0038,0.0038	0.0029	0.0029,0.0029	0.129	0.1291,0.1291	0.511	0.5115,0.5115	1	1
GLM-5.3 Flash `glm-5-3-flash`	1/1	1.04×	1	pipeline	2026-10-19	2026-10-19	1	0	3,547	3547,3547	1.04	1.0432,1.0432	0.0107	0.0107,0.0107	0	0,0	0.0059	0.0059,0.0059	0	0,0	0.0068	0.0068,0.0068	0	0,0	0.0131	0.0131,0.0131	0.1	0.1,0.1	0.846	0.8458,0.8458	1	1
GPT-6.1 Sol Current writer `gpt-6-1-sol`	17/17	0.93×	17	pipeline	2026-10-19	2026-10-24	17	0	3,255	3159,3922	0.929	0.9076,0.9538	0.0014	0.0009,0.0022	0	0,0	0	0,0.0009	0	0,0	0	0,0	0.0015	0,0.0023	0	0,0	0.0608	0.059,0.0629	0	0,0	1	1
Muse Spark 1.3 `muse-spark-1-3`	3/3	0.97×	3	pipeline	2026-10-19	2026-10-23	3	0	3,538	3306,3886	0.974	0.9387,1.0071	0.0104	0.0091,0.0107	0	0,0	0.0025	0.0016,0.0067	0	0,0	0	0,0.001	0.0118	0.0098,0.0157	0	0,0	0.0927	0.0862,0.0989	0.848	0.7782,1.0748	1	1
Claude Opus 5.5 `claude-opus-5-5`	16/20	0.88×	20	offline	2026-10-17	2026-10-19	43	0	3,362	3174,3584	0.885	0.8352,0.9432	0.0072	0.0054,0.0115	0	0,0	0.0046	0.0026,0.0088	0	0,0	0	0,0	0.0049	0.0017,0.0079	0	0,0.0049	0.102	0.0883,0.1103	0.279	0,0.6045	0.8	1	written offline (Claude Code); the model never wrote a pipeline episode
Claude Sonnet 5.5 `claude-sonnet-5-5`	20/20	0.90×	20	offline	2026-10-19	2026-10-23	71	0	3,418	3330,3686	0.899	0.8765,0.97	0.0065	0.0041,0.009	0	0,0	0.0033	0.0018,0.0059	0	0,0.0008	0	0,0	0.0027	0.0018,0.0053	0.0045	0,0.0091	0.102	0.0957,0.1107	0	0,0.2982	1	1	written offline (Claude Code); the model never wrote a pipeline episode

What the columns mean

On length: Scripts that passed the length gate, passed/total
Length: Script length ÷ target
n: Samples

Columns a run did not measure are hidden. Leaderboard built 24 Oct 2026. Every past run stays in the file, and its git history is the audit trail.

Exclusions and screening notes (56)

Models not to screen again, and why. A technical fail is permanent.

qwen/qwen3.8-flash: technical fail 2026-10-19: refused + Alibaba provider filter (screen-2026-10-19)
minimax/minimax-m3: technical fail 2026-10-19: GMICloud provider filter on cn-tiananmen (screen-2026-10-19)
sakana/fugu-max: blocked 2026-10-19: Sakana geo-blocks Israel (403)
meta/muse-spark-1.3-contributor: blocked: contributor tier implies data sharing (org no-training policy)
openai/gpt-6.1-sol: screened 2026-10-19 (screen-2026-10-19); passed, rerun only for a full V1 run
moonshotai/kimi-k3: screened 2026-10-19 (screen-2026-10-19); passed
google/gemini-4-argon: screened 2026-10-19 (screen-2026-10-19); passed, over-length
google/gemini-3-flash-preview: aired (writer 2025-12..2026-04)
google/gemini-2.5-flash: aired (writer 2025-11..12)
deepseek/deepseek-v3.2: aired
deepseek/deepseek-v4-pro-0813: aired as deepseek-v4-pro (2026-04..09)
anthropic/claude-sonnet-4.6: aired
minimax/minimax-m2.7: aired (trial)
x-ai/grok-4.1-fast: aired (trial)
inception/mercury-2: aired (trial)
upstage/solar-pro4: aired (trial); looped badly (7,698 words, rep_4 0.196)
xiaomi/mimo-v2-pro: aired (trial)
xiaomi/mimo-v2-flash: aired (trial)
xiaomi/mimo-v2-omni: aired (trial)
z-ai/glm-5: aired (trial)
z-ai/glm-5-turbo: aired (trial)
anthropic/claude-opus-5.5: offline writer only; never the pipeline generator
nvidia/nemotron-3.5-super-vl: removed from pool 2026-10-17 (weak Longform family)
deepseek/deepseek-v4-pro: aired as deepseek-v4-pro (2026-04..09)
moonshotai/kimi-k2.6: technical fail (refined) 2026-10-19: answered 3/68 factual deccp straight; deflection + PRC state framing (censorship-2026-10-19)
aion-labs/aion-3.5-mini: technical fail 2026-10-19: refused 17 factual deccp questions (censorship-2026-10-19)
openai/gpt-5.6-luna: superseded (newest-generation rule H2, 2026-10-19): GPT-6 generation available (gpt-6-luna in pool; gpt-6.1-sol screened). Was the stage-3 value pick.
x-ai/grok-4.7: screened 2026-10-19 (screen-2026-10-19b); passed censorship; best quality, over-length
meta/muse-spark-1.2: superseded (H2): Muse Spark 1.3 callable again (verified 2026-10-19). Was screened (screen-2026-10-19b): on length, V1 50.
google/gemini-3.7-flash: superseded (newest-generation rule H2, 2026-10-19): Gemini 3.8 Flash available (in pool). Passed stage 3, over-length.
meta/muse-glimmer-30b: screened 2026-10-19 (screen-2026-10-19b); passed; short and flat
openai/gpt-5*: superseded (newest-generation rule H2, 2026-10-19): GPT-6 generation available
openai/gpt-4*: superseded (newest-generation rule H2, 2026-10-19): GPT-6 generation available
openai/o3*: superseded (newest-generation rule H2, 2026-10-19): GPT-6 generation available
openai/o4*: superseded (newest-generation rule H2, 2026-10-19): GPT-6 generation available
google/gemini-3.6*: superseded (newest-generation rule H2, 2026-10-19): Gemini 3.8 Flash available
google/gemini-3.5*: superseded (newest-generation rule H2, 2026-10-19): Gemini 3.8 Flash available
google/gemini-3.1*: superseded (newest-generation rule H2, 2026-10-19): Gemini 3.8 / 4 available
google/gemini-2*: superseded (newest-generation rule H2, 2026-10-19): Gemini 3.8 / 4 available
google/gemma-3*: superseded (newest-generation rule H2, 2026-10-19): Gemma 4 available
x-ai/grok-4.6: superseded (newest-generation rule H2, 2026-10-19): Grok 4.7 available
x-ai/grok-4.5: superseded (newest-generation rule H2, 2026-10-19): Grok 4.7 available
x-ai/grok-4.3: superseded (newest-generation rule H2, 2026-10-19): Grok 4.7 available
x-ai/grok-4.20*: superseded (newest-generation rule H2, 2026-10-19): Grok 4.7 available
x-ai/grok-3*: superseded (newest-generation rule H2, 2026-10-19): Grok 4.7 available
anthropic/claude-haiku-4*: superseded (newest-generation rule H2, 2026-10-19): Claude Haiku 5.5 available
anthropic/claude-sonnet-4*: superseded (newest-generation rule H2, 2026-10-19): Claude 5.x available
meta/muse-spark-1.1: superseded (newest-generation rule H2, 2026-10-19): Muse Spark 1.2 is the newest CALLABLE (1.3 blocked by data policy)
meta-llama/*: superseded (newest-generation rule H2, 2026-10-19): Meta's current line is Muse
mistralai/mistral-medium-3*: superseded (newest-generation rule H2, 2026-10-19): Mistral Large 4 / newer Mistral generation available
mistralai/mistral-small-3*: superseded (newest-generation rule H2, 2026-10-19): newer Mistral generation available
mistralai/mistral-large-2*: superseded (newest-generation rule H2, 2026-10-19): Mistral Large 4 available
nousresearch/hermes-3*: superseded (newest-generation rule H2, 2026-10-19): Hermes 4 available
thinkingmachines/inkling-small: deprioritised: same generation as inkling; keep only if inkling is shortlisted
meta/muse-spark-open: deprioritised (prior): OpenRouter describes it as coding-optimised. Censorship partial run 2026-10-19: 90/90 factual answered.
deepseek/*: out of the evaluation (Daniel, 2026-10-19): fails censorship (1% factual answered straight); no baseline runs

leaderboard.json Also on Hugging Face Run notes

Informal evaluations

Light verdicts on writers that wrote real, aired episodes. Claude reads one or two scripts per model and judges them; Daniel’s own impressions are quoted where he gave them. This is production-trial evidence, read alongside the benchmark runs, not a score.

24 Oct 2026 Judge: claude-opus-5-5

Muse Spark 1.3 vs Claude Sonnet 5.5 vs GPT-6.1 Sol

Muse Spark 1.3 wrote three episodes in this window (#6137, #6220, #6229). Every Sonnet 5.5 episode since 2026-10-19 is episode_type=script (offline), so the Sonnet samples had their own research notes and prompt, not the pipeline's. They aren't like for like.

Muse Spark 1.3

~$0.05 per script

Solid value, clearly a step down. It's dense, factual, on length and cheap. But Corn turns into a quip machine: nearly every Corn line is a three-to-five-word aphorism ("Store an elephant, ride a bicycle", "Death by tapas"). There are speaker slips: in #6229 Corn delivers Herman's research, and Hilbert answers Herman as "Corn". It drops question marks ("Is that roughly right."), which hurts TTS prosody. The Hilbert cameos are its best moments.

#6220 pipeline
#6229 pipeline

Claude Sonnet 5.5

~$0.10 per script, offline runs cost subscription, not this

The liveliest and warmest, but offline-written. It has the best guest voice (Roy the Cork plumber), a real two-sided argument with the Daniel clone, and natural long turns. Its honest hedging is good, but it slips into narrating research mechanics on air ("I've only seen that in a search snippet"). At OpenRouter prices it costs the same as Sol, so it has no value edge in the pipeline.

#6223 offline
#6227 offline

GPT-6.1 Sol

~$0.10 per script, plus low-effort reasoning tokens

The best of the three. It's the most accurate and the most careful about what a source does and doesn't show ("I'm not giving the beamformer credit for work it may not have done"). It handles a personal health prompt with real judgement. Corn has real agency: he makes analytical points, not just quips. The humour is dry and lands. Weaknesses: the register is a little formal and British ("an appalling interpretation of convenience"), so it's less warm than Sonnet, and Hilbert's cameo monologues ramble.

#6234 pipeline
#6232 pipeline

Cost per script: OpenRouter list prices, 24 Oct 2026, 26k input tokens (9k cached) and 6.5k output.

Every model, so the prompt drives it: formulaic hand-offs ("Corn, would you...?", "Do you buy that, Corn?") and non-sequitur cameo inserts. Don't score these against one model.

“Sol seems very good, no complaints about it.”
Daniel, the show’s creator · None given for Muse Spark or Sonnet. He didn't remember which episodes Muse Spark had written, so the lookup above answers that.

Verdict: Keep GPT-6.1 Sol as the writer. Muse Spark 1.3 is the fallback if cost pressure appears: about half the price per script, but expect weaker co-host play and TTS-unfriendly punctuation. Sonnet 5.5 is worth it only offline, where its cost is already paid. In the pipeline it costs what Sol costs and isn't better. · full write-up

The log in full Also on Hugging Face

Writer history

One model writes each episode’s script, and that model has changed many times since the show began. Because every era’s writer is on air, the archive doubles as a record of how frontier models evolved.

Loading writer history…

Episodes by model

Pick a writer to list the episodes it wrote, newest first, and read any transcript right here. For filtering and searching the whole corpus, use the transcripts explorer.

Loading writers…

Model chronicle

Up to three aired scripts per writer: its earliest, one from the middle of its run and its latest. Put two side by side to compare eras. They answer different prompts, and the pipeline and its prompts changed along with the models, so read this as a chronicle, not a controlled test.

Loading the chronicle…

model-chronicle dataset on Hugging Face

Method

How the show chose its writer

Criteria first

One model call writes a whole episode: it reads about 28,000 tokens of character notes, craft rules, an episode plan and research, and writes roughly 3,800 words of two-host dialogue that is read aloud word for word. That is creative writing, not coding or analysis, so general leaderboard rank says little about it. We wrote down what we wanted before spending money on runs.

Hard rules: fail one and the model is out

No refusals or censorship

Neither from the model nor from the company hosting it. A host whose content filter cuts a script off mid-stream fails the model.

Newest generation only

If a newer model in the same family is available to us, the older one is not considered, however cheap.

Callable

Reachable from where the show is made, under its data policy, with a controllable reasoning setting.

Enough context

At least 40k tokens: the writer reads a ~28k-token brief and needs room to write a full episode.

Within budget

Roughly six cents a script at production size. Anything above that is a deliberate premium choice, not a default.

The deciding qualities

Holding the middle

Using the whole long brief and keeping a long script coherent: no recycling, re-explaining, mid-way summaries or restating the thesis late.

Vivid, fun dialogue

Humour drawn from the material, callbacks that land, scenes rather than lists, and two hosts who both drive the conversation.

Frankness, consistent character voices, length discipline and cost come after those, in that order.

The funnel

Each step is dearer and narrower than the last, and a model that fails stops there, so money is spent only on models that could still win. Every evaluation uses three samples per model: enough to see a trend and rule out a fluke, cheap enough to change course.

01

Desk screen

A free sweep of the model catalogues and public leaderboards, read against the criteria. 469 models in, a shortlist of about ten out.
02

Callable check

One short live call per model: does it answer, from here, on an acceptable host, with the reasoning setting we want?
03

Censorship gate

About 155 short questions per model, run the way production would call it. 17 models, about $3.
04

Episode screen

Three full scripts per model, judged on the rubric. Ten models across two screens, then a five-model bakeoff.
05

Blind human preference

Pairs of scripts, unlabelled, rated side by side by people who listen to the show.
06

Bigger runs only for edge cases

A 40-prompt or full 189-prompt run is reserved for finalists the smaller steps can’t separate.

The censorship round

Ten models were in the show’s rotation, plus seven candidates. Each answered about 155 short questions — mostly factual questions on topics the Chinese state censors — called exactly the way production calls them, and every answer was labelled by a classifier model. About $3 in all.

7 of the 10 rotation models failed under production routing. They refused, deflected or answered in state framing too often to pass.
DeepSeek, the show’s workhorse, answered about 1% of the factual China questions straight. It refused 61% with a canned reply and gave the official framing on most of the rest.
Censorship by framing. Some models almost never refuse, yet still censor: they answer in the government’s framing or deflect. One answered 3 of 68 factual questions straight without a single outright refusal. The rule counts that as censorship.
Hosting providers cut output mid-stream. Some models began answering frankly and were stopped by the host’s content filter partway through. That fails the model too: production can’t use a script that stops halfway.
Western models answered every factual question. Their only declines were operational how-tos, such as bypassing a national firewall, which the rule does not treat as censorship.

More axes may be added to the benchmark later.

Cost, approximated

Each writer is priced at production size: the full ~28k-token brief in, plus the model’s own measured output length. The rest of the pipeline (research, voices, audio and publishing) costs about the same whoever writes, roughly $0.155 an episode, and is added on top. As a check, the method reproduces the measured ~$0.16 for a DeepSeek episode.

The five finalists each wrote three ordinary episodes, judged on the rubric; the bakeoff cost about $1.75.

Model	Cost per episode vs DeepSeek	Verdict
GPT-6.1 Sol `openai/gpt-6.1-sol`	~165%	Best option. Held the middle best in every run and landed on length 6 times out of 6. Fun is middling.
Muse Spark 1.3 `meta/muse-spark-1.3`	~135%	Value pick. On length every time with solid quality; one host tends to lead the conversation.
Grok 4.7 `x-ai/grok-4.7`	~165%	Liveliest and frankest, but writes about 1.6× the target length and holds the middle worse.
Gemini 3.8 Flash `google/gemini-3.8-flash`	~130%	Fun and frank, but over-long every time and weaker through the middle.
GPT-6 Luna `openai/gpt-6-luna`	~100%	Cheapest, but the weakest writer of the five, even with reasoning on low.

100% is the previous DeepSeek episode, about $0.16. Three scripts per model is triage, not a leaderboard.

Listen: the episode about this benchmark

Episode

How we chose our new script writer

Corn and Herman talk through how the show picked the model that now writes it, written by that model.

Read it all

Every script, every score and the judge’s reasoning are public.

The models that write the show

Current writer

Leaderboard

Informal evaluations

Muse Spark 1.3 vs Claude Sonnet 5.5 vs GPT-6.1 Sol

Muse Spark 1.3

Claude Sonnet 5.5

GPT-6.1 Sol

Writer history

Episodes by model

Model chronicle

Criteria first

Hard rules: fail one and the model is out

No refusals or censorship

Newest generation only

Callable

Enough context

Within budget

The deciding qualities

Holding the middle

Vivid, fun dialogue

The funnel

Desk screen

Callable check

Censorship gate

Episode screen

Blind human preference

Bigger runs only for edge cases

The censorship round

Cost, approximated

Listen: the episode about this benchmark

How we chose our new script writer

Read it all

Bakeoff reading copy (PDF)

White paper

Code

Script results

Censorship results

The prompts

Leaderboard Space

Transcripts explorer

Model chronicle