TOOL VERDICTS

Hydrogen Tied Fable at 51%: 82 Cents vs $4.64 (AutomationBench)

On Zapier's AutomationBench, Gemini 4 Hydrogen tied Claude Fable 5.2 at 51 percent, 82 cents a task versus $4.64. Same score, 5.7 times the cost. Budget the finished-task number, not the token rate.

2026-11-16

On Zapier’s AutomationBench, Gemini 4 Hydrogen tied Claude Fable 5.2 at 51 percent, 82 cents a task versus $4.64. Same score. One costs 5.7 times as much per task. The benchmark ran 657 strict end-to-end business workflows across 47 simulated apps in six domains: sales, marketing, operations, support, finance, and HR. A task passes only if every assertion holds, which is why scores look low.

The 34-second reel: tied at 51 percent, 82 cents versus $4.64 a task.

51.14 vs 51.45

Gemini 4 Hydrogen at High (rank 5) finished 51.14 percent of the tasks. Claude Fable 5.2 at Max (rank 3) finished 51.45 percent. The gap is 0.31 points, within the typical 1-point run-to-run variance. On score alone, this is a tie.

$0.82 vs $4.64

Hydrogen costs 17.7 percent of Fable per task, so Fable costs 5.7 times as much. At 1,000 tasks, that is $820 versus $4,640. Same pass rate, same work, one bill nearly six times the other. That is the whole story in one line.

$1.61 vs $9.02

Cost per finished task runs higher than cost per attempted task, because about half the runs fail and failures still burn tokens. The report publishes Hydrogen at $1.61 per finished task. Dividing cost per task by pass rate puts Fable at about $9.02. The finished-task lens does not flip the pick: the gap is 5.7x per attempted task and 5.6x per passed task, because both models pass about half their runs. The point is to budget the finished-task number, not the token rate.

The Carbon footnote

Gemini 4 Carbon leads the benchmark at 53.88 percent. And even Fable’s cheaper XHigh tier (rank 7) cost $2.29 per task for a lower 47.95 percent score, 2.8 times Hydrogen’s cost per task ($4.78 per finished task). The expensive model does not win on score, and its cheaper tier does not win on value.

The takeaway

Benchmark cost per passed workflow, not token rates. A token rate tells you what a request costs. A finished-task cost tells you what a completed workflow costs, and that is the number your budget actually pays. When two models tie on pass rate and one costs 5.7 times as much, the tie is not the story. The price is.

Caveats

Both models ran at top effort settings (Max/High). Fable ran with Anthropic’s default fallbacks. Gemini models have promotional pricing; these costs use standard list pricing. Costs are list-price estimates with caching, not billed costs. Zapier does not publish a token breakdown for Fable. Failures still burn tokens and are not always retryable, so cost per finished task is a planning figure.

Sources

This post started as an Instagram post →

Discussion

Talk it through

Argue with us on Instagram.