I ran Microsoft’s TokenEconomizer against plain Claude Code on the same Web-Bench tasks, meter against meter. Nine tasks from the TABLE project. Both sides solved all nine. Plain Claude Code cost $1.10. With TokenEconomizer, $0.82, which is 25 percent less for the same work.
The 56-second reel: two meters, nine tasks, one masking rule.
$1.10 vs $0.82
The first three tasks were a dead heat. The plugin masks a tool once it has gone unused for three straight tasks, and from task four onward the TokenEconomizer side ran cheaper on every step. Task 4 is where it turned: $0.1655 on the Claude Code meter against $0.0682 on the TokenEconomizer meter. That is the first step where the paper’s masking rule can bite, and the lead never changed hands after that.
The pooled picture
Nine tasks is a small sample. Pooled across everything that ran, the plugin side covered 14 tasks for $2.31 against 13 tasks and $2.70 without it, including a tenth table task only the plugin arm ran, and the pooled saving was about 14 percent. Microsoft’s paper reports 18.4 to 30.9 percent lower cost across 277 SWE-Bench tasks, on the authors’ own benchmarks. Our pooled number sits below the paper’s range, on a much smaller sample.
The shipped plugin differs from the paper
Two differences worth knowing before you install it. Micro-skills are off by default in the shipped plugin, so the paper’s headline numbers come from a configuration you do not get out of the box. And the loop guard redirects a runaway run instead of stopping it, which changes what happens when an agent loops.
The takeaway
If your agent loads tools it never calls, masking them out of the prompt is the cheapest win available. Check which tools your agent loads on every run and never actually calls. That list is where the saving lives.
Caveats
Our own test, run Nov 18, 2026, at live Sonnet 5 rates. Nine tasks is a small sample. The Web-Bench evaluator ran in development mode, a 2-second Playwright timeout instead of 30, so the grades are stricter, and the same mode applied to both arms. The test cost $5.56 in API spend, including two stalled runs I had to re-run.
Sources
- Instagram reel: https://www.instagram.com/reel/Df-Xat8E_rJ/
- Threads post: https://www.threads.com/@dataeng.ai/post/Df-Xa0hTKTu
- YouTube Short: https://www.youtube.com/shorts/xIOSP95Du0o
- Paper: https://arxiv.org/abs/2610.22732
- Repo: https://github.com/microsoft/TokenEconomizer
This post started as an Instagram post →
- https://www.instagram.com/reel/Df-Xat8E_rJ/
- https://www.threads.com/@dataeng.ai/post/Df-Xa0hTKTu
- https://www.youtube.com/shorts/xIOSP95Du0o
- https://arxiv.org/abs/2610.22732
- https://github.com/microsoft/TokenEconomizer
Numbers above trace to these sources. If one moved, tell us and we fix it.


Talk it through
Argue with us on Instagram.