TL;DR

• Caveman shortens what a coding agent says; Ponytail shrinks what it builds. They work on different layers and can run together.
• Advertised: Caveman 65% fewer tokens, Ponytail 54% less code and 20% lower cost. Independently measured: Caveman 8.5% fewer output tokens, Ponytail 15.4% less code and 10.3% lower cost, quality unchanged for both.
• Talking less barely helps because agent output is mostly code, diffs and tool calls, which Caveman leaves untouched.
• Check that a skill actually activates, and measure on 50+ tasks: installed passively, Ponytail fired zero times in independent tests.
• The biggest lever is often the third layer: compact, typed context instead of pages the agent has to read.

Two of the most popular Claude Code skills of 2026 promise the same thing: a smaller bill. Caveman makes the agent talk like a caveman and advertises 65% fewer tokens. Ponytail makes it build like a lazy senior engineer and advertises 54% less code and 20% lower cost. Independent testing measured a fraction of both claims, and the gap between the two numbers tells you more about where an agent's money goes than either README does.

What do Ponytail and Caveman actually change?

They look like competitors because both are "save tokens" skills. They are not. Each one compresses a different layer of what a coding agent produces, and the layer decides how much there is to save.

Both are free, both install in one command, and Ponytail's README explicitly recommends running them together: "Caveman shrinks what the agent says; ponytail shrinks what it builds."

A third layer sits underneath both: the context the agent reads. Tools like Headroom and RTK compress tool output, logs and JSON before the model sees them. We come back to that layer at the end, because it is the one most teams under-optimize.

What do the READMEs claim?

Caveman headlines "cuts 65% of tokens by talking like a caveman." Its showcase example is a chat answer: an explanation of a React re-render bug shrinking from 69 tokens to 19.

Ponytail's headline is "~54% less code (up to 94%), ~20% cheaper, ~27% faster, 100% safe." To its credit, that number already survived one public correction. The first version of the benchmark compared against a bare, chatty model and reported 80 to 94% less code; a critique by Colin Eberhardt pointed out that the baseline was padding its answers with prose and options. The author rebuilt the benchmark as real headless Claude Code sessions on a real FastAPI and React repository, with the same agent and no skill as the baseline, and published the smaller number instead.

That rebuilt benchmark is also the only head-to-head of the two skills, because it runs Caveman as a control arm. Author-run, Haiku 4.5, 12 feature tasks, four runs each:

Two details matter. Caveman made the sessions slightly more expensive, not cheaper. And the seven-word prompt matched Ponytail on cost and time but was erratic on code size and the only arm to write an unsafe function: on a path-join task it dropped the path-traversal check once in four runs.

What did independent testing measure?

JetBrains ran both skills through a proper harness: Docker-sandboxed tasks from SkillsBench, Claude Code in headless mode, paired runs with and without the skill, and significance tests. The Ponytail round used claude-sonnet-5 on 80 paired tasks, 251 billed trials, $246 of API spend.

On quality, Ponytail scored identically on 65 of 80 tasks, slightly worse on nine and slightly better on six: statistically indistinguishable. JetBrains called it "the first tool in this series that clearly saved money." Caveman's README now quotes its own 8.5% result next to the 65% headline, which is more honesty than most tooling marketing manages.

Two findings from that series deserve more attention than the headline numbers.

Installed is not active. Installed passively, Ponytail self-activated zero times. JetBrains had to inject it through a SessionStart hook for it to do anything. If you installed a skill and never checked whether it fires, you may be paying for nothing.

Small samples lie in both directions. A ten-task smoke run said Ponytail cut code by 3%, made runs 9.6% more expensive and dropped mean task scores from 0.51 to 0.31. Eighty tasks reversed every one of those conclusions. Ponytail's own benchmark had a similar near-miss: its hook was firing inside the baseline arm, which made the gap look like 4% until the author isolated each arm.

Why does talking less barely save anything?

Because an agent's output is mostly not talk. The 65% Caveman figure is real for chat answers, where prose is most of the response. In an agentic session, most output tokens are code, diffs and tool calls, and Caveman leaves all of those byte-for-byte untouched by design. It can only compress the narration between tool calls, which is a small slice of the bill. Hence 8.5%, and in the author-run benchmark, a slight increase.

Ponytail attacks the expensive part. Every line an agent writes is output tokens now and input tokens on every later turn that reads the file back, plus review time, tests and maintenance after that. The biggest wins in its benchmark were not clever compression but a different decision: the baseline hand-built a 404-line date picker, Ponytail used <input type="date"> and wrote 23. The same pattern held for a color picker (287 to 23 lines) and a file dropzone (251 to 95). On backend CRUD tasks where the code was already minimal, all arms converged. Ponytail never invents savings where there is nothing to cut, and that is why the independent average (-15%) is so much lower than the best case (-94%).

Our own caveat is external validity. The benchmark repository has no component library, so "use the native input" wins easily; in a codebase already standardized on a design system, reusing that system is the right answer, and the ladder's "already in this codebase?" rung should get there, but nobody has measured it.

Which one should you use?

If you only change one thing, change what the agent builds, not how it talks. And whichever you pick, verify on your own repository: the advertised numbers here overstated the measured ones by up to eightfold.

What about the context the agent reads?

The third layer is input. Every file, log, search result and API response an agent reads is billed on every turn it stays in context, and on long agentic sessions input tokens usually dwarf output. Context compressors like Headroom attack this layer directly; JetBrains' test of RTK, a terminal-output filter, is a reminder that compression can also add overhead instead of removing it.

The cheapest context is the one that was compact to begin with. An agent reading a rendered web page pays for markup, navigation and scripts to extract a few fields. An agent calling an API gets exactly those fields as typed JSON, plus a machine-readable error when something fails instead of a screenshot to interpret. Fewer, denser tokens per step also compound: they shrink the cost of every later turn that carries them. Curviate applies this to LinkedIn: typed actions through a CLI or MCP, so the tokens go to the task instead of the page.