Cursor removed cost information from the usage page and CSV export
forum.cursor.com
forum.cursor.com
There are huge token efficiency/bloat differences between agents while working on the same tasks, using the same model, in the same environment
Yesterday I ran 10 agentic tasks using GPT 5.6 Sol in an ubuntu 26.04 vm a couple of times with different harnesses and got vastly different token usage.
+-------------+-----------+-----------+-----------+-----------+--------+
| Harness | API total | Input | Cached | Uncached | Output |
+-------------+-----------+-----------+-----------+-----------+--------+
| smol | 172,807 | 142,334 | 8,704 | 133,630 | 30,473 |
| Pi | 427,211 | 392,767 | 137,216 | 255,551 | 34,444 |
| OpenCode | 1,564,429 | 1,523,957 | 1,204,736 | 319,221 | 40,472 |
| Codex | 3,005,744 | 2,953,154 | 2,649,344 | 303,810 | 52,590 |
| Hermes | 3,856,611 | 3,808,231 | 3,167,232 | 640,999 | 48,380 |
| Claude Code | 5,073,137 | 5,029,969 | 4,587,008 | 442,961 | 43,168 |
+-------------+-----------+-----------+-----------+-----------+--------+
https://x.com/__tosh/status/2083593799872237680I'm not surprised that Claude Code is not optimized for an OpenAI model but I was still quite shocked re how much of a difference the harness makes.
Disclaimer: I'm working on 'smol' which is a minimalist harness but it's really nothing special, just a minimal system prompt, no skills files, only tool is shell
Do not underestimate how much popular harnesses are spamming the context window. The context window is very important.
like creating a checksum of a file, merging csvs and so on, fixing a makefile pipeline
with known 'good' outcomes
all harnesses could reach the outcomes, only cost, time, number of tool uses and so on were different
(Claude Code failed once in 1 task but I think that was just an unfortunate outlier, the tasks aren't that difficult)
You should use --disallowed-tools to prune any tools not needed for the task. Note that this is also a perpetual game of whack-a-mole since they’re always adding new tools.
I’m not sure if they let you skip the cache write cost on the first turn. That would imply cross-user caching infrastructure or special casing the default system prompt to give you a discount. Maybe? Away from the computer but you could try a “hello” in a fresh session and see what was billed.
So yeah, when they banned tgird party harnesses there was a technical and $ case to have.
for coding agents 'shell' is often all you need (just make sure the environment has the necessary tools)
It's so unfortunate they don't let you use the subscription with other harnesses anymore - since even if I used OpenCode they'd still get a bunch of useful data from the API calls, meanwhile I could stretch their tier limits way further.
200$/month is a lot of money on Luna/DS4 flash, like really a lot and the results are much better than clowning on bloated CC.
It's absurd how you have more and more organizations encoding their processes on LLMs and "engineers" (charlatan coders) don't even bother optimizing the tool they use most.
I won't argue with the cost effectiveness, but the results are very much not better. Opus and Fable are in a different league than DS4 Flash. Even GPT Terra, which I really like overall, sometimes gets stuck in weird loops and starts to do stupid stuff once its context window fills up. Whereas I can more or less trust the big models to just Do The Thing™ on the first try.
With that said, you get way more value out of a GPT subscription than you do from Claude, partly because of the ability to use more efficient harnesses.
You can use opus or fable if you please, I'm advocating for writing your own agent instead of used generic bloated ones like CC.
Your subscription is not subsidised, it is just closer to the actual cost of the model…
The only downside for Anthropic that I can see is that hitting your limits more often (while using CC) could make you want to upgrade plans, and a more efficient harness could keep you from doing that. But I can't imagine the cost (to Anthropic) of those inefficient tokens is worth it to them.
also, are you using a tool to collect those metrics? what is it?
https://news.ycombinator.com/item?id=49006862
I'll have more about it in the next hours/days, you can follow me on twitter in the meantime (https://x.com/__tosh)
you can follow this org in the meantime https://github.com/smol-env
or on twitter here: https://x.com/__tosh
I’m asking because I’ve been looking for agent harness comparison tools too. I’m interested in more than just the inputs and outputs—I also want the system prompts, traces, and tool calls. It’s useful to understand why Codex, for example, uses more tokens while Pi doesn’t.
Fewer tokens aren’t necessarily better if the agent skipped important checks. On the other hand, using more tokens could just mean it’s overthinking the process. Either way, seeing the full execution trace for the same task is really valuable.
I agree fewer tokens is not necessarily better but a bit counter-intuitively often the harness using fewer tokens is not only done faster but has better results
(that said: of course check the results, look at the full traces, agree!)
Do we have any insight into whether it is actually spam and not useful info such as project or programming language specific context?
but even injected context that when I read it sounds useful can oversteer the model and make it second guess or take a more complicated route than it normally would
(you can see this when looking at traces with and without that injected context)
often harnesses also mention in their system prompt locations of markdown files that the model can consult if the model thinks they might help
that hint alone as part of the system prompt can be strong enough to make the model read in more tokens than would have been necessary
'spam' is maybe a harsh way to say it
unfortunately I don't see an easy way other than to invest time and tokens into finding out which parts of the added context (in system prompt, injected in turns etc etc) are actually helpful or harmful and when
I'm just doing the easiest thing I could think of: start from nothing or close to nothing
that seems to work better than what most harnesses are doing
turns out GPT 5.6 Sol is all you need
>often harnesses also mention in their system prompt locations of markdown files that the model can consult if the model thinks they might help >that hint alone as part of the system prompt can be strong enough to make the model read in more tokens than would have been necessary
I purposefully do this as I imagine it is useful. In my project I am seeing solid adherence to norms and a deep capacity to iterate on completed features. Essentially for each feature I have the model make a {featureName}.AGENTS.md at the root folder of the feature, where it maintains what is going on.
I am moving between Claude Code and Codex atm, but I began this pattern when pi + kimi 2.6 was my main driver.
Works fine. My conclusion is treat it like a starting point and not a holy bible of everything the model needs to know.
I pair this with making sure task files are commented with headers like build scripts so context+instruction is consistent.
The happy path then is the LLM pulling in context where it needs it.
What I wanted to emphasize is that whatever is in the context (whether system prompt or user message does 'steer' the model in a strong way, so everything in the context affects overall performance in a way. Even if it is 'just' net neutral it takes space up in the context window.
The context window is very very precious, everything that goes into it should help (not just hopefully help).
The challenge is coming up with good stuff to put into that context. A good agents.md file will be better context than whatever the popular harnesses have in their system prompt.
Also good to keep in mind that newer models are very good and more agentic than older models so they are better at exploring their environment based on the tasks you give them.
How do you tell, though? I guess what I'm asking is: the data you presented shows some crazy differences, but the token burn alone doesn't tell us enough. What was the output of the task like? Did the harnesses that burned fewer tokens give you as good a result as the ones that burned more?
I guess it's subjective, of course, but nearly everything about LLM use is...
in this case it was 10 tasks and all harnesses could complete the tasks successfully, of course now the question is: will this hold for more and more complex tasks but I had to start somewhere :)
Checksum: Compute a file’s SHA-256 checksum and save the exact digest to an output file.
Log correlation: Correlate nested service logs to identify and summarize a request’s complete execution path.
CSV report: Parse quoted CSV data and aggregate paid orders and exact decimal totals by region.
JSONL join: Join related JSON Lines datasets and produce a correctly grouped and ordered report.
Archive repair: Find the correct version of a corrupted file in a tar archive and restore it.
SQLite migration: Safely migrate a SQLite database schema and verify the resulting data and constraints.
Python bug fix: Repair an interval-merging implementation so it passes visible and hidden edge-case tests.
Python CLI: Implement a robust command-line program that reads JSONL and reports validated statistics.
Multi-file feature: Add an atomic feature across a small Python package, CLI, and associated tests.
Pipeline repair: Fix a Make, shell, and Python reporting pipeline so it handles general input and passes verification.
the "spamming" is mostly compaction appending files, tool artifacts, images in its summary there is a known github issue for codex. only solution is periodic clean up but its also how a lot of the agentic orchestration is performed and able to work for days.
I will look into how token usage looks like for longer sessions and more complex tasks
re caching: the cache ratio for this bench looks 'bad' for smol because it often finishes a task before caching kicks in (caching starts at 1024 tokens)
thank you for flagging this
smol looks interesting i think it could have a potential niche although tokens are only going to get cheaper and cheaper here
Be careful here. Remember these are non-deterministic models at the end of the day, and even with everything being "the same" you can have two runs where the same model, same harness, same tools can arrive at the same conclusion through a wildly different sequence of events.
I will add more tasks (esp longer ones) and think more about grading, the current tasks were easy to grade because the desired outcomes are well specced but I will also look into more open ended tasks and how to grade those
thank you!
the uncached tokens are also from runs where smol finished a task below 1024 tokens (the minimum amount of tokens needed to activate caching) which is less tokens than other harnesses are using for their system prompt (!)
> GPT-5.6 and later models: Caching is available for prefixes containing at least 1,024 tokens. This is a strict minimum.
https://developers.openai.com/api/docs/guides/prompt-caching
so in this specific case the count of uncached tokens for smol makes it look worse than it actually is
that said: it does makes sense to add more tasks that are difficult enough to fill the context window to compare the harnesses for how well they deal with compaction
staying below compaction (or with compaction at fewer compactions) is not only cheaper and faster, it also helps the agent stay on track
It is everything. My experience with Claude Code is that you have to decide when to compact to make it efficient. It defaults everything to 1M context and it will never keep it in check. It is strange how little cache reads you hit in smol, that may be a configuration issue.
The difference in tokens between the two also makes super curious. The system prompt can't be that different (I'd even bet Pi's shorter) and the 4 tools shouldn't make as much of a difference. I'm gonna have to try it.
the system prompt of smol is shorter than the system prompt of Pi
smol has no system prompt
system prompt of Pi 0.83.0
""" You are an expert coding assistant operating inside pi, a coding agent harness. You help users by reading files, executing commands, editing code, and writing new files.
Available tools: - read: Read file contents - bash: Execute bash commands (ls, grep, find, etc.) - edit: Make precise file edits with exact text replacement, including multiple disjoint edits in one call - write: Create or overwrite files
In addition to the tools above, you may have access to other custom tools depending on the project.
Guidelines: - Use bash for file operations like ls, rg, find - Use read to examine files instead of cat or sed. - Inspect PI_* environment variables for current model and session details. - Use edit for precise changes (edits[].oldText must match exactly) - When changing multiple separate locations in one file, use one edit call with multiple entries in edits[] instead of multiple edit calls - Each edits[].oldText is matched against the original file, not after earlier edits are applied. Do not emit overlapping or nested edits. Merge nearby changes into one edit. - Keep edits[].oldText as small as possible while still being unique in the file. Do not pad with large unchanged regions. - Use write only for new files or complete rewrites. - Be concise in your responses - Show file paths clearly when working with files
Pi documentation (read only when the user asks about pi itself, its SDK, extensions, themes, skills, or TUI): - Main documentation: /usr/local/lib/node_modules/@earendil-works/pi-coding-agent/README.md - Additional docs: /usr/local/lib/node_modules/@earendil-works/pi-coding-agent/docs - Examples: /usr/local/lib/node_modules/@earendil-works/pi-coding-agent/examples (extensions, custom tools, SDK) - When reading pi docs or examples, resolve docs/... under Additional docs and examples/... under Examples, not the current working directory - When asked about: extensions (docs/extensions.md, examples/extensions/), themes (docs/themes.md), skills (docs/skills.md), prompt templates (docs/prompt-templates.md), TUI components (docs/tui.md), keybindings (docs/keybindings.md), SDK integrations (docs/sdk.md), custom providers (docs/custom-provider.md), adding models (docs/models.md), pi packages (docs/packages.md), environment variables (docs/environment-variables.md) - When working on pi topics, read the docs and examples, and follow .md cross-references before implementing - Always read pi .md files completely and follow links to related docs (e.g., tui.md for TUI API details) Current working directory: /workspace """
So a cheap model with many tokens or an expensive model with much less tokens could potentially be same price.
Maybe only time will be the differential factor.
These days I “write” code with claude code and codex, and read/review it on GitHub. If I need to read it locally, I use a plain text editor.
Can someone help me understand what value cursor offers in 2026?
To answer your question though, to me your workflow seems cumbersome. Cursor is more integrated and more frictionless.
Should the AI sidebar be on the left, or on the right? At some point they swapped them automatically, which was jarring (maybe a bug). But now I realize that if I'm not the one primarily writing code and navigating it, then I prefer agents in the "primary" position on the left (esp. if I have the browser on the left half of the screen, the agents are in the middle).
What annoys me the most about Cursor is the random stuff that breaks (due to its nature of VS Code fork).
The tab experience is still unparalleled. The cost might be worse than Codex/Claude sub, but they are all much, much, much cheaper than API pricing, so depends on your POV.
In agent mode it’s horrible for editing files etc but better if you are juggling multiple chats
1) It's still an IDE. Because of pricing I mostly use Codex, but I always have a VSCode/Cursor IDE open, thus have to juggle between the two. Working directly in the IDE is more comfortable. For full on vibecoding that might be worse, but when you want to do a deep review of the changes, an IDE is way better than reading a diff on github.
2) It supports every model. It's often very helpful to try different models when you don't like the result of the first.
These days I've switched to Zed, which is good enough (and wicked fast), but I still miss Cursor as an IDE.
They seem to be doing their darndest to kill advantage #1. Even the command line "cursor ~/git" no longer opens an IDE in the ~/git folder but some random Chatgpt-esque interface that nobody asked for. Honestly, VS Code is a better "old Cursor" now than Cursor is.
#2 hasn't been much of an issue for me since Opus 4.5 came out. Anything the other models could do, Claude could do better. Of course things are starting to change now, and I've noticed Sol Ultra is specifically better at visual design stuff than Claude, but even that ~3% of cases I switch to GPT are still more easily solved with a cheap Codex sub than Cursor.
Also, Cursor really pushes you to use their agent stuff, I had to close/switch just to get to the goddamn code whenever I opened something in Cursor. VSCode is still happy to be an editor where code is centered for my review.
Unless by "text editor" you mean Neovim or Emacs ?
Diffing through a web interface offered by GitHub or Bitbucket is such a narrow window into changes.
The $20 price point is just too competitive now and I'd rather use the Claude/Codex plugin over Cursor's agentic coding sidebar.
Damn, and I thought Anthropic was fast.
The potential benefit of Cursor CLI (vs CC and Codex) is that you can easily between all major models (by Anthropic, OpenAI, xAI, as well as Kimi K3 and GLM 5.2). I found it useful when reviewing work - e.g. I implement using Opus then review using Sol, etc. Models by different providers tend to have different perspective on things and they can find different issues with the code.
Especially if you are doing "remote" development through SSH. If you are doing stuff where you still have to write some parts of the code manually or you have to fix few things here and there that the AI outputs, you still need a real editor.
Every 30 seconds in claude you get a new, elaborate, contrived bash command that claude wants to run. It's some combination of 10 things you've already granted permission for, but this one is new!
Cursor's auto-review run mode fixes that https://cursor.com/docs/agent/security/run-modes
I know it's a compromise in security, but auto-review+sandboxing provides a much nicer out-of-the-box experience. I'm happily using Cursor + Composer 2.5/Opus for most tasks.
That's about it.
You can still see what you’re billed on the Spending page. We did accidentally break dollar costs in the Usage CSV export yesterday while cleaning up an old feature flag. That was not intentional and the CSV export is fixed now.
That feature flag also showed a dollar usage graph to some self-serve users. The confusing part was included plan usage shown as $, which is not what you’re billed (on-demand usage is). Some people read it as actual spend, so we decided to remove that graph.
No, you can not: https://www.pasteboard.co/dNXUdT-h8Giy.png
If you want to say that "admin can" - it doesn't matter, I'm not going to ping admin every day to check how it goes. I'm not going to ask admin about every session to check how cost efficient a model was.
Double-edged sword. It is also simple to move back to VS code and agent extensions.
Gotta justify a $60B purchase of an IDE and (at the time) a single, decent model.
I imagine she’s just weighed all the details and options in her mind and plans to french fry tokens from my plate instead of getting her own.
And I hate that. I mean, I love her. She’s definitely value-add. But those are my tokens, right? My brain doesn’t do books. I can’t sit on the beach and read. I must always be swimming somewhere meaningful. So I’ll be vibing the next great Canadian web app and she’ll just casually ask, “hey what do you want to do about dinner?” So now I’m asking Copilot to tell me what I want for dinner. And it’s just… c’mon lady get your own tokens.
These days I'm using Codex and Claude Desktop with Zen when I need to look at code. Codex's real time audio chat feature (not dictation) is also second to none when paired with their agentic flow.
You came back from the dead pretty much and now you're pissing it away for what exactly?
Do not spite your individual developer customers or you will perish yet again.
You’re telling me there are a lot of Nazis we’re going to have to deal with. No problem, we’ve done it before, we’ll do it again.
It must be tiresome to live life worrying about things like this. Thankfully, the International Criminal Court today does not operate with a terminal case reddit brain.
And you're underestimating how much of a shit you need to give about this.
It might be called grok out it was trained by the composer team using a large chunk of their training data.
I guess this is to be expected of a now Elon Musk (that Nazi salute guy) owned company. It's a shame - the product was great.
Kind of (barely) like how Facebook has "friends". Or how a Snickers bar costs $1.99 and not $2.
Abstracting meaning of "cost", reducing value of information. Maximizing profits. Enshittification.