Hacker News Reader

Comment by aetherspawn | Hacker News Reader

I discovered yesterday that the “amazing thing that comes out of OpenAI” is Sol, due to its token efficiency.

Dollar for tokens, Sol and Fable are the same price.

However, Sol uses (literally: in testing) around 10-100x less output tokens compared to Fable for the same task.

We run our frontier models nearly 24/7, so switching to Sol will save us around $500 per day.

And, due to less guardrails, Sol also performed better, and we lost less tokens due to guardrails shutting down sessions (I feel like it’s illegal to take $50 of someone’s token money and then shut down a session with guardrails before they get an answer, and yet Anthropic do it to us constantly… either take our money and commit, or trigger the guardrails immediately)

View on HN

We’ve literally saved tens of millions of dollars already (no exaggeration! already 8 digits) by switching to Luna for many workloads at my company.

The amount of workloads we can shift with an advisor model pattern continues to grow.

It’s seriously amazing.

Luna came out about a month ago, you're saying that the cost saving from switching to Luna has saved your company $20 000 000+ in 1 months spending on API usage?

OpenAI and Anthropic are believed to be making billions per month in revenue selling tokens to enterprises. It’s not that surprising for enterprise SaaS they would have individual customers making up ~1% of their total sales. Although, perhaps a bit more surprising those few customers are posting about it here.

Another interpretation would this is a counterfactual savings, like they previously paid $1M for y tokens, and now that tokens are cheaper they increased usage and paid $1M for 20*y tokens.

A second angle on the counterfactual savings would be Luna telling them not to pursue a potential session with the expected/extrapolated (from which sessions they did ignore Luna on, e.g. just to keep efficiency statistics current) sunk costs at time of getting shut down used to derive the quoted number.

A third angle is that it is not true and that's just some attempt at influencing an HN thread, which is happening quite regularly pro and against AI. We know that the pro-AI has a way bigger budget to burn with this kind of operations tho.

It makes sense at a company that sells AI as a service. Perhaps to make presentations...

The ones with terrible slides on their MTA ads?

Canva generates billions in annual revenue, it's not unbelievable

By "advisor model pattern" you mean https://claude.com/blog/the-advisor-strategy ?

How do you implement that outside of claude code?

the easier way to do this, with any harness, is to use an expensive main agent that is told to delegate all code reading, writing, exploration, research etc to weaker subagents to conserve tokens. It's an inversion of the pattern but the resulting split is the same

Yeah, that one is easy, but I was thinking of the other way around, where the weak model calls stronger one.

I append this to many of my opus claude code prompts

`You may use a Fable subagent to answer questions, solve problems, and provide an adversarial review of your ideas and code`

You can use a similar pattern in most any harness, and you can tell them to use other harnesses. In claude you can write `Use codex cli to have Sol56 Xhigh provide an adversarial review to your plan before presenting it to me` or `Use opencode cli with GLM 5.3 to verify all code review findings before presenting` or whatever you're doing, as long as those other tools are setup and ready to be called.

IMO: This isn't useful as a token saving pattern in my experience with agentic engineering, but it is useful as a quality-enhancer.

what has a single company accomplished with tens of millions of token spend?

A pelican on a bike accurate to a subatomic level

But the knees still bend the wrong way

higher valuation

What are you using it for?

Evidently, Claude's tokenizer vocabulary size is ~15k[1]. On one hand, it's quite mind blowing. On the other hand, Anthropic models' token (in)efficiency makes a lot of sense in that light.

[1]: https://xcancel.com/magikarp_tokens/status/20878591737488549...

Not just that, they normalize everything into lowercase and use a special character to capitalize words (what about languages with non-trivial normalization/capitalization?) and mark beginning and end of each word, all of that diluting already small vocabulary. That smells like manual tuning of what should be done statistically, I wonder what technical merit they saw in that - I know they mentioned better generalization, but this is pretty counterintuitive.

Sadly it's not too counterintuitive; remember the old "how many r's are in the word strawberry"?

Also different tokens for the same named entity/concept if they almost entirely exclusively occur in non-overlapping contexts, and are themselves rare/uncommon in the first place, will result in behavior that's similar to the speech/phrasing/vocabulary registers humans exhibit, where the aspects of the named entity/concept get largely compartmentalized.

The most severe case along these lines were the old BERT models that ran over straight UTF-8 bytes (plus a handful special tokens).

But for the modern post-GPT2 LLMs such radical simplicity seems to mostly not be considered suitable. Note that CJK (the big one in particular, so Chinese semantic and Japanese Kanji) encodes each one into multiple UTF-8 bytes giving some automatic scaling for semantically dense languages; similar effects also apply to e.g. APL code.

Sol is way cheaper than Fable by the token.

Wait isn't Fable like 2x more expensive if we compare under 272k tokens

The comment you're replying to almost feels like it was written by a bot or am I crazy?

Peak under your skin a bit. Something weird is going on. I think we are bots/robots(sic)

Related, Under The Skin with Scarlett Johansson is an incredible movie.

Peek*

beep boop, everyone thinks I’m a robot. :|

What kind of bot would say `less guardrails` instead of `fewer guardrails`? I guess someone could instruct an LLM to deliberately make mistakes, but isn't that too paranoid?

We’re reaching transvestigation levels of people trying to spot AI text everywhere they look

Hmm, this sounds like something a bot would say to prevent being caught.

...Does a bot know the word "transvestigation"? Would a bot be allowed to say it by its corporate overlords?

It does now that is on hn. Thank you for your input.

15 years ago i was astounished by the intellectual deepness of this community. Now i'm aware this has always been a cult and their former cult leader Mr. Altman wants to destroy this capitalist society. He isn't even hiding motives. People just stopped listening carefully.

In some romantic languages less and fewer are the same word, so it is a common mistake for people that have them as their mother tongue.

In German the distinction is also different from what English speakers sometimes make.

“include common grammatical imperfections and awkwardness common in casual message board interactions.” I’m not saying that’s what’s happening here, but a high school student told me that’s basically what they do to make papers not sound like AI.

I agree. Weird to use <“> and <”> characters. Maybe written on phone, but certainly not on keyboard.

Other languages use different characters for quotes, if anything that's an indication that's not written by a LLM because it's not favoring the standard English character.

https://en.wikipedia.org/wiki/Quotation_mark#Specific_langua...

Written “on an iPhone” - yep, seems to automatically switch the quotes.

Common on non us keebs though.

You could say the token usage is "load-bearing".

Its token usage has earned its keep.

One thing I've deliberately deferred is drawing conclusions from all of this.

Reply on news.ycombinator.com