Hacker News Reader

Comment by mmaunder | Hacker News Reader

“Each of the results cost roughly $100,000 in API cost to develop.”

And

“Over the course of a week, one Anthropic researcher worked together with Claude to develop the HAWK attack, and another researcher built a scaffold4 that allowed Claude to fully autonomously discover the AES attack.”

Spending $100k in tokens in a week is an impressive feat even with massive parallelization. I suspect the TPS their internal folks have access to is far higher than their bulk public endpoints.

There’s a tech aristocracy rapidly emerging in our society and it’s going to tear us apart.

View on HN

Today's state-of-the-art AI systems are the equivalent of late 1970s personal computers: bulky and expensive, but wildly more powerful than what came before. And look what happened: tech improved by orders of magnitude, and eventually computers were tiny and cheap.

I predict the same will happen with AI: certainly the latest and greatest will still command a steep price (yes, supercomputers are still a thing) but for most people who just need something reasonably fast and powerful, cheap (or free) AI will do the trick, especially when run locally.

So no, the aristocracy won't have a lock on the technology because tech is always being democratized. Until arbitrary computation itself is outlawed (and yes, I know, governments and industry are always inching us closer to that), we'll be ok.

1970s personal computers were "controlled" by the likes of Richard Stallman. 2020s AI systems are controlled by the likes of Sam Altman.

See the difference?

This is intentionally a wild and crude oversimplification, but I hope to raise at least one point against the notion that AI will be similarly democratized.

> 1970s personal computers were "controlled" by the likes of Richard Stallman.

Who are you talking about? Richard Stallman is all about allowing anybody to freely run their own software on their own hardware.

That's not really that ridiculous. Looking at my ChatGPT stats my biggest day of token usage was 1B tokens (seeing how far Sol Ultra could go on a difficult problem with a quantitative goal and eval harness that it could run on it's own that allowed it to keep going until it succeeded). I blew through my $100 subscription usage in that one day, but with a 80/20 token blend that's $10k in API billing. So, $70k in a week. With the higher cost of Mythos, that's not that crazy. It's more so a testament to how ridiculously marked up API tokens are, or how discounted subscriptions are (who knows which is true)

How much was valuable the output you got during the day?

Is it at least comparable to the $10k of cost?

Definitely not. It was my first time using Sol Ultra, and I did not expect it to run for 12hr+ continuously. Previously with 5.5 I had to coax it to go for longer than a half hour, so I just started a run and forgot to check back in on it. I think I'm learning how to use it more effectively now, and I could see a scenario where that could actually be worthwhile, but it's a bit of a shot in the dark if you let it go that long uninterrupted.

Is that including cache reads? It seems improbable that (a) you actually filled 1000 1M contexts and (b) OpenAI allowed that in a $100 subscription window.

I don't know, codex does not disclose that information, it just says "1.1B Tokens". OpenAI recently got rid of their 5-hour usage window, so you can burn the entire week in one go, which is what I did.

So $1-10k in Chinese model time, thus why we must ban them.

If a Chinese model can do it for $1-10K, then why hasn't one?

Why have all the mathematical (and now cryptographic) breakthroughs come from OpenAI and Anthropic?

Is it possibly because the Chinese models are so benchmaxxed they can't make novel discoveries?

Don't know if one has or not, but a lack of announcement is not a lack of success.

Do you really think that if a Chinese model had achieved a significant math breakthrough that it wouldn't be trumpeted to the global media? The "Deepseek moment" was great for China; this would be the same.

They're a few months behind the frontier, which, by the way, only started making these discoveries in the last few months.

That might be true, but it's irrelevant to the current discussion, which is arguing about whether or not Chinese models can do this now. One of the commentators above is arguing that it's not only possible but significantly cheaper (i.e. Chinese models are not only at par with frontier but several months ahead).

Ah, sorry, you're right. I would be willing to be that Kimi could do a decent job today, but definitely not for 1% of the price.

Perhaps since the Chinese companies don't have the economy of an entire global superpower riding on them, they can't afford to light $100k+ on fire doing random shit with the hope it'll turn out a research paper.

“I haven’t seen X, therefore it doesn’t exist.”

Absence of evidence is not an evidence for absence of something.

I haven't seen leprechauns, at least not without the aid of xiao ren ren mushrooms, but I'm still pretty sure they don't exist.

For one thing they are extremely GPU constrained due to export controls, so it’s unlikely a priority compared to training

So why didn't anyone with access to Chinese models do it. They're open-source, right? So you're telling me no-one could afford $1K to solve a problem like this?

(hint: it's BS, the Chinese models either can't do it or can at the same or greater cost)

Reply on news.ycombinator.com