Davidzheng

Well that's the only way to escape the permanent underclass, otherwise even Elon is not excempt;)

Post-training can have big gains no? I don't think the current sizes at ~1T are saturated in intelligence (it's like saying AlphaGo Master is just a post-trained version of AlphaGo Lee)

Evidence towards aside, the risk calculation depends on how much upside is from the 100B if progress starts the same rate. I think it's hard to bound that accurately

I don't think Anthropic is claiming it's world ending? Just that it has offensive cybersecurity abilities which can be dangerous

Is it completely clear to you what the purpose of this is? It isn't completely clear to me but it's very likely it's an issue with me. I feel that it can be part of some larger counteroffensive against certain actors in China in a way that is more than just the signalling here--like maybe there's much more we haven't seen. In any case, it certainly shows their willingness to use less conventional tactics against those they view as adversaries.

But it reads to me like the thread parent's point is that there are many unknown risks which can exist? I also wonder about long term effects to the health of the genome from IVF and other forms of fertility treatment as infertility could be acting as some sort of protection mechanism of the genome. But I suppose such objections form a continuum which extends to treatment of all genetic diseases or diseases in general--all of which probably applies some evolutionary pressure towards more healthy individuals but which we as a society have to balance against wellbeing of individuals and their human rights.

oh so they should team up

But in any case, I think more than 10% of information workers today can be replaced by current-generation models indefinitely.

Yeah it's not true that for every job, it is better than median worker of that job. But it is conceivable that for almost all jobs it is already better than the median human (not just workers of that job).

And what is it worse at than an average human today that can be done on a computer?

Maybe what can happen is that the people who benefit from it donate back into the organization of this room when they have the means decades later?

ok I guess they had other clues then if you do any sort of comparison vs Nex & Qwen probably a lot of weird coincidences will show up if somehow the three weights are not linearly independent lol

it's interesting that this was even guessed at

If they only opposed it they wouldn't have had these billions of dollars? Also I think they genuinely believe they cannot stop it because the Chinese companies are close behind (I also believe it's impossible to stop b/c of strong economic pressures selecting for those who will advance this tech and there are many who can)

Are you sure most of frontier cost isn't inference in RL environments?

How is it ironic to believe in a risk that actualizes

Theres quite a large number of people who believe it's basically impossible when the intelligence gap is too big

I agree with this. But i think Ilya and Dario hold these beliefs sincerely. Probably a sizable portion of Anthropic employees too

We're on track to get there globally and economic pressures will ensure it happens. It's not too early to worry about it

Aren't the models also distributed to various data centers--i think it's very easy with resources

Human user usage data is probably a tiny contribution to improvement of the models--it's mostly RL on environments

What is this comment? If they occurred we would face a huge disaster; isn't it better to err on the side of caution to make that risks as low as possible???

I think it's too early to understand the ramifications but I agree this is a huge deal.

Is the total compute capacity outside of meta, google, amazon, anthropic, oai and x is higher than even the capacity of any of them? In any case, there's no chance a public collaboration gets to anthropic levels of compute even if communication were no issue.

At some intelligence capability there can be catastrophic risk, the fact that we don't yet have any catastrophe doesn't mean the risk wasn't real. It's similar to new viruses which don't lead to outbreaks, the correct takeaway isn't "oh you were insane to panic bc nothing happened". There is small risk (and increasing) of huge harms with each improvement

No...

can I put open math problems on this site too with bounties

someone correct if i'm wrong, but I think the max models are usually non-open

My only gripe is that on my phone sometimes it takes like 30 seconds to load, which doesn't seem to happen for almost anything else

Deep think still makes many many many more mistakes than gpt 5.5 pro on math