Hacker News Reader

Comment by sometimelurker | Hacker News Reader

> like they've repeatedly pursued).

source?

Many, many, many public policy positions; for a clear-cut example, they eventually supported SB 1047 [1] which would have banned open-sourcing any model trained with over 10^26 FLOPS (i.e. what Anthropic reportedly used to train Mythos). Their "Responsible Scaling Policy" [2] — a set of policy proposals that includes recommendations for government regulation — specifically calls out requiring "third-party controls" on model weights to prevent access; for developers to prevent "modification of models" such as fine-tuning (obviously impossible for open-source or open-weight models); prevent usage of model weights in "Automated R&D in key domains" which they specifically call out AI development as a key domain (again, obviously impossible for open-source); etc etc.

They want to ban open-source AI and are not shy about it.

1: https://campustechnology.com/articles/2024/08/26/anthropic-a...

2: https://www.anthropic.com/responsible-scaling-policy

Deeply concerning. How likely is this to become reality?

America has somehow managed to hang on to the right to encryption, despite plenty of well-heeled opponents, so it's possible to hang on to the right to open-source models. But it'll take a lot of vocal support, since there's strong incentive for Anthropic to try to cajole the government into banning competition (and they've already crossed that particular Rubicon, whereas OpenAI to my knowledge hasn't and at least still releases some open-source models like gpt-oss-120b).

The nuance is not what they propose, but why, even according to them, they propose this. Honestly the proposals are appalling, but biosafety arguments are not immediately dismissible. Ultimate cyber threats we can handle by rewinding society 50 years back. We can’t undo a novel genetically engineered virus.

I mean arguably, we _could_ create conditions in which it is much less likely for people to start developing such a novel genetically engineered virus.

If you think about the factors that lead to people wanting to do such a thing, they're almost always tied to (perceived) inequality, (perceived) injustice or similar in some way.

I do believe that we could greatly reduce a whole bunch of such risks by just stopping to squeeze people as hard as we do right now.

But that would require a major refactoring I guess.

> ...ch less likely for people to start developing such a novel genetically engineered virus.

only people that would do this would be crazy bc it cant be controled after release.

Yes but ethical responsibilities are entirely irrelevant, because that crazy person would still bioweapon you, and you kinda do not want that.

So regardless of if they're evil/bad/crazy or not, you still want to have that not happen. Hence the systemic perspective instead of the focus on the individual.

Reply on news.ycombinator.com