Hacker News Reader

Comment by aspenmartin | Hacker News Reader

Curious what your idea would be here for a truly good actor in this space; no AI development?

OpenAI's training is better suited to developing models that don't have these tendencies

https://www.goody2.ai/

Not the direct person you asked, but my answer would be alignment, interpretability, and policymaking. Perhaps improving existing usage? Helping grandma create reminders doesn't require advancing the AI state-of-the-art.

They are state of the art at all 3! As are other labs. Of all the labs they seem to take alignment and interpretability the most seriously to the point where they are hampering their own revenue in service of trying to not cause problems while also being in an incredibly competitive space.

All AI companies are trying to do all of what you’re saying. The issue is you can’t do that for long without a frontier system. Or you become a completely different, far less profitable company.

Implied in my answer was "and not creating ever stronger AIs", which unfortunately the big 3 labs are failing at. And they might be hampering their own revenue by doing the rest, but they also know that rocking the boat too hard is even more dangerous for their revenue. I wouldn't call it selfless.

No it’s not selfless, but I can’t imagine a more shareholder minded CEO would not have done a slow rollout of mythos. The point is: creating ever stronger AI systems is what these companies do, it is integral to what they even are. If you think that’s bad, even if all frontier labs agreed with you, you’re in a horrible game theoretic position. Any player can gain an enormous advantage by breaking the agreement. Not to mention Xi would be absolutely thrilled; now China can take over the AI race, become the load bearing infrastructure of humanity. We live in a complex world where simple childlike ideas like “well why don’t we just stop developing AI” actually are more damaging than keeping things going.

You're right that shareholder mindset cannot fix this problem, but that's what policy and agreements are for. And leaders can be convinced that AI is a direct risk to their own citizens too. If everyone else agrees to stop, you have less reason to continue when this action is putting yourself at risk.

And note how your argument can also be used against any non-prolifreration agreements, which are demonstrably possible.

I agree, I would say the difference here is economies weren’t heavily resting on nuclear armament, but maybe that’s the wrong take.

“Alignment” as a goal always ignores the “with what set of interests”, because there is an attempt to maintain ambiguity for different audiences (particularly, users, and non-users who seem themselves as the arbiter of broad social norms) to read in their own interests, when the actual answer is always the interests of the actor pursuing “alignment”.

Which value system to align to is absolutely the right question both rhetorically and otherwise. These models have a fairly western bias due to the domain of the training data.

But also, these models are capable of adjusting their value system depending on the user. Not saying that’s what’s being done but at a technical level that’s fairly straightforward, though not obviously better or with less problems.

No matter what human set of interests you consider important, you'll need alignment research to have any idea on how to instill it. Otherwise you're overwhelmingly likely to get an AI with a set of interests that's totally alien to what any human would ever want.

I think at this point the "instilling" part is not nearly as challenging and thorny as "what values should we instill"; that part is hard to imagine going away as it feels pretty fundamental to humanity that wars have been fought over.

Unilateral disarmament doesn't work though. If Anthropic is worried about this, just letting OpenAI win does seem genuinely worse.

If I speak up, I'm in big trouble.

Probably MistralAI or any of the Chinese companies that aren't throwing billions down the drain while American society lacks healthcare, childcare, and good wages.

You want Anthropic to fund your healthcare or something? Also, have you seen the impact of these models on healthcare? Also most of our GDP growth this year is from AI buildouts, would you rather that be negative?

And not even considering: Chinese AI companies are the good guys???

Yes, yes I would prefer that. Better than a total societal collapse.

Anthropic are not the good guys either. So here’s to hoping the Chinese pop the bubble.

Nobody anywhere is a good guy but I don’t think you’ve managed to pick the lesser of the evils here

American society has higher wages than almost any other developed nation [1], so it's objectively incorrect to say the US doesn't have good wages. It chooses to make you pay for private childcare and healthcare, both of which are high-quality but stupid expensive. It's a tradeoff like anything else a nation/society creates and prioritizes.

No idea how that connects to the idea that Mistral or DeepSeek are somehow the "good guys" though?

[1]https://www.oecd.org/en/data/indicators/average-annual-wages...

I like how you use average and not median, also while completely ignoring how bad income inequality is (worse than the gilded age ffs) or that the American elites stole $50 trillion from the bottom 90% over the last few decades:

https://time.com/5888024/50-trillion-income-inequality-ameri...

I'm glad you mention the "trade off" where it's elites trading off the lives of American workers for money. Makes it quite apparent where you sit on the table of equality.

None of the money being spent by Anthropic was going to go towards healthcare or childcare.

Reply on news.ycombinator.com