Comment by glub | Hacker News Reader

Comment by glub | Hacker News Reader

> We have sought a middle way: to show that it’s possible to build carefully and succeed commercially, and to make safety something on which AI companies compete. In other words, to create a race to the top

> Transparency. Regardless of what commitments we make, the public deserves to know what is going on. Anthropic has been a supporter of transparency for a long time

And then it goes on tangent of how we should pace everything (not just AI, but also the ingredients of what goes into AI, whatever that means), but only within approved democracies™, and outright restrict everything outside approved democracies™, because reasons that are definitely not about succeeding commercially that is threatened by the most transparent instrument possible - open weights, produced by basically just China.

I wonder what Dario would have done if open weights weren't produced by US's geopolitical adversary. How would an authoritarian manifesto be wrapped then?

Suppose it was, I don't know, Australia producing SOTA open weights, because they believed it was the right thing to do for the benefits of humanity - would Dario propose making Australia a geopolitical adversary too?

> Suppose it was, I don't know, Australia producing SOTA open weights, because they believed it was the right thing to do for the benefits of humanity - would Dario propose making Australia a geopolitical adversary too?

I'd expect he thinks that people are capable of realizing that AI is very dangerous and also that democracies end up mostly representing the will of the people, from which it follows that Hypothetical AI Leader Australia would agree to ban it too. This argument doesn't work for countries which don't care what their citizens want, like China.

I do think that it's a questionable decision to alienate China this much in this essay, instead of leaving open the possibility of China agreeing to a treaty that'll limit their progress. I suspect Dario is doing this to signal his allegiance with the US government, in hopes to increase the chance they'll go along with him, which is an unfortunate choice but plausibly the correct one.

I don't think people care as much as we'd like them to care about dangers of technologies. It takes a single step outside of technological bubble to see that their opinion of SOTA LLMs is vastly different. To them, AI means ChatGPT and ChatGPT is mostly still the same ChatGPT that it was 3 years ago, with similar failure modes and nothing that would indicate it would kill them, or take their jobs even.

It's the same as it has been with privacy/cybersecurity for decades. Vast majority of population doesn't care about hypothetical dangers, no matter how many essays get published.

So democracies representing will of people doesn't really work in favor of Dario's case here.

History is also not on his side. Limiting technological progress in the name of safety has a pretty poor track record.

> This argument doesn't work for countries which don't care what their citizens want, like China.

you didn’t have to use China as an example, the US clearly does not care what its citizens want as the most popular policies are never even discussed or proposed in congress

meanwhile, China destroying their housing market to decommidify it so everyone can have housing…they seem to care about their people more

another obvious example is climate change, where China is way more committed to solving it than the US. the division of the world into good guys and bad guys by a rather arbitrary definition of democracy looks so naive, even stupid. Anthropic is not a Democratic institution either, should we trust it? if this proposal for pacing the frontier is to be taken seriously, Dario should give its competitors, also international ones, the same assumption of good faith that he demands from us.

Reply on news.ycombinator.com