Hacker News Reader

Comment by eknkc | Hacker News Reader

> which they clearly think is bad

These are companies with huge amounts of money invested in them. Their main responsibility is to their investors. If building better models (or making more money from the models they already have) would increase returns, they would do it. So they clearly can’t do it right now.

Climate change hasn’t stopped the oil industry, and concerns about forever chemicals haven’t stopped the chemical industry. Companies don’t usually give up profitable opportunities just because they might be harmful or kill us all in the long term..

View on HN

I think you fundamentally misunderstand what kind of company Anthropic is. I'm simply taking the path of least resistance when understanding what people at Anthropic have to say, and basically every public source points to Anthropic talking the talk as well as walking the walk with regards to AI safety and their ideas on AI governance.

Re the point on returning shareholder value, Anthropic is a PBC:

> Public Benefit Purpose: Under Delaware law, a PBC allows directors to legally weigh social and public good alongside maximizing profit for stockholders

Investors surely know this and Anthropic probably has pretty reasonable legal grounds to stand on if they chose to prioritize safety over profits.

Seems pretty clear to me that this is in earnest.

Oh I am terribly sorry, did not know Anthropic was formed as a a “Good Company”. Which changes everything ever.

Snark aside, maybe I’m too pessimistic and you are correct on this. But.. doesn’t that mean we are completely fucked? If they don’t have any ulterior motive and they truly are panicking.. We should start packing.

yes that is literally why they are sounding the alarm - because people think AI is a party trick and they aren't taking it seriously enough.

The decision to found Anthropic to compete with OpenAI's then concentration of SOTA LLM AI researchers, and then achieving/surpassing parity with OpenAI is pretty much directly responsible for creating the current capabilities arms race.

Google has shown they are not motivated at all to push the bigger and bigger SOTA frontier with Mythos-type super models. Even OpenAI isn't really even trying to surpass Anthropic at the top of the market, preferring to keep pace in benchmarks but at lower costs to serve. Anthropic with Mythos is now more or less alone in aggressively pushing the ceiling for the most "super-intelligent" models current computing can support.

It has made them and their investors a gigantic pile of money, but we're really supposed to believe their contribution to that arms race (which they now describe as a threat to humanity) has always been completely altruistic and that their corporate structure inherently makes self-interested decision making impossible?

If you've read enough lesswrong, this won't be new to you. The strategy always was "become the most capable model lab so that you have leverage to control AI development."

Reply on news.ycombinator.com