Hacker News Reader

Comment by lebovic | Hacker News Reader

I used to work at Anthropic. I fully believe that the folks mentioned in the article, like Jared Kaplan, are well-intentioned and concerned about the relationship between safety research and frontier capabilities – not purely profit.

That said, I'm not thrilled about this. I joined Anthropic with the impression that the responsible scaling policy was a binding pre-commitment for exactly this scenario: they wouldn't set aside building adequate safeguards for training and deployment, regardless of the pressures.

This pledge was one of many signals that Anthropic was the "least likely to do something horrible" of the big labs, and that's why I joined. Over time, the signal of those values has weakened; they've sacrified a lot to get and keep a seat at the table.

Principled decisions that risk their position at the frontier seem like they'll become even more common. I hope they're willing to risk losing their seat at the table to be guided by values.

View on HN

> I hope they're willing to risk losing their seat at the table to be guided by values.

that's about as naive as it can be.

if they have any values left at all (which I hope they have) them not being at the table with labs which don't have any left is much worse than them being there and having a chance to influence at least with the leftovers.

that said, of course money > all else.

This is a common logical fallacy. It's not true that the party A with a few values can influence the party B with no values. It's only ever the case that party B fully drags party A to the no-values side. See also: employees who rationalize staying at companies running unethical or illegal projects.

Employees and employers are not sitting at the same table, this is a category error. We're talking lab to lab. Obviously in a fiercely competitive market like this with serious players not sharing the same set of rules it's close to pointless, but it's still better than letting those other players do their things uncontested.

I don't hold the belief that it's always better to have influence in a group where you don't trust leadership – in this case, those who decide at the metaphorical table – vs. trying to affect change through a different avenue.

It's probably naive, but it's also the reasoning that drove many early employees to Anthropic. Maybe the reasoning holds at smaller scales but breaks down when operating as a larger actor (e.g. as a single person or startup vs. a large company).

> I joined Anthropic with the impression that the responsible scaling policy was a binding pre-commitment for exactly this scenario

Pledges are generally non-binding (you can pledge to do no evil and still do it), but fulfill an important function as a signal: actively removing your public pledge to do "no evil" when you could have acted as you wished anyway, switches the market you're marketing to. That's the most worrying part IMO.

If you're not willing to give up your RSUs you shouldn't be surprised that the executives aren't either.

The moral failing is all of ours to share.

I was willing to (and did) give up my equity.

I fully believe that Dario is 100% full of shit and possibly a worse person than Altman. He loves to pontificate like he's the moral avatar of AI but he's still just selling his product as hard as he can.

They are all the same given their motivations - Demis Hassabis is the only one who, to me at least, sounds genuine on stage.

Demis is a researcher first. Others are not.

I interviewed at Anthropic last year and their entire "ethics" charade was laughable.

Write essays about AI safety in the application.

An entire interview round dedicated to pretending that you truly only care about AI safety and not the money.

Every employee you talk to forced to pretend that the company is all about philanthropy, effective altruism and saving the world.

In reality it was a mid-level manager interviewing a mid-level engineer (me), both putting on a performance while knowing fully well that we'd do what the bosses told us to do.

And that is exactly what is happening now. The mission has been scrubbed, and the thousands of "ethical" engineers you hired are all silent now that real money is on the line.

> Every employee you talk to forced to pretend that the company is all about philanthropy, effective altruism and saving the world

I was an interviewer, and I wasn't encouraged to talk about philanthropy, effective altruism, or ethics. Maybe even slightly discouraged? My last two managers didn't even know what effective altruism was. (Which I thought was a feat to not know months into working there.)

When did you interview, and for what part of the company?

> knowing fully well that we'd do what the bosses told us to do [...] now that real money is on the line

This is a cynical take.

I didn't just do what I was told, and I dissented with $XXM in EV on the line. But I also don't work there anymore, at least one of the cofounders wasn't happy about it and complained to my manager, and many coworkers thought I had no sense of self preservation – so I might be naive.

The more realistic scenario is that a) most people have good intentions, b) there's a decision that will cause real harm, and c) it's made anyway to keep power / stay on the frontier, with the justification that the overall outcome is better. I think that's what happened here.

I do trust that you earnestly believe in the importance of ethics in AI - but at the same time, I think that may be causing you to assume that the average person cares just as much or similarly.

I've seen the same phenomenon play out in health-tech startup space. The mission is to "do good", but at the end of the day, for most leaders it's just a business and for most employees it's just a job. In fact, usually the ones who care more than that end up burning out and leaving.

The EU should invite them over.

The kind of principles you talk about can only be upheld one level up the food chain. By govts.

Which is why legislatures, the supreme court, central banks, power grid regulators deciding the operating voltage and frequency auto emerge in history. Cause corporations structurally cant do what they do without voilating their prime directive of profit maximization.

Reply on news.ycombinator.com