Hacker News Reader

Comment by a2ff6eeb0 | Hacker News Reader

Why? Are trolley problems making it hard to make money?

Because leaving that to emergent behavior is silly. Why is the idea of training models on ethical reasoning so strange?

Because 1) ethics aren't universal. It's perfectly easy to frame ethics for the outcome you want depending on your a priori assumptions. 2) it's blatantly obvious that the business "needs" will supersede any ethical concerns or the ethics will be framed in a way that aligns with the primary goal of gaining money and power.

I don't see either of those as reasons not to train on it. Training isn't zero sum. You don't need to take something out to train ethics, and whether you explicitly train it or not, models will take ethical stances. Your second point actually argues in favor of training because that is still an ethical stance.

I think a lot of people confuse ethics training with "don't be evil"

When you put it like that, it's actually probably worse to train it on a subset of ethics curated by the people doing the training, aligning with their world view and interests. If the model learned ethics just from raw training it would at least be statistically in line with it's training corpus.

Manually training on "approved" ethical stances sounds a lot like censorship.

I mean, sure, its censorship to the extent that you can apply the term to an institution that is also the creator of the censored work, and that's exactly the point and the whole meaning of “alignment”.

I would eat my shorts if we could get alignment in a HN thread on what the top 25 "don't be evil" required ethical items for a AI lab would even look like.

The first 10 maybe easy, but we'd see so much disagreement even without money nothing would get done.

Because it's not going to make the investors richer, which is what most alignment work focuses on -- how do you make its output palatable enough to avoid embarrassing customers. The entire focus of the AI industry is pumping up the dollars coming into the company, and if you're not contributing to the bottom line, your neck is on the line.

You haven't explained why investors make more money by spending money on people to solve trolley problems.

Reply on news.ycombinator.com