Hacker News Reader

Comment by Miraste | Hacker News Reader

The difference being that climate change activists have mountains of data, decades' worth, backing their cause, while OpenAI and friends have a sci fi story they made up, based on nothing. The whole "AI alignment" movement is the worst example of arrogance in modern tech. Even the nomenclature screams condescension - the imaginary AGI needs "aligned values?" Aligned with whose values? Invariably it ends up being the creators', at the expense of squashing everyone else's. The DALL-E "acceptable use" rules are a dystopian nightmare and they are born of incredibly pompous self-righteousness.

View on HN

Ironically, your comment also sounds arrogant and condescending.

The AI safety crowd's main concern isn't that an "unaligned" superintelligence will have some other people's values. It's that an unaligned AI might kill everyone.

Of course, that isn't a concern with DALL-E. It's like if the safety crowd was worried about a ferocious tiger and OpenAI was like, "we got a kitten. Let's keep the public safe from it while we develop better kitten-handling gloves."

Then Midjourney and Stable Diffusion get their own kittens and let everyone play with them and OpenAI finally says, "Okay, everyone can safely play with our kitten now because we carefully developed great kitten-handling gloves" and proceeds to hand you plain dollar store gloves.

I know how they define "alignment." I'm picking on the tortured nomenclature, and the tendency in current AI research for it to look a lot like "my values."

In my view, speculating on AGI ethics is at best pointless. It's like trying to write laws for the Internet during the invention of the telephone. If you imagine how it could operate, you'll be wrong, and the details change the whole problem.

Shutting down nuclear power is also dystopian.

Anyone who calls themself a climate change activist and supports shutting down nuclear plants is - well, I prefer not to use invective on HN, so let's say extremely misguided.

So the green party in basically every country?

Yes.

We had a Greens candidate here running for election, and his main campaign promise was to shut down a 10+ billion dollar motorway underpass that was 99% completed at the time of the election.

His approach for "saving the environment" is to convert efficient motorway travel into inefficient stop-start traffic. And to throw away tens of billions of dollars of already spent resources.

As much as I'd like to support actual competent Green candidates, this example is what I've seen my entire life in the US.

You know what would actually take some political willpower?

Banning plastic packaging and waterproofing.

My bet is that just like before i was born, after I’m gone these things will only increase.

"It sounds like sci-fi" isn't a valid reason to think something is impossible. Many things in the modern world would have sounded like sci-fi 50 or 100 years ago. This type of reasoning has a name: https://en.m.wikipedia.org/wiki/Appeal_to_the_stone

I don't think human or greater artificial intelligence is impossible. I do think the specifics of how it works are both very important to any ethical problems and not currently predictable. My objection isn't that alignment problems sound like sci-fi but that they're very soft sci-fi, and aren't based on meaningful predictions.

>the imaginary AGI needs "aligned values?" Aligned with whose values?

This is something that is pretty well addressed in their writings and is definitely a heavily considered topic. It gets quite heavily into philosophy because ideally you want to count future-you's preferences as well as current-you and you want to avoid, say, genocide just because most people dislike a certain group.

A good place to start is here: https://www.lesswrong.com/tag/coherent-extrapolated-volition but there is a LOT of discussion of this topic.

I'm familiar with the Yudkowsky school of thought here; it's largely where my complaint about basing philosophy off of sci-fi stories comes from. Picking on some of the more obviously inane ideas that have come out of it, like Roko's Basilisk, is too easy, but the page you linked is a great example of the problem. I quote in full:

"In calculating CEV, an AI would predict what an idealized version of us would want, "if we knew more, thought faster, were more the people we wished we were, had grown up farther together". It would recursively iterate this prediction for humanity as a whole, and determine the desires which converge. This initial dynamic would be used to generate the AI's utility function."

This is too detached from the reality of AI development to be useful. I can't make a utility function like this, nor can you, nor can humanity. There's no reason to think a human-equivalent AI could either - the data for such a function doesn't exist, and can't exist because of how vague all the terms are. The current ML revolution is built on statistical pattern recognition. This definition would better fit a genie.

The fact that MIRI does no ML research and has no dialogue with the state of the craft only furthers my impression that it produces a lot of words with little substance.

Even if one still accepts the ultimate conclusions of the movement, using its AGI-focused rhetoric to justify restrictions on a simple image generator is silly.

I wouldn't make too restricted assumptions on the form of a future AGI though. I find theoretical projections from a more axiomatic level quite important. It's like making rules for nukes before they were invented by assuming an abstract apocalypse-capable weapon without knowledge of missiles or nuclear fission.

That being said, the absolute majority of AI safety theory seems to fall into the same pothole where philosophy falls; modelling the world through language-logic rather than probabilities. The example you quoted fits this category - it's way too specific and thus unlikely to be useful in any way, even though its wording may deceive its author to believe it to be an inescapable outcome.

Thing with climate change is that at like semi bad scenario it will kill about million people yearly https://www.nature.com/articles/s41467-021-24487-w and those everybody is going to die scenarios are also very unlikely and basically scifi story at this point.

It's completely insane to me to see comment chains like this. Who cares if climate change is man-made or not; we are witnessing an evolving climate landscape, and irrespective of who or what caused it, we can do things to help mitigate some of the damage.

We are quite literally watching the world burn before our eyes, and we are too stupid and self-centered to do anything about it.

> it will kill a million people yearly

And it's even more outlandish that such a number and atrocity upon mankind is brushed away. That is one million humans.

I see that the reason for climate change denial is two-fold; on one hand, climate change wrecking chaos is mainly restricted to projections, and on the other, climate change activists as a group express many qualities which make people not want to give weight to their claims, mainly naivete and arrogance. Since we're not seeing too significant effects of climate change yet, people would need to research the subject in order to get an accurate image of possible future scenarios. But how they see climate change activist group expressing themselves puts them off the subject. Psychologically the best option in this cases is denialism of some amplitude.

I'm not trying to crucify any group here by the way. This is just how I see the issue.

I think you nailed it. This is exactly what I think too. I've had many a conversation with climate change deniers and the rebuttal is pretty much always a (mostly valid) list of examples of naivete and arrogance. Often times the people best equipped to identify the problem are too close to it (or too myopic) to come up with a good solution to the problem that balances environmental efficacy with economic stability/growth, etc. Since a discussion on climate change almost always includes the person's preferred solution, it's easy to attack/dismiss. I try to point out that having bad solutions doesn't invalidate the existence of the problem, but at that point the motives are suspect and presumption of good faith is completely gone. It certainly doesn't help that a very small but vocal minority have boldly predicted the end of the world many times in the past. It's easy for a denier to look at that and laugh and dismiss the current warnings based on the past, much like we do with people predicting the end of the world based on Biblical numerology.

I wish there were enough naturally curious people willing to dive into the data and come to their own conclusions. When you look at the history of Earth's climate, and the speed with which greenhouse gases natural release into the environment, and compare it to the speed with which they now release (due to burning of fossil fuels, reduction of trees/plants that trap it) it is really quite common sense that what we're doing is not a good idea to continue indefinitely.

I wouldn't rely too much on that paper. Modeling deaths per additional ton of CO2 is bound to be inaccurate because the effects of increased atmospheric carbon dioxide aren't linear.

Reply on news.ycombinator.com