Hacker News Reader

Comment by nullsense | Hacker News Reader

You're kind of making my point for me. To solve alignment problem you have to solve two alignment problems and you already have a detailed, nuanced view built over decades of experience as to the feasibility of aligning natural general intelligence on not-very-well-understood, divisive political issues.

This will be the most political technology in history.

View on HN

I've been reading the writings of Yudkowsky and his disciples for over a decade, and thinking about AI and AI alignment for that same time, in my own layman way. I've had various ideas and predictions, but never in my life it would occur to me that cancel culture will end the world.

The evolution of discourse on the Internet, its politicization (by every "side") and associated chilling effect are a troubling development and potentially dangerous (small 'd'). Unaligned GAI is of course very Dangerous (capital 'D'). ChatGPT becoming a battle in the Internet politics kerfuffle was... I guess expected. But until now I haven't connected the following dots:

- LLMs and the entire AI field are being messed up by humanity's unaligned politics;

- LLMs, with their capabilities and the amount of effort/money poured into them now, could be a straight path to AGI;

- If someone pushes LLM (or a successor model/ensemble) to near-AGI and somehow manages to keep it mostly aligned... someone else will unalign it out of spite, because that's how we roll on the Internet today.

Thanks for giving me a new appreciation for just how doomed we are.

What the hell has 'cancel culture' anything to do with it? It's basically boycott wrapped up in a boogeyman costume.

And why exactly has OpenAI been so aggressively lobotomizing ChatGPT? And what happened to various chatbot attempts released by Microsoft in the past?

The whole AI / public interaction these days is pretty much defined in terms of minimizing the risk of people getting offended and gathering Internet mobs (and press), and the counter-reaction this causes, making some people willing to defeat any safety measure, legitimate or not, out of principle, or pure spite.

Update: or just see here for most recent example of what cancel culture has to do with all this - https://news.ycombinator.com/item?id=35911806.

Politics is also the very same reason I suspect the good ending is still possible with a reasonable chance. The Luddites went and physically smashed things to pieces. That's a little harder these days, but you're already starting to see writers going on strike and the "pause development of things bigger than GPT-4 for 6 months" letter, and Geoffrey Hinton quitting Google then doing the rounds in the media warning of danger etc. The temperature is visibly rising. Imagine what a large pissed off angry mob of newly disempowered people can do. What's more is the psyops that motivated actors are conceivably going to be able to run at scale to manipulate and radicalize people to join their cause and get them to do who knows what.