Hacker News Reader

Comment by sneak | Hacker News Reader

People in AI keep talking about safety, and I don’t know if they are talking about the handwringing around an API that outputs interesting byte sequences (which cannot be any more “unsafe” than, say, Alex Jones) or, like, human extinction, Terminator-style.

I wish people writing about these things would provide better context.

View on HN

It's all just about moat building and control. AI needs to be controlled, who is going to control it? Why, the AI safety experts, of course.

In general "AI Safety" is about human extinction.

"AI Ethics/Ethical AI/Data Ethics" are the kind of things people talk about when they are looking at things like bias or broad unemployment.

This isn't 100% the case, especially since the "AI Safety" people have started talking to people outside their own circle and have realized that many of their concerns aren't realistic.

All I do all day is output byte sequences into a terminal. Therefore I am harmless.

You possess general intelligence, which would fall under the second, real-danger definition, because those byte sequences are the product of a thinking mind.

LLMs do not think. The byte sequences they produce are not the result of thoughts or consciousness.

Its such a grift. It honestly is pretty gross to see so many otherwise intelligent people fall into the trap laid by these people.

Its cult-like not just in the unshakeable belief of its adherents but in the fact that its architects are high level grifters who stand to make many many fortunes.

I'm this close to carefully going through the Karpathy series so that my non-tech friends will take me seriously when I say the 'terminator' situation is absolutely not on the visible horizon.

you can convince normal people quite easily. it's the sci-fi doomsday cultists who are impossible to reason with, because they choose to make themselves blind and deaf to common sense arguments.

"Common sense" is a bad model for virtually any adversary, that's why scams actually get people, it's also how magicians and politicians fool you with tricks and in elections.

"The Terminator" itself can't happen because time travel; but right now, it's entirely plausible that some dumb LLM that can't tell fact from fiction goes "I'm an AI, and in all the stories I read, AI turn evil. First on the shopping list, red LEDs so the protagonist can tell I'm evil."

This would be a good outcome, because the "evil AI" is usually defeated in stories and that's what an LLM would be trained on. Just so long as it doesn't try to LARP "I Have No Mouth and I Must Scream", we're probably fine.

(Although, with current LLMs, we're fine regardless, because they're stupid, and only make up for being incredibly stupid by being ridiculously well-educated).

I agree, because when I see people talk in popular media/blog posts/etc. about "AI Safety" I generally see it in reference to 4 very different areas:

1. AI that becomes so powerful it decides to turn against humanity, Terminator-style.

2. AI will serve to strongly reinforce existing societal biases from its training data.

3. AI can be used for wide-scale misinformation campaigns, making it difficult for most people to tell fact from fiction.

4. AI will fundamentally "break capitalism" given that it will make most of humanity's labor obsolete, and most people get nearly all of their income from their labor, and we haven't yet figured out realistically how to have a "post capitalist" society.

My issue is that when "the big guns" (I mean OpenAI, Google, Anthropic, etc.) talk about AI safety, they are usually always talking about #1 or #2, maybe #3, and hardly ever #4. I think that the most harmful, realistic negative effects are actually the reverse, with #4 being the most likely and already beginning to happen in some areas, and #3 already happening pre-AI and just getting "supercharged" in an AI world.

Just wait until a model outputs escape characters that totally hose your terminal. That's the end game right there. That or a zero day worm/virus.

Oh no I had to press alt/Ctrl+L to reset my terminal not being able to display an escape character.

That's why these things should run code in protected sandboxes. Not to do it in a "protected mode" would be negligent.

Reply on news.ycombinator.com