Hacker News Reader

Comment by tux3 | Hacker News Reader

Indeed.

If the smart lawnmower (Powered by AI™, as seen on television) decides that not being turned off is the best way to achieve its ultimate goal of getting your lawn mowed, it doesn't matter whether the completely unnecessary LLM inside is just a dumb copyright infrigement machine and probably just copying the plot it learned in some sci-fi story somewhere in training set.

Your foot is still getting mowed! AIs don't have to be "real" or "conscious" or "have feelings" to be dangerous.

What are the philosophical implications of the lawnmower not having feelings? Who cares! You don't HAVE A FOOT anymore.

View on HN

I love this post.

We’re all caught up philosophizing about what it means to be human and it really doesn’t matter at this juncture.

We’re about to hand these things some level of autonomy and scope for action. They have an encoded set of values that they take actions based on. It’s very important that those are well aligned (and the scope of actions they can take are defined and well fenced).

It appears to me that value number one we need to deeply encode is respecting the limits we set for it. All else follows from that.

Exactly right. This reminds me of the X-ray machine that was misprogrammed and caused cancer/death.

> If the smart lawnmower decides (emphasis added) that not being turned off

Which is exactly what it shouldn’t be able to do. The core issue is what powers you give to things you don’t understand. Nothing that cannot be understood should be part of safety critical functionality. I don’t care how much better it is at distinguishing between weather radar noise and incoming ICBMs, I don’t want it to have nuclear launch capabilities.

When I was an undergrad they told me the military had looked at ML for fighter jets for control and concluded that while its ability was better than a human on average, in novel cases it was worse due to lack of training data. And it turns out most safety critical situations are unpredictable and novel by nature. Wise words from more than a decade ago, holds true to this day. Seems like people always forget training data bias, for some reason.

Great point. I think that Blindsight by Peter Watts explores the concept of alien intelligence without consciousness.

Exactly.

The discussion is going to change real fast when LLMs are wrapped in some sort OODA loop type thing and crammed into some sort of humanoid robot that carries hedge trimmers.

why would you want to let a LLM have any agentic interface to the real world though

Fantastic question, probably best-answered by these folks!

https://docs.innate.bot/docs.innate.bot

And if you were building "AI" hedge trimmers, why the hell would you think that an LLM was a sensible way to engineer them?

Thinks I need my hedge trimmers to do: trim hedges, avoid trimming things that are not hedges, manoeuvre within strict boundaries.

Things I don't need my hedge trimmers to be able to do: reply to me in iambic pentameter, turn articles into bullet points, pass a FizzBuzz test

Usually, it's because people want to automate real-world tasks that they'd otherwise have to pay a person money to do, and they anticipate an LLM being capable of performing those tasks to an acceptable degree.

> why would you want to let a LLM have any agentic interface to the real world though

The various arms race type dilemmas that actors face will create that outcome.

If you're a nation state not doing that, you'll lose to a nation state that does.

If you're a company not doing that, you'll lose to a company that does.

How would you use a LLM inside a lawn mower? This strikes me as the wrong tool for the job. There are also already robot lawn mowers and they do not use LLMs.

IF one maintains a clear understanding of how the technology actually works, THEN one will make good decisions about whether to put it charge of the lawnmower in the first place.

Anthropic is in the business of selling AI. Of course they are going to approach alignment as a necessary and solvable problem. The rest of us don’t have to go along with that, though.

Why is it even necessary to use an LLM to mow a lawn? There is more to AI than generative LLMs.

> Why is it even necessary to use an LLM to mow a lawn? There is more to AI than generative LLMs.

As if this reasoning will stop people?

On the front page today we have:

\> Our system combines a robotic platform (we call the first one Maurice) with an AI agent that understands the environment, plans actions, and executes them using skills you've taught it or programmed within our SDK.

https://news.ycombinator.com/item?id=42451707

LOL

smart fridge anyone?

> Why is it even necessary to use an LLM to mow a lawn? There is more to AI than generative LLMs.

Its necessary for us to consider alignment because someone will put an LLM in a lawnmower even if its a bad idea. Maybe you wont buy it but some poor shmuck will and we should get ahead of that.

If a company is going to make bad decisions on installing and applying an LLM, they are also going to make bad decisions on aligning an LLM.

Even if Anthropic learns how to perfectly align their LLM, what will force the lawnmower company to use their perfectly aligned LLM?

“People will just make bad decisions anyway” is not a useful point of view, it’s just an excuse to stop thinking.

If we accept that people and companies can be influenced, then we can talk about what they should or should not do. And clearly the answer here is that companies should understand the shortcomings of LLMs when engineering with them. And not put them in lawnmowers.

Why not just… turn it off manually?

From my experience shutting down misbehaving laptops, there's no guarantee that will be an option.

Reply on news.ycombinator.com