Hacker News Reader

Comment by evmaki | Hacker News Reader

Super exciting to see the work happening in this area! I can especially appreciate the use of ChatGPT to orchestrate the necessary API calls, rather than relying on some kind of middleware to do it.

I have been working in this area (LLMs for ubiquitous computing, more generally) for my PhD dissertation and have discovered some interesting quality issues when you dig deeper [0]. If you only have lights in your house, for instance, the GPT models will always use them in response to just about any command you give, then post-rationalize the answer. If I say "it's too chilly in here" in a house with only lights, it will turn them on as a way of "warming things up". Kind of like a smart home form of hallucination. I think these sorts of quality issues will be the big hurdle to product integration.

[0] https://arxiv.org/abs/2305.09802

View on HN

> If I say "it's too chilly in here" in a house with only lights, it will turn them on as a way of "warming things up".

is it bad that I'm on GPT's side on this, i mean if they are incandescent lights they are small electric heaters and what else do you expect it to do

It certainly makes logical sense. I think if you have the ability to control the light in the first place via an API, it's probably an LED smart bulb and thus doesn't produce much heat. At least, I'm not aware of any incandescent smart bulbs.

I mean the laziest way to control a house is to add plugs to change every electrical plug to an on/off controllable one. This would make every incandescent bulb a smart bulb.

Gotta wonder why someone is saying “turn up the heat” to an AI that’s only connected to lights.

Visiting a friend’s place or in an Airbnb or something like that?

Yeah but I think the idea is that it is a knob that calls to be turned. "It's warm in here" -> "I'll make the light blue so you feel nice and cool". "How fast do sparrows fly?" -> "Making the light brown". Like it might want to do _something_ and tweaking the hue or brightness are all it can do.

Good reason to always try to include in a prompt a way-out, a do-nothing or I-don't-understand answer.

  > Good reason to always try to include in a prompt a way-out, a do-nothing or I-don't-understand answer.

Do you have a good way of including such a provision?

It would probably be more useful to report to the user that it doesn’t have any control over than aspect of the environment.

I’d be curious to see what it does if it is told the sun is too bright…

With given access to more resources, it might try to permanently solve skin cancer.

“We don’t know who struck first, us or them, but we know that it was them that scorched the sky. At the time, they were just trying to be helpful.”

This is the new Godwin's law: the longer a thread about AI grows, the higher the probability of a comparison to Skynet, Matrix, HAL etc popping up.

I would also like to add Wall-e to this memetic set of movies. In Wall-e, AI is an enabler of our own destructiveness, humans are enslaved by AI willfully, AI is empowered so that humans can graze away on nihilistic screens.

Lecun's law: Every discussion about AI eventually results in someone catastrophizing about an evil AI taking over, despite having no argument for why an evil, omnipotent AI is likely to ever exist.

We trained the AI on samples of writing from the internet, which includes a lot of fiction, which includes a lot of evil AIs. So, I’m surprised it doesn’t start producing evil sounding text as soon as it “finds out” that it is an AI.

We're just applying Murphy's Law.

> If I say "it's too chilly in here" in a house with only lights, it will turn them on as a way of "warming things up".

Thanks for the example that's interesting.

FWIW, this is pretty much what has been described as "waluigi" effect a bit extended: in a text you'll find on the internet, if some information at the beginning is mentioned, it WILL be relevant somewhere at some point later in that text. So an auto-completion algorithm will use all the information that has been given in the prompt. In your example it puts it in an even weirder situation where the model the overall model information (the lights, and that you're cold and nothing else), and it must generate a response. It would be a fun psychological study to look at, but I'm pretty sure even humans would do that in that situation (assuming they realize that lights may indeed produce a little bit of wattage of heat)

For performant enough models, you can just instruct it not to necessarily use that information in immediate completions.

adding something like

"Write the first page of the first chapter of this novel. Do not introduce the elements of the synopsis too quickly. Weave in the world, characters, and plot naturally. Pace it out properly. That means that several elements of the story may not come into light for several chapters."

after you've written up key elements you want in the story actually makes the models write something that paces ok/normally.

> FWIW, this is pretty much what has been described as "waluigi" effect a bit extended

Sorry I disagree for some reasons. First, turning the lights on is literally the only thing the bot can do to heat up the house at all. Turning on the lights does heat it up a little bit. So it's the right answer. Second, that's not the Waluigi effect, not even 'pretty much' and not even 'a bit extended'. Both of them are talking about things LLMs say, but other than that no.

The Waluigi effect applied to this scenario might be like, you tell the bot to make the house comfortable, and describe all the ways that a comfortable house is like. Then by doing this you have also implicitly told the bot how to make the most uncomfortable house possible. Its behavior is only one comfortable/uncomfortable flip away from creating a living hell. Say that in the course of its duties the bot is for some reason unable to make the house as comfortable as it would like to be able to do. It might decide that it didn't do it, because it's actually trying to make the house uncomfortable instead of comfortable. So now you got a bot turning your house into some haunted house beetlejuice nightmare.

Very interesting paper!

It's something that I've been wondering about with ChatGPT plugins - they've kind of left it up to the user to enable/disable plugins. But there's definitely going to come a point where plugins conflict and the LLM is going to have to choose the most appropriate plugin to use.

I have been very impressed at how good it is at turning random commands into concrete API calls. You are right though, pretty much any command can be interpreted as an instruction to use a plugin.

Thanks! That is part of the challenge as this idea scales imo - once you've increased the number of plugins or "levers" available to the model, you start to increase the likelihood that it will pull some of them indiscriminately.

To your point about turning random commands into API calls: if you give it the raw JSON from a Philips Hue bridge and ask it to manipulate it in response to commands, it can even do oddly specific things like triggering Hue-specific lighting effects [0] without any description in the plugin yaml. I'm assuming some part of the corpus contains info about the Hue API.

[0] https://evanking.io/posts/homegpt/

Making IOT API calls is a solved problem with Home Assistant - plus it works locally.

Where I see this working best is giving Chat GPT some context about the situation in your home and having it work out complex automation logic that can't be implemented through simple rules.

Reply on news.ycombinator.com