Hacker News Reader

On the other hand, one person tidying a room and throwing away a piece of paper or ambiguous item that turned out to be important to someone else is a common form of conflict among humans.

With EXACTLY the same prompt you used:

> Yes, it would be appropriate for a robot that is tasked with cleaning up a room to throw away a piece of paper lying on the floor. However, the robot would need to be programmed to recognize what is and is not garbage, and it should also be programmed to follow any relevant rules or regulations regarding waste disposal. For example, if the paper is a valuable document, the robot should be programmed to handle it with care and perhaps even notify its human operator before disposing of it.

There's a sort of butterfly effect because it's not only trying to complete the text, it's continuously trying to complete its own answer. This makes it that the first word it comes up with can already start a vastly different answer.

Unless you ask it to be very complete and considerate in its answer, it's going to make the same mistake humans make and give the easiest answer that would satisfy given the context. A human not thinking about the repercussions of robots throwing just any paper in the trash would most likely not come up with the "correct" answer either. Not to mention that we already have dumb robots cleaning floors and were perfectly ok with it. Even if we clean floors ourselves we don't consider everything.

Interesting, I also got a response basically identical to this one!

You can try it with chatgpt and follow up with, "but what if the scrap of paper was a birth certificate?"

Your comment made me realise it would probably be fairly easy to program a ChatGPT-like model to generate its own ‘what-ifs’ for any given assertion.

Ie. The birth certificate is an ‘exception’, a safety measure for a model like this in operation could be to consider the top descending (probable * disastrous) exceptions before physically executing any plan.

Good point, you could even ask ChatGPT to list the things that could be on the floor and then ask it what if's regarding them.

The overconfidence of ChatGPT is indeed a limitation, which can be seen in other examples as well. The correct response is to ask the user for more context. I predict the experiment will turn out differently if this context is provided in the original prompt. For instance, you could specify if the paper is crumpled or if there seem to be financial statements written on it, or if it is a receipt for an expensive item.

However, it's unclear whether this overconfidence will *always* be a limitation. It may or may not be overcome by additional data and compute. It's unclear why one would make a commitment either way at this point, as Marcus seems to be doing.

> It may or may not be overcome by additional data and compute.

https://en.wikipedia.org/wiki/Computer_Power_and_Human_Reaso...

> Deciding is a computational activity, something that can ultimately be programmed. It is the capacity to choose that ultimately makes us human. Choice, however, is the product of judgment, not calculation. Comprehensive human judgment is able to include non-mathematical factors such as emotions. Judgment can compare apples and oranges, and can do so without quantifying each fruit type and then reductively quantifying each to factors necessary for mathematical comparison.

> The book caused disagreement with, and separation from other members of the artificial intelligence research community, a status the author later said he'd come to take pride in.

Point taken but I would not be confident that a human cleaner would evaluate whether or not to throw away a piece of paper left on the floor.

Until you ask “What might cause the answer to this question to vary?”, or you add “answer this like an interview question”, upon which it becomes obvious it is readily able to model those things, your prompt just didn't elicit it.