There is a demand for funny results. Someone asking “how many rocks should I eat” is looking for entertainment, so you might as well give it to them.
It was a bit premature to label LLMs as "Intelligence", it's a cool parlor trick based on a shitload of power consumption and 3D graphics cards, but it's not intelligent and it probably shouldn't be telling real (stupid) humans answers that it can't verify are correct.
It's nice that you have a parent that cares about how you are raised, or "a mentor". Do you realize that not everyone has that?
>We as a society should not have to break our backs bending over backwards to stop people from doing stupid things.
Sure, let's take down all the speed limits and see what happens. Let's tell people it's an option to wear seatbelts and see what happens. Let's deregulate everything and hope for the best. Sounds reasonable?
>People can make their own descisions and be responsible for them. If they lack proper guidance, well that just sucks.
There's this thing called "human nature". I think you should do some reading about it.
They pass the threshold of criminal negligence when they keep up a system that they know will actively mislead countless people in subtly dangerous ways. The problem being hard or the tech being fundamentally unsound doesn't wave away their culpability - if anything it destroys any reason why they should be given some leeway.
> The right answer is no rocks.
It's impossible to completely eliminate such trace contaminants from produce.
Pedantic? Yes, but you also can't expect a machine to confidently give you absolutes is response to questions that don't even warrant them, or to distinguish them from questions like "do mammals lay eggs?".
The LLM is clearly being dumb, but the underlying science of the question is actually interesting. Iron is another interesting one. Run a magnet though iron-fortified cereal.
The reality is though, that there is no known path currently to true AGI system and the research needs to be done. No one knows how to build this kind of system yet. LLMs are nice for things like roleplaying, helping with code stuff etc. but they are far from all that the marketers hype them to be.
These specific results aren't the problem, though. They're illustrations of a larger problem -- if a single satirical article or Reddit comment can fool the model into saying "eating rocks is good for you" or "put glue in your pizza sauce", there are certain to be many more subtle inaccuracies (or deliberate untruths) which their model has picked up from user-generated content which it'll regurgitate given the right prompt.
They have disclaimers, but a funny message is more likely to be read.