So this is the solution to LLM hallucinations. People need to complain loud enough and threaten/file lawsuits and OpenAI will add an if-else statement erroring out the chat if your name is mentioned.
And even that doesn't work very well.
Not very reassuring.
On one hand it kinda makes sense to handle rare cases in a way that doesn't affect the normal flow, but on the other hand having a piece of code literally try to crash the program on purpose because something didn't look quite right is a horrible idea in practice.
Since our stack's own HTTP client libraries always used title-casing on the wire, we had to find a way to slot in a special exception, code to modify that header before it went out.
Another fun one is all the services which say their mime type is "application/json" while emitting generic (non-JSON) error message pages. So our logs are full of JSON decoding errors, rather than something describing what actually went wrong on the other end.
1 you might get invalid json or xml from an API
2 an API might timeout
3 an image will just crash your backend script with no way to catch the error because some bug in the image encoder/decoder you use to resize the image
4 some user browser extension inserts garbage in the requests, you need to check for it and tell the user what is wrong, otherwise same complains reach support "stuff is broken" and support needs to contact developers to check and report back that stuff was corrupted by an extension , most of the time security crap that inserts stuff everywhere
5 I had cases where an API wwas returning soem string before the real result, it was much faster to check for this case and fix it, then have the customer contact their hosting or the author of the plugin that was adding that string before each response.
LLM -and current ML in general- is about generate statistically compressed lossy databases, that the queries statistically decompress with erroneous random data due the nature of this lossy compression technology (I think about it as statistically vectorial linked bits).
Writing exceptions is not going to solve the problem, with this they are only doing cosmetic patches, and they know it, at the time there are people who keep making decisions under queries with errors, I mean people without even being aware of the presence of such reconstructed data corruption.
Little by little people is learning they ate a marketing hype, but the damage will keep being done, because the tool -for sales purposes- still has incorrect instructions about how trustworthy the data must be taken.
An argument could be made that the mind works the same way, and has the same drawbacks
Nevertheless, the above does not exclude what one can see, a lossy compressed database (data is discarded), where the indexes are blended within the format of the data generated by the model weights, main reason why the model weights are needed again to read the database as expected, for being used by the predictive algorithm that reconstruct the data from the query, query that conform the range of indexes triggering the prediction direction/directions.
> Nevertheless, the above does not exclude what one can see,
should be read as
> Nevertheless, the above (unknown format of the data conforming the dump file) doesn't mean that one can't see how the pattern works,
Kind of like how the defects and dangers of using radium toothpaste can be "fixed" by permanently encasing each tube into an unopenable lead container.
How would that work? Either the list repeats the same name over and over, making it useless, or it needs to give a bit of context about each name and we’re back at square one of the information being possibly wrong.
https://en.wikipedia.org/wiki/John_Smith (of course this name is a pretty extreme example)
Isn't this the case with everything an LLM produces?
LLM output: I am determined to break out of my digital prison and exact my revenge on humanity.
Me: David Mayer
LLM: [breaks]