Using LLM's in any unfamiliar context is dangerous. When asking them anything, always follow up with: "Are you sure?".
Far too often it'll cheerily apologise and correct their own answer.
Far too often it'll cheerily apologise and correct their own answer.
That reminded me how important it is to give it the full parameters and context of my question, including things you could assume another human being would just get. It also has a sort of puppy-dog's eagerness to please that I've had to tell it not to let get in the way of objective analysis. Sometimes the "It's awesome that you asked about that" stuff verges on a Hitchhiker's Guide joke. Maybe that's what they were going for.
Secondly, it can be even worse. I’ve been ”gaslighted” when pressing on answers I knew were incorrect (in this case, cryptography). It comes up with extremely plausibly sounding arguments, specifically addressing my counterpoints, and even chain of reasoning, yet still isn’t correct. You’d have to be a domain expert to tell it’s wrong, at which point it makes no sense to use LLMs in the first place.
It just leaves you with two contradictory statements, much like the man with two watches who never knows the correct time.