I suspect by garbage you mean not perfect.
To be more precise can you please give a topic you know well and your % guess how often the answers are wrong on the topic?
To be more precise can you please give a topic you know well and your % guess how often the answers are wrong on the topic?
Automatic proof generation is a massive open problem in all of computer science and not close to be solved. It’s true LLMs aren’t great at it and more is required for example as with the geometry system Deepmind progresses on.
On the other hand they can be very useful to explain concepts and allow interactive questioning to drill down and help build understanding of complex mathematical concepts, all during a morning commute via the voice interface.
Yesterday I was looking for some help on an issue with the unshare command; it repeatedly made bad assumptions about the nature of the error even I provided it with the full error message and one could already guess the initial cause by looking at that.
I guess such errors can be frighteningly common once you get outside of typical web development.