Haha.
Maybe someone with GPT-4 access can check if it has improved. But I find it essentially regurgitating its sources.
I am by no way an expert in questions I ask it. But I find the answers generally speculative and not helpful.
I have been asking bard and chat gpt once in a while "What's the internal representation of strings in Java"?
This is a good question to test its information retrieval capabilities. because your average blogspam site or tutorial doesn't cover the this.
They both somehow fail to convey me that it's a combination of char array and a cached hashcode.
At best, chat GPT told me a 3 paragraph sentence that java String was an array of UTF-16 characters, which is not quite the case since JDK 9 as I know. There's no mention of compact strings.
Other day I asked what's the GCC pass which deduplicates similar function definitions. Both chat gpt and bard made up their own pass name.
I am no expert in these topics, nor in ML or IR. But I don't believe LLM is the way towards information retrieval.
If it's ingesting everything it reads, it would be worse than pagerank, right?
Granted, it does normie stuff pretty well, like writing a data class or making HTTP request. But as soon as you need something deep, it is worse than useless, because it confidently claims incorrect stuff.