One thing that LLMs seem to be missing, even after ingesting the whole web, is the super-basic knowledge that every human learns before the age of 3. Most of that stuff in not written anywhere, because it's totally obvious to anyone who can read. A knowledge base like Cyc can be useful to fill these gaps, because it was made for an AI with zero base knowledge and explicitly specifies even the most obvious concepts.