Many classic statistical modelling techniques have ways to produce some measure of confidence for their predictions; perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate.

> LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capability and have the former just look things up in a database?

I think the beauty of LLMs is exactly that all we need to do is to feed them raw text --- the hope, I guess, had been that the models will be able to develop human-like insights by learning to understand and "speak" languages on its own. Introducing "feature engineering" (e.g. the distinction as you suggested) would defeat that goal.