That makes sense but I'm still not so sure—we have things like software architecture patterns that can be discussed at a high level without knowing the intricacies. Like you can be aware of load balancing and even use it but be unaware of how load balancing might work algorithmically.

Let's consider even the original example. > You cannot remove facts like 'paris is capital of france' from llm and have it just retain 'high level thoughts' like 'countries have capitals that you can look up'

Wouldn't the knowledge that countries have capitals precede the knowledge that Paris is the capital of France?

This says nothing about the accuracy of our own models based on these abstractions that lack the lower-level understanding.