If you want something that "knows how to think" then I don't think you'll find it in an LLM. You can't divorce knowledge from this kind of model, it guesses what word comes next based on its training on a corpus of knowledge.
E.g. the training data might look like "a fooble is to a tooble as a fabble is to a" with the answer "tabble".
So you feed it tons of these kind of nonsense training data that forces it to only learn the in-context reasoning part of language, and none of the world knowledge.
That said, it is unclear to me how much value such a model would be.
You could imagine a middle ground where it does have basic knowledge, such as dictionary definitions and things derivable from those.
E.g. "A tree has leaves. Leaves are green. Therefore a tree has a part that is green." type stuff.
So you could give it some amount of world-grounding and common sense knowledge, but nothing involving history, proper nouns, etc.
I imagine you could make such a model much smaller than these giant LLMs.
The evidence is that there are these "glitched" tokens : https://www.youtube.com/watch?v=WO2X3oZEJOA
if you watched to the end of that video, it explores a very plausible reason why such glitched tokens exist - it is because the training dataset has a bunch of "nonsense" (a large dump of the subreddit that just counts a number, as well as various other "data").
So my hypothesis is that if you attempted the in-context logic training dataset, it would produce large number of glitched tokens (different ones, but equally as "illogical" to the user).
GPT4 does not suffer from the same glitched tokens as GPT3, presumably because it uses a different tokenizer.
Furthermore, there are LLMs that operate on single bytes instead of multi-character tokens, totally obviating that problem.