If you want to increase the likelihood that answers will read like Wikipedia entries, sure.
With Stable Diffusion, you're able to use LoRAs to introduce specific characters, objects, concepts, etc. while maintaining the same visual qualities of the base model.
Why can't something similar be done with an LLM?