They should be encouraged to read and review the LLM output so they can critically understand it and take ownership of it.
Well, at least it used to be the bottleneck. Nowadays you can just ask an LLM. For all their faults, they are really good at letting you know about what tools exist in out there in the world, surfacing more than you could ever come to know about even if all you did was read about what exists all day, every day.
You are certainly headed in the right direction, but not quite. To be seen as an expert in the eyes of others you need to have had a vision for something and to have successfully executed on it. If the vision was dependent on calculus, then you will have reached a point where you had to learn something about calculus, of course...
But that's different to having a taskmaster tell you to learn calculus for no apparent reason. Even if you follow through and built up a huge wealth of knowledge from it, you would still not be deemed an expert by others. You're no different than an encyclopedia, which isn't an expert either. It is being able to see things others can't and the ability to act upon it that makes an expert.
Learning taking place when you need it isn't the same as never.
> Maybe when we have supremely reliable LLMs that can replace humans we might not but we’re not there yet.
Frankly, even Page Rank already replaced humans for this. But LLMs are even better at it. Humans are just that poorly performing. Like I said before, even someone doing nothing in life but looking for what exists in the world could not take in as much as databases that have indexed every written thing.
I think we're on the same page here. Experts have both vision and execution. Someone who has simply learned a bunch of things, or a lot about one thing, is not what we consider an expert.
> You can ask an LLM for a particular formulation but you cannot make wide spanning connections to come up with something novel until you have a good understanding of a given space.
I don't get where you are trying to go with this. Using an LLM (or Page Rank for that matter) to search for tools that have been created/discovered necessary to fulfill execution of your vision seems to have nothing to do with what you are trying to say. Nobody would ask an LLM to do what you are suggesting, if I am understanding you correctly. LLMs are most definitely not good at that. That is AGI territory.
I believe there is a mechanism for this already.