This is a good reminder that the output we get from these big transformer language model is a side effect. It's not what they are trained to do per se, it's an empirical observation that "large language models are few shot learners". There's nothing automatically wrong with that, just that's it's dangerous and irresponsible to put any real trust in this emergent behavior and treat it like you've intentionally build and verified what the model does.
And soon true expertise will have been reached. Why not quit your job? It’s not like you can go back to grad school faster than Deepmind can train a new model.