Exactly. GPT happens to hallucinate a lot of "facts" making it unreliable for a large breadth of tasks. However it's quite adept at many NLP tasks and can be fine-tuned to further improve its domain expertise.

In any sensitive application like clinical charting, one would also want to include a workflow for reviewing GPT's output for erroneous data before welcoming it in.

I have the feeling we really don't know what it's good at for certain. Due to its highly statistical nature, and completely opaque data structure, we can only observe some behaviors, empirically. But it's a total black box. Nothing can be proved nor introspected about it.

Building anything related to people's health on top of that is just wrong imho.