MentalHealthBench

I can’t argue this, but I can build an LLM based care provider today with likely decent results in patient outcomes for those adapted to it and good cost dynamics, and across a variety of critical use cases in the care vertical from providers to end delivery. I can not however change American society, our economic structure, or political dysfunction single handedly, can you? Should I not do the former and let people suffer in protest because the latter is not possible, and hold my breath hoping it’ll change? How many lives must be destroyed in protest hoping everything changes first?

Fine, let’s have socialized delivery of mental health care. Let’s go back in time 20 years and establish a pipeline of capable providers that doesn’t exist. Let’s do those things if we can. But I don’t see it happening in my life time.

I do however see an opportunity to take the technologies presented to use in the immediate present and providing a mechanism to bring help to people who need it today with good efficacy where it’s appropriate. Is it OpenAI that brings that? I doubt it - they’re selling shovels, not digging for gold. Is it a general model that provides direct care? Doubtful IMO. Is it a chatbot interface ? I doubt that too. But I think if we are creative and thoughtful we can use these technologies to address huge swaths of the care providing problem, which is more than talk therapy. And we can do it today, and don’t have to wait for MAGA/MAHA to accept reality on realities terms.

I think the expectation that ChatGPT will be your therapist is wrong, however the expectation that you can build a therapeutic system using LLMs is not necessarily wrong. You are right that there are regulatory aspects, and liability, as well as guard rails built into our system (but less for mental health than medical as it’s largely a neglected aspect of our care delivery).

Also, it’s a different mode of delivery. Some people work well remotely, others better in person. Likewise some people open up more easily to a machine than a person. One doesn’t replace the other.

Further, LLMs can be very useful for diagnostic work in addition to a psychologist to help accurately classify mental and personality disorders. This is my fathers speciality and he finds modern LLMs to be fairly adept at classifying.

A lot of psychologist work in a clinical setting is case notes and an LLM can be quite skilled at producing standard expected case note structures from clinical notes, improving the time spent by the professional in care vs paperwork.

All of these are opportunities for use of LLMs as specialist models, agentic frameworks around general models, guard rails, delivery mechanisms, etc. I wouldn’t look at the world being OpenAI and Anthropic, but as model providers as providing screws and bolts and it’s up to us to build the products. LLMS are trained with an enormous corpus of witting on mental illness, but also writings by the mentally ill, care professionals, and all sorts of other materials. By eliciting all these dimensions the applications to mental health are enormous - it is a giant learned model of human thought and experience, not just of factual information, but the entire complex.

Here’s an idea I’ve talked to my dad about - how about using a model set trained on the writing of many people of specific mental illnesses. Then you can interact with them and they will produce responses likely of someone with that illness. This can be used to train clinicians, but also to build better inventories and diagnostics since you know the classification of the responses to some reasonable degree. This should accelerate research into mental illnesses.

The possibilities are vast, limited by our imagination and biases.