If someone goes off their meds, you don't just abandon them to the streets. You send a trained mental health nurse and/or social worker to talk to them, several times a day if necessary. If they don't want to engage, you work with them to build a relationship of trust. You work with that person to get them back on a suitable course of medication, or to help them manage their symptoms and manage their life while unmedicated. You provide supported housing and supported employment or work-related activity wherever it would prove beneficial. You keep working with them again and again and again unless you're damned certain that they're truly beyond help.
Of course that's expensive, but it's also remarkably good value for money compared to the alternative. The net difference between a borderline functional person and a completely dysfunctional person can exceed a million dollars per year in policing, incarceration, ER visits and other costs.
This model of community care isn't new and it's proven to work and to be cost effective. There are plenty of places in the world that don't rely on mass institutionalisation and have remarkably low rates of homelessness.