"..it may not be safe for you or your other employees to work out of the same location.."
It saddens me this individual's mental health issues seem suggestive of potential violence to _others_, when I don't read anything which portents of violence to _others_ in the OP. That's not to say the situation is not concerning. The OP says there is property damage. The OP says the person is disruptive to other employee's productivity. I don't imagine it's ever appropriate for an employee to be housed in the office.
I'm not a mental health professional, but I do have some exposure in this area as a volunteer for Community Access in New York City [1]. CA is an organization helping individuals exiting the health system after a mental health crisis who are also homeless. (I found the opportunity through Idealist.org. I was out of work and volunteering once a week was a great way to spend some of my extra time).
CA has several programs, and individuals had to apply to be accepted into the program where I volunteered; It was a big step forward to be accepted. Services included job and housing search assistance. As you can imagine this is very stressful situation under the best of conditions.
At the start of my volunteering there was training. One of the terms we learned to describe disruptive behaviors was 'decompensation'[2].
The lesson I take, and what I'm working to explain here, is unless you have some training you are probably not equipped to understand and deal with such an employee. Without experience or knowledge you will jump to extreme conclusions ('not safe' to work with). It's an outlier employment situation.
People in this situation need help and understanding. I applaud the OPs efforts to hire outside of the "normal", compliant, pool of candidates. If the OP is serious about continuing this effort, they should seek training. I can't speak for services CA offers to employers, but it might be a place to start an inquiry about the state of services supporting employers in general.
[1]: https://www.communityaccess.org
[2]: https://en.wikipedia.org/wiki/Decompensation