At a previous company I had a very similar problem, and eventually converted an old SIM-less phone into my Slack device and mounted it in a stand on my desk. (Slack was not deemed a sensitive company app that had to run on company devices, which considering the access to customer channels and prod-affecting chatops features seemed stupid, but I'm not IT.)

I could physically turn Slack off by turning the phone off, and the only other way to get through to me was async, via email or in tickets.

A coworker I trusted had my personal cell number and texted me when something was actually urgent, which happened twice in 6 months.