Probably not a helpful answer to your question, but I feel your pain, at any rate.
Uninstall slack from your phone. It's a matter of life or death and I am not even kidding.
But if you, unlike me, actually have hobbies or planned activities for your free time, deleting might be better.
Notifications can't all be disabled. You're at the mercy of any message from a dude who's at the office 2 hours before or after you.
Unlike a phone call, people don't realize that a slack messages is disruptive and require you to be online 24/7.
Yeah, unfortunately that's the problem. It's probably worth doing what you suggest for multi-day vacations though (and un-doing when I'm back to my desk).