Turn off all Slack notifications for mobile and make it a "Pull" interaction with the service vs. a "Push" interaction.

It's what I do with a lot of services these days. I'm selective about what services can actually push to me.

Examples of services that are important enough to push to me include iMessage (large group chats muted), Uber/UberEats, Lyft, Instacart, etc. Basically notifications that are immediately actionable or important in the moment. Lots of delivery services fall into this category.

You could just not install it on your mobile device.