This is the real problem, IMO.
Slack shouldn't be replacing e-mail or meetings or phone calls. It's great for impromptu discussions with a lot of back-and-forth, but once it becomes a serious conversation with multiple parties you need to escalate to a call or meeting or e-mail. Casual, asynchronous chat is great for low-importance conversations that aren't time sensitive. It's terrible for coordinating and making important decisions in a timely manner, though. Don't be afraid to schedule a synchronous communication session in whatever flavor you prefer.
Once you start making Slack the center of communication, you stop making deliberate decisions about who is included in e-mails/meetings. Now everyone in the channel has to skim everything to make sure they're not missing out on anything important. Busybodies love it because they can be a fly on the wall for everything. Heads-down workers hate it because they have to choose between focusing on their work or checking into Slack all of the time. It optimizes for the wrong kind of engagement.
Trying to enforce a lot of rules around threading is a band-aid, IMO. The real solution should be to create a culture where people aren't trying to force every interaction into Slack or avoid meetings at all costs. Excessive meetings are a problem, but going out of your way to avoid all meetings will waste more time than it frees up.