True, a company has to enable it, though I wouldn't bet on employees necessarily being notified. Slack's help site page about message export doesn't mention anything about notifying employees on higher tier paid plans (though that doesn't necessarily mean it doesn't happen) [1]. Either way, I'd be willing to bet a lot of money that Slack would happily disable that feature if a seven figure deal were on the line.
All that said, it's also not really true that "your boss" can read your private messages, in practice most of the time. Most companies that do this kind of data export just dump it to a warehouse and hold it for safe keeping just in case. Some random director of engineering probably couldn't get access to it unless they were willing to jump through a lot of hoops. Though, Slack's Discovery API does seem to make it easy to create user-friendly interfaces for searching through private messages. But I doubt a lot of companies are doing that.
[1] https://get.slack.help/hc/en-us/articles/204897248-Guide-to-...