I was the A/B test engineer for one of the most well known dot com brands about a decade ago.
A/B testing media (advertising) is extremely misleading compared to testing against something transactional. The goal of a transactional sequence, called a conversion (payment in exchange for a product or defined service), is a supremely simple metric. Although the goal is simple to define and recognize the costs associated are wildly complex. Media is the inverse of this where the goals and consequences are not immediately clear or known but both the costs and revenue are immediately straight forward.
The reason the financials of conversion are complex is because you need to account for the expenses to third parties to pull a user in, the purchasing horizon in time, potential for cannibalism against other purchasing decisions, and various other factors that cost the business money.
The reason metrics associated with media are complex is because users HATE it. Increased advertising presence drives traffic away. Shifts in traffic occur over a wide duration due to a variety of factors so it is almost impossible to measure just how toxic advertising is to a given product oriented business in isolation in accordance with a variable time horizon. Its actually much worse than that for a variety of technical factors. In the past legitimate ad placements have been delivery vectors for content and logic of malicious criminal activity. Advertising logic is frequently poorly written. Many websites will isolate their advertisements into iframes in order to protect their website from accidental defacement of presentation and/or broken logic. Advertisements come into the page super slowly which completely pushes out the waterfall for what might have been a quick loading page.
When I did A/B testing I would describe the media side of the business in terms of illegal job deals. I did this because of so many parallels in the transactions and risks in those markets, not because they are both filthy evil things.