Hacker News Reader

Comment by rm999 | Hacker News Reader

Back in the early 2010s I worked in ad tech on a data science team, and one of the things we were pushing for was causal A/B testing; basically turn off a campaign's advertising to a % of people and correlate it with sales to measure ROI.

As we were kicking this off I was at a conference chatting with an executive at another ad tech company. His response: "oh yeah I know a guy who tried that, he's not in the industry anymore."

We almost immediately came to realize our launch clients were getting negative ROI, sometimes severely so. AFAIK our efforts fizzled out, and I believe none of the people on my team are in the industry anymore.

View on HN

Similar time frame for me, worked at an ad-tech startup, when retargeting was first becoming a big thing and we were pivoting the startup from the bad business idea it started to to doing retargeting as a demand"-side-platform" (DSP) on ad various exchanges. Tried myself to confirm whether any of it had any positive effect, I couldn't really discern any increase in the click-through-rate (CTR) for various approaches, but I'm not a stats expert, etc. so talked to the founder about how we should employ someone with a stats background. That convo went nowhere, and for that and other reasons I was out the door within a couple months.

That was the era of the ad-exchange DSP bubble. After that I went to work at another company that was on the other side of the exchange pipeline and I could see all these DSPs just plugging away doing their thing and none of it looked (to me) like it was accomplishing much. It was all bottomfeeding off of lower quality inventory but I suspect making big promises to investors.

That startup eventually pivoted a couple more times and sold to a bigger player a few years later, making some money for the founder but I suspect no value to the buyer.

Reading about startups that pivot dramatically multiple times make me cringe so hard.

The goal of pivoting isn’t throwing spaghetti at the wall until you hit something before running out of VC or angel money. That just shows terrible product/marketing leadership.

Your prior story is unsurprising combined with that.

What do you mean by the two references to not being in the ad industry anymore? Quit? Fired? Why?

I might be misunderstanding... but, negarive ROI? I.e, the more they spent on ads, the less sales they got?

Feels like a missed something, as that sounds... counterintuivie.

(Not in any way related to the ad industry so pardon my ignorance)

No. A negative ROI just implies that the ratio of benefit/cost is less than one. If I incur a cost/investment of $100 but it creates value of $200 then I have an ROI of 100%. If this same expenditure instead only produced $80 value then I would have -20% ROI as in the value I’m realizing from my investment is 20% less than the cost of the investment.

I would expect:

  ROI > 1 = profitable ad spend
  0 < ROI < 1 = ad spend does not cover its cost
  ROI < 0 = losing money because of the ad spend (the ad is damaging to your brand)

Ratio can't be negative, they can either be above of below 1. What you are talking about is "$benefit - $cost", not "$benefit / cost".

ROI is (net benefit)/(cost), where net benefit is (gross benefit - cost). Net benefit can be negative, making ROI negative if cost exceeds benefit. GP was using benefit as a shorthand for net benefit, which can certainly be negative.

Even gross benefit could conceivably be negative, if the ads are bad enough! (I.e., a terrible ad might prevent a sale.)

We tend to call most rations below 1 'negative'. Its technically wrong but most people get what you mean when you say it.

Positive and negative are the right words on a log scale :D

Ratio can't be negative, but ROI can. Positive ROI is the percentage of the ratio being 1 or higher. Negative ROI is the percentage of the ratio being less than 1.

Or, from a "common sense" perspective, if I spent $1000 on advertising, and earned $100 in revenue from that investment, my ROI is definitely negative.

As I read it, the amount they were spending on the ads themselves was more than the converted revenue from ad clicks. While the decline was probably (guessing) not linear, spending more on ads led to less than proportionally more revenue. If that had happened I can imagine calling that negative ROI.

That makes more sense!

Thanks

Did they leave the industry because they realized their product wasn't providing a positive ROI, or were they fired for pointing that out to other people? I think the former is probably what you mean, I just want to be sure.

Yeah, it could be that many startups try to do advertising ethically, realize that it doesn't work, and drop out. That's necessarily going to leave all the firms willing to sell snake oil behind.

You can get firms selling stuff that doesn't work that are "trusted" simply because they've been around for years; there are plenty of distinguished brands selling homeopathic remedies, audiophile speaker cables, timeshares and MLM schemes.

Good question. In the first case it was left vague but I understood it as being fired and then not trying to find a new ad tech job. In the case of my team it was finding better industries to work in.

As a person with just shy of a stats graduate degree, the early 2010's were a nightmare of trying to get ad tech companies to grasp their incompetency. I met a dizzying number of slick talking frauds and became quite jaded.

Ok good, it wasn't just me being overly negative (because I usually am). I am not a stats grad, my math is weak, but it didn't look good to me, and I tried to get my employer to bring a stats specialist in to analyze. That got a cold response.

Reply on news.ycombinator.com