How to Lose Money on Paid Marketing | Hacker News Reader

How to Lose Money on Paid Marketing

jamiequint.com

53 pointsjamiequint29 comments

I'll also add that the attribution window is totally relevant as well.

In the multi-touch example given, the 5-day reporting window does not account for the fact that tomorrow's revenue will be sprinkled into today. That means that your ROI for today's spend isn't final until you're beyond the attribution window.

Isn't that an argument for assessment on a campaign basis? I'm still wrapping my head around his calculations. It looked like he included the cost of the ads prior to the reporting window in his multi-touch model. That's a fair way of measuring the true cost of user acquisition, but if the ads you are paying for today won't pay dividends until you're out of the reporting window, it would skew the ROI.

I'm not sure if this is the same thing as what you cited, as my argument is about the cost association during the reporting window as opposed to the future revenue association. I can't figure out a way to back those out in real-time (i.e. in enough time to make adjustments to the campaign) though.

In long-running campaigns, I still typically want to be able to measure changes to performance over time.

With a fluency in the ramifications of the attribution models and windows, I like to look at multiple attribution models (last-click and parabolic multi-touch) to draw conclusion about performance.

My takeaway from the post was simply, "Be aware of how much attribution models influence metrics," and I think that's a critical point.

Indeed, that is the most salient point. I like your suggestion of looking at many attribution models to assess true performance. This seems especially relevant in larger organizations where there is an incentive to game the budgeting process - presenting just the best possible case is what marketing is all about but can lead to skewed perspectives of success.

This is definitely true, you can think of that as Day 1 ROAS/ROI vs Day N ROAS/ROI. So when looking at your marketing spend you can see how it pays back over time since depending on how you attribute it could change forever (e.g. if you give some value of each conversion back to the original conversion source/sources).

Yep. And when looking at data you have to be very clear on whether the conversion data you are seeing reflects "date of click" vs. "date of conversion." "Date of Click" data will continue to change throughout the duration of your reporting window whereas "date of conversion" does not.

This can get sticky in things like AdWords where the main UI reports on "date of click" and keeps changing, whereas Search Funnel reports report by "date of conversion."

Fortunately, most bid management platforms like Marin let you make that distinction in reporting.

Date of click is fine as long as you are understanding your reporting over fixed timeframes. e.g. What is my Day 7 ROAS on spend from 2 weeks ago, that will never change once you get past 7 days from the window of spend that you are looking at.

Date-of-conversion has the different (and in my opinion worse) issue of being 100% accurate in terms of current cost-per-conversion but under-values all marketing channels. This is because it attributes the cost of visits only to conversions that have already happened, while more conversions may later be attributed to those visits. This means that a conversion based model is actually worse for channel analysis.

Agreed on both points. Ultimately, everyone should try to gain a better understanding of the duration and number of touch points in their sales cycle. This directly impacts the lookback window you set. It is also important to know what these are when comparing any data across platforms.

For example, right now I'm going from a 90 day window for certain goals in AdWords because I know that while the majority convert on that goal in ~30 days, there's a long tail. Unfortunately, Marin restricts their window to 30 days, so any deeper digging will need to occur within Google Analytics or data from our data warehouse.

Would be great to learn more about solutions you've seen in the wild since it seems like you have a lot of experience on this topic. If you have time for a chat my email is in my profile (tried to find you but shostack is ambiguous :)

This is a great argument for having a data scientist or other statistician run your marketing department. Tech is really changing the way people think about marketing expertise. What counted 10 or even 5 years ago is quickly growing irrelevant.

Revenue per visit really? If a visit does not convert how does it have any value (yes I understand the math)? Why not convert Cost-Per-Visit to Cost-Per-Conversion (total spend / # conversions). I am going through this now with a client. The CPC gurus need to justify the CPC budget and surprise surprise the more cpc dollars spent the more they make. What perverse incentive system they have set up for themselves.

If a visit doesn't convert it has $0 value. You can convert cost-per-visit to cost-per-conversion instead (the #s will come out the same) but its more complicated to do technically.

(total spend / # conversions) is pretty easy to do

You can't do it that easily.

If you just take total_spend/conversions the question that leads to is what spend do I measure over what timeframe and what conversions do I measure over what timeframe? For example, if I want to know cost-per-conversion for my Facebook traffic then which conversions should I consider in your equation above?

a time frame is assumed in my equation. Calculating conversions and ad spend over a time frame is easy if you have your analytics tracking set up properly.

You can't just add up marketing spend and conversions over the same period of time and divide them. That's not an accurate method of measurement.

The conversions and the visits they lead to them are not guaranteed to happen over similar timeframes. If you calculate this the way you're suggesting you are making all sorts of implicit assumptions about variance in visit volume, marketing spend, conversion rates, etc.

For the sake of sanity keep your budgets the same/channel over a time period. The variance in traffic attribution will thus even out so you can add up marketing spend and conversions over the same period of time. I subscribe to KISS

That works if you have short conversion timeframes, but if you're running e-commerce, B2B, or anything with longer than same-day conversion timeframes you're counting conversions that were driven by visits outside your analysis time window that way, it may be simple but its not very accurate.

I disagree, ceteris paribus the conversion associated with each channel will be about the same once you past the longest conversion window. We are not attempting to land on the moon, a high degree of precision is not required. This is cookie based analysis so there goes any degree of precision, there is more precise "user finger print" type of analysis but that gets creepy.

All other things aren't held equal in the real world. You may add a new campaign, or stop one that is currently running. You might change up the audiences or add to them. Basically any activity you would do to optimize a campaign is going to potentially change a channel's behavior. Either you limit yourself to not making those changes, or you severely slow down your decision cycles by constraining them to an amount of time based on the longest conversion window.

That may be an acceptable way to run marketing with small budgets (say under $20k/mo) since you don't have the spend volume to come to many new statistically significant insights about what is working and what is not in short timeframes, but once you get to larger budgets the method of hoping that all other variables are equal-ish is a great way to lose money on marketing. If you run substantial marketing accounts it trivial to see the differences in practice between the model you suggest and a more accurate model, you shouldn't just hand-wave away the differences.

Not factoring in Cost of Sales is also a good way to lose money on Paid Marketing.

I couldn't quite follow what the problem is.

I would define the revenue in the ROAS formula as revenue per time period, and the marketing spend also per time period

Then I would qualify the revenue, as that part of the revenue that can be tied to the marketing activity (cookies, refer URL, etc)

And then the ROAS ratio becomes an ordanary division, because both values are known; no need to infer them as in the rest of the article.

The whole issue is how you tie revenue to marketing activity. In a case where you had multiple touch points before a conversion, it's a question of which channel gets credit and in what time frame.

Respectfully, this is a submarine PR piece. From the last paragraph:

>"It’s a lot of work to build out accurate multi-touch attribution, but there are third-party solutions out there that can simplify the process. My company Interstate makes a pretty good one in my opinion!"

That said, the importance of the topic, cross-channel attribution, cannot be emphasized enough for any company doing any sort of serious digital marketing these days.

A fantastic piece that goes much deeper would be Avinash Kaushik's article[1] on various modeling approaches. It is a great primer for both people new to the concept and approaches, as well as people with familiarity who are looking for new ideas.

The broader challenges these days are shifting towards the cross-device dilemma as well as reliance on 3rd party cookies and browsers' and users' increasingly aggressive stance against those.

-Cross-Device Dilemma-

I visit initially on mobile and then convert on desktop. Even Google has challenges with cross-device accuracy here, and depending on your mobile usage, it could dramatically impact how you view your performance. There are a variety of technology solutions out there that attempt to solve this, but ultimately there is no great solution that doesn't make me truly question the privacy aspects of what it takes to get to accurate data here.

If someone signs up for a trial and gives you a unique user ID, great, you can join that to the data from usage on other devices and understand that user's path to conversion. If they don't take an action that requires logging in, it is much murkier (not counting cookie onboarding services like Liveramp etc.).

-3rd Party Cookies-

Google and Facebook are excellently positioned because most browsers recognize their tags as 1st party. Many other networks and adtech platforms are boned here unless they can have their tags served as 1st party. I've seen variances of around 15-18% or higher in some cases between 1st and 3rd party tracking setups. While it is REALLY hard to control for all of the unknowns (no two web analytics platforms will ever give you the exact same data), my hunch is that 3rd party vs. 1st party cookies are playing a significant role here along with cookie deletion. Would love any other data folks can share on this.

Beyond that, the author talks about static models, but does not address their real limitations. At the end of the day, these look at aggregate data, when the reality is attribution occurs at the user level. Every touch point has some value, and that value is different for each user. For example, seeing a display ad on one site does not necessarily have the same value as seeing it on a different placement within that same site. Static models are very helpful directionally, but dangerous when taken as religion.

The future is in dynamic attribution models that constantly shift based on the data and whatever contributing factors you can feed in as supplementary data to the model. I'm super excited to see what Google starts doing with Adometry (check out the TV release they just had), and the other big players like VisualIQ and Convertro will likely have their technology trickle out through other platforms over time. FB's Atlas acquisition is also a big statement that they take this seriously.

Net net, attribution is the hardest challenge in the marketing/advertising world today. Period. End of discussion. It is plagued with data that is messy as hell and incredibly nuanced based on the infinite variables of everyone's unique setups. Look through a variety of attribution models as different lenses to help inform directional decisions, but be cautious on saying "this is the model we will use across everything." It might help initially, but without constantly checking it against other models, you can miss shifts in your efforts.

[1]http://www.kaushik.net/avinash/multi-channel-attribution-mod...

Or stop all media buys for a month and see how much sales are effected. Then turn them on one at a time. Do this every now and then but change the order in which you turn each channel on.

On/off tests like that can give you a macro view of things, but in general are not an ideal way to go about this in many scenarios.

First, you risk the hit to your acquisition efforts. For many companies in growth mode, that's the LAST thing you want to do, even if you could gain some valuable data. If anything, you do a plus-up test to increase budget in a measurable way (narrow geo-targeting for example) and look for some lift there. But that's just one approach.

Second, attribution isn't just at the channel level. To REALLY be useful, you need to understand it at low enough levels to make more tactical optimizations to your efforts. A simple channel-level on/off test will never get you that, hence why attribution models are really useful.

It's always increase your ad send, the solution is never to decrease the budget. A macro view seems like the best place to start. Also the anlysis is never based on profit it seems to be a ratio of gross rev and medis cost. There are other variable costs like cost of goods sold that must be considered. If cost of conversion is $15 and it generated $60 in rev wow we have a winner but the COG is $45 oops we made no money.

That would be mad to do with a large account... I guess if you just began and aren't sure...

That's common in TV ads.

One month show ads, one month off, or 15 days on, 15 days off.

Reply on news.ycombinator.com