Comment by rfergie | Hacker News Reader

Comment by rfergie | Hacker News Reader

Parent Full thread

I am aware of the significant attention this paper got about a year ago. In my filter bubble (and I am in a massive filter bubble with regards to this kind of thing) the attention was similar to the rest of your comment - focusing on ebays failures of execution rather than the failure of measurement.

Other advertisers do not make the same mistakes as ebay with regards to landing pages, adverts or keywords selection but many make the same mistakes when it comes to measuring their advertising value.

Now, perhaps the errors caused by faulty measurement are not as important for other advertisers and I can see why this might be the case (for the reasons you list about how ebay is running their PPC) but what should be an opportunity for the SEM industry to look at how their work in valued and measured has instead turned into something else

> As a control they take MSDN/Yahoo vs. Google

They also used a geographic split as a control. E.g. if in the past areas A and B have performed similarly then pick one of them at random to stop showing ads in.

AdWords geo-targeting combined with this technique is a way for other advertisers - regardless of scale - to run similar tests. But we won't see many doing this because of the attitude that it is ebays execution that sucks rather than their measurement

The paper sure is interesting and should add value to the debate and approaches to SEM. To discard it by scoffing at eBay's execution would indeed be your own loss.

My biggest take-away is the saturation of advertisements and the effects it can have on running ad campaigns. Yours seem to focus on the common correlation-causation misinterpretation, and faulty measurement approaches.

I do think there are enough problems with the paper to not grant it the authority to speak for an entire industry (as Slate put it). eBay is just one (and a very big and unique) advertiser in this arena, and every move it makes (even to measure it) changes the entire ad network.

> They also used a geographic split as a control.

Thank you for this correction.

Aside: The problem with eBay's non-profitable advertisements seems to me a beautiful problem: The paper authors subscribe to the benefits of saved costs. I do not think that the marketing department of eBay subscribes to this benefit. For them, and external ad agencies, the more ads your run, the more clicks you get, the more you get rewarded, the better your monthly reports look. People at eBay's online marketing department must be smart enough to realize that many of these ads were worthless to the company, yet it was worth it for them to empty the monthly budget on a list of generic keywords.

People at eBay's online marketing department must be smart enough to realize that many of these ads were worthless to the company, yet it was worth it for them to empty the monthly budget on a list of generic keywords.

I read somewhere that alleged cookie-stuffers claimed that eBay's staff knew about the cookie-stuffing all along, and encouraged it so long as it increased the sales attributed to their own work in courting affiliates.

If true, these are both cases where incentive design (and measurement) are as important as people's skill with their craft.

I now agree with most of what you say - as you point out we were just looking at different parts of the same thing.

Your aside comment is really interesting. I'm not sure what the solution here is - more accurate systems of measurement lose the tight feedback loop that make online ads so successful, but simpler measurement is wrong. Some sort of combination of the two is the answer, but how that exactly works in practice I have no idea!

Reply on news.ycombinator.com