Hey Yahoo, You’re Optimizing the Wrong Thing
hilarymason.com
hilarymason.com
As someone who has been in the belly of the beast, I'm going to try to summarize the various ways that this article is getting it wrong.
* Most Yahoo Mail ads generate revenue based on impressions, not clicks. So Yahoo isn't directly making more money by grabbing these "random" clicks.
* Where clicks do matter, major ad platforms including Yahoo throw away a LOT of clicks as fraud, and accidental clicks tend to disproportionately get thrown out along with them.
* In general, random clicks are considered a real PITA for major ad networks, as they confuse the heck out of ad optimization. While small players do tend to soak up that revenue, the big players really, really hate the phenomenon because it makes them far less efficient.
* It turns out what most impacts ad effectiveness online is whether people actually see the ad. An incredibly number of ads are just never seen by the audience. As a consequence, a good publisher will try to find locations for their ads that are highly visible. Highly visible and likely to evoke accidental clicks are, unfortunately, highly correlated.
* Most advertisers who are paying per click are very performance driven. They look at ROI, which means they look at conversion rates. When you charge an accidental click, it's almost certainly not going to convert, so in the end you look worse and they pay less for your clicks.
* One unfortunate bit of truth: advertisers do pay too much attention to clicks and CTR (click through rate). Even advertisers doing brand awareness campaigns, which are not looking for immediate response from their audience, tend to look at CTR.
* Yahoo has actually tried hard to establish other metrics that they should look at, like "Bounce Rate", which attempts to factor in whether visitors immediately exit after clicking. They use those metrics internally for optimizing ad performance, so accidental clicks are likely to discourage showing an ad more than encourage it.
'She' has a name, Hilary Mason or @hmason.
If it had read, "He's a data geek..." it would have only have been offensive because they got the gender wrong.
I understand the line of reasoning, however the tone of the comments about the article are negative and not up to standard. [1] It is not a question of gender but how to discuss, acknowledge & discuss an authors work in a civil manner.
You are criticizing the people who are behaving in a civil manner for the behaviour of those behaving uncivilly. Try to focus on the jerks, there's an off chance they might get it.
Yes & no. I'm don't think I'm being that critical, 'why don't you refer to someone by name.' I take your point though. As for focusing on "Jerks", I ignore them. I'd rather point to measured responses like your own illustrating to "Jerks" how to respond.
There are obviously a number of ad revenue driven pressures on Yahoo Mail's design. The display advertising market is very different from the search engine market. It is only tangentially about the clicks. The biggest pressures are:
"Premium Locations": these aren't necessarily the places that get high click throughs, but rather the locations which are prominently displayed and noticed by the users, which causes them to stand out in a world awash in banner ads. This competes with good UI design, which wants to put important functions in these exact same locations.
"CPM": just getting way higher impression counts. This translates to filling the screen with as many ads as possible, stealing screen real estate and really disrupting eye flow.
"CPA": Lots of ad networks/DSP's try to arbitrage between CPM and CPA. Misclicks don't generally help much with that beyond hindering optimization efforts to use clicks as signals. Mostly what this does to design is encourage "rest points" in applications where users are likely to actually take the time to go all the way through to a conversion (think of it like TV commercial breaks in sports).
"Uniques/F-Cap": You want to raise the bar on the minimum # of impressions seen by every visitor, because those early impressions are far more likely to be people who haven't seen the ad before. Advertisers pay a lot more (10x more is not unheard of) for that. So, there is pressure to spam the visitor with as many impressions as possible before they are likely to leave (which makes it hard to serve ads in a way that is proportional to the value the user derives from the experience).
"View based conversion": The underbelly of the adverting network. In theory this is a good thing. In practice, it creates an incentive to shove ad locations in to the page that the visitor never even notices (ironically the opposite of the problem the article contends).
Some people have apparently gotten the impression I was questioning your credentials, your intelligence, or somehow attacking you personally. At least in terms of intent it couldn't be farther from the truth! While I think your naïveté about online advertising undermines your article, all it might imply about you personally is your curiosity about online advertising is low as compared to other pursuits. If anything, I'd say that is a compliment. If I conveyed or you perceived anything other than that, I certainly apologize.
And these kind of cutesy oneliner descriptions of one's skills make me cringe: "Simply: I make beautiful things with data".
OK, she works as a "chief scientist" (that's not even a job, for us, old time folks, but anyway) at bitly (a URL shortening service, i.e as far away from a real business as you can get, that aspires to be a "bookmarking service" also.
Please define 'real business'. What does a company need to have to be a 'real' business? Users? Revenue? Profits? A product that at least some people are willing to pay money for?
I agree that bit.ly is a very simple concept, but I take issue with the disdain in your post - even URL shortening stops being straightforward once you expand from n = 1 to n = 10,000,000. And bit.ly does a lot more than 'just' that.
This reminds me of McCain's 'real America'. You may not like {NoVa, these kinds of startups} and you may think that they don't capture the 'true' spirit of {Virginia, business}, but at the end of the day, their {votes, dollars} count just as much as 'real' {America, business}.
You may not think Hilary's qualified to talk about the details of Yahoo's particular advertising model, but don't extend that into what's bordering on an ad-hominem attack against a very legitimate startup, as well as a key figure at said startup.
Revenue and eventually profits. Even if "url shortening" is a "real business", it still is in the very far outskirts of "businesness". A very marginal value adding service, that exists solely because of some very marginal deficiencies of other services. Even Twitter has trouble monetizing, a "url shortening service" 100 times more so.
>even URL shortening stops being straightforward once you expand from n = 1 to n = 10,000,000.
I fail to see how. Even a simple setup can handle 10,000,000 shortened urls with aplomb. And url shortening is the most sharding friendly use case you can get, ie linear scaling is trivial in a url shortening service.
>You may not think Hilary's qualified to talk about the details of Yahoo's particular advertising model, but don't extend that into what's bordering on an ad-hominem attack against a very legitimate startup, as well as a key figure at said startup.
Well, she may or may not be qualified. I just pointed out that the page the parent pointed to as proof of her competence on the matter doesn't _prove_ her qualifications at all.
I'd like to ask you to think twice before publicly questioning someone's credentials like this. Whatever your intentions, picking on someone's CV just because of a blog post you disagree with is not only rude, but it sends a message -- particularly to women in tech -- that if they speak publicly, if they offer up their opinion, they will be attacked not about the content of their point but about their competence to speak at all. I believe this kind of attack has real consequences on our field, and I would urge everyone to show everyone the respect they'd want for themselves.
Dave
Oh, don't worry about that. I'm an equal opportunity insulter! I didn't even care that she is a woman or not while making my point. Nor do I think that women in tech need special treatment. We all get what we all get. Do you see exchanges between males being any more courteous?
Notice also how I avoided to speak about their opinion on Yahoo at all -- they might or might no be 100% correct.
I only responded to the statement by some parent poster, that her CV "proves" that she is especially knowledgable in the ads field.
Is he legit in the "titled world?" :)
Which is very common in modern business, but shouldn't happen when you have "science" in there. Lives a bad taste.
Science is all about clarity and SPECIFIC fields of study. Nobody does "science" in general.
You say "Nobody does 'science' in general", but you are wrong. Many people work at organizations that have a broad range of scientific pursuits in a broad range of fields of study. The Chief Scientist's job is "ensuring that [the organization's] research programs are widely regarded as scientifically and technologically well founded and are appropriate for their intended applications".
It's largely equivalent to a PI role of a university research project, except that there are things going on in the company other than pure research and so there are chief officers of other things too. Nobody thinks that "primary investigator" is a fluffy title on the grounds that people don't do investigation in general; it's clear they're investigating specific things.
A lot of people with her background come to the online ad business. While brilliant, they find they have a lot to learn and a lot of assumptions that they need to throw out the window. Fortunately, the brilliant ones have no problems learning so they come up to speed quickly, but this doesn't help them to understand things prior to digging in a bit.
Ms. Mason has confirmed her lack of domain expertise, so I guess that settles it.
In the process of attempting to highlight an important lesson about applying data analysis, she has highlighted another important and related lesson: regardless of the level of one's data analysis expertise, without domain expertise to contextualize the data, it is terribly difficult to do any kind of useful analysis.
"...experiences like this are great reminders that data only takes us so far, and creativity and clear thinking are always required to find the best solutions..."
This might be very basic knowledge for everyone, but many researchers in machine learning & data mining community do not think like this. They believe that data tells the entire story.
I come from a data understanding background, and the cool
thing I liked about this article was the following statement:
"...experiences like this are great reminders that data only
takes us so far, and creativity and clear thinking are
always required to find the best solutions..."
> * It turns out what most impacts ad effectiveness online
> is whether people actually see the ad. ... Highly visible
> and likely to evoke accidental clicks are, unfortunately,
> highly correlated.
I'm not sure what your end point here is, as it's all over the map, but you seems to simultaneously be arguing against the author's point and for it as well.
Sometimes the truth doesn't entirely support or undermine someone's argument. It's one of the sure signs that it is the truth. ;-)
I wasn't trying to make a specific point, but rather to inform everyone, since the article didn't.
I don't mean to be coy, so I'll share my opinion:
The article presents this as a bug, which implies it is clearly wrong. It isn't. Misclicks are always going to create undesired outcomes. If you didn't have an ad there, the misclick would trigger some other action. Showing an ad landing page is probably one of the least harmful things you could reasonably do with a click in that location, so I question the notion that this leads to a "terrible experience". It might not be a great experience, but unless Yahoo Mail is expected to fix mice and operating systems, it's probably one of the better experiences they could provide in response to a misclick.
Ms. Mason seems to believe that the false clicks are driving Yahoo to think that they are getting superior ad performance. That isn't the case, as the data itself does indeed provide some clues as to the relative likelihood that an ad received a misclick.
The notion that this is evidence of "the data only takes us so far, and creativity and clear thinking are always required to find the best solutions" isn't well supported by this problem. In reality, clear thinking about the data itself has indeed lead to about as good a solution as Yahoo Mail could provide.
As the article describes, people aren't misclicking because they've looked closely at the ad and mistaken it for an application function. Misclicks are predominantly people who really aren't sure what they are clicking on, don't know they are clicking, and/or didn't intend to click at all.
Either way, in practice Yahoo's optimization efforts if anything create incentive to minimize misclicks, not the other way around.
perhaps not, but surely if I was to buy such an ad and pay per impression, I'd probably like to know what the impression/click ratio was. no doubt Yahoo advertises a low one to potential buyers...
http://www.conversionvoodoo.com/blog/2010/04/test-your-horiz...
That ad unit is available only through Yahoo's CPM program - not self serve and I know for a fact that ad unit sells out months in advance with huge CPM volume commitments.
Modern ad platforms optimize media with backend ROI tracking via pixel-fires / cookies and Yahoo optimizes their inventory to that metric with their advertisers
Pain for users in this units placement? Maybe yes. Could do better? Always.
But pressing first priority problem for a business that can't afford to screw with their profit centers, no.
And it's not just me; I watch other people use laptops, and their usage is generally littered with unintended taps; they just don't seem to care that they keep randomly misplacing their keyboard focus or switching to other applications.
I mean, I get that it's slightly easier to tap on the trackpad than it is to click with your thumb. But how can that possibly make up for all the time lost to accidental input?
I don't have any trouble with errant clicks, but I may have altered my typing style. Right now my hands are both entirely off the keyboard (elbows resting on armrests on chair). I also understand there are some improvements in Mac OS X over other operating systems with respect to errant clicking.
Then again, I'm using a Macbook Air, which has a highly-rated trackpad.
If you have a touchpad with crappy drivers, tap-to-click can be painful, yes, but good touchpads are pretty decent at discarding accidental taps. Many drivers also have options to adjust tap length, etc, if you're interested in fine-tuning it, but it's never something I've had to do personally.
I don't know how companies can be convinced to do ad placements like these, or if they simply rely on getting enough conversions from accidental clicks from unknowing users to make out in the end.
Either way, it's a bad practice and something I'd equate with a torrent site or rapidshare.
In the end, the advertisers don't really care (at least if they understand the math, which many don't). They care only about their ROI. If they pay 20 cents a click and make 40 they're happy, if not they aren't. Sure you could perhaps get rid of all the shaky grandpas, pay 40 cents a click, and make 80, but it's the same ROI.
What you really end up with is a cottage industry of people making websites that can convert shaky grandpas better than real advertisers because just like spam, some non-zero percentage do convert. That's why you see some stupid ID fraud ad there. Shaky grandpas are terrified of the evil hacker who wants to steal their identity.
I think shaky grandpa should be the internet standard term for worthless clicks.
The contrast on the background is much lower than the federal 508 standard for contrast and I think has changed to over the years to a lighter shade as Google "optimizes" it.
One is an ad and one is a search result, is there much difference? Given the average quality of monitors, I think those are designed to fool even otherwise sharp eyes.
On a CRT this isn't a problem, but I imagine that many people have their LCDs at an angle which isn't perfect for viewing such contrast differences ... and if a user (like my parents) are not as accustomed to playing "spot (and ignore) the ad" based on content as we are, then they might completely miss the visual cues as well.
Getting people to click on the ad instead of the search result makes Google about $20 per click because of the keyword, so no wonder it's designed to be almost invisible.
If you click on an ad that isn't relevant to you, there are three major harms caused. One, your time is wasted closing it and going back. Two, the advertiser may have to pay extra for a useless click. Three, Google (for instance) has to consider the likelihood that this is a useless or fradulent click, and possibly reimburse the advertiser if so.[1]
All of these are bad for Google (for example). The offset gained by a few dollars in revenue is probably not going to counter that, because what they really want is users to continue using their search engine and they want advertisers to be happy with the clicks they pay for. Those are really important for them to keep. Tricking you into clicking useless links is bad, bad, bad. (Adwords hosted on someone's website is another story; that person might not feel the same way.)
But if, on the other hand, the ad actually is relevant to you, then it's great for all three parties if you click it. So I'd hesitate to attribute these UI choices to trying to fool people.
Disclaimer: I can't remember the last time I clicked on an ad anywhere on the internet, except by accident. But that doesn't happen to me on Google or other reputable advertising-based sites.
[1] see e.g. http://support.google.com/adwords/bin/answer.py?hl=en&an...
First, if anything, this seems like an accessibility problem that would be hard to solve. Shakiness affects many of us as we get older, but it is hard to design a website around this constraint.
Second, there is an implication that this ad placement is somehow "surely wrong," and a step further, is likely only successful due to pseudo-fraudulent interactions. I want to rebut this, but she does not actually substantiate it. The most I can say is that the top-left corner of the screen is the hottest hotspot, so placing an ad there is savvy at worst.
I too have been reading her blog for a while, but this piece caught me off-guard as awkwardly personal and lacking in substance.
Is it? Large link targets surrounded by empty negative space seem like an easy way to solve this problem. The issue is only an issue because Yahoo puts the links in tinytext and crowds the ad link right up next to the "Inbox" link, so a twitch can send the cursor scooting past the link you want to click on to the ad you don't.
That's correct. Adding a tasteful mouse-over effect also serves as a nice additional visual cue.
(Speaking from experience: I have a very shaky hand due to an old injury.)
Also, do you use a mouse or trackpad?
Seriously, I'm curious.
First, I use many pointing devices. With my home standing desk, I use an Evoluent vertical mouse. With my workplace sitting desk, I do better with a trackball (it's huge). When I need to take my laptop on the road, I just use the trackpad. They all have their pros and cons depending upon the task at hand.
I think the hover effect helps by making the click a more reflex-driven event. Even if my cursor is skittering about a little bit, a mouse-over effect helps the lower parts of my brain "pull the trigger" at the right moment. I'd describe it as a lot like playing an FPS or some other fast-moving action game. Sounds lame and totally unscientific, I know, but I definitely notice the difference.
Are we to ascertain the shakiness level of the user to determine the proper amount of white space? A CSS media query maybe?
@media min-shakiness: 0.5 and max-shakiness: 1
But then, I'm just being rude here, sorry.
The other case seemingly ignored here is that the clicked link in no way resembles the intended link. Assuming the user can read the link it seems he did not read what he's clicking on. Granted, life puts limitations on us as we get older but I don't understand how one can predict behaviors of people who do not fully read what they are clicking on.
There's a great deal to be said about examples of bad design causing people to do things they did not intend, I don't believe this is one of them.
"Decent" for whom? For you, maybe, because your motor skills are sharp. For those whose aren't, maybe not so much.
Are we to ascertain the shakiness level of the user to determine the proper amount of white space? A CSS media query maybe? ...But then, I'm just being rude here, sorry.
Yes, you are. Degraded/impaired motor skills aren't just something older people deal with, they come with a range of illnesses and disabilities too. Snarking about a media query for "shakiness" is sort of like a retail store owner snarking about whether he needs to put a camera on his store door to check if people using the ramp are really in wheelchairs.
There's even an intended barrier between the two in the form of a horizontal rule.. I don't understand how one can predict behaviors of people who do not fully read what they are clicking on.
These comments indicate that you don't really understand the problem we're talking about here. It's not that the user doesn't know the ad link is a different link. It's that she tries to click the link she wants but ends up clicking the ad link accidentally because the close placement of the two links makes it easy for a bump on a trackpad to send a click intended for link A skidding over to link B instead.
I admitted I was being rude. The point is that we cannot possibly account for every kind of limitation that people may have. We do the best that we can realizing that we cannot account for everything. The only answer to that question is to remove the ad altogether, which is not a solution.
Ok, good point, the problem being seeing one spot to click on but clicking on another by accident. Now explain to me exactly how one is supposed to predict that and account for it? Accidental bump on a trackpad is not a problem inherent with the design of the site, it's a problem of the hardware and its usage. It's the same problem with the vertical row of links that belong to the app in question, despite the ad placement. Are we suggesting that the menu links on the left should be separated by at least fifty vertical pixels?
So, my original point, what's the optimum distance between two links to avoid accidental clicking due to physical limitations and hardware problems? There is no way to determine that.
The example given of Google using low contrast colors to separate search results from ads is a good example of taking advantage. This Yahoo example is not because the two links in question do have a noticeable separation and do not bear any resemblance to each other. People clicking on spot fully expecting that they are clicking on another is a completely different topic and is not indicative that Yahoo is taking advantage of anyone. How can one claim that Yahoo is somehow taking advantage of people by tricking them into clicking on ads that are "close" to an intended link? What exactly do they benefit from this?
See, you've misunderstood the root of the problem. This person knew exactly where he wanted to click, he aimed for those targets, and his physical limitations caused him to miss those targets. User intent is not the problem.
The problem here is the penalty for missing: he gets shuttled to an off-site advertisement that doesn't have anything to do the task at hand. If he misses and just goes to Drafts instead of Inbox, that's a much easier mistake for the user to understand and correct; the intended target link is still prominent on the unintended page, it's easy to connect cause with effect ("Stupid shaky hand, oops!"), and it's easy to try again. Unfortunately, in this case, that advertisement's placement makes it a very expensive miss for the user. Before the user even knows what has happened, he's on another site entirely. And that ad is placed between what are arguably the two most important links in the interface: Inbox and Compose.
In the example provided the ad link has a decent amount of white space between it and the inbox link. There's even an intended barrier between the two in the form of a horizontal rule, granted one pixel. What more do you want?
He probably missed the targets because they're skinny and/or long and require a good deal of "vertical accuracy", which can easily devolve into "vertical inaccuracy". The targets should be given some additional vertical size. This is anecdotal based upon experiences with my own shaky hand, so YMMV.
I fail to see how going to an ad page is a worse offense than going to Drafts as opposed to Inbox. Either way the person in question has to correct the mistake. Expensive in what way? Money, time, embarrassment? I don't see the difference.
The ad is between Compose and Inbox, so what? Are you suggesting that Yahoo somehow benefits from an accidental click on this ad? What's the purpose of them putting it there? Could it be possible they have it there because they consider that you are more likely to see it there and not that you'll accidentally click on it?
And again, what's the optimum distance that's acceptable? Fifty pixels? One hundred? I would have to assume that if this is the problem then it is a potential problem for nearly every website and application ever made. The reason this bothers me is because people are saying the burden is on Yahoo to fix this when the problem does not necessarily lay with them, but other factors are in play they have no control over. Some people are even suggesting nefarious reasons for this ad placement to "trick" people into clicking on it. For what benefit?
I too sometimes click on something other than what I intended. Most of the time I don't blame the app, I blame myself. There are times that people will employ tricks to get you to do what they want, that's when you complain; this is not a good example of this.
At first it's going to be confusion. It's the difference between quickly being able to understand the mistake (because the whole interface is still on the screen, with the pointer maybe in the same position) vs. suddenly being on a completely different site. Keep in mind that this is a missed link, so the user may not immediately know what has been clicked.
After the guy has been using Yahoo Mail for a while and the peril of that ad is well understood, it's just going to become really really annoying, getting sent off-site, every time he hits it. And he will hit it.
Could it be possible they have it there because they consider that you are more likely to see it there and not that you'll accidentally click on it?
Sure. But is that the best choice? That's the question. Like you say, even fully able-bodied people sometimes click the wrong link. It seems like a good idea to take that into consideration when placing what may be the two most trafficked buttons within your interface. I'm not assuming nefarious purposes on Yahoo's part.
I can't say what the optimum vertical height of a button should be for a shaky person. Gmail serves my purposes fairly well without making many design compromises.
In the particular case of Yahoo Mail, their crappiness has driven off large numbers of high-end users. They also lost a lot of great staff who were tired of working on crappy things for managers who didn't give a shit about the users or the workers as long as the numbers were good and their political power increased.
Yahoo's a classic example of what happens when you know the price of everything and the value of nothing.
Yahoo has already solved this problem.
- Use a Thinkpad with a trackpoint.
- Disable the touchpad.
- Use ad-blocking.
- Don't use Yahoo.
Know a few folks who have to deal with Parkinsons, which a lot of computing devices don't accommodate particularly well.
My tremors are not that serious though, so I'm perfectly OK with a touchpad (I'm a huge fan of Apple's glass touchpad). With trackpoints, it's hard to get a good level of sensitivity.
Typically with an A/B test you'll test something like clicks. With a user test you'll have people in to try it out to see things that you can't see in the data. In this case, I think you could look at an A/B test and say, "Ad clicks are up! Great!" But in the user test you might say, "Oh, these clicks are accidental, so although it makes more money in the short term, it will decrease the value of the click and the value of the property.