Google Instant Proves Google's Design Process is Broken
fastcodesign.com
fastcodesign.com
"Testing can only tell you so much -- and it often only reveals that people only like things that are similar to what they've had before. But brilliant design solutions convert people over time, because they're both subtle and ground breaking."
The article provides no evidence to support its claim. For all the anecdotes, not a single one describes a situation where the intuitions of "designers" outperformed a solution based on testing alternatives.
However, the pro-designer propaganda flies thick and fast, repeatedly implying designers would have made a better solution, or seen the solution as obvious without the need for testing:
- "While that solution seems obvious and not particularly elegant"
- "Is it just us who find our eyeballs spinning in their sockets...?"
- "Obviously, none of these (prototypes) were going to work"
- "A little design know-how would have made that obvious"
- "(They eventually settled on a blue that is basically the average of all the blues used in hyperlinks across the web. Duh.)"
- "Google's 'solution' to providing instant results still seems so primitive and ugly"
- "But brilliant design solutions convert people over time, because they're both subtle and ground breaking."
- "testing artificially limits the worldview of the people"
- "has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? Have either of them become a pleasure to look at or play with? No."
The author isn't interested in making a coherent argument. The author's interest lies in unashamedly gratifying the readers' sense of importance. It is a website for designers, after all.
Firstly that often results in horrid UX experiences, as Google Instant is. It is also, weak for the following reasons:
1. No innovation happens (which is why Apple came up with the smartphone touch UI and Google just copies it)
2. People don't like change, so will favour things close to what they had before
You have fallen straight into the same odd thinking dominating Google, that UI is incremental, testable and easily measurable.
You want precise and measurable. That's not design. Jonathan Ives is worth his weight in gold for his design skills as much as Linus Torvalds is worth his weight in gold as a programmer. And we as engineers have to accept that.
Google will never come up with a great design with the way it approaches the problem. While I love their products, none of them have ever blown me away because of their elegance or coherence. They only know how to do simplicity and are beginning to forget how to do that too.
Am I the only one that loves the UI for Google Instant and thinks it's the single biggest time saver of recent times?
I'm not saying that designers' intuition can be replaced by testing, I'm only saying that testing can not be replaced by designers intuition as the article implies. A designers' intuition, after all, hopefully comes from the experiences of informally testing out designs in real life.
What really annoyed me about the article was the attitude of "We designers come up with brilliantly ground-breaking yet subtle designs, and if testing says there is a problem, then ignore the tests because we are always right. Isn't that right, boys? YEAH! WE RULE!"
But a designer's there to sit down and go, right, what about z. And j. And 77883. And still not test them, but decide on one. And then refine it and then you test it some and refine it some more.
And even better is that the designed product will have a flow, a coherence because it's not about a bunch of tiny improvements and changes, it's about a vision.
It wouldn't necessarily test well against a to begin with. People don't like change, so say here's Ives' search and here's the instant that's almost exactly like the existing google and you've got a lot of initial resistance that will skew the testing.
But people who love new stuff and then evangelize, mavens I think Gladwell called them, will result in more people trying it and it will end up successful. The people who helped push Twitter and Facebook and Google itself. Not that I buy all of tipping point, but there are some good points in it.
Incremental design is not about design at all, it's more about fear. Designing can include A/B testing, but not at the start. It just seems the wrong way to go to me.
As for the tone, I didn't notice it, but then again I realized after reading it I already agreed with it, he was vocalizing something that has been dawning on me.
Innovative products rarely test well, overall. If you test it to a group of visionaries who are comfortable thinking outside the box, then yes, it'll test well. But if you test with a group of regular / casual computer users, non-techies and the like, many innovative or important design decisions may not test well.
Take, for example, USB. The technology came out and was sparsely added to new computers, but always in addition to COM and Parallel ports. Apple saw that USB was the future, so they made the iMac which—gasp!—only contained USB ports. No COM, no Parallel, just USB.
You think that decision would have tested well with users?
"I can't use my existing printer?! That's stupid!"
Except… it was this very decision that made USB successful in the market, because it forced all peripheral manufacturers to make USB devices if they wanted to sell to Mac customers (a large enough market to be worthwhile for virtually all of them).
And so, USB succeeded in the market. But it wouldn't have if Apple had used Google's test-driven approach, because that decision would have tested quite poorly.
So not all Apple decisions work, like PC's over Macs. Apple lost that war as the open standard won.
And then sometimes it is the better design, like iPods and iPhones over their competition that wins.
My point is I'm not sure your argument holds in general, sometimes its this, sometimes its the other thing that works.
How do you know that Apple's designers aren't also testing their designs with real world users?
I don't think "design vs real world testing of experiments" is a binary choice.
Having worked there, I can vouch for this. Apple employees (and, most notably, Steve himself) are the real world users they test with. Every important or critical product only ever sees internal testing, but much more important is that the rationale behind Apple's design process is that of an actual designer thinking about solving problems from a user’s own perspective, whereas Google's "design process" is based entirely on trying to solve problems from an engineering perspective.* Often, they take it so far that they only solve engineers' problems, not even general users' problems.
* That's bad because the vast majority of people on this world are not dedicated engineers.
If that is true then Google's immense success proves "Design" is not the only way to design and strongly points towards big D Design with traditional designers being not the best way to design.
UI is more than looking pretty - gmail is popular because it stays out of the way and lets you focus on content, which it handles quite well; in contrast, iTunes increasingly feels like a train-wreck because they swap icons without fixing any of the quirky behaviours and non-standard interface conventions which users must memorize. It was also a confusing comparison because a UI for managing a tightly-curated, highly-structured list containing on average a few thousand items pre-selected by the user isn't obviously relevant to arbitrarily searching across billions of unknown items.
As for iTunes, obviously, if you read the piece, I wasn't comparing it to Google. I was comparing the process that produced it -- and how Apple has always gotten better in UI, with each generation, while adding complexity. I don't think Google can really say that for themselves, as the profusion of Google products shows.
Google Search is an information service, and IMHO, it seems they maintain a healthy respect for information itself. Google feels like they let the information guide their UI, which I appreciate. -I love my german car because in so many places, I can experience utility trumping style. Makes me want to drive.
"has your G-mail or Google Reader gotten any easier to use, or less stressful on your eyes? ... No. Just look at the music listings in new, redesigned iTunes."
If iTunes was OSX-only, it could spray itself all around OSX as a bunch of cute one-screen utilities with clever integration hooks:
1. The App Store would become part of Software Update (which would thus become a general Mac App Store and App Update manager—hopefully buying out AppFresh and giving MacPorts a GUI);
2. Contacts, Bookmarks, Notes, etc., and the transfer of media to the iConsumerElectronics in a friendly, GUIful manner, would all be a part of the iSync utility (yes, that exists—it's the ghetto for synching phones that aren't made by Apple);
3. Podcasts would just be a client program that relies on the same background-downloading daemon that System Updates do, with a modification to read arbitrary feeds, and extract enclosed media files (or torrents!); once downloaded, iSync would just see them and sync them;
4. iTMS would just be a website, which would expose special content types that Safari would know what to do with (audio/x-apple-ringtone = save to the Music/Ringtones folder, etc.);
5. and iTunes would be left to be a music library, consisting in its entirety of Playlists, Genius, and perhaps the Radio (and hooks to send events to Ping, if it likes.)
If iTunes was OSX-only, it wouldn't need to know how to burn CDs; it could just allow you to export a playlist as a folder of MP3s, and then integrate audio-CD burning as an option in the OSX Burn Folder menu.
If iTunes was OSX-only, it wouldn't need to have sections for TV Shows, Movies, Books, Ringtones; those would just be folders on your hard drive, which iTMS (through Safari) would write to, and iSync would read from.
There's a thousand other ways it could be better and slimmer—but, if you'll notice, none of these things could work given the restriction that they have to work on Windows as well.
An integrated iTunes as an all-in-one media file manager linked in to the iTMS is also pretty integral to Apple's desire to drive people to that store - they _want_ people to think about buying more music and movies when they go to update their podcasts or back up their phone. Much like an instant search that serves more ads or an unnecessary password prompt when downloading a free app update, it makes a lot more sense from the perspective of encouraging particular consumer behaviour than from enhancing their productivity or enjoyment.
As far as package management goes, the problem here is that Apple simply does not care. The Mac sysadmin community has been asking for better solutions for years but it's just not a priority for Apple - even App Store updates, which theoretically are more important, have been broken[1] for something like the last 4 major iTunes releases but since it's merely clumsy and doesn't prevent sales it obviously hasn't been as important as a new version of some non-standard window controls.
[1] The process is now: click on Apps. Click on "Get Updates". Click on "Get All Updates". Wait. Dismiss erroneous "The information on this page is outdated and must be refreshed" dialog. Click on Apps. Click on "Get Updates". Click on "Get All Updates". This from a UI powerhouse? The phone almost gets it right except for the gratuitous password nag.
Yes, Windows does have hooks to add functionality—but hooks aren't enough. The reason Apple could remove components from iTunes was that it could, itself, integrate them into all shipping copies of OSX. Apple doesn't decide what drivers and plug-ins get shipped with Windows, so anything they'd install would be third-party and after-the-fact (which is what already happens: e.g. the CD burner driver bundled into the Windows iTunes installer.) You can't slim down Windows iTunes because you have to ship all the programs that make up the functionality of iTunes, whether modularly or monolithically. However, you can slim down OSX iTunes if you just start saying "this will be an OS feature, not an iTunes feature."
Also, on a completely unrelated note:
> The phone almost gets it right except for the gratuitous password nag.
I've always taken that to be a sudo escalation prompt. You don't want your kids picking up your phone and buying things on it.
I remember a study done by Google that showed that by shaving off something like 1 second from their load times, they were able to increase the number searches people did substantially, which for Google = millions of dollars in extra revenue from ad clicks. Seconds still matter.
What really matters to me is how quickly can I find what I'm looking for and go on my way. This is the only metric that really matters.
Total time = search result + visual scanning/deciding
Right now, for me, the total time has increased because the time it takes for me to scan the page and figure out if I want to click a link has increased. Also, because you're being bombarded with extra links as you type, I'm more apprehensive about clicking a link, because I'm wondering if there was a better choice using a different variation of the query.
I loved the older method of as you type updating the query box, but this is information overload.
Slightly different topic, but I like Wonder Wheel for searching as well. I start with something intentionally vague if I don't know the exact phrase I want for describing something. Then I select the best choice from the Wonder Wheel.
I'm tying in the search box and I see a whole bunch of results flying below. But by the time I get to the end of my text, I look down and there are no results, and Google is telling me to hit ENTER. I feel like I'm literally wasting seconds because I had an expectation of results, but instead got nothing instantly.
The UI is probably great for people who are hunt and peck typists. But I think most people will find it to hinder more than help.
After each search don't you actually open a few websites and see if they have the information you need?
There's no inherent value in refinement without inspection. The little snippets of text under each link are only 20-30 words long. They rarely convey anything of value.
Perhaps I'm looking at this wrong, but if people want to find out about the Nottingham food and drink festival do they start with 'Nottingham'. Then 'Nottingham Festival'. Then 'Nottingham Food and Drink Festival'? As that seems like an odd process to me.
Also, if you're not finding what you need by inspecting sites as you go, then Google isn't making any money as you're not clicking on any links.
So either tack you take on defending it, the opening comment still makes no sense.
After each search don't you actually open a few
websites and see if they have the information you need?
There's no inherent value in refinement without
inspection. The little snippets of text under each
link are only 20-30 words long. They rarely convey
anything of value.
My experience could not be more different. Probably half the searches I ever do on Google are complete without opening any of the links, because the answer I'm looking for is in the summary. Say I use Google only fifty times a day (many days it's more than that, but some days less). The vast majority of those searches are for trivial things that I would never have bothered to find out until after Google:
"What's a more businesslike word that kinda means flighty... oh, capricious, right."
"4.1 million rupiah? What's that in USD... ok, a little over 400 bucks."
"What's this error message from mysql|apache|PHP mean?"
"What's this error message from I-don't-know-what mean?"
Now that I think of it, the summary answers my immediate question well over half the time. You appear to make an argument that that's bad for Google.
Yes — for Google to make money a lot more than for you as end user.
I'm hoping that future iterations of instant can bring some polish to the UX, because right now it seems messy and unnecessary. If I was to continue to work as rapidly as it does now I want some more context to the results it's spitting out, like in the 1st design iteration they show....
I feel like they could have hit it out of the park with some real magic, but instead they (b)punted and put out the ugly untested beta.
This is the search design for the people who use google to get to their facebook login, and not for those who are truly searching for information.
Do you personally even encounter 'Google Instant'?
If I'm not using my browser's search bar, I've got my personalized iGoogle page open.
They've made changes to the vanilla Google homepage which is most likely, predominantly visited by the sort of people who use Google to get to their facebook login anyway.
I'd say most times I search through my chrome bar, but sometimes I do end up on the google page for whatever reason.
Even if that's true, there's more of them than there are of us.
Also, in the comments I made about a second mattering, it was exactly my point that while seconds are important to engineers, they may or may not be to users. And a focus on actual user stresses is the essence of design.
Anyway, thanks for reading and commenting!
I totally disagree with your premise. If I put it in the nicest possible way, you are suggesting that engineers can hit some sort of local maxima for their design decisions with testing, but that 'professional' designers can get them out of that with bold, intuitive changes.
It's nice to think so. It's certainly a possible difficulty in design: many designers have made their names reworking a tired design into something iconic and revolutionary. (This article about Tag Heuer's digital stopwatch is a great example: http://lenovoblogs.com/designmatters/?p=3748).
In this case though, I think you're missing it. If you believe design's God is usability, that is that design is about providing function to actual users of the design, then you're on the wrong side of this conversation: the side with only your opinion, arguing against the side with billions of datapoints.
Since search is the only way Google makes money, let me ask you a rhetorical question: how long do you think Google Instant will stay turned on if it worsens the user experience? By worsens, I mean that empirically users demonstrate less advertising engagement, or show that they are getting worse search results?
So on the seconds conversation: seconds matter to USERS. Seconds only matter to google's ass-kicking advertising engineering team because they matter to users. Google is accidentally leaking some proprietary search information to you in those videos -- they're telling you: ultra-ultra-fast search makes Google more money because users like it better.
Where good design is less easy to statistically evaluate, say with Google Buzz or News, this approach falters, but I think picking on Google Instant/Homepage search is choosing exactly the wrong section of the company to complain about: these guys are the absolute best in the business in the search results / advertising world, it's their bread and butter, and they test the hell out of stuff like this before it launches.
As to the matter of personal speed preferences, a counterpoint: speed matters to me a lot, and I want to use a search engine like it's an extension of my brain. Case in point: one of the computers on my desktop uses duckduckgo for its search bar. It's noticeably slower than google for searches. Maybe .2 or .3 seconds vs .05 seconds. That's annoying. I notice it every time I search with DDG; it takes me out of the flow of whatever I'm doing that required some searchable information. I would guess that I'm not alone.
Google in particular is a data driven company. Most decisions are made by results from experiments. Google is able to do this because the economies of scale. They have tons of data. As a matter of fact, the "experimental traffic" is higher than 100%, indicating multiple levels of experiments are conducted in every search (on average). I remember the quote from a session "A/B testing is for marketers. Engineers run multivariate experiments."
In terms of traditional designers: their process is more of an art than science. They think, and trust their instincts. They traditionally operate in a setting that can't afford this kind of experiments.
The reality is people are weird and unpredictable. It's statistics and hard science when it's backed by data.
The challenge, of course, is that it is easy to iterate incrementally using measurement data. But for disruptive changes, although you could measure it, the change itself (the alternatives) is often not obvious.
Higher bounce rates, lower conversion, changes in dominant navigation path, lower average order value, etc. can be determined from A/B or multi-variate testing, and a lot of companies are much less interested in whether a user is confused than they are in whether any potential confusion is causing a reduction in CR% or AOV.
If the article's rant had argued "A/B or MV testing is a hill-climb and incremental changes backed only by test data virtually assures you'll find a local maxima, but only a daring design decision is capable of moving you to a different, higher, hill," I'd be more inclined to agree.
As it stands, it reads to me like a designer who is frustrated by not being able to scratch his own personal itches, without regard to the underlying business results: http://www.artlebedev.com/mandership/140/
You certainly can. You can measure bounce-rate, abandonment, etc. They are all proxies for the real thing, but are much better than anyone's instincts.
If your assertion that "you'd be challenged to find a single person on the internet who prefers the new Google News" is true, than more testing is exactly the right approach - which again is the opposite of the advice in this article.
With that said, I agree, I hate google Instant. It's a shiny object that may have limited shelf-life.
Personally I think Google Instant is great, and this guy's design suggestions for making it more readable pretty much gut it.
But hey, I'm an engineer, so I like efficiency.
I'm all for efficiency, but I prefer clarity.
The single most important factor Google should be concentrating upon is the quality of the search. If the search returns exactly what I'm looking for at the top of the first page, then I'm happy. I don't care if it took Google 3 seconds longer to produce that quality result. In fact, if by taking 3 seconds longer, the search result quality were to jump by 2x or 3x, that would be much better than trying to flash results in front of me as I type.
I don't see why this would be subject of cricism. User testing is the one and only validator for good design decisions. Every designer knows this, especially when the decisions closely pertain to User Experience.
A better approach would be "hire a professional designer, and generate some significant ideas, and then take an engineering approach to testing those ideas."
In other words: who should come up with the B's for those A/B tests-- engineers, or designers? (Or both?)
* Vertical scrollbar at width of 800.
* The dotted gray border right to the text is annoying (it flickers on my TFT).
* At lot of vertical space wasted on the left.
* I don't find it very pretty at a whole. (But that is of course subjective.)
I've designed apps for iPhone and Android and what ends up happening on iPhone apps is that you have to put additional buttons in your app to compensate for the lack of buttons on the iPhone.
Take the virtually omnipresent back button in iPhone apps. Most applications need a back button. Hence on the iPhone, they draw this button on the top left corner. I much prefer the Android phones' hard back button, not only because it makes single-handed operation possible, but also because sometimes soft back buttons don't cut it - what do you do when you want to switch to the last app or what if your app lost focus to another app because of some unsolicited event and you want to go back? Same for search button.
Now I'm not saying Android got all things right. In fact Android UI, UX is nowhere as pleasant as iPhone's. But, I don't agree that a single button on iPhone was a good decision. I'd like to know if there have been any real-life studies or empirical data on this.
I agree sometimes a back button seems like it would help, but after using iOS products for a long time (this is on an iPad, I don't miss hardware buttons much.