Vertical vs. grid product listings - a surprising AB test
westiseast.co.uk
westiseast.co.uk
Testing multiple variables in a given trial makes it impossible to discover which change (or combination thereof) actually improved the conversion rate.
From experience though, an 'uglier' site getting more clicks is actually fairly common. I've seen the results on 100,000's of visitors across a multitude of landers. The rationalization that seems most common is that people click to 'get away' or get to whatever they want.
So which format resulted in a higher revenue? You'd be silly to make the assumption that greater click through = greater revenue.
Although in this case, my justification is - the 'distance' between this product listing and an actual purchase is maybe 4 steps total (ie. view product, add to basket, give details, pay), and there are decreasing response percentages at each step. If I tried to measure actual sales with my current tool the test would take much much longer to conclude. Rest assured, I've got AB tests running elsewhere in the process too :)
People could have been clicking through more because one was better at conveying information than the other, so they didn't need to click through to make a purchasing decision. Maybe the list meant that they ignored the text?
What if the end result is you still only sold 20 boxes of tea for each? Your assumption would then be wrong, the grid would be better because it's serving less page views.
What if because the consumer had to click around more you actually had less sales? They took too long to find what they wanted?
"What if because the consumer had to click around more you actually had less sales?"
That's very difficult to measure-- you can't assume that a customer at the browse portion of the funnel is able to find what they're looking for or not, just by looking at the number. You might want to look at the number of add-to-cart actions per product view for a better answer to that question.
Performance is based upon how well the conversion funnel performs. Revenue of course matters, but the way you get that revenue is to get people to go through the conversion funnel. What this A/B test shows, though crudely, is one optimization of one portion of the funnel. When you get more people to click on, say, a product page, you're "widening the funnel" at that particular point.
It's entirely possible to widen the funnel higher up, and still have the same amount of revenue; however, this doesn't mean that it was a useless optimization. Whenever you widen the funnel, you get the potential for higher revenue, especially with further optimizations. That makes for an increase in performance.
Lastly, widening the funnel at any point also means that you are getting increased page views, increased product views, and increased stickiness. That is _always_ a good thing, regardless of whether you increase your revenue or not. It means that people are more likely to interact, and buy from, your site in the future-- users are naturally more likely to go to a site that they know. So, greater click throughs to a product page, though you may not immediately see an effect, will almost always lead to greater revenue.
For example, would you rather have 1000 people through the funnel with a 1% average likelihood of purchasing (for an expected 10 sales) or 100 people through the funnel with a 50% average likelihood of purchasing (for an expected 50 sales).
It may well be that you can get more people through the funnel but it doesn't make it a good optimisation if the people you are enticing are unlikely to buy anything from you.
Real world example: Apple products. Apple specifically target the top end of consumers (narrow funnel) who are happy to pay a little bit more, whereas other producers go for the mass market (wider funnel) sales at a lower margin.
That isn't 15% better. That's a 60% increase in click-through rate. That's massive.
We had already planned to remove a grid display from our app and replace it with a linear flow, but this seals the deal. Thanks!
There's basic underlying usability rules that permeate through everything.
The human eye scans like an F.
It's almost the golden rule of design. Always build your layouts in an F structure for the most important items.
A list design goes hand in hand with complimenting the F average.
I too guessed and have always believed vertical listings are "more natural" way of consuming information than the grid listings. Grids are too overwhelming to digest any of the information and make a decision. They are probably good to browse sometimes.
I also wanted to say that a common perception that prettier is better is rarely true. Google is a great example and so was Plentyoffish. Utility and usability pretty much make the biggest difference and can make up for lack of pretty design. Infact sometimes pretty design is often not the most usable design.
A prime example is yahoo.com VS yahoo.co.jp
White space is an important tool in controlling that density.
In cases where there is enough visual difference between the products so that they can be distinguished between without any text, then I think a grid is probably a better layout since you can compare more products at a time. So while the results are certainly interesting, I don't think I would try to generalize them too far.
I'm used to looking at very visual samples in a grid format because it allows for more large images to be shown in a row. Things like furniture, clothing, templates... For things that require some text descriptions (such as tech specs), I think a list view is superior.
Simple example: if I have a study of 20,000 people and I find that 51% of people prefer foos to bars, that may be similarly significant to your study of 20 people, all of which prefer foos to bars.
I just used this:
http://www.prconline.com/education/tools/statsignificance/in...
The author should plug the numbers in here: http://www.usereffect.com/split-test-calculator
I'd go as far as to consider the data to be worthless without the confidence value
A multi-dimensional grid takes you in a Z shape, whereas a vertical lineup's images and text are all aligned and easier to distinguish.
I think I'd overestimated "pretty" and underestimated "functional" in this case, so the results were a wakeup.
This paper from Yahoo Research contains an interesting analysis of the problem:
Optimizing Two-Dimensional Search Results Presentation
http://www.chierichetti.name/papers/optimizing2d.pdf?attredi...
I'd still expect vertical to do better even if the grid was equally well executed, but reality beats expectations.
All a function of how many products you have really, ideally your sending people to pages that are focused enough that they don't need to trawl through many products.
Also the eye is attracted first to the product images then moves naturally to the right to read the description, with the matrix the eye has to move less naturally downward to read the related product info.
Amazon grids don't always use images that are quite as effective at indicating discrete products as the book-cover screenshot you've linked (just try browsing through headphones), but for a surprisingly sizable portion of their catalogue image-oriented browsing is both effective and enjoyable.
People like looking at pictures - but showing effectively the same picture over and over again for different products makes me (as a user) feel like I'm wasting time at best, mislead at worst ("the items in this list of unique things can't possibly all look the same in person, can they?")
But with teas, it's perhaps not so obvious, so the vertical listing seems to work better to allow people to choose.
If your experience of Chinese tea is getting a pot of Jasmine in a restaurant, try these, it will be a revelation.
Edit: It looks like there may be exceptions to the grid view within departments (see "Amazon Instant Video", "Books", and "Video Games").