Expedia on how one extra data field can cost $12m | Hacker News Reader

Expedia on how one extra data field can cost $12m

silicon.com

88 pointsrobin_reala47 comments

To be fair there's absolutely no frickin' way that you could predict that people would enter their bank in place of their employer.

No, but there should be some way or another that those who create large, international websites should know that their websites are going to be used by non native english speakers :-)

Still there is lots of web forms with a Country field and a State field. And in the state field you'll only find American states.

Or 5 digit "zip" codes.

// I wonder how many Internet residents live in 90210? I know I do in a couple of places.

:) Yup that is my standard zip code when I need it. What I find even more frustrating is that some US stores ask for your zip code when you make a purchase. (Is it ok to refuse?). Being English, I just state that I don't live in the US, but it's not a practice I've ever encountered in the UK.

It usually confuses them if you refuse. When they ask, I live in the zip code 90210 and my phone number is 867-5309.

Oh that was mean mean mean. I don't know how long it will take that song to leave my head now :(

Yeah, I usually refuse, and they simply enter 99999 or some such. It's only for marketing research, so they don't actually care.

It's happened to me in the UK (and in Austria) from time to time. Mostly seems to be shops with a relatively large catchment area. (something like Ikea, though I'm actually not sure they have ever asked me) Asking for the UK postcode is of course pretty sinister as it narrows it down to a couple of houses. Sometimes they only ask for the first bit. Based on the reactions I've received, I doubt many people opt out.

It happens in Denmark quite a lot, but I usually just give them a real zip code I don't live in.

Exactly, which is why review and testing is so important.

In the abstract, this ought to be obvious.

But as it relates to this specific example, I don't see how to test it. Surely you're not going to just randomly remove fields from forms, to see if anything good happens. The only way I can see to deduce this particular problem would be if the site is storing all of the failed checkout entries. And I think that's the wrong thing to do in this case, because of the privacy implications

Who doesn't store failed checkouts ? - I can't see why you would consider that any more of a privacy violation than storing successful transactions.

Especially in the travel industry where credit card fraud is very high, storing failures is important for auditing/anti-fraud activities, not to mention customer support.

When a customer calls you to say "my transaction failed", and you respond "We don't have any record of your transaction" at best that makes you look unreliable and at worst the customer thinks you're calling them a liar.

Who doesn't store failed checkouts ? - I can't see why you would consider that any more of a privacy violation

It seems to me that a common failure mode would be entering a value into the wrong field, and that includes putting the credit card number into address or something. I want to do everything I can to avoid storing a credit card number (except for the one place it's saved, encrypted, at the customer's discretion).

The process doesn't have to be random. Nobody likes to fill out forms, so the goal is to make them as simple and short as possible. What value did "Company Name" add to this form? Someone thought it would be a good idea to collect this data, but I wouldn't be surprised to learn that it was never used in any way that contributed to the bottom line.

Well, the testing of that form might have been as simple as asking few people (ex. clients) to buy something and watching as they performed the task (and how they fill in the form). It could have been arranged for a few bucks less than that $12m. That magic approach is called usability testing.

On the other hand if they weren't trying to gather as much information about users as they possibly could this wouldn't even have been a problem in the first place. What possible benefit does an optional "Company" field provide?

"It would be great if we knew what percentage of our customers were business travelers. It would really improve marcomm.". "The web guys say they can ask, it won't take five minutes.". "Brilliant! Do it. Next order of business..."

Six years later, nobody has checked the form since. After all, that isn't their job.

In the event that there are several companies sharing the same address. E.g. my address here is <name>, <company name>, <building name>.

That $12m/yr might be added revenue, but it's hard to see how it could represent added profit (as claimed in the article).

Say Expedia earns $200 profit per transaction (total WAG, but intended to sound high). Then $12e6 means 60,000 people a year - or 165 people a day - were failing to complete their transactions because they thought "company" meant "bank name". Similarly, $500 profit per transaction means 66 people a day were thrown off by an input field that's actually pretty common on the web. Neither of those seems even vaguely plausible to me.

Take another tack. According to this article (http://www.reuters.com/article/idUSTRE69R56X20101028), Expedia had $177m net income this past quarter. That's framed as a big jump from the past, but say they make 4 x $175m or $700m a year. Before this change, they would have made $688m per year. So 12/688, or 1.7%, of their customers were being thrown off by the "company" field? I find that hard to believe.

If, on the other hand, that $12m is added revenue, then (taking the article's $6.89b revenue per quarter figure) we have 0.044% of their customers failing to complete due to this problem. That sounds a bit more plausible.

I don't know what's more surprising, that one small change is worth millions or that A/B testing is still news to some companies.

Wasn't there a famous post a few years back by amazon about their a/b tests resulting in 60million in sales increases, just by changing the button colors?

Everyone working on ecommerce sites of this scale should be expected to know about this stuff

I have spoken to a lot of companies who have A/B testing somewhere on their radar, but some combination of "That's nobody's job, so we haven't actually started doing it", (sometimes) organizational infighting (marketing wants to start, engineering has higher priorities and refuses to allocate resources to "twiddling text", etc), and uncertainty about outcomes means they don't start it yet.

One of my big tasks at some consulting clients is convincing people to buy into A/B testing. The easy way is to show them how their website is leaking staggering sums of money, and present A/B testing as the cheap, reliable solution to that problem in the future.

These are smart software companies who transact most of their business online, by the way. The broader economy which isn't primarily web-focused has many, many, many opportunities to improve their execution.

Yup, it's shocking. However, is this more of a problem with A/B testing or just bad user experience/design? I suppose an A/B test would've been to remove the Company field from the payment process and see how things have performed.

I think companies are more concerned with more obvious A/B tests (button color, page headings) than with testing large changes to their site or testing more in depth things like user experience.

A/B anything (testing or dual projects at once etc) is seen in some places as a "lack of skills, otherwise you would know beforehand".

(which I don't agree with, in case someone doubts!)

I don't know what's more surprising, that one small change is worth millions or that A/B testing is still news to some companies.

Yeah, my company has been doing A/B (we call it Champion/Challenger) testing for years.

This wasn't A/B testing. This was fixing UI mistakes based on UX analysis. There was no 'B'. They weren't comparing A and B.

In this case, the A/B testing would have been with that field and without.

How would you know which fields to use?

UX analysis and A/B testing go hand in hand. There'd be no way to know that the Company field was a UI mistake if it had not been tested.

A/B testing probably wouldn't have caught this.

It occurs to me that instead of removing the field (which I'm sure people who used Company credits cards are looking for) they could simply have made it clear that it was the user's billing address they were looking for, instead of the bank's.

It might have resulted in more revenue if they had done it my way, since there may be people on business trips that now think they can't use a company credit card.

Or not. That's the thing with dealing with the public... Without paying really close attention to what's really happening and people are saying, you are just guessing.

They don't really care about the companies name or if it's a company credit card.

What they should do is a randomized control trial in which they either keep the field and make its meaning more explicit or get rid of the field at all.

Then they should look at the differences in sales by comparing the two groups.

Nothing wrong with being a bit academic..

Problem is, that this field is question is actually useful for some subset of clients. Making the intent of both "Company" and "billing address" fields is probably most meaningful way to go, possibly hiding the "Company" field behind checkbox like "Bill my employer".

For those curious Expedia actually captures a stupendous amount of information down to session level activity. They can play back failed sessions to discover areas of opportunity. This was discussed at an analytics conference I attended on how they implement testing across a range of systems to find possible areas of friction and improve the business.

Convenience is king and this is an excellent example of how easy it is to go too far when collecting data from users. I already expect to jump through a few hoops when using a credit card online, so asking me for my life story will only encourage me to bail out and never return.

You've clearly not comprehended the subject article - I'd suggest reading it again.

I did read the article, and the title says it all. They lost money because they got greedy for data that was unnecessary to complete the transaction. As a result, another point of failure was added and frustrated users bailed. The body of the article may try to spin it as a case where analytics boosted the bottom line, but the truth is that it was a bad design choice that cost them money from the start, because they ignored the KISS principle.

According to the article the sales they lose were due to customers who didn't understand the form and therefore filled in the wrong information, not because people couldn't be bother to provide the data.

I wonder what the percentage was of customers who made that mistake and then corrected it in a second try. The increase in sales after correcting the problem implies there were plenty who didn't, but how many were stubborn enough to try again after making the bank mistake?

Don't forget Amazon's $300,000,000 button:

http://www.uie.com/articles/three_hund_million_button/

Now, if only they fix all the other issues on their site. I tried twice at the weekend to make a booking and the whole convoluted process simply did not work (saying there is errors on the form even if there was not etc...).

Confusing customers costs money: news at 11

While designing the database, it is better to have a notes column in every table so that we can use it in future without changing the schema and breaking the application.

Wow... seriously?

Why not? I'm a big proponent of the SQL K-X model. You have two text fields in each table: key and xml. The flexibility of this model is incredible.

Can you go into some more detail to explain what you mean? I completely don't understand.

Reply on news.ycombinator.com