Hacker News Reader

Comment by brookside | Hacker News Reader

I have never understood how the Consumer Reports model of self(reader)-reported reliability can produce can statistically valid results.

There are many biases build in to self-reporting surveys. Longtime Consumer Reports readers who buy a Toyota because they believe it is the most reliable make want their choice to have been correct. Their answers about problems with their car will reflect that.

View on HN

There was a time when CR regularly rated Buick at the top of the charts. At that time, my mom and all her little old lady friends (I'm not using those adjectives lightly) had Buicks and literally used them to drive to Church on Sundays (and occasionally the grocery store, which was about five blocks away). The cars were terrible, but they were treated well.

CR also uses their own tests and maintenance records.

There are definitely flaws with self reporting but it’s just a data source with its limitations.

Statistical validity is based on what is done with the data, not whether it’s self reported or measured accurately. What you may be thinking of is whether their study or logic is sound.

I wish there was something more objective used like to quantify how many vehicles are still on the road after X number years. I guarantee Toyota/Lexus/Honda/Acura would be at the top if you compared how many vehicles would still be daily drivers after 25+ years and 250,000+ miles.

That, in my opinion, best describes a BMW owner. Is it a more reliable car than Alfa? Well, if you service it at the higher requirement and price that a BMW is most often services at then maybe it is.

Data around cars is notoriously complete bullshit. One example is that one of the main metrics used by a very big name, maybe CR, is “reliability compared to *expectations*”. Obviously cars that people have high expectations for (Teslas) are going to score horribly here even if they are objectively more reliable than other brands. And of course they bury this little fact way down in the info and everyone walks away with “welp teslas just fucking break all the time”

The linked article is pretty explicit about how they calculate it. A detailed explanation is literally the first section.

And as they explain, they ask people about the actual problems they've had in the last 12 months with their car; nothing soft or fuzzy or relative to expectations at all.

If I ask you about your experience with a Rolls Royce you'd probably mention in the door rattles a little bit between the speeds of 55 and 60mph. If I ask the same question about an entry level Toyota I doubt people would think anything of it or mention it. This is especially impacted if you're used to buying cars that cost $75k (a lot of Tesla owners) versus used to owning preowned cars bought for $10k

Reply on news.ycombinator.com