Hacker News Reader

Comment by callmeal | Hacker News Reader

Wirecutter is a pay-to-play operation. I stopped heeding their recommendations after I discovered that they were aggressively persuing kickbacks and refusing to review products that do not pay. See https://www.xdesk.com/wirecutter-standing-desk-review-pay-to... for example.

Also note that wirecutter in their response do not deny this but try to weasel out by claiming that the word 'kickback' was misleading.

https://www.nytimes.com/wirecutter/our-response-to-nextdesk/

  One valid criticism NextDesk raised was our use of the word “kickback” in our business communications, which is a misleading description of the affiliate business model because it implies an illicit transaction. In our company’s early days, we misused the term to describe a straightforward affiliate relationship, but we have since changed how we talk about the affiliate business, which is one we continue to stand behind.

View on HN

I wanted to link to an older comment (https://news.ycombinator.com/item?id=24714286) I made on this topic, because I keep on seeing this article. It's turned out to be one of my most upvoted comments on HN, to my genuine surprise.

Reproduction:

I keep on seeing this link pop up. Since no one is replying, I'd like to point out that I think the Wirecutter is actually in the right here. Another HN member did some investigation and found that Xdesk is stretching things: https://news.ycombinator.com/item?id=22144078

In general, having been able to talk with some of the people there, I'm convinced that WC was focused first and foremost on truth-seeking and quality at this point in their life (pre-acquisition) — however, the consensus seems to have been that after the NYT acquired them, they started becoming more incentivized to grow revenue, and started to jump the shark.

> I stopped heeding their recommendations after I discovered that they were aggressively persuing kickbacks and refusing to review products that do not pay.

Let me offer a counterexample - VPN services. They didn’t get a cent from Mullvad, which they declared the winner. Number two was IVPN, which also doesn’t have an affiliate program.

The review: https://www.nytimes.com/wirecutter/reviews/best-vpn-service/

While my experience with Wirecutter has been decidedly mixed, Mullvad really does seem to be the best VPN around.

Thank you!

Ahh I didn't realize you were one of the founders of Mullvad! Thanks for all your work :)

Why do you think they didn't get money from them?

Mullvad: I know it for a fact, because I'm one of the founders.

IVPN: I don't know it, but I've met Nick Pestell (the founder) several times, and he's told me they don't pay for reviews. I trust him.

I have an abandoned tech blog and I usually get asked by marketing firms to include links in old articles as part of SEO services they have.

I hope you enjoy the random cards of cash I send every month! I quite enjoy your product!

Ahh, this is why one must be careful when responding to seemingly random people on hacker news, there's a pretty good chance the CEO, head researcher , worlds fore-most expert on the topic, etc. is actually here reading the article.

To be fair I think is_true's question is excellent. If anything I should have clarified where I was coming from in my first post. Ultimately I opted for keeping it shorter.

I agree though, it is a real joy to lurk a HN thread on J. Random topic and be treated to high-signal conversations. :)

>Mullvad: I know it for a fact, because I'm one of the founders.

I cant stop laughing :)

Forgive me if I have asked this before. Have you ever considered a Pay as you Go model with minimum top up fees? $50 for 50GB with no expiry you can use it for life?

Or is that model that generally dont work because it seems all PAYG VPN services go bust.

> Forgive me if I have asked this before. Have you ever considered a Pay as you Go model with minimum top up fees? $50 for 50GB with no expiry you can use it for life?

Yes, but I can't recall the reasons we discarded the idea. Something like this perhaps:

In general I like the idea of VPN as a utility - anonymous internet on tap. Utility-like services in functioning markets compete on price to the benefit of the consumer. On the other hand, things that look like one-dimensional utilities to the consumer isn't always. A service category where there's plenty of innovation potential left shouldn't pretend to be a utility. Especially players who aim to improve state-of-the-art. A pay-as-you-go model encourages consumers to compare quantities of something that is quite disconnected from the quality of what they get.

Paying per month makes more sense to me for another reason as well - It's similar to what we do with our suppliers.

> A pay-as-you-go model encourages consumers to compare quantities of something that is quite disconnected from the quality of what they get.

Thanks, that make sense. The reason I asked is because I only need a VPN once every few months for 10-20 min. Currently I am basically setting up my own VPN Server every time I need one. It is just quite a bit of hassle.

That they didn’t do this for a single product category doesn’t mean very much, though.

It does mean something. I can only testify to how they acted in our case of course, but what I can contribute is this:

In a product category which is famous for its high-paying affiliate programs, they chose two players who are smaller and less well-known. Neither of them paid for their reviews. Most of their competitors mentioned in the article do have affiliate programs.

Well, to be fair, their guides focus much more on physical products than SaaS.

They can get third party affiliate revenue from Amazon, Walmart, Home Depot, etc in a program covering lots of products. Is there a B&M store analogy for SaaS?

[The rest of this comment is in response to parent comments more than your's directly.]

IMO The xdesk guy seems to be stretching reality way, way, way too far to claim that Wirecutter guides are influenced by whether or not affiliate revenue is offered on the purchase links just because Wirecutter repeatedly tried to get an affiliate program in place. Of course, they need a business model of some sort, and affiliate programs are what most, or at least many, review guide sites seem to choose.

Also, Wirecutter has never specialized in recommending the highest end options. A lot of their guides recommend a best choice for most people at some tradeoff of quality and cost. It's no surprise at all that the standing desk guide was iterated in that direction. And, even after no affiliate program, they didn't dis-recommend the xdesk desk, they just made it an upgrade pick recommendation.

Personally, I am skeptical that a typical Wirecutter reader is actually interested in spending $1500 on a great standing when they can get a pretty nice one for $500. The update makes reasonable sense to me.

It's worth noting that this whole thing is from 2013/2014. Since then, Wirecutter has been sold to the New York Times and has completely different management. I don't personally think Brian Lam did what Nextdesk accused him of (disclosure: I've met Lam socially), but I also understand why it seems improper to have that kind of outreach.

Based on my personal knowledge of many people who have worked at Wirecutter past and present, I don't believe they are a pay to play operation at all.

Regardless, that has no bearing on whether or not their employees are owed a fair labor contract or not.

The whole affiliate marketing space is a dirty business. Saying affiliate relationship is just softer language for kickback IMO. The incentives are the same. I think all reviews with affiliate links should have to be labeled as paid promotion.

This [1] sleepopolis saga is an amazing read if you’ve never seen it.

1. https://www.fastcompany.com/3065928/sleepopolis-casper-blogg...

We need a new mattress, after about half an hour of searching for reviews I realised that there was no way I was going to find any genuinely honest reviews anywhere. Search results are packed with full-time mattress review sites that are all clearly getting affiliate payments of some form or other, many of them seem to have the same content.

There might be some honest ones out there, but they are drowned out by all the ones that are clearly just affiliat link farms.

Maybe we should have like… a place you can go feel them in person. Like a building where there would be a few of each model and you can test them BEFORE you buy one?

/s. Mattress stores and mattress shopping is the fucking worst. Ultra scammy and there is a clear reason that the industry was turned upside down once someone figured out you can fit a mattress in a box and ship it.

The last time I went mattress shopping, I was told by serious and expert sleep professionals [sic] that I should get a mattress with embedded diamond particles because it’ll help keep me cool, and silver thread at the seams that will be more hygienic.

For what it’s worth, I took a chance on a mattress and love it. I have never slept somewhere I like better. It’s an Avacado, but buy whatever you can that has a real return policy and try it out.

Yep! I don't know what it is about mattresses specifically, but every part of the industry from highstreet shops to the online products all make me feel like I'm being ripped off in a way that no other product does.

Bed salesmen are worse than car salesmen.

There was a 2018 bit on Planet Money about mattress stores (and specifically Mattress Firm): https://www.npr.org/transcripts/676543180

tl;dr - Turns out the markup at a mattress store is about 100%, which helps the economics of store rent vs profit. Also, Mattress Firm went on a debt-leveraged buying spree of their competitors (why there are so many physical stores). Oh, and also Steinhoff (think South African IKEA) bought them in 2016, for what seemed an inflated price. And it turned out Steinhoff was (or was soon to be) under investigation for accounting fraud: https://en.wikipedia.org/wiki/Steinhoff_International#Contro...

That explains a lot. I must have missed that episode, thanks for the pointer.

Consumer Reports reviews mattresses, and they're independent and (imo) very trustworthy. A subscription costs money but some local libraries offer access.

At least with CR, their subscriptions are pretty trivial to cancel, and I think they even pro-rate the refunds.

I don't subscribe to them on a regular basis (I don't buy things often), but I have subscribed to them a few times to make a better informed decision on an upcoming purchase.

> There might be some honest ones out there, but they are drowned out by all the ones that are clearly just affiliat link farms.

I think the most interesting part of your comment: Is how do we [objectively] classify the two?

Perhaps the tendency to recommend products not recommended in spammier affiliate link farm guides is one metric that would be interesting to look at.

In comparison to WC, CR is not free; but, to play devil's advocate, [why] does a guide being non-free presuppose that their reviews are of higher integrity?

[In general, I don't think that it does. We just had a post about this the other day actually.]

In CR's case, they've been around for decades and are independent / non-profit, so that's something. But also, I don't think longevity and the legal business entity being a non-profit are necessary preconditions to writing review guides with integrity.

If we were trying to find the "CR of startups today", what would we look for?

I don't understand the problem with affiliate links, esp in this case. Wirecutter's recommendations, budget pick, and also great items are often all through Amazon, which as the largest online retailer isn't suprising.

It isn't like they're recommending some brand over another because Amazon pays them for the click-through traffic. I don't see the kickback connection here.

That said, I largely like Wirecutter recs - but also do external research. When those things align I've almost always been very happy with my purchase.

That's a pretty sensationalist article. Wirecutter doesn't hide the fact that they make money off of affiliate fees. Almost all content sites do.

Are there better review websites than the Wirecutter? I've found them to be solid and helpful.

I like RTings, but they only review a limited selection of products. Consumer Reports is not what it used to be, and I always wonder if they're about to go out of business. I'm not aware of anything better, though I'd love to hear other people's recommendations.

I have wondered if there is even a way to sustain an organization large enough to review a wide variety of products, and have a business model that does not raise questions about your motives or integrity. I'd guess that such a business would need a lot of people and time to comprehensively review products well, and that costs money. In the Wirecutter model, where they claim to keep their reviews up to date as new products come out, it's kind of a recurring cost, too. But if affiliate links are ethically dubious, and advertisements are even worse, and if not enough people will pay for a subscription to their site, then what is the alternative?

> I have wondered if there is even a way to sustain an organization large enough to review a wide variety of products, and have a business model that does not raise questions about your motives or integrity.

Costs: {average number of new products released per unit of time} * {cost per product}

Revenue: {volume of purchases per unit of time} * {affiliate marketing per product purchased}

Advertising is probably difficult, due to the "Buy something other than what we recommend!" issue with advertising on review sites. Although I guess there are options for cross-selling.

I can't believe there are many products that makes sense for, without the Consumer Reports "Pay for the report" model. And even with it, the economics seem dubious, and I feel like you'd always be barely getting by.

At least originally, they presented it more as "we link to large retailers and get affiliate fees, so we don't have a motivation to recommend any specific product in particular over competitors". Which is fair enough I guess, but "we want affiliate fees from you, if you disagree we downrank your product" is quite different.

Eh, correlation ≠ causation.

It takes a huge leap of faith to claim that their product went from main recommendation to upgrade recommendation (not really a downrank) because they don't have an affiliate program.

To me, the xdesk guy is reading extremely hard into two independent events to try to claim cause-and-effect where it doesn't really exist.

Their pricepoint is high and the standing desk space has commoditized a lot. There are plenty of good affordable options these days. It sounds like they just make a niche high-end product; I was surprised to hear it was Wirecutter's initial recommendation for most people in the first place — I don't know too many people that read Wirecutter guides and are also looking to spend $1,500 on a standing desk.

Fair enough, but the suggestion that it is a motivation is always there once they leave the range of products that are well-captured by generic affiliate programs.

Oh shit. I used to consider them to be one of the most reputable sources.

Oh no! I did as well.

Who is the current honest broker of product reviews?

Anything that contains affiliate, referral, or commission links I do not trust.

What monetization models do you trust more?

I trust publications with product-independent ads and paid articles. Generally these publication shouldn’t profit from promoting a particular product. That can include free samples and any dependence on a manufacturer’s good will.

An example for relatively trustworthy reviews would be (German) Stiftung Warentest (“Product Review Foundation”)’s https://test.de, you can subscribe to all articles for a monthly fee or buy them individually.

No clickbait, no pop-ups, just no-bullshit, user respecting, fairly objective tests.

Seems like it's time to go back to Consumer Reports.

How about The Strategist?

Complaint threads on reddit probably. I don't have to watch video game reviews anymore at least, I just wait for the community on reddit to explode positively or negatively towards the game developer then I make my purchase or not. If you only went off the puff pieces in game reviews you'd be buying every title to come out. This works for a lot of stuff thats popular on reddit at least. Not sure where else I can find good communities online of people complaining and calling out shortcomings in products.

googling "$product_name sucks" or similar

What does this have to do with The Wirecutter employees trying to get a fair contract with The Times?

> Also note that wirecutter in their response do not deny this but try to weasel out by claiming that the word 'kickback' was misleading.

They've been getting a kickback from affiliated links for years.

You are late to the party. There was a thread a while ago debunking some of the misinformation in that case https://news.ycombinator.com/item?id=22141719

Is there any site that you do use now for recommendations? In the game of capitalism it seems everybody becomes a sell-out.

Consumer Reports has a long history of subscriber-funded independent reviews. They refuse all advertising, free/discounted items, etc. to prevent conflicts of interest. They review everything from cars and lawnmowers to mattresses and electronics, but depending on what niche of electronics you're looking at they may or may not have the kind of reviews you're looking for.

https://www.consumerreports.org

consumer reports is a great resource when you just want an example of X that isn't a complete lemon. that's not a criticism, most purchases are like that for most people.

but if you care about getting a really good X, they're not a great source. most of their buying guides have a lot less detail than Wirecutter's, and of course pale in comparison to dedicated review sites for popular niches.

it's unfortunate that you essentially have to trade trustworthiness for detail.

Consumer Reports has less prose, but much more objective data - in sortable tables. The data is from their labs, where they have domain experts using the scientific method to evaluate products.

Available in many libraries.

They rate products in a bunch of categories using a table and a circle which would be various levels of full color depending on how a product rated. which meant sometimes you wouldn't just go to the highest rated if one part of the product scored lower in a category. Its an interesting data visualization method. Plus the table usually has specs too.

https://www.consumerreports.org/consumer-reports/we-put-ours...

As long as the categories can be boiled down into reasonable, discrete levels, I'm a huge fan of radar charts for quick comparisons. They communicate multivariate differences pretty well, compared to alternatives. https://en.wikipedia.org/wiki/Radar_chart

Unfortunately, they also require summarization, so they're only as good as the individual quality projections into discrete levels. Powerful when used correctly. Useless when incorrectly, or your audience isn't visually-oriented.

As far as I'm concerned, America's Test Kitchen is the gold standard, with Consumer Reports just a little behind them.

Reply on news.ycombinator.com