Hacker News Reader

Comment by boringg | Hacker News Reader

Interesting product the problem with the model is if this ever takes off then the underlying system will be gamed. If it doesn't then its just affiliate advertising in another mechanism putting the reviews of other people to use for the website owner.

It's like cream skimming for funding programs which then leaves the core product worse off. It's extractive in nature.

View on HN

This system is already being gamed, and it has been for years. You can find comments on Reddit by marketers who describe how they create organic-looking accounts, and then find recommendation threads and promote their brands there.

People know how to write content that looks organic. Some people get caught, but others are fairly skilled at slipping marketing material into ordinary threads. I'm not even talking about the karma-farming bots out there--it's much easier for a bot to farm karma on Reddit than for an unaided person.

We will soon see a post titled “HN’s favorite products in one place”.

I'd love to see this purely out of curiosity.

There should be a way to see the list exactly as it was on the day the list app was released, avoiding any gaming of it.

They already did that, it was called Product Hunt, and it quickly became a landfill of low-quality side projects .

And self-congratulation of indiehacker networks. Those making $10k-$20k moving to twitter though, tweets like guru now. Those making $40k+ becomes business guys who talk un-related stuff for those who are finding good content and product.

As someone who's struggling on market research right now, the current situation of these site is pretty bad, hard to get real insight.

I stopped reading IndieHackers the moment it was acquired by Stripe. It's nothing more than a marketing tool now, and the quality of the content has shifted accordingly.

This exists, in a way, but it's not all in one place

This one comes to mind: https://hackernewsbooks.com/ https://news.ycombinator.com/item?id=12365693

I think in niche subreddits are pretty resilient to this kind of gorilla marketing, at least in my experience. Folks who join them are usually pretty vocal when they've used a product and had good or bad experiences with it.

Even in large niche subreddits like /r/coffee, /r/homebrewing, and /r/skincareaddiction which are all over 1m followers, there's pretty good self-moderation and calling out of astroturfing. /r/skincareaddiction might be the most susceptible to it since there are just so many products and skincare is pretty specific to the person, but I still see folks calling out shill posts and comments.

I don't think this is accurate.

People call out astroturfing, sure. But when do they call it out? When it's obvious.

Sometimes the callouts won't be apparent. Someone will PM the moderator, the mod will review and remove that post.

Obviously, I only have anecdotal evidence. But I see it regularly happen on the subreddits I'm a part of.

Equally anecdotal--I've seen lots of examples of astroturfing that hasn't been caught. People who run social media marketing accounts have occasionally explained exactly how they create their marketing accounts and how they make marketing posts. The account will have legitimate Reddit activity--basically, because somebody will be using it as a personal account. And when someone asks for recommendations for hiking boots or whatever? "I've been using X for a while now, they seem pretty reliable." Anyone looking at the comment history or post history will see what looks like a normal Reddit account.

How much of the astroturfing is called out? 10%? 90%? Certainly nobody thinks it's 100%...

    The account will have legitimate Reddit activity--basically, 
    because somebody will be using it as a personal account. 
    And when someone asks for recommendations for hiking boots 
    or whatever? "I've been using X for a while now, they seem 
    pretty reliable."

This would definitely work, but it seems like a large amount of work for a single organic-looking "I've been using X for a while now, they seem pretty reliable" post.

But at what point is "marketing" just engaging with the community? If you go out and develop a suberb product that some niche reddit community might love, you post it...and they like it... OK, good for you?

What I am saying is, if your product / marketing is sooo good that it passes the smell test of the relevant subreddits..maybe your not shilling / astroturfing but helping?

Similar to how a lot of SEO advise is: "Create good content". Where is the line here?

Just like it's really helpful for a waiter at a restaurant to suggest you get the <product with largest margin that is about to expire> if they, without you knowing, get a cut of the savings for doing so!

How great

If you are a member of these forums over time you see the same products / etc. get recommended.

For example, the /r/coffee forum always recommends 1-3 particular espresso machines. People are posting hacks where they add arduinos, etc. to control it better. There are github repos to describe all of this. Either its an incredibly elaborate marketing ploy (to sell a couple more $500 espresso machines..) or its organic content.

Also, redditers are incredibly harsh when shilling, etc. is discovered. In the /r/diablo2 forums, a moderator started a Discord for everyone (to play and trade items for Diablo2 Resurrected). He posted an ad (in the Discord) for a site to buy items (frowned upon in /r/diablo2). Redditors did a thorough investigation and kicked this guy out of the sub. So his $100 of profit from that ad lost him a moderator spot and his Discord community.

The risk/reward of sneaky marketing on Reddit is just not there (for many scenarios, I'm sure it still happens sometimes).

> redditers are incredibly harsh when shilling, etc. is discovered.

Ehhh that depends the subreddit, big corporate products as Disney are highly astroturfed and get downvoted in to oblivion if you call them out. That's why you see alternate subreddit as r/saltierthancrait and r/freefolk.

Their astroturfing gets more agresive as they lurk and comments on those subreddits too.

I don't think it's one size fits all. I moderate a niche hobby subreddit and we have been fighting astroturfing for years now. We're over 5,000 subscribers but this has been a problem for the mod team since we were around 1,500 or so.

I actually think if we were larger we'd have more subscribers who even knew what astroturfing is, how to spot it, and why it's important to shut it down.

Once products take hold as being "good" according to the groupthink Reddit's strong rewards for parroting groupthink make it almost impossible to go against the grain and get heard to listen to anything negative.

Anyone complaining about M18 tools, Toyota vehicle, Kitchenaid mixers, speed queen washers, etc, etc will find their comment shit upon by people looking for a few quick virtue points for dunking on the dissenter and downvoted to oblivion.

Try shitting on javascript or react in /r/webdev your karma will be heading downhill even though you only speak fundamentals.

Meh, maybe “shitting on” a piece of tech just isn’t that interesting and your points aren’t as novel as you think? “Javascript bad amiright guys? :D” isn’t even so welcome here anymore.

Nah, I'm not "this thing sucks" kind of guy. More like urghh it's too late now to speak out about "this is unhealthy, don't you see, it needs fixing, shall we?".

Simplifying issues into kids' fistfight isn't even so welcome anywhere.

This perfectly sums up the culture of reddit.

fwiw s/gorilla/guerrilla/

Nah, there are literal gorillas doing the marketing. Haven't you ever heard of the infinite monkey theorem?

But the marketers are monkeying around, swinging from thread to thread. Pounding their chest and hooting.

/r/coffee is a never ending stream of expensive paraphernalia. I don’t know how you’d tell the difference.

Looking at the top 20 hot posts on /r/coffee right now the only one that has to do with expensive equipment is 1 post where someone is asking if the Mazzer Super Jolly that they got from a business is overkill to use at home [1] (it is but if you got it for cheap then might as well use it) and another post about a calibrating a refractometer [2]. The rest are questions about tasting coffee and troubleshooting cheap pieces of equipment (pour over brewers that are all <$40, moka pots, french presses, etc.).

[1] https://www.reddit.com/r/Coffee/comments/x7lo4x/mazzer_super... [2] https://www.reddit.com/r/Coffee/comments/x75a35/refractomete...

If it’s posible to recommend a product earnestly, then you must take for granted that it’s possible to recommend a product dishonestly.

There’s no way to tell those two things apart when done well.

Catching the obvious cases only means you’ve caught the obvious cases. Reddit is really naive here. They’ll punish “look what I made” but not “look what I found” . We did it, Reddit! We beat ads!

> Catching the obvious cases only means you’ve caught the obvious cases. Reddit is really naive here. They’ll punish “look what I made” but not “look what I found” . We did it, Reddit! We beat ads!

Also catching the obvious cases can easily create an illusion of competence, which quickly leads to overconfidence.

    If it’s posible to recommend a product earnestly, 
    then you must take for granted that it’s possible 
    to recommend a product dishonestly.
    
    There’s no way to tell those two things apart when 
    done well.

This is true for some things, less true others.

For products that are almost purely subjective (books, movies, etc) yeah. There's really no way to spot well-concealed astroturfing.

As things become more objective the astroturfing gets a little bit difficult.

If I'm an astroturfer and I say, "I think brand XYZ leaf blowers are the best" but unless I go into some details that recommendation is not going to carry a lot of weight. And if I falsify objective details, things will start to smell more and more like astroturf to anybody that is actually knowledgeable.

Reddit in particular with its very strong groupthink/circle jerk loop is really susceptible to this. Marketer's content gets popular/upvoted, then people parrot it, they get upvoted, repeat.

Recently, I successfully shut a 10,000+ karma reddit user on /r/webdev up because he spreads false information on particular html spec (that comment got 10 upvotes, unbelievable).

True. I've been reading about CMS/headless CMS on reddit especially /r/webdev for the last 2 years. Not authentic at all. Even if it is, reddit population seems to be very outsider (can't find word to replace "newbie"), though it also depends on particular sub. /r/asianbeauty is quite authentic but lots of products aren't there, they seem to have discovery problem. Thus, reddit is good for market research where you wanted to read/hear newbie opinion (the mass).

Reddit has a lot of very senior newbies in a lot of its subs. That's the way I describe what I think you're saying.

There are some subs with great expertise, but a lot of it gets drowned out by the former.

I think pro level of particular topic tend to have their own website with articles / videos. So they don't need inbound by posting reddit so often.

Cough cough...Strapi...

Yes, those who got load of VC money to comfortably allocate some huge marketing budgets.

I think the moderators could easily prevent this. Just block comments from new accounts or accounts with karma less than some value.

Karma-farming bots exist. It's trivial to make bot accounts and farm karma on meme subs until you're over the threshold, and then go and comment in product-related subreddits.

Technical solutions are extremely difficult. A verified purchase mechanism is difficult to do unless you control the purchase platform.

The "easiest" solution is a web of trust, but that has a significant non-technical component by definition.

I'm sure the reason is "too complicated and not worth it" but I've always been surprised that Reddit doesn't have per-sub karma.

It does and you can limit posters to in-sub karma. afaik the only way to implement that solution is to write a custom bot though. Reddit's tools for moderation are basically garbage.

The Reddit karma algorithm is already a black box, so there's not much harm in making it more complex, thus:

Measure the general up-votiness of each sub, and scale cross-sub karma based on this value. A +100 post on a sub with 1000 members on a technical topic where people tend to downvote for minor errors is worth way more than a +1000 post on a fast-moving sub that's mostly reposted memes.

It is never as simple as "just" doing something. A lot of people have multiple Reddit accounts because it is a forum for everything. Then next is how to determine that arbitrary karma value, what if a lurker has some valuable experience to share? Besides, these are relatively low barriers to entry. Check @weird-eye-issue anecdote below.

You’re only thinking of obvious spam. You need to think bigger to ads that fly under everyone’s radar. You don’t need to burn accounts to do that.

You can even buy reddit accounts that are aged(>1 year for like 5 dollars)

The catch here is that real users also make recommendations. I recommend things on reddit all the time, and I'm not some nefarious marketer.

I've also had a lot of good stuff recommended to me on reddit over the years. If that was by a marketer, then frankly, the best marketing in the world is "hire subject matter experts to go into amateur forums and recommend good solutions for them" because that's what it feels like I'm getting on a lot of good nitty gritty subreddits.

Yeah, I make recommendations on Reddit fairly often. I try to provide enough detail to make it clear that I’ve actually used the product in question, hopefully that helps separate my posts from those of astroturfers.

I agree. I'd even go a step further - I'm a mod on r/budgetaudiophile and we're even open to company reps as long as they're from reputable companies and are transparent.

    subject matter experts

The trick in a lot of hobbies is that it's so hard to have anybody, even a professional reviewer, stay current with this stuff.

There are a lot of audio products out there. It's hard to say what's good and bad without putting each one through some objective measurements, and for some products (ie, DACs, amps, etc) this requires $50,000+ pieces of equipment and/or hours of time.

The balance between free marketing and access to community-loved brands is an absolute razor's edge to balance. Each sub does it differently and there's no correct solution. One person's astroturfing is another's excitement for an AMA with a company or retailer.

Having spent some time on these subs, especially /r/edc and /r/coffee, I could have guessed 8 or 9 out of the top 10 in each.

The more interesting things are the obvious ones that don't get discussed, but form some sort of an "accepted canon"

For example, the v60, despite being out of the top 10 on /r/coffee, it's assumed if you do pourovers, you have at least some form of this classic coffee maker.

    The more interesting things are the obvious ones 
    that don't get discussed, but form some sort of an 
    "accepted canon"

Well stated. My experience on r/budgetaudiophile and r/audiophile matches this as well.

The most obnoxious are the obvious ones in subs like /r/oddlysatisfying and /r/blackmagicfuckery where it's some cheapo product with a staged product video from one poster, then the inevitable plant post of "OMG where can I buy this amazing thing!" Followed by some helpful link to a drop shipping store, sometimes not till substantially after the post has fallen off the front page to avoid scrutiny.

A good example of this is the Reddit post below. The post looks somewhat natural but if you read the text critically it sounds very suspicious. This post had 8k+ upvotes and only a few comments calling it out at the very bottom of the thread that most people would never see.

https://www.reddit.com/r/OutOfTheLoop/comments/wfa6q6/what_i...

Makes me wondering how many of those thread replies are part of the same campaign. Some are so insanely praiseworthy of the venture. This also reminds me just how atrocious the reddit UI is.

I did this a decade ago on Reddit when I had a website drop selling watches. Made a post on a couple subreddits (men's fashion advice was the best iirc) with a nice wrist shot and in the comments casually mentioned where I bought it. Directly drove hundreds of sales (it was only $80 or so)

I was 18 so cut me some slack ;)

This is a valid concern for the whole review space. That's why most people don't trust a single source and look at multiple opinions (4-7 websites on average). One single source will never provide "just enough" research for a product you plan to spend a lot of money on and use for a long time. There is often at least one deal-breaking issue that you will find after reading through different sources.

We belive that a product research engine that provides honest and transparent recommendations from trusted sources is still a massive improvement over the current process.

We're not immune to the "garbage in - garbage out" problem, but we do our best to keep track of and remove sources caught getting paid to write fake reviews. Some categories are more prone to astroturfing than others, and we account for that by restricting the sources. Actively curating the sources is part of our daily work.

I doubt "most people" go out of their way to 4-7 websites on average to buy anything.

Not to mention Google results are so bad now that you'll frequently find the same sponsored blog spam review sites showing up, or worse, ones that just list top bought products from Amazon with affiliate links. 4-7 reviews sites will just get you more of the same unless you're looking for things like Consumer Reports, rtings, etc...

But how are you compensating people for the content being scraped?

It already is gamed. Almost anyone who has a brand targetted at high class consumers already astrotufs the living crap out of Reddit.

In Reddit's case this is made even more odious by its propensity for group-think and dogpiling so you can't even call it out once the liking for the product has taken hold.

Good use of dog piling / group think. Very little independent thought on reddit once something gets rolling. Though i guess thats human nature in a nutshell.

The devil is in the details.

Borrowing money almost always involves interest but there's a big difference between a government bond and an usury payday loan. Likewise I think that there is a big difference in critical thinking ability between the "reddit" blob of humanity and say, the US senate, subscribers, chemical engineering professionals, people on the Debian mailing list, etc, blobs of humanity. While the lowest common denominator in all groups may be similar I think the mean and medians are in very different places.

It seems like no online review platform can escape Goodhart's Law.

It's a race to the bottom where I can't get there fast enough and I never wanted to be.

Isn't an easy way to prevent this from being gamed is to discount user's who don't have a large post history?

Marketing firms are already aware of this and they use what is called "organic" marketing. This means the accounts will post all over the website generically so it seems at a glance to be an actual person. I imagine it can still be detected with some kind of heuristic, but its a classic arms race between spam and detecting spam.

Ah, didn't know that. That's frustrating.

But agreed - this _does_ feel like something that could be detectable, over time.

My guess is that such posts can be identified by lack of depth and few or no mention of negative aspects of the product. Even the happiest customer will have a gripe or two, even if it’s minor.

Just buy real reddit account

Thank you for this. I wish more people would suggest solutions instead of just being a Debbie Downer constantly and crying about how the system can be gamed therefore it's all futile. We didn't create technologies we have today because people sat on a chair and kept pointing at the flaws without trying to make improvements.

Reply on news.ycombinator.com