Show HN: Summarizing product reviews into simple bullet-point lists with GPT-3 | Hacker News Reader

Show HN: Summarizing product reviews into simple bullet-point lists with GPT-3

buyforlife.com

123 pointshubraumhugo41 comments

Hey HN!

Finding and researching good products can be very time-consuming and frustrating. Every time I want to buy a product, I waste hours reading reviews and researching the quality, durability and maintainability of it. Wouldn't it be great to have a service that does all this for me?

That's why I built the AI-Reviewer. The AI-Reviewer summarizes product reviews from all over the web into simple bullet-point lists.

#How does it work?

1. Scraping reviews from trusted sources on the web

2. Running it through a fake detection

3. Doing a sentiment analysis

4. The AI-Reviewer generates a brief and concise summary of all the reviews by using GPT-3

#What sources do I use?

I asked users where they look for product reviews and focused on the most trusted sources.

Besides the reviews of buyforlife, the sources are: Reddit, Wirecutter, Amazon, GearLab and some other.

#How do I prevent fake reviews?

This is a question that always comes up. There is no satisfactory solution to this problem, but I'm trying my best. A few things I'm doing:

- Running Amazon reviews through fakespot.com

- Diversifying sources and cross checking them

- Adding weight to reviews from trustworthy sources like Reddit and buyforlife.

#What's next?

I plan to continuously increase the number of products with an AI-Review. In addition, I can think of a few more use-cases:

- Summarizing warranty terms and conditions of brands into simple bullet-point lists

- Summarizing maintenance and care instructions

- Shopify App & Google Chrome Extension

- Ability to compare products

Read the full blog post here: https://www.buyforlife.com/blog/548RijnkRdPwn1cAI5RDjw/make-...

Not really related to the GPT-3 thing, which seems cool, but a point of confusion and / or criticism:

> if warranty = lifetime, then price/expected lifetime

The #4 item showing for me is a tape measure, which costs $20 and has an estimated monthly cost of $.28. That means you are estimating this tape measure with a lifetime warranty will only last 6 years.

The #3 item showing for me is an iron skillet, which costs $180 and has an estimated monthly cost of $.25. That means you are estimating this skillet with a lifetime warranty will last 60 years.

Later on down the list you have another cast iron skillet, which cost $20 and has an estimated monthly cost of $.29. This means you are estimating this skillet will last less than 6 years. I happen to own this skillet. It's made of solid iron. I promise you no one who is taking care of this thing at all is going to see less than 10 years of use out of it. I plan to hang on to mine for decades.

What's the basis for this estimation? Pretty much every one I've seen seems completely random and mostly unjustified. E.g. if my tape measure with a lifetime warranty breaks after 5 years I'm definitely taking it in for a free replacement, so what's the deal here? (My high quality tape measures have never broken that quickly anyway.)

All that said I think the implementation of "badges" was really neat and what I can see of the GPT reviews so far look pretty good (although I'm a bit worried that scraping certain review sites may lead to a garbage in, garbage out problem). I'll be checking out your site again in the future.

Multiple things to think about here:

1. For summarizing, in my experience GPT-3 still has some ways to go. It gets it right a lot of the times, but when it misses, it misses bad.

2. Assuming that after scraping we feed all the scrapped data as a prompt from which GPT-3 generates bullet points, that will be a very big prompt. Since prompts are also counted as tokens, it might end up costing 10cents minimum to generate one summary.

3. I think the core USP from the process the OP has detailed is in steps 1,2 and 3. Step 4 is a good hook to get people to try this out, but have to test it properly and check the costs also.

1. Can confirm. I'm currently manually checking every summary before pushing it to production. And there were some really bad ones.

2. You are right, the prompts can become quite big which increases the costs.

3. Let's see where it goes once it gets bigger :)

Looked at a few products I'm familiar with, and it appears you're doing pretty damn well at summarizing the high and low points for them. Nice work, bookmarked. It's pretty rare to find an affiliate model type site that actually adds value.

Edit: You'll want to watch out for people scraping your work and hawking it as their own though. I did something kinda similar by taking poor OEM images and cleaning them up, adding some rotation/depth, improving product descriptions, specifications by hand, etc. Scraped and copied by competitors pretty soon thereafter. Then a whack-a-mole DMCA game of getting rid of the ones that didn't bother to even change the copy.

Thanks for the feedback :) Sad to hear about the copycats, anything you tried to prevent that besides DMCA? I guess it's unavoidable in the long term, but by then the competitive advantage is hopefully big enough.

The problem with trying to use the DMCA for this is that, provably, no human creativity is involved, and so it's likely that the output of this is uncopyrightable.

It tends to work in practice anyway. Competitors that stoop that low aren't likely to actually contest it. I understand there's a legal liability there, but again, if you research who is doing it, the risk can be very low. And if you focus just on google SERPS, they often don't notice their pages got yanked from the index.

You can, of course, play tricks with javascript to thwart the scrapers, or serve them deliberately bad data, block ip ranges, etc. Depends on how tenacious they are, it's a war of escalation.

Looks cool, but I do think there are ethical concerns with "scraping" reviews other people write and then not giving them credit. It reminds me of a talk I watched where a guy was saying how AI is essentially just taking work from others and taking the profits. For example, no one cares about the thousands of translators that lost their jobs due to AI translation, but those services were built on their work.

One of the things that makes these GPT3 startups hard, and in my opinion most likely non viable, is that GPT3 is not ready for production.

It can produce text that at a glance looks like it could have been written by a human, but a human that is not very competent at the task it set out to do and makes basic mistakes. For example:

https://www.buyforlife.com/products/efa8b0c1-d7da-4485-b7f0-...

- Blade will rust easily unless oiled.

- The locking mechanism is not as secure as it could be, and can be difficult to disengage when the blade is being used for heavy cutting tasks.

- The blade is softer than most stainless steel blades, so the knife will dull quicker than others.

There's also no way for the startup to fix these issues, unless they have the resources to actually improve GPT3 itself.

What am I missing? What is wrong with those? I read the complete review and honestly, if I didn't know better, would have thought that a human wrote it.

> - The locking mechanism is not as secure as it could be, and can be difficult to disengage when the blade is being used for heavy cutting tasks.

This sentence seems particularly bad.

A locking mechanism on pocket knives holds the knife open. "Not as secure as it can be" implies that the knife closes in on itself (possibly snagging your fingers) under heavy use.

But... the next sentence says "hard to disengage during heavy use", which is the exact opposite (the knife will stay open after heavy use, and its difficult to close).

---------

- The blade is softer than most stainless steel blades, so the knife will dull quicker than others.

- Blade will rust easily unless oiled.

These two seem to contradict each other, but that's my knowledge of knife-steels talking. Softer steels are less prone to rusting but dull more quickly.

> A locking mechanism on pocket knives holds the knife open. "Not as secure as it can be" implies that the knife closes in on itself (possibly snagging your fingers) under heavy use.

> But... the next sentence says "hard to disengage during heavy use", which is the exact opposite (the knife will stay open after heavy use, and its difficult to close).

I own Opinels, and these points are actually not mutually exclusive. The Opinel locking mechanism serves to keep the blade closed as well as keeping it open, and it is sometimes not as secure as it could be, causing the knife to open when you don't want it to (e.g. in your pocket), but it can also be difficult to disengage when the knife is open (still love them though!).

> These two seem to contradict each other, but that's my knowledge of knife-steels talking. Softer steels are less prone to rusting but dull more quickly.

I don't think this is necessarily GPTs fault—it seems to be a pretty divisive issue. If you Google now "stainless steel vs high carbon knifes", in the top few results you'll find articles claiming that stainless holds an edge longer and others claiming that high carbon holds an edge longer. I always thought that stainless held an edge longer while high carbon is easier to sharpen (and rusts easier, of course), but maybe I've been wrong on that.

Opinel also makes carbon steel blades that rust if not properly maintained. It looks like one of those reviews may have slipped in to the list.

Is this an issue with GPT-3 itself or with how it's aggregating multiple (potentially contradictory) reviews?

> A locking mechanism on pocket knives holds the knife open. "Not as secure as it can be" implies that the knife closes in on itself (possibly snagging your fingers) under heavy use.

> But... the next sentence says "hard to disengage during heavy use", which is the exact opposite (the knife will stay open after heavy use, and its difficult to close).

I disagree entirely with this assessment.

There are plenty of badly made mechanisms that fail (eg, the knife flexes enough to slip past the lock) but are stiff and difficult to disengage.

As a specific example of a knife with this exact problem the CRKT M16-01 has complaints about it being stiff to disengage[1] and complaints about the lock failing[2]

[1] https://www.bladeforums.com/threads/frame-lock-way-too-stiff...

[2] https://www.bladeforums.com/threads/crkt-m16-14sf-lock-failu...

I'd say the bigger challenge is building something that is defensible. It's not that hard to design a GPT-3 prompt that summarizes reviews or writes ad copy, so you quickly find yourself with a bunch of competition and not much to differentiate on besides price.

It would be good to have links to the original reviews which were used for bullet points.

The reviews are actually listed/linked at the bottom of each product page.

Possibly, the parent commenter is looking for per-bullet links? (I'd want that as well.)

Ah I see, just added this feature to my backlog :)

Really well done. What are you using as a target for GPT-3? I see where you got the reviews, but where do you get the bulleted lists to learn from?

This is awesome and love the site. Especially with Amazon’s fake reviews issue, this is very much needed

Hello, Interesting product. I am Adrian, the founder of https://FeedCheck.co a product review monitoring solution. Are you open for collaboration?

Funny thing, my name is Adrian too :D Just send me a DM on Twitter or email me (address on the about page of buyforlife)

Is anyone selling (or open sourcing) a solution like this? I would love to feed our company's reviews into a program that could generate a bulleted list like this to summarize what our customers are saying.

That's acutally what I plan to do - offering this as a B2B solution where companies can summarize reviews of their own website/products. Feel free to contact me to talk about this.

Looks awesome and good job on hands-on demos.

Is it possible to extend it to run statistical analyses to spot fake reviews without third party tools or at least discrepancies and contradictions among reviews?

Thanks for the feedback. Improving the fake review detection by using statistical analysis is definitely on my list once this gets past the MVP state. If anyone has inputs on how to implement this, you are welcome to contact me :)

Great work! I'm a person that thinks of every purchase in terms of amortization, or my periodic cost for lifetime, this suits me!

>Summarizing product reviews

Dont know about you, but I dont give a flying F about random peoples reviews regurgitated by black box machine learning. When I buy something I either dont care about quality at all, or I look for an opinion of someone I actually trust.

Letting some company feed me recommendations is no better than believing in claims straight from product commercials. You are giving up your agency to a party treating you as a product.

This is fantastic.

Could you please provide alternative shopping links? Amazon.com links don’t resolve for Amazon.ca customers

The goal is to link directly to the manufacturer if possible. But right now the affiliate links from Amazon give me a few $ per month to cover the infrastructure costs.

Right. But for those Amazon affiliate links can you also provide an Amazon.ca affiliate link?

This is super cool, great start!

Anyone have any insight into elapse time on the waiting list for GPT-3 access?

I just submitted the official request form on https://openai.com/ and two days later I got access :) Not sure whether they are checking every submission and use-case manually or not.

My guess is that they are opening it up a bit more as the models and the API are becoming more mature. Their slack channel currently has over 20k people in it.

Amazing!! Keep going!

Nice concept

This is awesome. Great work.

Reply on news.ycombinator.com