Show HN: We built a Rotten Tomatoes-style website for VPNs | Hacker News Reader

Show HN: We built a Rotten Tomatoes-style website for VPNs

seekvpn.com

74 pointshubraumhugo64 comments

What you'll find difficult is establishing trust.

I have no reason to believe your review site is a shill for another company, but I have no reason not to believe that either. It was recently revealed that Kape Technologies, a former malware distributor and now owner of PrivateInternetAccess and ExpressVPN, are doing this.

The first step would be clearly identifying ownership and benefactors.

I fully agree; we need to be more transparent about our process, intentions, and ownership. We'll work on that.

Here is a good Reddit post about the fake reviews from Kape Technologies: https://www.reddit.com/r/VPNTorrents/comments/r160ll/spottin...

It's curious that your #1 listed VPN NordVPN has a pretty tasty affiliate program but I can't see Mullvad (no affiliate program) on your front page at all in spite of it being most peoples' favorite ;-)

I tried to start a site similar to this one in the past. I gave up in the end because honesty and $$$ just wouldn't mix. No matter how hard I tried I couldn't square this circle. Economically I think it doesn't even work on a theoretical basis. The mullvad of most industries just wouldn't have affiliate programs whereas the scummy companies would have the best affiliate programs.

Maybe it's a recent change, but I typed in mullvad and got this:

https://seekvpn.com/review/mullvad-vpn?keyword=mullvad

It has a fairly high rating overall.

And yet ExpressVPN is rated 5th on your site. A reputable review site would have ExpressVPN and NordVPN rated 0.0, or even not rated at all and shoved into a separate section, "VPNs to Avoid."

We're working on improving the site and the data quality. Keep in mind that it's still an MVP.

...but you are a review site... The data quality is your MVP.

This.

Why to avoid?

Youtubers are still promoting ExpressVPN, unfortunately.

Just one data point, but the "No, I'm not interested in joining a fun community or a chance to win an awesome VPN" text on the solicitation for my email after ~20 seconds on the site was enough to get me to leave. I am currently unhappy with my VPN and shopping for a new one when my annual subscription ends early next month.

I cannot recommend Mullvad enough. From past posts on this website, I think many here will agree.

Mullvad doesn't even ask for your email address!

You're right, the popup feels quite annoying when first visiting the page. We removed it.

It's still there on blur. It seems to be useful, because it correctly gets across the impression of the sort of VPN you recommending.

Awesome, thanks!

VPNs are a mess. It's becoming a very commercial market, and companies are bought up by less reputable and bigger ones, making it hard to know what to trust.

The biggest pain points when researching a good VPN are:

- Sponsored content / Ads

- Fake reviews

- It takes time

- Inconsistent metrics and info across sites

To make finding a good VPN even easier, we've aggregated over 500 online reviews, 3k Reddit posts, 4k HackerNews posts, and 400 Youtube videos.

## How it works

Using GPT-3 and BERT, we've built a service that collects VPN reviews from all over the web, filters and aggregates them, and presents the summarized results in a short and understandable form.

## What's next?

We're constantly improving SeekVPN and will soon add new features like:

- Building a community for VPN enthusiasts that helps each other

- Running statistical analysis to spot discrepancies and contradictions among reviews and filter out fake reviews

- Increase catalog and add more filters

- Improve summaries (pros and cons style)

We eventually plan to do our analysis for other categories of SaaS products. It should become the Rotten-Tomatoes for software products.

Let us know what you think! What should we add or improve?

Why would you use NLP to collect reviews that are astroturfed? Garbage in, garbage out

Just go wirecutter style and actually research the products a minimal amount to give an opinion

This is what a good VPN roundup looks like.

https://www.privacyguides.org/providers/vpn/

This "RT-style" looks exactly how I would expect a "machine written sourced from affiliate blogs" review to look like.

Wirecutter is a great resource, and we'll soon include their lists for our analysis. But most people visit 5 different websites and spend many hours doing research before they buy a product. Our goal is to build a service that gives you a holistic view about what the internet thinks of products.

Again, garbage in, garbage out - how is rampant astroturfing and review fraud taken into account?

Edit: the site highly ranks the ones I see pushed on YouTube, so I’m guessing this is just “how often is this vendor referenced” and not “how quality is the vendor”.

Agreed, aggregated reviews are not helpful when you can't trust the source. For example, I just looked at the NordVPN page and all three of the highlighted YouTube reviews are pushing the NordVPN affiliate link in their description. Can I really trust these YouTube reviews when the creators are getting paid for each referral?

The way I see this going is that a bunch of people are writing VPN reviews because these companies pay high commissions. This site is going to aggregate these untrustworthy reviews, and also try to earn a commission in the future. As the end user, I don't see the value here, because I don't trust anyone in the process.

Fake reviews are certainly a big challenge, even for companies like fakespot that try to detect them.

What we plan to do:

- Running statistical analysis to spot discrepancies and contradictions

- Weight reviews based on the trustworthiness of their sources. E.g. Reddit, HN or Wirecutter are known to be very trustworthy sources.

Disagree that specific forums are more trustworthy than others. To me, a trustworthy source is a known entity that tests themselves and keep a track record of accurate reviews. For specific forums, keeping track of users that post honest reviews would be a good way to start.

> Fake reviews are certainly a big challenge, even for companies like fakespot that try to detect them.

Not that big: If you cut away all 5 and 1 star reviews you'll have stripped roughly 99% of astroturfing (for and against a company).

How do you figure either HN or Reddit (anonymous, unverified accounts) are trustworthy when there’s money to be made by astroturfing? The incentives are stacked against you.

I dare say that Wirecutter used to be a great resource before NYT bought them... not so much these days, it's mostly a pay-to-play machine. https://www.nytimes.com/wirecutter/our-response-to-nextdesk/

Don't even get me started with Wine Spectator and wines in general... https://freakonomics.com/podcast/season-11-episode-6/

Like other have pointed out trust is difficult to obtain these days for review sites but I hope you persist and have a successful product.

How do you propose someone actually review VPN services? All of them work, have many countries available as egress points, and are fast enough.

The only actual differentiation is in attributes that are impossible to measure from the outside: Do they log accesses? Make logs available to governments? Sell it to data brokers for advertising purposes?

That's pretty much the only difference between good and bad providers, and there's no way to tell, so in-depth reviews are an impossible task.

One could include honesty in marketing, that seems like a pretty clear thing that this industry would need and that few providers actually do.

> Using the newest NLP (GPT-3, BERT) technologies

Maybe remember your audience here and drop marketing bullshit like that. If you're using specific techniques, great, tell us. But don't use hand wavy marketing buzzwords.

Good input, fixed it.

Haven’t been on the website for one minute and you already shoved a popup in my face asking for my email. The options are “submit” or “No, I'm not interested in joining a fun community or a chance to win an awesome VPN”.

If you’re going to be as annoying and manipulative as every other scammy website out there, why should we trust you? My reaction was to close the page. Considering you’re posting this to HN, I doubt I’m in the minority.

We removed the popup and we'll add some more information to the about section to make us more transparent and trustworthy.

The #1 result NordVPN had servers hacked to root and then attempted to covered up the huge security breach rather than disclose it. [1] Not to mention their concealed relationship with Tesonet, a shadowy data mining company. On top of that, key personnel of Tesonet (including the CEO) are stationed in Lithuania, which has a "cozy" relationship with the CIA, which extends as far as knowingly hosting rendition sites there.

Isn't this an immediate disqualification of the site's methodology?

[1] https://en.wikipedia.org/wiki/NordVPN#Criticism

Well, the site has caused some really insightful comments from knowledgeable people, which will definitely helpme choose a VPN.

So in that sense, the site has already succeeded.

All they need to do now is prettify this HN thread and replace the current garbage with it. ;)

The fact that NordVPN is somehow #1 immediately discounts whatever algorithm you've designed for this.

Under no circumstances would I ever consider a VPN that's been breached even once as a #1 choice.

Especially not if they tried to hide the fact that they just got hacked.

I thought the general consensus was that the only VPN worth a damn was Mullvad?

I'm not interested in a review aggregator. Individual reviews are the problem in the first place. What I want is someone to have a set of criteria and then rate the services individually based on that criteria.

Also, your family of review sites are for VPNs, backpacks and knives. This sounds like a writing prompt.

Reminder that if you only need 1 specific location you need to appear in, you can get a VPS for less than a monthly VPN subscription.

These VPS, while still distinguishable from normal residential connnections, are more likely to pass filters etc, it won't also trip up IP-change-detection mechanisms as you'll get an stable IP address.

Rotten Tomatoes works because (1) its audience generally trusts some (maybe even most) of the reviewers, and (2) some movies are good. I don't see how Rotten Tomatoes for VPNs can work.

What rotten tomatoes considers the "best films of all time" is laughable across the spectrum.

Some gems from the top 10:

4. Black Panther

6. Parasite

7. Avengers: Endgame

9. Knives Out

10. Us

Its surely just coincidence that the half of the best films ever made in human history were produced in the last 5 years.

https://www.rottentomatoes.com/top/bestofrt/

They're doing something fishy with that "sorted by adjusted score."

If it sorted by tomato-meter value instead, the esteemed classics would rise to the top, and the entries you took issue with wouldn't even crack the top 50.

Whatever "adjustment" they are doing seems to strongly favor recency.

I get the impression that VPNs massively overcharge for what they are; every YouTube video that is even moderately mainstream is sponsored by some sort of VPN provider. Is the idea behind choosing this industry in particular to charge a referral and suck away some of their sweet, sweet profit for yourself? Seems like a good idea.

Take a look at mullvad. There's no real way of knowing if they're telling the truth, but I think the fact the have no affiliate program or even advertising and don't collect your email address is a good thing.

(They give you a token, you send them cash or use a credit card to add money to the token, you input the token in the app.)

How do I pick a VPN from this website? Let's say I go with the #1 Trending VPN, it scores very well, but the Reddit threads are negative. The "Reviews by Topic" are then suspiciously almost all positive. Do you audit the reviews?

To make it easier for people to pick a VPN, we plan to add lists for Top X vpns for absolute security, top ones for privacy, top ones for reliability.

About the reviews:

We collect the reviews per topic by doing a web search. Unfortunately, many VPN reviews are bought and paid for, so we need to improve the collection, filtering, and ranking of the reviews. We did this for other products, but VPNs seem to be a very tricky category.

I'll save you, and everybody else, the time, and just say that Mullvad is the best VPN currently available, and the day they get inevitably bought out will be a sad day. But until then, they beat all the competition hands down easily.

The ownership and future of Mullvad VPN

"Mullvad VPN is here to stay, and we are not interested in ever selling it."

https://mullvad.net/en/blog/2021/9/16/ownership-and-future-m...

Avoid ExpressVPN & NordVPN because of problems with support and shady practices. I suggest you delete these two from your list and/or show prominent warnings.

I currently use SurfShark which works well for me.

What shady practices do NordVPN engage in?

March 5, 2018: Hacker gets root access and key material on a NordVPN server.

404 days after hack, NordVPN finally discovers server is compromised

594 days after hack, a twitter user boasts of experiments with NordVPN private key material that had "been floating around mostly unnoticed"

595 days after hack: NordVPN finally acknowledges they were breached. Apology hammers on two themes, "it's not our fault we had no way to restrict access to that server" and "you should still trust us because we confirmed this guy didn't do many of the awful things we allowed him the capability to do"

That's a whole year of running a compromised server, and 27 whole weeks of cover-up between discovering the breach and getting called out on twitter.

Wow... the immediate email nag is pretty catastrophically bad already, but then your "top" VPNs are... Nord? ProtonVPN? I'm not here to point fingers or assign blame, but I've gone through a fair handful of VPNs and I can tell you firsthand that those are faaaaaar from the best options, especially if you're trying to get the most out of your service. Mullvad and Azire are head-and-shoulders above anything I see on the top. Kinda disappointed.

Edit: Also, your "about" page is surprisingly non-informative.

Do you earn affiliate fees if I sign up for something after being referred by your site? If so, I won't trust it.

Are you going to disable or reweight user scores when they dunk on a VPN you're in a strategic partnership with, in the name of "preventing review bombing by white supremacist alt-right trolls"? Because that'd be necessary in order to achieve feature parity with RT...

I notice that page 3 of 4 doesn't have a Next button to go to the next page - that seems wrong. In fact, the second-last page (even in a search result) doesn't have a Back button, but the last page does have a Previous button. Classic off-by-one error it looks like :)

Should be fixed now :)

Is there any info on parent companies? I don't know much about VPNs but I thought ownership was conentrated into just a few firms, and that some of them are owned by chinese companies - both of which could be useful information when making a decision.

You can find information about the ownership in the "Transparency: Ownership" section when expanding the specs on a specific provider.

No you built an affiliate links site where you say they are all "great".

Obligatory vpn.md https://gist.github.com/joepie91/5a9909939e6ce7d09e29

NordVPN as #1 means I know for sure not to trust this website.

Thanks all. I have cancelled my NordVPN subscription. I was not aware of all the shady stuff they were doing behind the scenes.

Why should I trust this in any way?

Needs DIY VPN option, being on HN and all.

Reply on news.ycombinator.com