Hacker News Reader

Comment by ArtTimeInvestor | Hacker News Reader

Wow, that is a bummer!

Alexa is my go-to place to get a first impression on how much traffic a website gets.

I have never looked into how they get their data, I always assumed they get it from internet providers?

Is there an alternative? SimilarWeb publishes their data with a huge delay, so it is not of much use for me. From my experience, it is also less reliable.

It is crazy how valuable internet properties get closed down when they are owned by giant companies. According to Alexa, Alexa is still a top-5000 site:

https://www.alexa.com/siteinfo/alexa.com

It seems insane to just close it down.

View on HN

I started a news aggregator that at the peak was ranked in the top ten Alexa sites in a country (was doing 100M page views / month). Alexa was the standard that people would use to judge how successful a site was.... Until people figured out how to massively game it.

For a pretty small sum, you could approach illegal download or porn sites to embed an iframe of your site or do a pop-under ad, and instantly be in the top 10. People aren't actually visiting your site, you're just getting "shadow traffic". Then you'd go tell advertisers "look, we are one of the biggest sites in the country".

We didn't play that game but our competitors did and it was really frustrating.

Anyway, the point is their rankings weren't very reliable (this was 8+ years ago, maybe they've gotten better at detecting traffic fraud).

I've loaded in Alexa, Majestic, Cisco DNS, and Tranco rankings into this site:

https://domain.glass/ycombinator.com#dns_rank

Not the prettiest, but I use it a fair amount myself for researching domains.

Damn, with that data TikTok.com is ranked #1 O_O

The world suddenly makes sense doesn’t it?

No a world where TikTok is the #1 website makes no sense at all.

Though I suppose it's the sewer of the internet so it has to be everywhere.

I've never looked at TikTok but I look every once in a while to a user comment section of some pretty reputable world / local news sites. This alone leaves the impression that the world populace is but a bunch of dumb indoctrinated rabid racists and haters.

My gut feeling is that most are nation-backed trolls.

Way too primitive for those I think.

I would probably have made the same off-the-cuff comment before I signed up.

You can make the same comment about YouTube or Twitter, but it all depends on what content you ingest. TikTok's algorithm actually serves me incredible content that is both educational and entertaining and I thoroughly enjoy using the app.

(And I don't consider myself some sort of floozy - I've been using the Net since before the first commercial web sites appeared)

Also interesting doubleverify.com is ranked 14th, the first I did not hear of.

Quickly looked them up, some sort of anti-fraud company, working with TikTok apparently. There website doesn't load for me, I'm wondering if they are target of DDOS attack which might actually move them up the rankings haha

It's an ad tracker. It's loading in ads that people see, but isn't the source website generating the traffic.

Ah, that makes sense. I take it that its name is a play on "DoubleClick"?

I've notice some non-native English speakers use "double verify" where native speakers would use "double check" or "verify". It could be a play on that.

Or they had a meeting where the founders asked "what domain sounds twice as good as verify.com?"

thats a superplusgood question

DoubleVerify is an adfraud and brand safety tracker. Basically, they ensure that your ads are displayed to people and not on questionable content if you dont want to. Most agencies and large direct advertisers add this 3rd party to ensure actual humans are seeing the ads, and thus the advertiser is paying for “good” eyeballs instead of “bad”. It’s essentially a 3rd party checking that the ad network doesn’t do anything naughty, which some sites have done in the past.

TikTok is big in the Western world, but it's even bigger in China.

But isn't it douyin.com in China? I would think that would be separate in the rankings since the rankings are based on domain.

TIL about https://bookface.ycombinator.com/

I’m perplexed about “ whoredom.groups.ycombinator.com” ? From grandparents link

Is "groups" the private messageboard for yc participants?

groups.ycombinator.com redirects to Google groups.

The traffic seems to come entirely from Umbrella DNS requests? https://domain.glass/whoredom.groups.ycombinator.com

Neat! How often do you update this?

apple.com being so high is (to me) a little counter-intuitive. microsoft.com too for the same reason. They don't seem like the sites people would be using a lot in their day-to-day lives. I guess it's counting dns lookups from devices and not necessarily "human" web page/app requests?

It's been forever since I've seen this first hand, but iirc apple.com and microsoft.com are both default homepages in Safari on macOS and Edge on Windows, respectively. That may artificially inflate the numbers a bit too.

That’s a good point. Both companies also offer single sign on services too so again maybe that could keep redirecting users back to their sites.

If you’ve worked on or around an enterprise gateway team, you’ll learn pretty quickly that 1) Microsoft, Akamai, Cloudfront and Apple traffic appear to represent 95% of traffic in a given organisation, and 2) providing meaningful reports on internet usage for executives is *hard*

Somewhat over two decades ago, the #1 site in the web cache stats for the web cache between an ISP dialup pool and the internet at large was "icons.hotmail.com" (may have been a number on "icons", actually). With probably 2-3 x the hits than the first porn site.

how many iDevices and their apps/services ping back home via a *.apple.com domain? In other words, is it possible that the traffic isn't from people visiting the website in a browser?

Think docs.Microsoft.com etc - some people spend all day every day on there!

I spend quite a lot of time looking at new MacBooks :)

"Am I on a captive portal?" tests might make up a decent chunk of requests, I believe both Apple and Microsoft have equivalents to http://detectportal.firefox.com/success.txt

This is a terrific tool, thank you. Bookmarking for reference.

> Alexa is my go-to place [...] I have never looked into how they get their data

Right, this is a problem with all sorts of data sources that provide numbers (and use lots of SEO) but don't talk much about their methodology. CelebrityNetWorth is another example of this.

I've come to assume that celebrity net worth sites are mostly just made up numbers. Sometimes you can look up the payout for some specific jobs, but not all, and most of the time, they seem to just ballpark a guess based on that.

These website ranking websites like Alexa were also pretty close to ballpark guesses.

> I've come to assume that celebrity net worth sites are mostly just made up numbers.

That’s how I increased my net worth to 100 million USD. Now I’m using rolling out an unstable coin trading under the symbol UCCT (UCrayCrayToken) backed by my CelebrityNetWorth.com value. I’m going to burn half the tokens soon, so now is a great time to buy tokens from me before they double in value.

> Alexa is my go-to place to get a first impression on how much traffic a website gets.

Alexa hasn't been a reliable source of traffic data for many years. It's gotten worse as mobile devices, private browsers, VPNs, and tight-fisted companies (like Facebook) have become more widespread.

If you own a high-traffic site and check Alexa, it's not even close. One of my sites wasn't in the order of magnitude.

What's the best alternative for website traffic estimates?

SimilarWeb is generally correct - but again, it can be pretty off in some cases. Another one that is sort-of accurate is SEMRush

Tranco list [1] is considered the most accurate source for relative site ranking by traffic. It is a result of triangulation of data from several sources (one of them is Alexa) and it what we use at Kagi Search for domain information [2]

[1] https://tranco-list.eu/

[2] https://kagi.com

Alexa was the strongest Tranco input, actually think this is quite bad for them.

How does Alexa work? It seems to me it’s hard for anyone except Google Analytics or Cloudflare to build such a service.

Handwavey "proprietary methodology" using data from "millions of Internet users using one of many different browser extensions" and "direct sources in the form of sites that have chosen to install the Alexa script."[0]

Translation: trust us

[0] https://www.alexa.com/about

Initially, it was mostly by monitoring the traffic of users of the alexa toolbar, https://www.alexa.com/toolbar

I think they moved on to more broad data sources, possibly purchasing data from ISPs.

Read between the lines as to how this similar product from Cisco works.

http://s3-us-west-1.amazonaws.com/umbrella-static/index.html

Thanks! Goodness, only raised more questions:

1. What is Umbrella? See http://info.opendns.com/rs/033-OMP-861/images/OpenDNS-Global...

2. What is Passive DNS? See https://docs.umbrella.com/investigate-ui/docs/passive-dns

They used to have its own toolbar that tracked which websites users visit.

After that, they've paid other browser add-ons to add their script and some websites voluntarily gave them their data by adding their tracker.

It says so right on the about page (warning: unusable on mobile): https://www.alexa.com/about

You'd assume that this would give a super biased sample of the population that would skew boomer+.

It was always biased, and it's only gotten worse over time as browser toolbars have ceased to be a thing and data collection in browser extensions has been heavily discouraged. I have to wonder if that's one of the reasons they're phasing the product out.

Yeah, it was also really handy for figuring out how to attack various web properties, as they frequently indexed administrative pages that were secured through nothing but obscurity, as the toolbar was most popular with amateur (and (im)professional) webmasters.

Ah, the good old days.

Interesting. I wonder if Apple privacy policies killed it. Or if the fact everyone is using the same 10 web sites for everything has eroded the need.

Google Analytics would only work on sites that run Google Analytics, which would exclude sites run by the other big tech companies. Alexa worked because their toolbar addon would record every site their users went to, regardless of what was running on the site.

I remember vaguely that they got their metrics from a browser toolbar.

How fondly people remember a company that totally installed tracking software into browser add-ons.

I thought they had a browser toolbar/extension which they use to collect data from a very very small subset of internet users, which is probably incredibly biased to a certain audience. (e.g. boomers who don't know how to not install random toolbars when downloading stuff).

they had a browser toolbar to collect user agent signals

Reply on news.ycombinator.com