WordPress usage worldwide | Hacker News Reader

WordPress usage worldwide

github.com

58 pointsandros46 comments

By only profiling the top 1M sites, I wonder if this may be sampling from a set not normally distributed? I suspect the frequency of use of WordPress might go up the further down the list you go — some random blog is less likely to be on the top 1M list yet be more likely to use WordPress.

I suspect this is so. The top million (specifically a subset of that) is where the sites fall whose owners can afford to hire people to build their own custom stuff. Small independent sites seem more likely to rely on something like WordPress.

> Small independent sites seem more likely to rely on something like WordPress.

I never understood this. I've also been suggested Wordpress by a web guy.

The only time I had Wordpress on a pretty basic website I got a notification it had been hacked, while on holiday!

I've been using static websites ever since.

I suspect a lot of the current small Wordpress sites could be just a well and more securely served by some static thing.

Not every site is static. Sometimes you need forms, processing, or other back-end business logic. Some people advocating using saas widgets for stuff like cc payments, but you're just farming out the dynamic parts to someone else. That gets hard to maintain with scale and with increased complexity.

I see a lot of people on this site debating the pure merits of a technology, but what others know and can work with is just as important as the technology itself. People know wordpress.

Getting harder to maintain with scale and increased complexity is not something I would associate with a small site though.

I agree with your remarks. There certainly are sites out there in need of dynamic behaviour. But that long tail has a lot of restaurant websites and such where nothing is really needed.

That is definitely not true. I think that statement is correct roughly for the top 50,000. The fall off in economic viability is fairly abrupt.

There are some websites in certain markets where the earn a disproportionately high amount of revenue per visitor, and these are the exception. But, given the nature of web and the difficulty in how the user actually found that site, a good chunk of their margins may have accrued to another party such as Google.

From what we know about software platforms - it's a winner takes all/most business.

Wordpress isn't there(but 34% is huge).

Why is that ?

Most of the top 10,000,000 websites are the internet presence of businesses that sells things in the real world.

I said "a subset of that." The top 50,000 is absolutely a subset of the top million. But you don't need to conjecture about that number 50,000. Again I'm saying the top million most-visited sites is where all the profitable sites likely fall. (Not that they're all profitable.)

W3techs profiles the top 10 million sites which shows a percentage of 34% [1]. You might be right.

[1] https://w3techs.com/technologies/overview/content_management...

The article says:

"Afirman que usan una base de datos de 10 millones de dominios sacadas directamente de Alexa, pero cuando vas a la fuente no existe. Alexa solo proporcionan 1 millón."

Translation:

They [W3techs] say they use a database of 10 million domains taken directly from Alexa, but when you inspect the source, it doesn't exist. Alexa provides only 1 million.

Alexa provides a list of the top 1m... but you can also page through the top sites (100 at a time) using their 'top sites' api... and I don't see a limit on that.

Edit: confirmed by aws: https://forums.aws.amazon.com/thread.jspa?threadID=167457

The test is likely to be under-reporting WordPress installations as it stands.

The script seems to detect a WordPress site by looking for a meta generator tag containing WordPress:

https://github.com/tanrax/calculate-wordpress-usage/blob/5aa...

It's pretty common to remove that meta tag — popular WordPress theme frameworks like Genesis do it by default.

A more reliable test would be to look for additional strings in the source that point to the use of WordPress, such as “wp-content” and “wp-includes”.

A faster way that avoids string searches would be to send an HTTP head request to `/wp-login.php` and check for:

Set-Cookie: wordpress_test_cookie

(/wp-login.php doesn't always appear in the root directory and it's not always accessible to all IPs, but that setup is most common).

It'd be interesting to do all that you said (and more) and then determine what's the combined amount, as well as what % of sites do some sort of obfuscation... and why?

The way this works seems to check if there's the "generator" meta tag in the <head>.

Custom themes etc. might choose to omit that so it's not a 100% reliable check

you can compare it to this: https://www.isitwp.com/

It doesn't work, I introduced the url of my page (programadorwebvalencia.com) made in Jekyll (a static site) and it tells me that it is WordPress.

also headless website using WP as BE only

The article says that they also look for "wp-admin" et al in robots.txt

I suspect (hope) that means none of the WordPress sites I maintain will be in the tally.

Customizing the metadata, stripping out unnecessary cruft, and moving the user and admin login pages behind IP-restricted firewalls (when possible) are among the first things I do.

I don't think there's an unquestionable way to know if a site is made in WordPress, although you can intuit by some clues: header, meta tag, robots.txt, cookie login, sitemap ...

you have to use a larger sample. There are 300M+ domains. The longer tail will surely have more wordpress in it.

Some other notes: 1) you're not checking subdomains like blog.company.com or paths like company.com/blog 2) if you use something like zgrab you can do 1M site crawl in a couple of hours. Consider checking it out.

Or more Wix, Squarespace, and Shopify in it.

> the list of the first million domains with the most visits

How is that a random sample?

are you using http or https? following redirects? how many? This is all going to change the study.

Only ? Looks like a big number for worldwide usage, no ?

WP itself usually puts the estimate at 25-30%.

W3techs measures the top 10 million sites and has stats like that. See https://w3techs.com/technologies/overview/content_management...

The WordPress.org home page says 34%: https://wordpress.org/

Right, originally I read 1% and was surprised, but that's massive usage

Google Translate link for the article at the end of the repo:

https://translate.google.com/translate?sl=auto&tl=en&u=https...

The Readme says "Warning that it can take a long time: between 20 to 30 days."

How in the world can it take so long time? The csv file seems to be 24mb in size and the computation performed can't be that advanced. Did the author do something seriously wrong?

It just processes everything sequentially.

I saw that it's actually going through all the website urls and makes a request for each. To be optimisitic, each request takes 1 second, that's 1m seconds. That's ~10 days, so doesn't seem unrealistic it would actually take ton of time.

> To run it you'll need either 2Gb of RAM

What? I guess this is a toy program used to learn clojure or sth - it even uses sed for line parsing. A 10-line php script could do the same with a few MB of RAM

What is sth?

I do have to agree this sounds relatively inefficient, especially considering you could just immediately flush the result out to disk.

I agree, even a Bash script would be more efficient. Everything was born as an exercise to learn Clojure. I can tell you that it actually uses 1.1Gb, but for safety I recommend twice as much. Why does it use so much RAM? Ask Java :) (I am the author of the script)

Performing a web request to 1M targets takes a long time.

However, this is an [embarassingly parallel](https://en.wikipedia.org/wiki/Embarrassingly_parallel) problem, and renting out some machines would speed it up.

This is a pretty amazing feat. The top 1 million sites includes many who have the money to afford custom sites and yet Wordpress is still almost 1/5 of sites.

WordPress is the software HN loves to hate, but while it certainly has plenty of warts it's also a very flexible, pliable system for building the kinds of web sites that most people want to build. It'll never win any architectural beauty contests, but market share is driven by utility, not beauty. And WordPress can be very useful software.

I can afford to have all of my clothes professionally cleaned, yet I have a washer and dryer.

so they studied the top 1M websites and found 19% is using wordpress? man, that's a good chunk if you ask me.

It's amazing. So we can say that 19% use PHP (at least) and MySQL.

I’m wondering why this takes 20-30 days to run all up? Seems crazy for 1M requests. Could one make this a concurrent task and get much greater efficiency?

Reply on news.ycombinator.com