How HN crushed David Walsh's blog
davidwalsh.name
davidwalsh.name
The secret? I have a lightweight theme with minimal dynamic content, and I use LiteSpeed cache on the server. That's it. Easily handled 20-40 thousand pageviews over a couple of hours.
This post[0] was #1 on the front page for a day and I had 0 issues serving requests running on the cheapest shared Wordpress hosting with the LiteSpeed Cache plugin enabled.
[0] https://andreschweighofer.com/agile/anxiety-in-product-devel...
What makes MySQL so bad at handling queries? I have never worked with it personally, but it seems like a core feature of a database should be handling many concurrent requests
The software that is used to generate the queries. Inadequate indices, writes on each page load that lock shared tables...
In fact, the database layer is often forgotten when you talk optimization in the WordPress world — page cache is usually seemed as the holy grail, with MySQL left to fend for itself.
I am not saying that implementing page cache is bad — it is essential for WordPress —, just that page cache is not the only thing that matters, but seems to be the go-to solution for performance issues with WordPress, when in reality, you should look at the picture as a whole.
That scenario (in general, not referring specifically to WP anymore) makes database indexing/optimization much more expensive at webhost scale, because you potentially have columns that range in size from empty to 1MB.
That's why most WP users have gone deep down the rabbithole of view-level caching, because optimizing an uncached result is so much harder in that environment.
But if you call yourself a managed WordPress hosting company and your marketing material says your entire stack is optimized for WordPress, you should be held to higher standards than a shared hosting company like HostGator, for example.
Properly tuned, a database is able to handle millions of requests per second.
* blazingly fast for readers with no chance of slowdown
* very low hosting costs
* linkfarmers never steal my content
* I don't have to moderate comments
I recommend this approach to everyone.
Oh well, everything else is true: Blazingly fast, low hosting costs, no comment moderation, etc.
>_<
Don't feel bad about taking your allotted paid time off days and sick days (if you need to) - it is expected that you will disappear for a week or two periodically.
Marmite is supposed to be spread very thinly on buttered toast. Vogels is a good brand of bread for this.
Despite what you might hear on the street, Pavlova is not good.
Want to know the rest? Hey, buy the rights.
And yet that means so many fascinating blogs go unread.
A while back, I scraped the top-level comments from a "dear HN, what's your personal blog" post, and found SO MANY AMAZING BLOGS!
The tool's here: https://random-hn-blog.herokuapp.com
I'd much rather read a small obscure blog with very little traffic (well, any small obscure blog but mine) than something that regularly gets deluges of traffic.
Something about knowing they're writing to a potentially huge audience changes the writing, I think...
I'm starting to think that it's time for people to stop using WordPress+MySQL and move over to something more performant.
I've found it amazing how much you can do these days with even a cheap VPS, and how many requests you can serve as long as you're disciplined about not going overboard with excessive dynamic content and knowing where your scalability pain points are (sometimes this isn't obvious until you really get a lot of traffic!).
That's not an acceptable server-side response time at all, regardless of how dynamic, or not, a blog post page or index page ought to be.
Even now, I'm seeing 500-600ms+ server-side response time from Europe, and 800+ms in the US.
When did this become "good enough", nevermind "normal"?
Based on many data points from our monitoring of website performance, this is a very common range.
4 to 6 seconds for effectively static content is absolutely insane, and only justifiable for exceptionally rarely accessed dynamic data queries.
It shouldn't matter how "common" this response time is, when you can load a huge static HTML/CSS/PNG site in hundreds of milliseconds.
Frankly, our entire industry should be ashamed that anyone thinks 4-6 seconds is an acceptable amount of time to render a fucking website.
Unless the website is literally loading an entire page of High Res images/video content, 4-6 seconds is atrocious
"dns": 158.37, "connect": 328.503, "send": 0.148, "wait": 856.465, "receive": 91.914, "ssl": 175.243
"Wait" is the time between when the last byte of request is sent, and the first byte of the response comes back. This is measured all after the DNS+TCP+TLS stuff has happened, and is basically measuring the latency to the server, the backend processing time, and the latency coming back.
800ms+ is... not good because this site is supposed to be behind a CDN (lower latency) and with a supposedly optimized backend. I also am still seeing the "cf-cache-status: DYNAMIC" response header, so whatever optimization was made didn't stick.
(Also, That connect time is oddly high. 300+ms of which 175ms is the TLS handshake. Something to look at as well.)
FWIW I'm in the web performance industry as well, and Todd is correct, a 4-6 seconds for window.onload is common for sites (not just web apps, but sites). Of course modern site development practices (lazy loading images, deferring/asyncing scripts, font fallbacks, etc) have made using the "onload" time essentially useless as a good metric.
(PS: Todd I'm a big fan of your videos on building Request Metrics)
On top of that, testing caching is challenging - replicating between local, staging, and prod is ultimately a very manual and error prone process, so there's no real way to figure out how to test and if what you're doing is the right thing. Since caching is not an immediate thing (it can take time for a CDN to pick up an asset, for example), it can be unclear if what you've done works, or if you need to wait five minutes and try again.
I wrote a blogpost a few months ago about this and other issues. (https://solitaired.com/why-we-switched-from-wordpress-to-nod...)
Maybe I'm doing it wrong? ¯\_(ツ)_/¯
Edit: I see you mentioning cloudfront on your blog post, what problems did you encounter when using it with wordpress?
Also, any reason for not using sanic, strapi or any of the headless CMS and building from scratch?
I eventually solved it by futzing with all the options.
Re why I didn't use Strapi/the others? Once I switched to node / markdown, I told myself I'd fit in a real CMS, but never got that far researching it. However, I like the options you mentioned. Thanks for sharing!
We picked whatever the most popular cache plugin was at the time and turned on the default settings, then went from there. Honestly, the difference between any caching strategy and no caching strategy can be huge, especially when you get a surge of traffic.
We put staging behind the cache too, but not local or dev. Staging was just to test that a caching configuration change would not take the site down. A bunch of the tuning was actually done on production and back-ported to staging.
I think a hidden factor here is what people know. If a person/team knows node.js really well and doesn't know Varnish, then of course it's going to seem easier to move to node. That seems pretty common to me; it's more like web development, whereas caching/Varnish seems more on the sysadmin side of things. My team had to learn it, which took a little while.
But in my experience, well-tuned caching in front of WP can scale really well for content that changes rarely like blog posts or articles.
Today we pay WP Engine take care of it all for us and it's money well-spent.
My biggest learning was that CDN != caching. While it can be used as such, it's not automatically set up that way out of the box.
The strategy is the same even for complex sites, full page cache at the server daemon for almost every page hit unless a user is doing an action that isn't "View this page".
Obviously I'm glossing over cache busting strategies and how you handle dynamic actions like add to carts, sessions etc, but they're all far more simple than rolling your own application level caching strategy.
You will get so much more out of your hosting if using an FPC, which gives you way more headroom for the requests that do need to spin up the application.
I even made an open source site you can just fork and use in a few seconds:
I wonder if this is intentional trolling
From that perspective, not being able to generate a blob of basically static HTML text in 600ms (100 requests per minute according to the article) seems insane.
From that perspective, not be able to make a decent UI that more people can use without getting frustrated seems insane.
We all have our focuses in the areas where we can get the most impact. Being able to serve a website in 100ms instead of 200ms simply is not as important as the 100s of other things us web developers have to think about.
Although I do agree with you, the web is bloated right now.
"Higher Security: With server-side processes abstracted into microservice APIs, surface areas for attacks are reduced."
I'm sorry, but how does microservices make something more secure? You've only outsourced your security problems in hopes a 3rd party will resolve it.
"Jamstack is an architecture designed to make the web faster, more secure, and easier to scale. It builds on many of the tools and workflows which developers love, and which bring maximum productivity." W.T.F.
And yeah. Microservices ~= webserver + backend
JAMstack isn't a move to microservices. In some places, it's about eliminating services entirely, or severely reducing their scope.
Highly recommend trying it. I'm seeing the vast majority of visits now are not hitting the origin server at all — for assets or the page itself. At least it's a good stopgap until we can convince everyone to move 100% static...one can dream, right?
[0] https://blog.cloudflare.com/automatic-platform-optimizations...
The problem with long cache times is that then some readers might see an out-of-date version of your page. I would argue that that is a small inconvenience to avoid no readers being about to see your article at all.
I know there are a lot of WordPress plugins that effectively generate the pages ahead of time and just serve those. I think perhaps that should be the default way WordPress works, at least for single articles. There is little gain in generating the whole page on every hit and everything to lose.
Using a TTL isn’t great if you have a long tail of content, but pretty awesome if it’s a limited surface area that’s taking the load even if your dynamic backend is something heavy like Wordpress.
Aren't ETags exactly meant for that though?
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ET...
They can help let a intermediate cache know that content has been updated, but the cache has to be configured to ask for it.
Plus, for dynamic content even the cost of calculating the Etag is not zero.
https://twitter.com/requestmetrics/status/132144452901530010...
Edit: actually, seems the author of this post also works for/owns trackjs.com, who owns requestmetrics.
Sad to see David Walsh' blog this way, of hidden sponsored posts. Really wish people could stick to ad-free principles better these days.
(Tone note: Technical discussion about the modern era of development that just happens to be prompted by this article, not a criticism of the targeted site. I've written that sort of website myself enough times!)
Yeowch. That's barely faster than a page view per second.
I have noticed in several sites (generally APIs rather than end-user sites but the same principles hold) I've built lately that as nice as databases can be, there's a lot of places where things are coded to do a query per page view for things that just have no reason to be doing a query per page view. Even a "no sql" database can slow you down a lot vs. an in-process memory structure. I took one site from being able to serve a few hundred per second to tens of thousands per second by simply taking the relevant DB tables and slurping them wholesale into memory. Whenever someone makes a change to the underlying tables, a "several times a week" operation, I simply slurp the entire database tables in again from scratch. Slurping in the entire DB takes ~.25 seconds for all of its tens of thousands of rows on a low-end RDS and a low-end EC2 instance. Precomputing the answers to "all the questions we saw last hour" (as this is a service queried hourly by a lot of machines) takes another half a second or so. During the second this is happening it's fine to serve stale results from the previous version of the memory contents.
My point is I see a lot of residual code and frameworks and habits from an era that come from an attitude of 5 megabytes being a lot of stuff, but it really isn't anymore. Obviously you can't do that to thousands of things without some issue, but almost every application has these little tables like a sidebar or the list of types of X or all kinds of other things where you're better off just slurping the table into RAM and slurping the table into RAM again if there are any changes rather than constantly hitting a network database over and over again, because even if it's a completely cached query it's still vastly more load on your systems than a hash table lookup. (There is a bit of trickiness around making sure you detect changes, but one nice thing about "just reload it all from scratch again" is it's feasible. "Update just the changes" always turns into a problem because of the way an error, once made, echos forever, but "just reload it all from scratch" is a feasible level of complexity.)
I also blame the "shared-nothing" architecture for hanging on longer than it needed to. It is OK to use the architectural patterns without literally throwing everything away on every web request. I think what I describe above can still just be considered a glorified DB cache if you do it correctly, which is fine to "share". There's a ton of websites like this in the world where every page load makes dozens or hundreds of DB queries that don't change their results more than "several times a day" and as a result are very slow for no good reason.
(Many of these websites could also just run the queries every hour and serve the results with little to no loss in most cases. You want your "published stories" to update immediately, but you probably don't need adding a site to the sidebar to be reflected instantly, etc.)
Your server needs to be able to cope with the peaks, not the averages.
Thinking of "events per long time period" as "an average of events per small time period" is an easy shortcut to overloading your servers.
I have a blog written in my "micro" framework [1] that doesn't use any caching, and is hooked up to MongoDB, it has handled being #1 a few times without falling over, or even slowing down.
The secret? 1 query per page that pulls all the relevant information into the view model. Also has a hidden benefit that I can count my pageviews by just looking at the number of queries against mongo for the day.
MySQL has some performance quirks but I wouldn't say that it's outright 'a well-known performance limitation'--is this particular within Wordpress circles?
This honestly seems like a rookie configuration mistake that was overlooked during some migration between webhosts. At this point, tuning WordPress is pretty well known. The reason MySQL gets blamed is due to WP's poor DB schema. Very few indexes and the data is not normalized.
On small sites with few comments/posts, it's never a problem, but at scale, you'll see issues start to popup as the DB has to scan entire tables for each page load. This largely drove the rise of comment services like Disqus and FB comments a decade ago. It seems in recent years a lot of people have opted to just not have blog comments, instead driving discussion to dedicated forums or social media groups.
I wonder if it would survive an HN hug. I would think it would, but will have to wait and hopefully see someday. I spent too much time setting up a static blog, that I just went back to WP so I could focus on the content and not the setup.
"at least 2 database-touching requests for every person reading the post" should never be a blocker for 2 requests per second unless you truly do not understand your application infrastructure at all.
on edit: The better solution here would be to figure out where the MySQL server was being put under load -- hint, it probably was 99% idle, because unless that sidebar file was making an unindexed query there's no way things would take 500ms -- and then realize that you need to bump up the number of database connections in your php pool to be MYSQL_MAX_CONNECTIONS so that php isn't blocked on obtaining a new connection. Problem solved.
> This site was set up to be cached by Cloudflare at one point, but over time things changed. Somewhere along the line from a WordPress plugin or hosting upgrade, the cache-control headers were changed, and the caching was broken.
I find the assumption that it once worked a bit optimistic. Could be true, but people do stuff that doesn't actually work all the time. Easy for me to see someone setting up cloudflare without either verifying that it has the desired effects of reducing latency and surviving load or digging into details like request rate to the backend and cache-control headers.
I'm mildly curious why each request had 500ms latency at best. I know PHP isn't the fastest out there, and talking to MySQL on every request doesn't help, but still that's pretty slow. Also, no parallelism? That's a bit sad.
If the content isn't truly dynamic, I'd recommend to anyone just using a static website generator like hugo. A cheap VM can easily do thousands of queries per second without requiring cloudflare.
I work with a large ecommerce client that uses WP and our TTFB on an extremely dynamic page with user specific products and hitting external micro-services is still about 1 second. WordPress unfortunately makes it really easy to become very slow unless you diligently stay on top of things.