My blog can’t keep up: 500 errors all over

30 to 50k uniques per month really isn't that much. At a rough guesstimate, I'd say I've got at least one hosting customer that does that much with Wordpress, gracefully, on a site that's not optimized even a little. I charge them $10/mo.

Static site generators and everything else are lovely and all, but it's hard to beat Wordpress when it comes to being able to just plug in a new feature. Which, of course, also gives some folks the rope they need to hang themselves. As other folks have pointed out, there could be a plugin issue.

But my guess is it's the hosting provider. They can have professional-sounding support but still be running an architecture that hasn't been updated very much in the last 5+ years. mpm-event and php-fpm alone really changed the performance characteristics for bog-standard LAMP hosting.

The error logs should give a number if memory's exhausted. I bet it's super low, because a lot of shared hosting providers still try to cram as many sites as they can onto a beleaguered colo'd box. At a glance, that does look like the csoft.net architecture (they have "Two Dedicated, Redundant Servers (with Private Gigabit LAN)").

There's certainly a broader point to be made about the gross, inexorable bloat of web-based applications, but there are also plenty of people running even more bloated, busy stuff for less money with less trouble.

Oh, and you can install the traditional editor for Wordpress, they've made it available as a free plugin and it works just fine.

There are plenty of people around who can sort out stuff like this. I'm one of them, but I'm heading to bed. Email's in my profile, or just accept help from anybody else that knows what they're doing.

Consider moving away from a CMS like Wordpress. It's great at what it does but I think you've out grown it. Static content generators might help here. They let you edit off line and compile to a static version of the web content you currently serve. I'd suggest looking at Hugo or one of the many specialised for blogging. https://opensource.com/article/18/3/start-blog-30-minutes-hu...

You can then push the content to something Amazon's s3 and put cloudfront in front to speed things up.

You'll have to then offload the comments to a service but that is relatively easy and cheap. A service like discus is only $9 a month. Moving to s3 you'll save that in hosting. (I run two sites like this and pay $0.07 on average per month). Not to mention those services are pretty good at filtering spam.

There are some other options around too. https://darekkay.com/blog/static-site-comments/

Wordpress is nice but it will never out perform static content.

EDIT: I just read this in discus "Free For small, personal, non-commercial sites who do not run any ads. This plan includes everything in Plus except Direct Email Support."

$50 seems quite a bit of money for a wordpress blog, when you can have wordpress.com host you for $5/mo.

I am running multiple Wordpress customers on digitalocean $20/month plans, and some of them get more than 50k uniques per month, never had any issue. Never did much optimization. I am pretty sure the issue is your hosting. Also check your Apache error log, maybe you are using some plugin or theme that is overloading the memory with some crap code like majority of free Wordpress plugins/themes.

p.s make sure to update to php 7+. The speed difference was very noticeable when I upgraded

I'm just going to say one thing: EasyEngine.

If you need help managing/setting it up, I can do it for $15 a month + DO cost - or AWS/Google/Azure. I'll manage your stack, depending on time to migrate, there might be like $50-100 for migration.

I'm looking to start a WP management service, and hosting company -- mainly to get some MRR during my slow months.

I'm currently a Full stack laravel/vue/react freelancer.

For reference, I ran a political blog during 2016 w/ 100k hits an hour for a few weeks (traffic died down a ton), but my site wasn't functional at all on shared hosting.

After easy engine on DO's lowest tier, everything just worked. Nginx + redis cache is amazing.

Just drop me an email : patrick @ zvive.com -- linkedin: https://linkedin.com/in/patrickcurl

That’s just a bad hosting service.. I manage somewhere in the neighborhood the OP gets with a basic Wordpress deployment. It’s extremely stable, and only ~7/month. I use limited plugins and have optimized for speed.

My guess is a plug-in or theme is bugging out here. Easy solution is to remove all and slowly add until the problem exists again.

In 2015 I had “Crayon Syntax Highlighter was Killing my WordPress” for instance:

https://austingwalters.com/crayon-syntax-highlighter-was-kil...

Ignore all the comments telling you to switch to static content generation, that's generally a large time endeavour and for a blog that is getting 50k uniques a month should be pretty easy to serve all requests with a little CloudFlare tuning.

First, let CloudFlare do 99% of the work. Don't worry about optimizing the linux box, PHP, MySQL, PHP Opcode Cache, WP-Cache at first.

First, let's check that CloudFlare is caching and serving all static assets such as js, css, images, .xml, .ico. I don't see a cache-control header being returned on js, css, images, etc. Make sure this is setup in CloudFlare with Caching Level => Standard and Browser Cache Control => 1 month. Also, consider enabling Auto Minify => Javascript and CSS. Finally enable Brotli and make sure HTTP/2 is enabled under network.

Now, here is where you can really make things fly. It is possible to have CloudFlare cache the result of generated html pages, so requests don't even hit your shared hosting provider. This is done with Page Rules. The trick is going to be crafting a rule that you can match posts and posts only. With my site, I leave .html extension on my files, so my rule is simply:

https://imgur.com/a/4iNpB4V

Your challenge will be creating a matching rule while excluding admin backend pages. Also, most certainly you probably going to want a much lower cache time than 8 hours. My site content does not change often. Perhaps something like 30 or 60 minutes to start.

Good luck, let me know any questions, my contact info is in my profile.

Although a lot of the recommendations are great, and to me it also looks like there are specific hosting problems at play, possibly serious overcommit by the provider or some plugin problems, I do think there's a deeper issue:

> What am I missing? How can I be in so much trouble in 2019 with such a relatively modest task?

Something is not quite right with our web-platform when we have these sorts of problems, and the solutions also require quite a bit of sophistication.

As an experiment, I recently switched my http://objective.st site to run on Objective-Smalltalk itself on a Digital Ocean $5/month droplet (their smallest). Surprisingly made it to the HN front-page and didn't hiccup, despite an egregious memory leak on my part (fixed now).

This was reported as "blazing fast"[1], despite being a single binary hosted in Frankfurt, having no CDN support and no static files, with everything served from the embedded web-server. Heck, I haven't even hooked up gzip-compression

I don't log much, so I don't really know how much traffic it's getting. I do know that hitting it with wrk remotely I got around 1000 requests/s, locally on the same box it was around 8K. So I think there's plenty of headroom. Oh, and the logs do show continuous attempts to break in POSTing to what I presume are php vulnerabilities.

I am emphatically not saying I did something amazing or even particularly good, everything was somewhere between straightforward and egregiously simple-minded, and there were other (mostly easy) ways of accomplishing the same thing. Previously the same site was served using static files running the same code as a static site generator.

However, it does confirm my suspicion that we've path-depended ourselves into a corner.

[1] https://news.ycombinator.com/item?id=18965054

Which PHP version is being used? If it is still 5.* I would consider upgrading to 7.*. Just estimating now: 45000 users would be like about 1 user/minute, so maybe 10 requests per user per minute, could equal on peak max. 1-2 requests per second.

Some shared hosting shouldnt run into trouble.

For 50$ (Canadian, I guess), will not someone like wordpress.com do better? Their business package is USD25 per month.

I come across colleagues in the same boat as the OP, but the truth of the matter is that a $5/mo VPS hosting WordPress is really not going to handle a surge of high traffic. Been there, done that, when I had a $20/mo Digital Ocean droplet with my blog on it. Anytime I posted a link to it on HackerNews or Reddit, it used to die under the 'hug of death'.

Since moving my blog to SquareSpace, I've never had that issue. In years. Note: Not an advert and not an affiliate for SS, just a happy user.

We still have a WP blog on a VPS for our SaaS, but I am keen to move that over to another platform, because I have enough on my plate to develop code for my SaaS itself. For my blog, I just want to post my thoughts without having to wade through a WordPress Admin console and see dozens of flags, warnings and popups about a myriad of plugins that need attention. Seriously - our WP admin console causes a lot of stress just logging in, which sucks all the creativity out of me.

SquareSpace (and even Medium, for all the negativity surrounding it) makes it so easy to start with a blank slate and just write and let the ideas flow.

Anyone else use Ghost? I've got Ghost running in a Docker container on a DO droplet. It's the smallest droplet there is. I don't have a ton of traffic to my blog (at least, I don't think—I haven't found a privacy-respecting analysis system that I like (maybe roll my own... just count page request)) but it does the job.

If I recollect correctly (haven't used WP in years), the bulk of error 500-related problems on WordPress are due to poorly written plugins -- of which there are many. One of the better ways to track down a problem is to disable plugins one by one until the problem goes away.

50 CAD a month (33€) for such a small site and then it doesn't even work properly. It boggles the mind. I'm sure you can get a working setup for 5€/month without breaking a sweat.

Having a company that deals with your Wordpress installation is only beneficial if they get it right.

WordPress might be slow, but it's probably just the host or not enough PHP memory assigned to it. There is nowhere near enough traffic to bring down a WP site, unless you have a ton of bots pinging your site that never appear in your actual google analytics (because they are boths) .

* look at an actual server access log. cPanel offers one if your host is offering cPanel. * try and increase your WordPress memory limit: https://docs.woocommerce.com/document/increasing-the-wordpre... * check your server's PHP error log and see if there is an error * set WP_DEBUG to true in wp_config.php

Hugo. It's completely static and great out of the box. You can add new posts in Markdown.

Oh, good old traditional LAMP shared hosting and WP. Honestly, I doubt that combo will ever work reliably. WP hosting is bit of a dark art in the best of times, and operating under shared hosting does not help. That being said, self-hosting on something like AWS probably would end up much more expensive, and would take also some effort to set up and maintain.

Personally I would either start looking for fully managed WP service (wordpress.com), or switching to a different, easier to host, platform

I use wp-stateless to offload images, and cache-control to serve most of the content through CDN. (I also put a serverless reverse proxy infront to mixin other stuff on a common domain). I get great performace off shared hosting. https://corepox.net/devlog/simple-extensible-performant-low-...

Caching 'anonymous' requests at edge with Cloudflare workers - no need for the $200 p/m business plan

https://www.mmaton.com/2018/07/02/cloudflare-cache-anonymous...

You can export to static pages (one for every blog) with a plugin. You'll lose the comments functionality, but views will work.

An easier/quicker solution is to just stick a cdn in front, that way you can cache pages and still have to back-end work.

No offense intended at all but I can't help but wonder how a technologist/computer scientist has ended up asking for help resolving 500 errors on his blog rather than hypothesizing what the problem could be and testing his theories.

A quick Google search on the error he's getting says it's an Apache error either from invalid file permissions or running out of memory. His provider refuses to transparently increase memory so the hypothesis seems stronger here. Now just test it by spinning up a simple load balancer, clone your website on a different server, split the traffic 50/50, and see if the errors continue. Repeat this process until you've isolated the issue. Trying to solve the problem by adding a cache layer isn't going to help before the problem is understood.

Seems like Wordpress has outlived its usefulness. It's like the Windows XP of the internet. It was great and simple, until the security problems and feature creep plagued it to death.

This won't end well for csoft.net

Reading this, I feel they should hand this job on to someone else. There comes a point where if you're not experienced enough to do this, no amount of Googling and investing more time will help you.

tl;dr I get the feeling that this person has been sold the idea that throwing money at things will help (why is he using argo for a smallish blog, it's not worth the money?)

I've had similar issues with several wordpress installs.

Some of the less known issues I found that can add to this problem:

Spiders/Indexing Bots Brute Force Login Attempts Hidden Hacked Pages

Helps to look at logs sometimes, if you get hit with all three of the above at the same time, it's a tricky mess to clean.

You may find relief making an aggressive noindex robots.txt file (there are tons of bots to block which are worthless and use up your resources. - it may be best to say * disallow all, then specifically allow google, bing, etc (would love to find a way to tell archive.org it's okay in that situation))

Even when doing the robots block, I've found some aggressive spiders or bots that use some of their user agents need to be blocked via ip subnets and cidrs with htacess for example.

Brute force login has many options for slowing down access. I love a combo of ipgeo block, sucuri, graphic captcha, shield security

plugin for disable xml-rpc or use the setting for such with the shield security plugin (this blocks massive login attempts) to block all xml-rpc use.

Hidden Hacked Pages If you find bots spidering a bunch of product pages for things in other countries in your logs for example, you may have been hit with a bunch of pages being created that you may not see on your may front end, or even in the backend admin area of your site.

Removing those pages is a start, and then a plugin like 404 to 301 redirect may be beneficial - but you will likely find a bunch of bots coming to index them, and other hackers who have been sold access to your site trying to login and make more of them.

For me it was more log checks and looking up ips to add to the server block lists to slow it down. Just make sure you don't add google's or bing's ips to the list.

With a few sites I had that got some of these issues I decided to pull down wp-login.php - and in an effort to lighten up the load on the shared server, I decided to use one of the plugins that make wordpress into a static site (there are two good ones in the repo now)

Once I generated the static html and css, I created a new wp-login.php that would log all ips and user agents / date to a separate text file, and display a message on the screen warning the visitor hacker that I was casting a spell on those who access the login page.

The amount of bots trying to login, and trying to access the old hacker-created-pages that were made over a year ago is still crazy. Their bots are still coming and trying the same login page and trying to access the same pages that have been removed over a year ago.

If I had left those pages available as php pages, each one of those visits would of been taxing my sql, even if they were showing the wp 404 page, or failed login.

I hope your issue is simpler, like a bad plugin. Certainly I've run into that issue with other problems not related to these as well.

Pssh, all you need to do is install docker, convert static files and blog post into lambdas, then route each unique http request to a serverless IPv6 address located on the nearest continent.

I can recommend you Publii CMS (https://getpublii.com), it's excellent static blog tool I use from over one year and manage four blogs.