Wordpress on AWS: smooth and pain free | Hacker News Reader

Wordpress on AWS: smooth and pain free

cloudonaut.io

193 pointswiddix111 comments

This is one way to do it, though I can't recommend it as the best way.

The first step is building your WordPress site correctly, and for me, that starts with using trellis (https://roots.io/trellis) and bedrock (https://roots.io/bedrock).

He's partially correct about S3 for media and uploads, but you want to take it one step further and use Imgix for actual image (jpeg/png) delivery because it'll do all of the transformations, cropping, etc. for you without WordPress mucking it up. At that point you have a fairly stateless WordPress installation and can spin up any number of instances of it. Here's my plugin for handling both S3 and Imgix: https://github.com/jawngee/ilab-media-tools/

CloudFlare is really the best thing to happen to WordPress since Imgix. Using CloudFlare's ability to cache at the root and sending correct caching headers from WordPress to CloudFlare, you are virtually indestructible.

We run a news site that gets about 500K uniques a month off a $20 instance on Digital Ocean this way. Granted, it's static, eg no user directed dynamic content like comments, votes, etc.

We build a lot of WordPress sites this way.

And avoid caching plugins. Most of them are garbage.

If you're going to add (full-page) caching, Batcache [0] is the best solution. Combined with a Memcache plugin [1], you can use this to store your page caches in AWS Elasticache.

[0]: https://github.com/automattic/batcache [1]: https://github.com/tollmanz/wordpress-pecl-memcached-object-...

You see Taylor Lovett's SimpleCache plugin? nice solution to those too https://wordpress.org/plugins/simple-cache/

You really can't use WP without some form of caching (be it a plugin or proxy).

I do wonder how much energy is wasted from poorly optimised websites & systems?

The root post of this thread is pointing out that you should always run WP behind a CDN like CloudFlare. That will handle all of the caching that you need. If you're running a truly static site, then you're going to have very few actual hits to your server.

I would recommend Fastly for caching. It's brilliant.

Please try to avoid CloudFlare when possible. StackOverflow recently dumped CF.

http://www.slashgeek.net/cloudflare-making-internet-little-b...

$20 a month for the DO droplet, but out of curiosity what's the monthly bandwidth bill from AWS? That's always the kicker - bandwidth from AWS is not cheap.

Why would he pay AWS anything?

He says he's using S3 for media and uploads? Unless I'm misreading it.

> I’m not a fan of Wordpress, as it is neither cloud-ready nor serverless.

Well that's a funny opener.

TL;DR: rather than configure your uploads to go to S3, one of AWS's most battle hardened and time tested offerings, use a service that was launched 10 months ago and can definitely scale perfectly and will never have any issues. Then you can even keep on managing your htaccess files using a WordPress plugin and skip that pesky version control!

It's always validating to see someone else write a blog post of the same decisions you made. This guide is good. For doing the uploads folder onto s3, I couldn't recommend S3-Uploads more it has served me well https://github.com/humanmade/S3-Uploads

I work at Human Made (owner of the above repo, and an enterprise WordPress agency), and we run all our sites on an all-AWS stack. We have a few other plugins for related AWS pieces, including email via SES [1] and a dynamic image thumbnail generator that runs on Lambda [2] that might also be relevant.

Using S3 for the uploads solves our problems without needing EFS, as we disable file system changes through other methods. Plugin installs or upgrades are great, but aren't tracked in the git repo, so we do those via local upgrades and deploy out to AWS instead.

The article also mentions installing WordPress as being a pain, but you can easily automate this using wp-cli [3] if you want to. Your RDS containing the WP data is going to be shared though, so there's no real need to install more than once.

[1]: https://github.com/humanmade/aws-ses-wp-mail [2]: https://github.com/humanmade/node-tachyon [3]: http://wp-cli.org/

Hey thanks for your work, I wanted to mention wp-cli, too, which is especially excellent. Especially with one of my applications where the content portion of the site is wordpress, and the "app" portion is other tech, I needed a way to integrate with the wordpress site, and found wp-cli made some tasks incredibly easily.

Great information, thanks - and I have a question: have you had issues with S3 being only eventually consistent for some usage modes [0]? If so how do you work around them? My first intuition is that this property of S3 could cause occasional problems with plugin upgrades, file changes, etc.

[0] See "Amazon S3 Data Consistency Model" on this page: http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction....

Can someone chime in why hosting this behind Cloudflare is a bad idea? For something which is a blog/CMS (put another way the # of readers >> # of creators), I'm envisioning using a single instance behind Cloudflare with heavy caching turned on? Cloudflare caches and serves everything. For free (you can pay them if you want). Host your assets in S3 or GCS, also put them behind Cloudflare. If your instance goes down, you have a rough chunk of your caching window to bring your instance back online (which if it's well encapsulated should be a few scripts to spawn a new instance and re-point the cloud provider internal DNS which will update ~1s)

I use Cloudflare for mine (https://www.garysieling.com/blog) - they catch 75% of the requests. For sections that are rarely hit, you'd still get 404's from downtime, and there are more valid URLs to pages on a Wordpress site than you'd think.

> Can someone chime in why hosting this behind Cloudflare is a bad idea?

It's not a bad idea. Who said that?

Put your blog behind cloudflare. It will cache most of the requests and save a lot of load and bandwidth on your instances.

This guide is spot on for WordPress on AWS.

The persistent and shared EFS volume was the missing piece for running WordPress without frustrating patches.

And CloudFormation is the AWS best practice.

Convox has a few serious WordPress installs. The only difference is that it uses ECS so those EC2 instances can be utilized for more apps.

Also you can hack on the WordPress site locally with Docker via 'convox start'

https://convox.com/docs/wordpress/

Disclaimer: I work on Convox.

Do you have anything to say about EFS and WordPress (or other PHP CMS that are basically designed for a single instance?)

Latency on stat() - PHP is often a oop-monster these days with thousands of fs reads as it loads the code? (there are certainly ways to dodge that overhead though).

Have you seen the filesystem stall for any reason? Any caveats with concurrent writers (such as cache/ directories or generated images with deterministic paths etc).

You say without frustrating patches - are there any general limitations with using EFS?

Have you used self-managed NFS for the same purpose before? Any comparison with EFS?

I have seen some users file system calls stall due to the EFS quotas.

This was on a workload much tougher than WordPress.

I haven't run my own NFS for comparison.

I recommend reading the NFS v4 spec. AWS has all the pieces to implement the protocol as well as anyone can.

We have some notes on small file performance for EFS and ObjectiveFS at https://objectivefs.com/howto//performance-amazon-efs-vs-obj... that you might find interesting. Scaling out PHP on AWS is definitively easier with a shared file system.

IMO the better way to run is with _no_ shared file system at all. Use CodeDeploy to ship the deploy files to all ec2 servers, and then use S3 to store all uploaded files. WordPress doesn't need to do any file writing outside of the `uploads` directory.

This allows you to sidestep a large amount of pain points with shared filesystems, and every ec2 server has local disk performance.

I tried EFS and it seemed very slow for the wordpress site I tried to host. I didn't think EFS was meant for hosting a webserver but mainly for large files?

Before you hop on board with this "pain free" solution, you should accept the fact that you may end up maintaining this: https://github.com/widdix/aws-cf-templates/blob/master/wordp...

There should be easier way, e.g. in Azure you can start WordPress (includes MySQL, limited to one app service instance) with one click, not sure what are the costs of maintaining that, but at least starting one in that case is pain free.

Infrastructure as code offers a lower maintenance cost over time in many scenarios.

Sure you can click to set up WordPress on other hosts.

But how do you scale capacity of the servers or the database? How do you peer the network to your other cloud resources? How do you mirror the stack in another region?

With CloudFormation resource management is a few API calls away.

You can also use CloudFormation in point and click mode to create a stack and update its resources.

It may be overkill for some cases but for serious systems it's the best practice.

Quick shout out to Sandstorm.io where you can run Wordpress and many other useful apps fairly easily...

https://apps.sandstorm.io/

I gotta shout out wpengine which has been hosting sites for me for years. Scales up with demand and managed so I don't have to play devop. I don't consider myself a WordPress guy but it does fit a number or use cases.

Funny, I had the opposite reaction to that file - appreciation that the whole stack is encapsulated so well.

Maybe that is from having been through the ad-hoc or "well, its easy to spin up, but this only works for a single node" setup roundabout a few times.

> Before you hop on board with this "pain free" solution, you should accept the fact that you may end up maintaining this [...]

True, although ongoing maintenance of a CloudFormation template isn't significant. It's essentially a specification of the state of the infrastructure, which AWS is then responsible for making come true. It's also their stuff from CloudFormation back, so they're on the hook for making sure nothing changes that would break it (relative to Terraform or other alternatives).

Also keep in mind that AWS services are usually encapsulated from each other, which is a good thing. The result, though, is that defining something like this involves a lot of parts. You need to define its private network, its subnet masks, its EC2 instance types, its database type and configuration, etc. The author also took it a step further to define CloudWatch alarms and scaling policies, which is great.

It is something that requires understanding if changes are necessary, but it's not something that you would expect to need tweaks over time.

> There should be easier way

There is--he could have started Wordpress by just launching an instance with userdata that installs it and sets it up. This isn't really comparable to that because it involves multiple servers behind a load balancer, a CDN, a hosted DNS zone, a separate relational database service instance, and autoscaling.

But Azure has terrible MySQL offerings, which really limit the appeal of running WordPress there.

And this cloud setup is going to cost probably for few hundreds of dollars a month.

Has anybody working towards serverless version of wp?

Something like https://beta.getshifter.io? Unfortunately only as 'full SaaS' available, so the 'build process' and hosting is managed by them.

Not if you have many domains/customers.

You can easily manage to get this setup at 30$/mo with Scaleway for example.

The ELB is $20/mo. A micro RDS is $12/mo. 3 x t2.micro EC2 instance is $15/month.

So you're looking at ~ $50/mo.

For a setup that can now scale to any volume independently on the HTTP, compute and database tiers.

Overkill for a personal WordPress. Just right for a serious business.

If a Wordpress site is valuable enough to justify paying an engineer to set this up, hundreds a month are probably not a concern.

> Has anybody working towards serverless version of wp?

I'm curious on what a "serverless version of wordpress" consists of, please enlighten us on the technical details of a serverless wordpress, sincerely. And how would you have a "serverless shopping card" too ? I'm really curious about the how it. If you're talking about a static blog generator then it already exists (Hugo,...) is this what you call serverless ?

I read "serverless" in this context to mean a statically generated site, yes. I also infer that dynamic content like authentication, shopping carts, forms, etc would be third party solutions via Ajax and CORS.

I built a service [1] for this latter part (authentication, CORS sql database, and static file hosting). The pieces one could use to build anything for a statically generated site. Adoption has been... non existent up to now.

[1] https://www.lite-engine.com

By serverless I mean version of the wp which could use Google functions (https://cloud.google.com/functions/) or Amazon Lambda (https://aws.amazon.com/lambda/) or Microsoft Azure Functions (https://azure.microsoft.com/en-us/services/functions/)

It would provide unlimited and automatic scaling, simplified setup/architecture and we would pay only for what we use.

Most blog content is static, why would you need to use lambda for anything ? you already get automatic scaling by putting static files on S3. What Wordpress functionality would need to be implemented with Lambda ?

I run a static blog on S3 using Lambda. I upload a markdown post to an S3 bucket, Lambda runs Hugo (could be Jekyll, or whatever) to regenerate the appropriate files, then uploads it to an output buckets, invalidates the files it needs to in Cloudfront, etc.

Not infeasible to think you could use Lambda to generate a static cache of a WP site onto S3 and update files on demand.

Yes, you can do this. But, what about a staging environment? How do you keep data and code in sync between environments? My company runs a lot of wordpress instances, and we've wound up shipping most of them to Pantheon. (pantheon.io) Not only does it take care of environment issues, but it gives you redis and implements caching as a service. As the post says, wordpress fights AWS a bit. We leave it to the people who are experts at winning that fight.

(I know this reads like an ad. I'm not affiliated with pantheon except as a user. Pantheon comes close to making wordpress not suck.)

Plug that site into GTmetrix.

Requiring the $100/mo plan and charging an additional $30/mo for HTTPS seems.. dated. It makes me wonder if they're behind the times with other things too.

I can host several WordPress sites on a 5$ / month droplet. How much would this AWS version cost? In 99,99% of the cases it's not every an option

This is clearly for sites that require more than a fraction of a droplet to run. Just because WordPress can be run on a tiny instance doesn't mean that every WordPress user does -- some sites are actually popular.

Stick CloudFlare as a CDN and configure it, enable some caching plugin on WP and you'll need some serious traffic to saturate that small droplet.

What's serious traffic?

The first page of HN can send 10k users to a blog in about one hour. (don't know the peak rate per second).

You could host several WordPress sites on a free tier AWS EC2, too. Where things get a little more complicated is when you're getting real traffic and want to be fault tolerant and redundant.

RDS is here because keeping your database on the same server as your web server is not ideal at scale.

EFS is here because at some point you want to scale horizontally and it's one method to share underlying code (and perhaps cache).

ELB is here because if you have more than one web server, you need a load balancer

And as you scale, you could add services. This guide isn't for anyone who can host a tiny site on a tiny instance. You'd do that in AWS the same way you would with Digital Ocean.

I'd love to see the bill of materials for that.

Its _only_ running 2 PHP instances + 2 RDS instances + EBS storage costs + S3 buckets fees + ELB fees + CloudFront fees + traffic costs.

Came here to say the same thing - I'd love to see a monthly cost on a setup that looks like this... or anything on AWS. I can run many many WP sites on a $10/month box (not AWS) with proper config tunings.

What service do you use?

Linode's $10 plan recently bumped to 2GB RAM so I've been using that (Digital Ocean only offers 1GB)

FYI: This post is not about your personal 2-reader a month Wordpress setup but for sites scaling to the size of TC.

FYI: You can run a professional blog on wordpress.com for about $100 a year (highest plan) and it will take millions of visitors.

The smallest production instance on AWS is about $100 a month (c4.large) for 1 core (+intel HT) + 4GB ram + a few GB of EBS volume.

The setup requires at least 4 hosts, thus that's a start price of $400 a month + the other fees (hardly 10% more :D).

It's f* insane to pay more than $400 per month to run a blog. No matter the traffic.

  >It's f* insane to pay more than $400 per month to run a blog. No matter the traffic.

It's insane if your site isn't being monetized at all and downtime doesn't cost you anything. What you spend on infrastructure should be a formula of risk vs. reward taking into account availability vs. lost revenue for downtime or visitor bounce due to errors, speed, etc. Site owners will happily pay whatever the cost is as long as they are not losing sales and their net income is positive.

Wordpress.com is more available, more stable and more dependable than a hacky setup on AWS you made yourself.

Not to mention that guys who can do that kind of custom setup are going for $150-300k a year nowadays. And it's gonna take man*months of work and maintenance.

I am one of those guys who can do that kind of custom setup for large scale high volume sites and the actual setup is not as complicated as it sounds. The maintenance is literally a day a month, if that. You can even automate it if you like. And if you have me on a maintenance retainer then you are not paying the equivalent of my $X00K per year salary. All of my clients who don't have their own corp DevOps team like having someone they know and can call at any time with issues instead of submitting a support ticket and waiting. Again, it all comes down to what percentage of our income do we feel comfortable spending on infrastructure? It has nothing to do with oh my - $X00 or $X000 per month sounds expensive.

If you're aware of the particulars of how they work, why are t2 instances not a production instance type?

Because t2 are NOT production instances.

They are cheap, overrallocated, burstable instances with a low quota of CPU usage (about 10% for a t2.micro). They can burst over the quota for short period of time and they can be interrupted for short period of time.

In short, the instance can pause anytime and stop processing traffic.

Try running a production database on that see where it gets you ;)

If you're using a t2 instance for a heavily used production database that has a relatively steady amount of traffic, then you obviously don't understand the use case for it.

There are plenty of production workloads that are burst based in nature, and for those, t2's make plenty of sense. The micro is also hardly the only instance in the family - if you go to the opposite end, the t2.large has a baseline of 60%.

They also are not interrupted or paused, so I don't know where you're getting that. If you run out of credits your performance is throttled to baseline gradually over 15 minutes.

t2s are fine for production if you understand their strengths - workloads with low levels of constant usage and periodic bursts of higher resource utilization.

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/t2-instan...

I'm not talking about your personal 2 reader a month setups either. There are high amounts of traffic, I just have proper caching setup. Administrative interface is the only thing you can't cache and that's low traffic.

I have to disagree with the author. I have setup Wordpress using OpsWorks and it is a much better way to do this especially considering all of the stack can be auto managed via Chef. Updates are handled automatically and if you get a surge of traffic, autoscale kicks in.

The only issue is getting the plugins to auto update on all of the servers, which I am fairly certain you can do with Chef or WordPress just handles it.

In my humble opinion, WordPress is something to be surpassed, that is why it fails on being cloud-friendly. If someone makes a system as customizable and flexible as WordPress it would be a huge success.

I've been waiting on this for awhile, currently I run most sites through jekyll and for clients that need CMS options I hook them up with http://cloudcannon.com/

I've gone the WP a number of times and it's been more headaches then it's worth. Everything seems like a hack on a hack. There are things like sage/roots that make it a "little" better.

>> Everything seems like a hack on a hack

My experience with WP has been the same. Seems like you get a basic theme installed, and then whatever functionality you need you just download, install and configure a dozen or so plugins. Can't find a plugin that you need? Oh, then you can just write a plugin that you download, install and configure on your own.

The bloat from that approach just drives me nuts.

I hope your right, but part of the problem is the traction WordPress has. We tried to go with something other than WordPress, but ranbinto all sorts of issues mostly because everything the marketing people wanted to use was already built for WordPress. We also had a hard time finding contractors to make themes and stuff because they all knew WordPress but didn't want to commit to something else.

My company used WPEngine for a while until we wanted to move from a blog. subdomain to a subfolder for SEO reasons. WPEngine doesn't support this, so we decided to host our own.

We are on AWS and have a pretty sophisticated Chef setup, but I want to stress that if you are a SaaS company, you do _not_ want your Wordpress install as part of your production network. By running Wordpress yourself you're taking on upgrade responsibilities, and you need to be pretty diligent in upgrading in order to avoid exploits. At minimum you should put Wordpress in its own VPC; we ended up running on Heroku instead to keep it as far as possible from our production infrastructure.

If I were setting up a new wordpress blog for a company I'd go with WPEngine if possible, if not I'd host on Heroku, and only if it needed to scale a lot would I consider hosting it myself on AWS.

Most blogs consist of trivially addressable content (that is, everything has it's own URI, and the page is essentially "functional" -- the same URI will return the same content each time) and is thus trivially cacheable. Cloudflare or its many competitors will handle this extremely well.

It's trickier when there is content on a page that has to be dynamically generated fresh (or nearly fresh) per page load. Most blogs never get into this territory (or have no reason to, at least) and most of us have experience with WordPress at the "look y'all, ima start a blog" level of sophistication, so there's this kneejerk toward feeling like this is total overkill.

But it's not if you have the problems that owners of complex, interactive, heavily-trafficked WordPress sites have.

Unless you pay for a larger instance where the cost turns out to more like 50 bucks a month you get very limited bandwidth out of AWS, which is why I stopped using it. I suppose if you are running a corporate blog and cost isn't an issue then AWS probably works fine but for smaller sites and individual users there are vastly cheaper very good hosting options out there. I don't want to seem like a shill so I won't mention them by name. It was the super crappy bandwidth on AWS where basically my site would die regularly with just one or two visitors that made me abandon the platform.

Why use CloudFlare? If you're already on AWS why not just use CloudFront? I've been using CloudFront on my WP installations for a few years now, ever since they've started supporting GET and POST requests.

  Why use CloudFlare?

<3 for captchas. Maybe?

The article doesn't mention CloudFlare?

The common argument is pricing, many people seem to prefer a relatively reliable monthly amount (that in the beginning might even be zero) to CloudFront's potentially expensive traffic costs.

I do similar for my personal website on Azure, they have a Wordpress plugin that maps the upload folder to use a blob storage instance. It's a bit dated and doesn't support things like custom domains using https (for the blob.) Something I hope they address (it's 2016 guys.) I don't know if this Amazon solution does the same thing or not. However currently I have to use my Blob's CNAME domain to serve assets over https. I use Cloudflare to manage my certs instead of Cloudfront like they do here.

I'm looking for a solution for my agency to put sites onto hosted instances like AWS or Azure, but my hesitance is that they aren't managed solutions and I'm afraid of getting hacked. Does anyone have any insight on how secure these solutions can be? My Azure instance has been running for years without problems, but that's my gamble with my one site and not hundreds of customer sites. I use CloudFlare's page rules to add extra security and caching and have WordFence just in case.

I'd love some insight into how I could scale a solution like this to be hundreds of sites on cloud instances while maintaining security. Perhaps they should all be separate? Would my personal solution be good for my company's clients? Managed solutions like WPEngine are just outrageously expensive for some of the sites we have that only get a handful of traffic (Azure offers Wordpress on Azure websites.)

I'm assuming you're on Debian or Ubuntu. You can set up automatic security updates on Apt, and if you have a firewall you're halfway there. Then I would make sure each website has its own user, group, and www-user and using file permissions, make it so each virtualhost doesn't have access to others' files.

If you wanna be extra safe you can have each site on its own vps. Although what's nice about keeping a bunch on 1 big vps is that you get to save a lot of time managing it as it can take a while to propagate changes across many VPS and if they all have different setups it can get confusing.

To do 1 site per vps properly, you should have fleet management going like Ansible or Puppet so that you can maintain them all easily. That way all the code is hosted on a build server, and individual environments get built for each site.

DM me or shoot me an email, e@ericwaldman.ca if you have any questions about it :)

I do performance WP hosting and would like to try out EFS but it is not available locally yet.

What I do is use redundant RDS, ELB etc but on all the WP installs have an inhouse plugin that with any change it ioniced rsyncs that vhosts tree to all webservers and ssh's a command to them all to dump their WP cache.

It may seem hackish but the sites absolutely fly, have tried several clustered file systems but all that complexity hits reliability and speed.

Why not roll your own NFS, then?

And if your cache is all on disk, a very quick win would involve moving that to memcache, no? That also makes forced expiration very easy.

Of course there are managed services for all of these things anyway.

I was thinking where is the memcache in all those diagrams, AWS has very simple redundant memcache setup.

The thing is that users update their sites very little and memcache is another expense.

Please could someone estimate, what would be a cost of hosting a site like this, with around 100K articles and 5M monthly pageviews?

EasyEngine would be easier for creating wordpress websites :p

ee site create mysite.com --wpfc --php7 # install wordpress + nginx fastcgi_cache, with php7

ee site create mysite.com --wpredis --php7 # install wordpress + nginx redis_cache with php7

I have yet to see a site of mine, which breaks my limits on a 5$/month droplet.

Great short/quick piece. I also do multi-EC2 instances in AWS, but do NFS instead of EFS, and another CDN. WordPress still has its many challenges, but it definitely scales for many different web publishing businesses.

No thanks. I'll stick with Roots. [1]

[1] https://roots.io

I appreciate this article. It gives me another solution when contemplating how to host CraftCMS in AWS.

kindda surprised he wasn't using SES (sendy) for mailing list.. instead mailchimp...

I've run a sporadically popular (100s of thousands of uniques on some days) blog on Wordpress for years. All running on a single spot instance (if it was killed I have a script to roll over in minutes).

The database, Wordpress, the file storage. Entirely self contained.

I occasionally persist the AMI so I can spin up a new updated system base, and have regular backups of the files and database. My monthly cost is ~$9 on AWS. I've had literally minutes of downtime over the past several years.

Obviously this guide is geared to a more serious installation, but there is the danger that the more reliable a system becomes, often the less reliable it becomes. EFS and NLB configuration changes or problems taking the system offline, for instance.

It's WordPress, not Wordpress.

Someone needs to convert `capital_P_dangit` to a userscript :)

https://codex.wordpress.org/Function_Reference/capital_P_dan...

You mean like this? https://github.com/jonathanbardo/capital_P_dangit

Nice! I figured it existed, but a cursory search didn't find it. Thanks.

Reply on news.ycombinator.com