Doesn't sound that simple to me (as a complete sysadmin noob). Somebody should write a book about this.
And thanks for the reference to Digital Ocean. Never heard of them before. Seems great, might try using them :)
> Do your VPSs already have things installed or a GUI?
Nope. I prefer straight-up Arch linux. ssh in and go from there.
To add some concrete-ness to the mix, I was installing ejabberd. When it came time to ping the server... no response. I did the exact same steps on my Digital Ocean VPS and everything went fine. I had done whatever commands EC2 expects to open the right ports...
A raw EC2 instance is identical to your average VPS offering. You don't have to use all the extra alphabet soup of ELB, SQS, SNS, SES, etc.
> When it came time to ping the server... no response.
What were your security group settings? ICMP (ping, traceroute, etc.) are blocked by default, you have to enable them.
See, you say that, but then you later say
> What were your security group settings?
I dunno, man, I just want a server. Every other VPS provider works, and I can set up my own iptables etc.
I'm not saying EC2 is _bad_, as I'm sure things being extra mega locked down by default is good overall. But I don't think it's fair to say that EC2 is the same as a VPS.
EC2 fits every definition of VPS I've ever seen.
Furthermore, I don't feel very comfy when doing 'sysadmin via Google,' who knows what stuff I'm screwing up?
I had never used Digital Ocean before. But getting going with them was the exact same as my previous Linode, Rackspace Cloud, prgmr.com, and every other VPS provider I have tried. I don't need to install special command line tools, or set up security groups, or generate .pems... I ask for a server, they send me an email with a password, I log in, change it, and set up my ssh prefs. Super easy, using the same stuff I use everywhere else.
The documentation I was following made no mention of this at all.
I'm a large user of AWS but, in general, I never feel like I'm in a true VPS. My last experience:
Out of the sudden one of our EC2 instances could not connect to another, causing our HA solution to spun dozens of instances and eventually crash too. It was clear to me that the dest machine was behind some firewall, we went to the security group, the machine was supposed to accept any connection, from any port, any host. The instance itself had no active firewall.
Out of desperation I added the very own security group to itself. It worked for a few hours, then stopped again, I removed, it came back to work and still working (8+ months now)
This is only one of various mysterious events I've seen happening on AWS.
For me it that restarting an EC2 instance deletes all the local storage. I have had good success just getting a big ass VPS, and running the database locally, and pushing text backups to S3. It is trivial to manage, and in the real world, downtime is more likely to be caused by configuration wonkiness than hardware failures.
You also have to have a huge amount of traffic to overwhelm a 24core / 96GB ram server. Why not put off the managing the complexity until you really are doing 10M page views per day?
The thing is that for whatever reason, data wasn't being written to the EBS (the virtual hard disks for use with EC2 instances) and was instead being written to the "ephemeral storage", a really big local data store that every EC2 instance has that is basically a `/tmp` directory. If the server restarts, everything in the ephemeral storage is destroyed.
Local storage on a $5 Digital Ocean plan will do 2000 IOPS, where as that would cost $200/month with Amazon Provisioned IOPS. I know it is not an apples to apples comparison, but it is worth thinking about. Running a database on a local SSD is a good option for many people, and it is not an option that Amazon offers.
Essentially you pre-run a puppet manifest and have it do all the first-run processing and then store THAT as the AMI/instance. This way you have an instance that only needs a few seconds to get itself ready while also integrating configuration management (which really you should be doing these days). Just some suggestions from a sysadmin.
There are 3 general levels you can take with AMIs:
1) Vanilla instance AMI, have puppet/chef install everything for you, then fetch your app code and configure that
2) Use another tool to have your instance built with puppet/chef and then capture that into an AMI. Once it spins up it just needs to get your app code. Idea here is your services arent likely to need updated as fast as your app code.
3) Same as two, but when your production code is in release/maintenance mode, bake everything into an AMI. When you need to deploy new code, you need to create a new AMI, but you are creating an AMI with scripts so its no big deal :) All you have to do now is update your cloudformation
Obviously if you have a setup with rapidly deployed code changes you'll want to have your puppet manifest grab the latest version before deploying live when spinning up a new instance.
There's a reason many Vagrant boxes I see use Puppet to instantiate though: It's friggin simple to get started with. More advanced configurations might even end up requiring you to build your own modules and extended manifests, but I expect this configuration detailed in the article would probably amount to a couple hundred lines for the entire manifest.
I think this also underscores one of the things I like the most about Puppet (instead of Chef, which is a fine CM system as well); It's easy to get started with but powerful enough to get things done down the road. Honestly, there's never a better time to start working with Puppet than from the very beginning of a project!
If you wait until you've got a more complex configuration, you're fighting both the tool syntax and system setup requirements. Not to mention the clock that's ticking and telling you that you needed to be all up and running yesterday.
Check out the code snippets I linked to from that blog post. It's pretty easy, I promise -- the tricky thing is just figuring out the various APIs.
Maybe we should just invent something new.
- Create a new EC2 instance (Literally click "Launch instance" and select Ubuntu. - Login and install any dependencies (sudo apt-get update && sudo apt-get my list of awesome packages i need installed) - Go back to the EC2 dashboard and click "Create AMI" which is just imaging the server. - I can't attest to the ease of ELB and auto-scaling rules as I haven't used them, but I would assume it's fairly straight forward - and there is a ton of resources on AWS to help you out :-)
Have fun!
That's why I don't buy books from Packt unless I can find good reviews of it written by members of the community around the software the book is about.
Quite configurable via scripts too (install additional packages, etc). Under the hood it's basically the same - Elastic Load Balancer and EC2 instances.
Elastic Load Balancer = software based load balancer or traffic director
auto-scaling rules = if number of connections >50 or bandwidth > 100k or reponse time >10ms start a new instance and automatically update the load balancer so the new instance takes connections