How we got our AWS bill to around 2% of revenue

It comes as little surprise to me that the AWS lightsail offering saved them money over traditional AWS services, just as they’d save even more using two dedicated servers with any reasonable provider and being able to have a hot-failover for everything.

At DNSFilter we’ve gone through the evolution of Heroku -> AWS -> VPSs -> Dedicated -> and now are setting up our first colo rack.

I did a price comparison recently between Dedicated, AWS, and Colo. Dedicated is 15% the cost of AWS for our needs, and colo will be 42% the cost of Dedicated for us.

Now keep in mind we have dedicated DevOps staff and are running at a very different scale from OP, and such a solution is not for everyone. But I personally have never understood the folks who love to brag about having spent much time and effort optimizing AWS to spin up and spin things down, when for the same cost I could just have a 10x more powerful server sitting there, on all the time to handle a spike in load, and I can utilize the extra resources or get things done faster with faster gear at 1/10th the price.

The author completely failed to do his homework. An a1.large instance is only 37$ if it's an on-demand instance. You pay a high premium to be able to pull the switch in the exact minute you cease to need one.

If he's willing to go with AWS lightsail with it's monthly plan, the same a1.medium instance type is about 11.75$/month as a reserved instance, and can be had for about 131$/year as well.

1. elastic beanstalk, no docker: EB comes with really nice defaults so that you can quickly whip up a flask app, upload it and it just works. It provisions a small ec2 instance by default which iirc costs 10ish a month at best. Importantly, any operation you do with EB will by definition be ready for continuous deployment since you don't get an option to ssh into the machine to deploy. It's extensible enough to add whatever extra stuff you need as well. Only thing it can't do is simple caching ( if you scale to more than one instance that is), but that can be solved by having a separate eb deployment for a worker that can take care of all these aux stuff (elasticache is expensive I think). In a pinch, it also scales well (though the default cheap deployment does not have a load balancer).

2. Just suck it up and go with RDS postgres. Again, I see too many things going wrong with spinning up your own db in an ec2 instance especially if that instance goes down. Im too lazy to write backup scripts and keep track of them! The cheapest RDS postgres costs 13 a month or so, but I just suck it up to power whatever side projects I do. Postgres means I'm working with something I know, and I get full text search and pubsub for free. And whatever I write is not locked code in anyform, and can be scaled up if needed as well. More importantly I'm only spending my time in technologies that are relevant for me in my day job so that's a win.

3. Github Actions to deploy to eb. just a few lines of yaml and you instantly get continuous deployment directly from your repo for free! Really can't beat that.

I have meant to try out heroku since it could be cheaper from what I have read. But I couldn't figure out what their S3 alternative is or how different they are from the canonical cloud offerings.

I'm sure it can be so much more cheaper, but I'm not good at advanced networking or sysadmin, and I'm too lazy / bored / disorganised to write deploy scripts or sshing into remote machines. I'm also always afraid of if/how long I need to re-provision a vps/ec2 if it goes down. Not that they do, but they can, and that scares me.

... if you did things well, it's just starting another instance and installing / copying things over with some minimum downtime. But if you don't have a 100% documented stack, you don't know which configurations you touched to make things work, you have files lying around, then you're probably going to pay back all your savings and more in the workdays needed to migrate.

At my current employer we never have time for maintenance, and we don't have professional expertise (the "I worked exclusively at this for many years" kind, I mean) at webhosting, security, system admin or dba, so I heavily lean towards more "managed" and cloud solutions.

On the same note, in case someone is digging into how to reduce CDN bills, I wanted to share that we are quite happy with BelugaCDN. It distributes objects stored in S3 using, in a hacky way, referrals as authentication method. Lots of money saved there.

Is there a way that the hypervisor allocates a dedicated portion of the shared L3 cache to just your instance, or is it a free for all for all of the L3 cache space against potentially noisy neighbors?

Do you have more than one, and DNS load balance, or do you just live with the risk?

One of the main reasons why I use an ALB/ELB is so that I don’t have that SPOF. If you found a way around that, please share, I would love to know, so I can save some money :)

They are using Lightsail to create pets, it has not much configuration. When you need a cattle, you need more config and complex setup which costs money.

As they say in the end, this is because they are a micro startup and don't need a huge scalable infrastructure.

It's funny as it seems to be the inverse of economy of scale. The more you grow, the higher the marginal cost. But I think that Amazon gives discount to large users

But, my advice tends to be Lambda first if it is really low volume, LightSail second, and full AWS third.

As far as Lambda, I often recommend proxy integration, where you can just use the standard API framework for whichever language you choose (Django, Flask, Express, ASP.Net, etc) add three lines of code and push the entire thing into Lambdas. This gives you the flexibility to deploy to a VM later with no code changes.

For your static assets use S3, except for the case of Lightsail where you get plenty of free bandwidth.

Besides using the free tier and other AWS services which are practically free at low uses, you can use cost effective options like Fargate Spot Instances and EC2 Reserved Instances. I highly recommend Fargate over running instances. Use Lambda with CloudWatch triggers if you need to schedule occasional jobs (or use Fargate's feature for that). Try to avoid heavy reliance on caches, ElastiCache is kind of a rip off. Move as much content to static as possible, use CloudFlare to reduce bandwidth costs. If you're gonna serve over S3, you might as well front it with CloudFront as it's actually cheaper due to caching at the edge, and also more reliable. ALBs are expensive but very useful for APIs as well as autoscaling (if you have to run instances, run them with an ASG, which also means having versioned AMIs)

I wrote a PostgreSQL DBaaS Calculator that got some traction here a while ago, and I just updated it this evening to add Lightsail to see how it stacks up: https://barnabas.me/articles/postgres-dbaas.html#calculator

No surprise, Lightsail is similar pricing to RDS with a 1-year commitment, but it's month-to-month. It's a pretty good deal until you need more than 8 GB memory, 240 GB storage, 2 cores ($115/month Standard plan). But Azure or AWS RDS are the ones to beat.

Its amazing to me that this is all included for basically free except for Application level filtering by these providers.

In general I'm not sure you can really compare instances from different service by just look at cores, RAM, and disk.

I used to have my small website and email server at Rackspace, on the smallest, cheapest instance. I ended up getting out of mail hosting (moved that to Fastmail), and putting the small website on Lightsail.

The Lightsail instance and the old Rackspace instance have the same nominal specs for virtual cores, RAM, and disk. (Actually, Lightsail may be better on disk--I don't recall if the Rackspace one was SSD like Lightsail).

The main thing the website actually does is host some graphs showing the temperature in my house. An ESP8266 temperature monitor I made uploads a sample once a minute. A script was running once a minute on the server that using gnuplot to make graphs of temperature over the last hour, 3 hours, 12 hours, 48 hours, and over all samples.

At Rackspace it ran that fine, while at the same time gathering my mail from several services via fetchmail, receiving mail for my domain on its smtp server, and running spam filtering.

On Lightsail just handling the website it was locking up every few hours. I managed to catch one pre-lockup and found the load average was something like 300.

What was happening was sometimes that once a minute graph generation task was taking longer than a minute, and that would slow down the next one, and so on. Oddly, it didn't seem to be gnuplot that was taking too long, but rather the script that took the file containing all the samples, and extracted just the samples newer than a specific threshold. At least, running everything by hand that was the only step I ever saw take unusually long.

I changed it to every 5 minutes to temporarily stop the frequent hanging, and then added a check to my script to skip regenerating the graphs if a previous instance of the script was still running to fix things permanently. I also changed it from storing the data in just a big file of samples sorted by time to an sqlite DB.

In my opinion, that approach sacrifices pretty much all the benefits that cloud proponents usually talk about, like only paying for what you actually use or scaling up and down on demand.

In fact, I'm not sure what the actual difference would be between Lightsail and a good dedicated hoster that provides backup and failover services.

- kill services with low usage

- downgrade instance size

- downgrade instance type

- merge databases

- schedule services with obvious usage patterns to shut down when not used

- use EC2 spot instances

Most importantly, it requires an aggressive culling mindset. If drastically reducing the AWS bill means staying afloat, then make bold choices.

Edit - Alsocontainerize your app from the start so that if it does take off you can slam it in ecs or k8s.

I wonder how do you cover the cost for unlimited outbound messages in your free plan...

Thanks

Author is talking about few hundred dollars here.

> I would like to put emphasis on the fact that we are a micro-SaaS product that solves a small and specific use case and therefore this kind of AWS setup worked for us. This may not work for big organisations or products where the traffic is erratic.

> This setup will also not work for folks who have a ton of stuff to do already and would prefer to use managed services and not take the additional headache of monitoring, maintaining and provisioning hardware resources on a regular basis because this has a time cost to it.

It is refreshing to see a post that doesn't pretend as though its recommendations are the holy grail for anyone and everyone.

I personally prefer saving time and effort, over a ~$500 monetary discount. But it's nice to read posts like these and learn more about alternatives such as Lightsail.

> where I learnt how to scale an application to serve ads to millions of users a day

Please lets stop this 'race'.

I would like to have a workflow of really lightweight QR/barcode transactions... but with what seems to me currently a complex implementation:

1. you have a prescribed barcode ParentA 2. you scan that barcode and create a QR code ParentB 2.a: you ascribe that QR to an object ChildA 3.n: you have that ChildA go through many iterations 3.n+: ChildA may be conjoined with ChildB,C,D etc... -- but I want an iterative QR code that follows the family tree back to parent A and keeps history of all transitions...

(I know I am wording this poorly... just thinking out loud...)