Hacker News Reader

Comment by boba7 | Hacker News Reader

By not using AWS I have saved thousands. This one simple trick.

Ah yes you're at the fifth level of AWS cost management consciousness. You may have skipped the first four levels.

First is the simple test case using something random like Lambda and S3 after dragging through the Whizlabs course. This costs you $5 a month.

Second is the migration of something not particularly complicated but a bit meatier which works out cheaper than your capital expenditure coming up so you can write it off without having to fill in a purchase order and argue with accounts again.

Third is the overconfident architectural approach of multi-account, multi-AZ with peering all over the shop as recommended in the best practices, certification and architecture documentation. Approving nods all around on delivering this, despite the operational expenditure being slightly higher than predicted on your hacked up and not totally complete Excel spreadsheet for cost management.

Fourth is the first bill. This immediately points out your inter-VPC, inter-AZ transit and shitty shared tenancy CPU provision you had to crank up quickly at the last minute, costs more a month than your entire infrastructure capex for 3 years did before you got AWS resulting in sad kitty faces all around and a scramble for a cheaper option while trying not to get fired. This is all while Bezos dances on the flames coming from the dollar bills he's burning in a giant bonfire cackling loudly.

Fifth is several months later after being on the job market, eating ramen and searching for a company which "doesn't do any of that cloud stuff". You eventually find a position herding a couple of 1U supermicro boxes with CentOS on them which require the odd disk replacing here and there and some PHP updating without going near Terraform, Jenkins or any of that shit. Your entire infrastructure upgrade is automating your entire job into a few ansible playbooks and spending 6 hours a day inspecting the insides of your eyelids.

Hilariously true. Yes, AWS has some services that are cheap and not really replaceable, like S3, but once you come near high performance EC2 and RDS and add multi-region in, you’ll really have a bad time. Believe me, Ive been there and in the end had to migrate most of the applications to another provider or host on prem.

Eh. S3 is easily replaced by Wasabi. And less easily by minio.

Damn that’s cheap on wasabi. No egress fees! Thanks for pointing this one out.

Do note that wasabi is aimed for long-term retention of cold data. There's no egress, but there is a fair usage policy (2x data stored IIRC).

I use it for live data, but I put it behind BunnyCDN. I serve a few terabytes per month using that setup, no problem.

Most real world uses that I’m aware of follow the 80/20 rule, meaning you’ll store much more than you serve. And your hot paths are quickly cached by a CDN.

Wasabi has an important 'gotcha': they charge you for a minimum of 90 days of retention, even if you delete an object seconds after creating it.

https://wasabi.com/paygo-pricing-faq/#minimum-storage-durati...

That detail isn't mentioned anywhere on their pricing page or cost comparison calculator: https://wasabi.com/cloud-storage-pricing/

I don't find that pricing objectionable on its own, but I'm wary of shopping with a vendor that advertises price as their main selling point, but buries such a potentially costly pricing detail.

I've helped with two migrations, and warned them; they didn't listen.

Last place also did datadog for log aggregation. So Bezos' hands in their pockets, datadog's paws on their money.

Logs are something I didn't love self-hosting, but I also don't have a good sense that ELK was really an ideal setup for that. I can imagine running my own hardware, what's out there for logging?

(DD is super expensive, as you've said)

The only thing I can actually suggest is to avoid the hell out of logging if you can. It’s a really expensive concern and should be treated so from day one. If your system has nothing actionable to log them don’t log it.

At a high level from observations of trying to handle 100Gb a day.

Cloudwatch is inflexible and expensive.

ELK is expensive to run and administer. The commercial variants are even worse.

Splunk is expensive and slow.

Datadog is expensive.

Loki is expensive to run and administer.

ryslogd and grep starts to feel like a viable solution eventually. Then you realise that you need about 50MB of that 100G a day of logs and enlightenment comes.

Thanks for the response, yeah that resonates. I think the sheer volume of logs has been what necessitates expensive solutions.

I've found that my current team really relies on those and an event store for observability, and it's just clunky and awful. Logging for, well, logging purposes instead would be a dramatic improvement.

If any AWS service has replacement at this point, it has got to be S3 (unless locked-in via peripheral aws services).

https://wasabi.com, https://backblaze.com, https://min.io some of the ones I've seen frequently mentioned on news.yc

It's fucking ironic how AWS selling point was how it was more cost-effective than managing your own infra.

Yet another classic SV-style rugpull.

There’s a very subtle difference between “managing” and “operating”. There’s also a rather large jump from paying AWS to manage and hiring a complete operations team.

But you are right, running on AWS isn’t cheaper. It can be, but you have to design your software to take advantage of the platform.

Me too. The data transfer costs would be the killer for me (video conferencing).

Reply on news.ycombinator.com