Hacker News Reader

Comment by Veratyr | Hacker News Reader

Hmmm... 39M requests per month = 15 requests per second. "20.3MM out of 39MM" of those are hitting a cache, leaving 7.2 requests per second that actually need to be run. The article says requests typically take 2.4s but it's a scraping API and many of those seconds are going to be I/O rather than CPU time. Assuming that there's say 1.0s of CPU time used per request, you should only need an average of ~8 cores/hyperthreads for this.

I suspect you could fit this into a pair of dedicated server s from somewhere like OVH or Hetzner for <$150/month (two servers to provision for peak load, which I'm assuming is ~3x as high as the troughs).

Is there any reason why this wouldn't work?

View on HN

The transactions are likely very peaky though, so it's difficult to map transactions/month to transactions/second without knowing more. For sure, a straight average isn't useful.

WordPress is notorious for non-cached requests being very heavy as well. Hundreds of PHP files and database queries for a single page isn't uncommon. And the PHP opcode caches suffer from an avalanche problem at cache expiration.

It isn't Serverless, it doesn't use any of the current buzzwords, and you can't run it on the browser side (who doesn't want to run a backend scraping API on the browser), so you'll get zero buy-in from any of the current load of entrepreneurs.

I see only negatives, don't think about doing it that way.

All of your negatives are about the perception of server/serverless, not about the actual benefits. Who knows if this company has investors or if it is actually self funding (judging by the costs, maybe not hard?).

I suspect Aeolun's comment was somewhat sarcastic, no?

While there are benefits, the biggest one the OP pointed out was reduced costs. The GP post was pointing out you could still likely achieve the scale the OP posted about for half of what they're paying on traditional hardware.

This is some budget thinking, but 20 $5 instances, 10 from Vultr and 10 from Linode in 4 different locations and 2 load balancing servers for tops $30 with DDOS protection etc. to send the requests to proper servers would easily handle this and scale without the problem. Would that work? We would be left with $130 easy to scale, huge bandwidth and computing power network.

What would you use for load balancing, where would you put it and how would you run it (say, DNS wise)? It's been a few years since I looked at it, but I couldn't find a cheap, reliable way to load balance a web app between providers.

Tbf, AWS can't do that inter-region well either which is effectively what we are talking about. They rely on DNS-based solutions and so do many CDNs.

Everyone does DNS load balancing unless you're ready to do BGP and announce/withdraw your net blocks based on health.

Anycast is nice, but niche.

CloudFlare?

I get about 150/requests/sec on a $25/month google cloud instance. Each request queries or updates a mysql database. The database is about $200/month though. Although the database doesn't break a sweat until about 1000/requests/sec

Yeah...my takeaway from this article is that they shouldn't be using AWS Lambda. If you wanted to go through all of caching and load balancing responsibly you could serve their load for <$100/month.

I'm sure it would work, but you wouldn't get a lot of the benefits like a global distributed cache (API Gateway uses Cloudfront as it's cache), and you're obviously dependant on support from a very low cost supplier in the event of a problem with the hardware.

Would it be worth introducing that support dependency for ~$200 a month? Maybe, I guess it depends on what you're making from the service.

So using a CDN? I wouldn't list that as a 'benefit' of a cloud-based solution, anyone can use a CDN.

you still have to run the cache from somewhere, and you probably want to be able to deploy during peak load times.

Not sure about the price, to me $75 for a decently beefy server seems low? Compared to the cheap stuff that most people use because most people are only at like 1 or 2 r/s.

Also serverless might sound silly but isn't it nice to not have to do ops works? Surely worth $200/month.

> Not sure about the price, to me $75 for a decently beefy server seems low?

You can argue whether Hetzner is reputable hosting or not, but $75/month will get you a v3 Xeon with 32GB of ECC RAM.

> Also serverless might sound silly but isn't it nice to not have to do ops works? Surely worth $200/month.

Hosting something on Amazon doesn't absolve you of operations work, it's just a different kind of operations. Instead of managing the OS/software updates, you need to manage S3, IAM, CloudWatch, optimise time/memory consumption to reduce cost, etc.

Maybe we just have different definitions of "ops works" but it bothers me when people claim that you don't need to have anyone minding AWS because it's serverless. You still need to have people monitoring, managing permissions, optimising to reduce cost, pushing updates, etc.

> Maybe we just have different definitions of "ops works" but it bothers me when people claim that you don't need to have anyone minding AWS because it's serverless. You still need to have people monitoring, managing permissions, optimising to reduce cost, pushing updates, etc

Thank you. Instead of being able to find a server ops guy from a large pool of experienced folks, you now have to find someone specifically with AWS-ops experience - a growing but much smaller pool of people, ime. Getting IAM permissions correct for complex stuff is just not simple, for example, and there's some use cases IAM stuff doesn't (or didn't) cover.

Had a client who couldn't give me EC2 access because - at least at the time - they'd have to give me access to all their EC2 instances, not just the ones I needed. Now... this may have been IAM-inexperience, although, at the time, my digging seemed to indicate it was the correct conclusion.

I keep recommending people look at places like Hetzner over AWS. People keep wanting to do AWS anyway, and then end up paying 2x-3x as much for the servers and more for the ops work. I shouldn't complain - people who opt for AWS even when it doesn't make sense are what my profit margin is made of.

(I have clients I recommend AWS to as well, but reducing ops time and/or reduced cost are rarely if ever part of those discussions; AWS is the luxury option)

out of curiosity: What makes you recommend AWS in those cases?

It would typically be if they have use cases where they either want to do large batch jobs, or have other needs where "add-on" services AWS provides is worthwhile. Often it's a matter of urgency, and AWS will be a good first approximation, and we can start migrating onto a hybrid or fully dedicated environment depending on what's most cost effective down the road when they know more about what they actually need.

Sometimes it's a matter of politics and trst - AWS is trusted, and if their hosting cost doesn't add up to much of their total costs, then that trust is sometimes more worthwhile than squeezing the cost down. E.g. I'm working on a project for a VC now, and their yearly hosting costs are likely to be below what I'm charging them for the initial development of their system, so spending time selling them on a cheaper provider is just not worth it for either them or me.

> Instead of being able to find a server ops guy from a large pool of experienced folks, you now have to find someone specifically with AWS-ops experience

Yes, and too many companies don't realize that by moving to AWS, they are not only narrowing the pool of developers/operations people they can hire from, but the salary of these people is higher.

I'm not against AWS, there are absolutely businesses where AWS makes sense (as the article describes). However it's more than just a question of dollars and cents to move to AWS, you also have to factor in the HR consequences of losing access to dime a dozen developers/sysadmins.

For most businesses I have worked with, I would argue losing access to developer/sysadmin talent outweighs their perceived benefits of moving to serverless.

replying to myself - the client at the bottom had a "move to AWS" mandate from management, they moved, then found out that at least some of their use cases weren't supported. As silly as it sounds, research a tool before you commit to it. Some things are certainly not discoverable until midway in to a project - life happens - but moving ops to AWS in ... 2014, 2015, 2016 - there's enough "known" caveats and gotchas already documented that basic things shouldn't be a surprise. The cost of working around those gotchas often surpasses any first year savings from a migration, but as I've learned - "that's someone else's budget" is sometimes a valid excuse (from some perspectives anyway - as a shareholder, I wouldn't want that justification).

OK well here's the sort of ops work that serverless kinda absolves you of:

- making sure you don't run out of inodes

- having to manage DB backups

- making sure your cache didn't stop working

- updating your box for security updates (beyond just your application requirements)

It's a tradeoff (architecturally as well), and for a lot of stuff you'll end up wanting full-ish control of your box. But for getting the ball rolling it's nice to not have to first spend a bunch of time getting postgres to start consistently on boot.

You still have to actually manage deploys themselves, of course. And keep an eye on costs, too. But a lot of these services are getting close to "the code that you are running locally, but in the cloud". A far cry from "OK, first lets get apache up, then set up some proxy servers, then set up supervisor" etc etc.

> OK well here's the sort of ops work that serverless kinda absolves you of

I wasn't trying to list every bit of ops work for server/serverless. My point is that as a business you cannot expect that once you move to serverless you don't need anyone to manage it anymore, or that the time to manage it drops to zero.

Yes, serverless does have its distinct advantages over rolling your own, there is no question about that. But serverless still requires experienced people to configure/manage/monitor it. I've seen way too many managers get giddy at the thought of having their developer/ops time completely free to perform other tasks, and that's not the case. You still have to budget for people and time to manage serverless architecture.

We use Azure, when you deploy it actually goes down, like proper down-down, timeout errors. Long periods of down-time.

That doesn't happen when I deploy on dedicated servers.

It really does depend on your setup and cloud guarantees nothing.

Also, 2.37s for parsing a HTML doc and then making it more readable is awfully slow, assuming it gets sent the HTML doc.

Do you mean Azure goes down or your application?

Do you mean if you do an in-place upgrade? or a VIP swap? A inplace should just go through the update domains so if you have your extra 20% provisioned as normal it should be fine and a VIP should be pretty quick and painless.

I use linux on my desktop and have for almost 20 years. Managing a server is a piece of cake for me. AWS is much more difficult for me.

Nope, but how much are you paying the sysadmins?

You don't need a fulltime sysadmin to handle this. If you can't set up the servers yourself, you can set up a contract with a sysadmin for initial setup and ongoing support, which can be billed hourly.

It would be my impression that 1s of cpu time used per request is a very very high value for the given problem. Am I incorrect?

For whatever reason lambda cpu time is much slower then test on your laptop time. Something that takes your laptop 5ms may take lambda 100ms. Some may be oversubscription, some could be cache thrashing...

Takes time to bring the container up to execute.

No, I also suspect it shouldn't use nearly that much CPU time. I just wanted to make sure it was high enough that I wasn't underestimating it.

I guess there will also be some cost reduction if you can skip the server management. You will still have to monitor how your code is doing but no longer have to patch them.

The requests are not uniformly distributed throughout the day, so you can safely triple the calculations you just made.

You realize he actually did take that into account, and tripling his estimate is exactly what he did?

Reply on news.ycombinator.com