Ask HN: Cloud vs. Dedicated Hosting | Hacker News Reader

Ask HN: Cloud vs. Dedicated Hosting

52 pointsjack_pp96 comments

I work at a small startup (~5 devs) and right now we are hosting everything on Azure because we have some credits. Our use case does not really need instant scaling all that much since we're a B2B business and know ahead of time how much compute we will need.

I've been trying to convince my CTO that we would gain more benefits from a dedicated hosting solution (Hetzner) but haven't managed to convince him yet.

Right now we have around 5 small dedicated VMs for each dev (2 cores, 4gb ram) and 2 production VMs (4 cores, 16gb ram).

His arguments are that things are easier to manage in the cloud and he doesn't want to have the extra burden of manually configuring infrastructure since none of us have any real background in systems administration and in the future when we will scale to more production machines it would be easier to just start another VM than wait for a new dedicated server. We also use Azure managed MySql which helps us to see performance easier than if we rolled our own.

Do you agree with me and if so what arguments should I use? Would you recommend something other than Hetzner?

View on HN

Your CTO is right. Your job and his job is to provide value to customers fast to generate sales to justify the next round before you run out of money. There is no other goal. Saving on infra cost is marginal compared to saving on time. Your salary and other developers salary has more impact on the burn rate than infra costs. Being able to deliver your “killer features” on time is significantly more valuable to the company than saving infrastructure costs. When you are a mature company, then thinking about infra costs might be great to revisit. Too early now unless your business model is tied to infra costs and this has a direct pact on your bottom line.

+1 to this.

If you're concerned about operational and cost efficiency, keep a thought in the back of your mind that one day your scale would need you to host things yourself. Your design choices should work with you at that point rather than lock you in some proprietary platform. But, the key is to get to the point where that becomes a problem. That day is not today. Azure VMs, AWS something whatever, get to your product features and get yourself to a spot where you need to worry about scaling and the savings it would bring.

Does cloud save on time?

Depending on how you handle it.

Sometimes it does outright, see Heroku, or AWS Lambda.

Sometimes you try to bite more than you feel comfortable to chew, and get bogged down configuring kubernetes or terraform when you don't need as much.

In the simplest case, I think, the cloud is also an inevitability. Can you imagine starting up by buying some used servers, colocating them in some datacenter, calling an operator when they need tweaking a BIOS setting, scheduling a visit and downtime when you need to replace a failing NIC, or a degrading HDD? This is what life without the cloud used to look like. Yes, a EC2 instance is cloud.

Of course you can try a middle ground, dedicated hardware managed by a provider, e.g. what Hetzner offers. If your load is very stable and predictable, your stack is very simple and mature (an RDBMS + Apache + PHP, ore maybe Erlang / Elixir), and you are ready to either keep a hot spare or sustain rare but prolonged downtimes, you'd be fine with a small amount of dedicated hardware.

A large amount of dedicated hardware may be economical, but only at quite large scale (see, well, Facebook).

You are underestimating the reliability of modern servers by a lot (otherwise your EC2 instances or random VPS would randomly reboot way more often than they do since if the server crashes the VM is still gone).

Also failed PSUs or hard drives don't require downtime, since those are hot-swappable and redundant in servers. If you want to optimize for (physical) hands-off operation you can even add an extra 1-2 hard drives that are configured as hot spare that the RAID (be it hardware, software or ZFS) will automatically use in case one of the active ones fails.

And the performance difference between dedicated hardware (be it dedicated server or your own) can be quite staggering compared to what you actually get from the cloud providers.

No, and it is way more expensive. But it is fashionable :)

It's not fashionable, and it adds real value, but it's definitely costly because you have to spend on operations. I'm working on a solution to gives you an experience like Heroku on AWS without breaking the bank.

Still, this depends a lot on what features you are actually using. If your use case is a few dedicated servers and maybe a https load balancer, you can have that at Hetzner for half the money. Paying the premium for any of the large clouds only makes sense if you really plan on using their advantages, i.e. well-integrated managed services like RDS and SQS or quick scaling.

AWS also has the advantage of reassuring customers/insurances/investors. “Did you encrypt at rest? Is your db port accessible from the internet? Are you backups autodeleted?” “Yes, we’ve implemented this check and this check, and concerning encryption at rest, we ticked that box in AWS RDS.”

Same here, +1 for this.

As someone who is freelancing (and using Hetzner-cloud + dedicated) and also working as a cloud-ops (using AWS and mainly Azure as cloud providers), I can say that going with a cloud provider like Azure is simpler.

Saving cost is definitely important, however, seeing that there are multiple dev/salaried, I doubt it's really a big concern (you also don't seem to have that many resources).

Most of the time, with resources from Azure, you would get features already completed and existing (I am looking directly at Application Insights for example) - with resources like App services, Web apps and Function apps you get ease of debugging (yes, there is a cost associated, however, in the long term when you scale, you definitely see the advantage).

I've personally suggested my colleagues to try out a dedicated server from either Hetzner or OVH (that specific one needed at least 128GB RAM) so going with Azure would of been quite expensive (and probably less than $200 per month from either OVH or Hetzner) - but they've decided to stay with Azure and I can confirm that they are not paying less than $900 per month.

It seems like cost isn't always the main factor, but rather ease the time it would take to prepare/set up everything.

Perfect answer.

We're working on a solution to gives you an experience like Heroku on AWS without breaking the bank, and I'm pretty sure we won't be the only one working on such a solution.

Flexibility of cloud and simplicity and convenience of services like what we're building will boost the speed massively, plus you'll never be concerned about outgrowing the solution as everything is built on top of AWS.

You're grossly overestimating the cost of managing infra for a small to mid startup. You can serve A LOT of customers with very little tech. Especially if your code is not a pile of crap which waste tons of resources for no reason.

I love Hetzner. Why not use their cloud offering? You can also combine that with dedicated servers if you need a lot of computation power. We use 4 VMs + on AX161 in the same network for Pirsch [0] and it just works.

Spinning up a new VM takes a few seconds, just like on Azure, and ordering a dedicated server is only a click away if you really need it. So you get the best of both worlds.

I don't believe that you can be "too small" to save on infastructure. Hosting our setup on Google Cloud would probably cost 10x what we currently pay for, maybe more considering that our largest server has 32 physical cores...

[0] Blog article about our setup: https://pirsch.io/blog/techstack/

I'm using Hetzner cloud right now and it's affordable and quality is good. You can do snapshots and have monitoring, etc.

If you're just running plain Linux VM's running some software without need for additional AWS-like services or fancy orchestration, it's good value for money.

I also have a couple of dedicated Hetzner hosts - also remarkably stable, but it will of course take longer to rebuild if it goes down. You should have a continuity plan in place.

Yep, we'll add another database instance in the future. But 250€/month for the AX161 is something we first have to justify...

Btw - your website looks really slick. I could easily understand the product by just casually browsing. And the site is fast too.

Thanks! It's always nice to hear positive feedback :)

fyi, both inline code snippets and full code blocks appear as as solid white blocks to me in light mode.

Oh, thanks for letting me know, we'll look into it!

At that scale the absolute cost difference is pretty small, right?

None of your dev team has a background in sys admin or managing a DB? You're going to be in for a period of difficulty while you gain those skills and that's going to be felt quite heavily on a team of 5. Now if only one of you picks up that skillset now you've also got a high Bus Factor (1) in the event something goes wrong with your self-hosted infra.

It's entirely possible that you're right in the long term but in the short term it doesn't make sense because your team is too small and the cost difference, while probably proportionally large, is almost certainly tiny in absolute terms.

I would explore the possibility that you want to be able to easily move off of Azure in the future. This may or may not be worth planning for but it likely makes more sense than going to self-hosted right now. If you can get buy-in that you shouldn't be on Azure long-term then you should avoid Azure specific services that will make it harder to switch. This would mean keeping your stack simple - compute and DB where possible. Avoiding things like Event Grid, their PubSub product and possibly even their CI/deployment offering.

Again - even doing that very possibly does not make business sense and you shouldn't approach that analysis with a specific conclusion in mind. Does your product need something like their AI offerings? If so it's probably a lot faster and more effective to lean into the products they're offering than trying to run an alternative yourself on their compute.

(1) - https://en.wikipedia.org/wiki/Bus_factor

>Our use case does not really need instant scaling all that much since we're a B2B business [...] His arguments are [...] we will scale to more production machines it would be easier to just start another VM than wait for a new dedicated server.

For most B2B businesses offering a SaaS product, I don't think on-demand variable "web scalability" is the decision framework for AWS/Azure/GCP.

Instead, it's really about faster product development iteration. If new SaaS product features can be delivered to market faster because it can leverage many of the higher-level managed services in the AWS/Azure tech stack portfolio like DynamoDB or Cosmos DB -- without your 5 dev team building the equivalent tech stack from scratch, that's when paying the profit margins to AWS/Azure are worth it.

In other words, your CTO and the devs have to look at your future product roadmap and see which features would require extra developer costs in re-inventing aspects of AWS/Azure that may negate the cost savings of a "dumb" IaaS like Hetzner.

> In other words, your CTO and the devs have to look at your future product roadmap and see which features would require extra developer costs in re-inventing aspects of AWS/Azure that may negate the cost savings of a "dumb" IaaS like Hetzner.

If there are actual development costs from moving from Azure to dedicated servers then they've already coupled to Azure and at a very early stage. The point of giving credits paid for for Microsoft. It would also seem they've unknowningly coupled too which should be worrying. I would be very worried about a CTO who unknowningly coupled to anything.

Also security, compliance, disaster recovery, many geographic / data locality issues, and maybe most importantly comfort-level for enterprise IT departments. For example we have a product in a regulated industry and running on AWS gets us past many of the hurdles our large customers throw up.

You have a very small environment, what do you expect to save by moving to hetzner?

your aws infra: 600-1000 month? hetzner: 200 per month?

Is 800/month gonna make a difference to your organization?

Imagine you launch new feature with performance regressions and suddenly your database is at 100% CPU usage

Managed database service will allow you to scale up the instance in a few clicks, so you can redo the feature with a better plan, instead of rushing to roll back or deploy a hot fix.

If 800 is not gonna make a difference, why not increase the dev's salary with 800? Apparently he is concerned about cashflow, so he probably doesn't get paid enough to not care, or things like a better working machine are difficult to get.

Hetzner provides cheap and performant VPSses. Depending on the geo, it might be a better fit.

CTO says they have to expertise in "systems administration", yet they're running VPSes, and not managed services. This is exactly where some dev installs some software which is accessible by default. There you have your little dataleak.

So imo, CTO is wrong, but post doesn't have to be right per se.

Because a developer already costs a lot more than 800.

I understand the value being assigned to ease of scaling up but I’m curious about this.

In the early stages of product development, how often does this scenario occur where your database is at 100% CPU utilization and the solve is not a new database index but instead a hotfix?

At previous project, we have a team of 15+ people.

AWS bill is 5k plus per month

One deployment lead to regressions (^2 growth growing along with usage of the new feature)

When it became a problem I just upgraded the instance and made a ticket

the ticket was solved within next sprint, so 2 weeks or so minimum

I pay 258 euro/mo at Hetzner where I paid about $2000/mo at gcloud for databases and Kubernetes. I did the switch 3 years ago which means my business saved about $61k during this time period.

in these 258 euros I have 4 dedicated servers that never went down so far. One of the 4 is only for the database while the other 3 are for kubernetes which is managed and deployed using Rancher. Out of the 3, my cpu usage is 4 out of 28 so plenty of room to grow as everything is on overkill.

I do daily backups of the database and upload them to an S3. I know this is not the perfect solution but, for a company that small (mine is similar) $61k are not something to be ignored. I used that money on Adwords which brings much more value than spending huge amounts on something which rarely "shines" for my use case. In fact, I had more downtime in the cloud than I had with Hetzner - mostly because they perform updates (to kubernetes, to database engine etc) or they simply have outages.

Everything is subjective.

moving from k8s to dedicated

not the same as VM to dedicated

These price differences are way too small, I would expect to go from ~3-5k to ~200. If that "doesn't make a difference", you have too much money.

That's a rough estimate based on his numbers

5 small VMs, 2 big VMs, 1 managed db

I believe your CTO is right. You are too small a team to care about saving infra cost and if that was a business problem he would already be looking for solutions (e.g. hiring a dedicated engineer)

To me looks like your are trying to solve a non-problem.

>none of us have any real background in systems administration

If you don't have sysadmin that could tackle dedicated hosting but you have knowledge to keep things running in cloud - stick with the cloud for now.

You don't actually explain what benefits you would gain.

If it was my money, yes, dedicated FTW. Egress charges... nien danken.

If it's you boss' money, what's the point arguing it. Just let him have his cloud bills.

This started from him telling me that we need to keep in mind RAM usage for a language model we were planning to use. I said that we shouldn't worry about RAM right now because RAM is cheap but apparently it isn't so cheap in the cloud so.. here we are :)

Sounds too me like you're saying "Let's take Hetzner so we don't have to be smart" and your CTO is telling you to be smart. In a way, he trusts you're smart enough to figure out a better way then just throwing hardware at the problem.

Also, while cloud pricing can be daunting, once you reach a point when Azure costs to much, you can look at optimizing things. There are a lot of techniques to keep cloud cost acceptable. Look up "FinOps" if you haven't already.

Sometimes it's cheaper not to be smart. It's often the case. Because the reality is, most poeple aren't smart. They're average. If you're entire case depends on "we're smart" then more often than not you're going to end up with egg on your face.

Your solution is to spend time and money on reducing costs. This only makes sense when the costs to move outweigh the costs to reduce costs.

Maybe that’s a another interesting question. “Hi, I have a big language model I’d like to run on AWS. What’s the best and cheapest way to run it?”

Here is the thing, why isn't don't use cloud computing use dedicated servers often not considered the right answer. Also, it could be a case of they do hybrid hosting where they dedicated servers for the heavily lifting background work and cloud for the customer facing.

Make a 15min spreadsheet of $/month server cost and developer cost for differents usage scenario, include things that can speed up developpement. Instead of fighting.

Don’t argue with him, provide numbers for the perspective he might not see. Tell him you want to help him make the best decision. He might loosen up on budget for cloud if he see that you are fumbling around to save a few Go of RAM.

> provide numbers

Even a spreadsheet with such numbers is primarily speculation and your upper management knows this (they make such things all the time). As such, if they did not want to agree with you before this spreadsheet won't make a difference. If they want to agree but don't want to take the time to present exactly that spreadsheet to their compatriots, it is worth making it. If you don't work for a very large company (essentially a bureaucracy), this situation probably will not occur.

His goal on keeping RAM low is probably for growing to thousands and millions of customers. And he’s right- right now it would look like RAM is cheap, but when you start having much users, it adds up.

There is a line between premature optimization and total resource ignorance.

Personally, if your CTO is already talking about worrying about resources, then I would be telling them they're moving to dedicated servers.

You can get a dedicated xeon + 64GB DDR if that's enough for $50/mo (random google: https://www.binaryracks.com/dedicated-bargain/#plans ). Maybe just put the NLP part on a couple of those.

But it sounds like your boss has the idea cloud is where it's at, it's hard to shift. I'd put numbers on it like how many less connections the cloud he wants to pay for can handle, vs the hosted solution for less $$$.

Its a game of tradeoffs, you are going to be spending money somewhere. Consider:

- You ignore resource usage and move to dedicated hosting. You are now wasting money on inefficient processes. To remedy this you apply software engineering practices to understand the model and design it correctly.

- You ignore server unit costs and stay on cloud, but improve your runtime characteristics. You are now wasting money on the hardware, but your software is efficient. To remedy this you have to learn how to manage infra and take it on as constant maintenance.

It seems on your team it makes more sense to me to just pay for the cloud and stay in your software wheel-house.

Why are you developing on cloud vms? Those should be on local dev boxes rather than in the cloud.

Also, you can use vms from other vendors. You don't have to go full dedicated.

You have two small prod servers. Yes, other cloud services would be cheaper but it's probably not worth the migration time. Stepping on dollars to pick up pennies.

Realistically, where you host is a business decision because it has costs. If you want to couple to a cloud provider, etc is also a busines decision. The CTO's job is to provide you with an expert opinion. He has to convince you not the other way around.

In this thread, we have a lot of technical people who would want this to be a technical decision. But it is not. It's a business decision and has long term consquences often lasting longer than the technical staff who made them. The reason cloud providers give credits is for vendor lock-in. It's an entirely apporirate decision to want to avoid vendor lock-in and when people are selling technical products to companies they sell the business value first and the technical value second.

The business decision here is do you want to spend time and money just now on moving or do you want to spend time and money later on. If you want to avoid vendor lock-in go for a cheaper hardware solution then you need to spend time and salary money on setting it up. If you want to stay with Azure then from what I've read in this thread from what you've said you're going to have to spend money later on for resources and you're going to have to spend time and money later on managing the costs. This seems literally like technical debt.

While it may be frustrating to technical people that a business person is making a decision that effects how they work this happens in every other industry. Business people decide the vendors, they decide the materials, etc all based on business reasons.

Your CTO is correct. Generally speaking you start building using the path that can deliver value the fastest to customers. This is usually something like Heroku. At some point you run into platform limitations and have to migrate to another platform. Generally this would be AWS or some other cloud. Eventually you outgrow the cloud provider and go into a data center. By that point you’re talking number of racks vs VMs.

Another thing to consider, as a developer, is how to make your app as independent from vendor requirements. For example, if you build an app in such a way that it only works using SQS you’ll run into issues down the road.

Also, why y’all using dedicated dev VMs? IMHO it sounds like there are other issues to solve before you migrate the app somewhere else. If I was in your position I would work on making the app easier to setup and deploy. Your CTO has a lot to consider and being a CTO at a startup isn’t an easy thing. How long does it take to deliver a feature to customers? How can you help speed that up? What are the engineering goals of the CTO and how can you help with those goals? Make his or her life easier and work with them. Continuing to push an issue like this will cause you frustration and make your CTO dismissive of your ideas and suggestions in the future.

> Eventually you outgrow the cloud provider and go into a data center. By that point you’re talking number of racks vs VMs.

How many companies actually hit this stage? I can only think of a few, and usually it's because they have very specific hardware requirements (e.g. Dropbox's whole business is file storage, or if you're doing something that requires tons of GPUs).

Rarely companies hit this. I’ve done work in a few startups and we never hit that point.

Also, it’s about priorities and goals of the company. Security and control is the main reasons I see companies migrate to data centers. Generally things like GitHub Enterprise are being used.

> How many companies actually hit this stage?

In practical terms you should expect to never hit it. The point where such scaling (also the main value add of aws/azure) really matters and you start looking at an entire DC to lease, you've arrived in the realm of speculative fiction. You should not plan to get there, just as you should not plan to get a winning lottery ticket.

Another vote for "listen to the CTO." In my mind it's a tie of you're paying $0 at Azure since you're on credits, and you don't yet have the expertise to do anything else, by your own admission. The company grows by the product, not the infrastructure. Ship things, get money, hire sysadmins when Azure becomes cost prohibitive (read: not necessarily "not free anymore," but too expensive to stay), then migrate.

>since none of us have any real background in systems administration

For the sake of security and performance you're better off using azure (especially the hosted database).

Otherwise you'll need to configure iptables, backups, software updates, high availability (assuming it's a requirement) all by yourself. If you don't have experience in these things you don't want to be learning how to do them with the thing that generates your revenue.

I briefly did devops with Softlayer in 2011 (dedicated hosting) and now I do devops in Azure.

To be quite frank: Both hosted and cloud have a learning curve. The difficulty is about the same.

Furthermore, many businesses shoot themselves in the foot worrying about scalability too early. Are you really going to jump up to Google scale overnight? (No one does. Chances are your software has bugs that will prevent this, even if you are running "in the cloud.") When your business is well-run, your scalability needs will be predictable enough that you can provision hardware in advance.

That being said: I really like Azure. If you want to "save money for the company," focus on making sure that your software is reasonably efficient. This way you can pay for 10 servers instead of 100 servers.

5 VM for each dev ? That sounds a bit strange given that you only have 2 prod VMs.

Ignoring the weird dev VMs. The setup you have at the moment is tiny so cost isn't the issue here. In terms of dedicated you'll likely have to buy that for a least a year and so I don't think fully dedicated is the correct move here.

There are smaller cloud providers that'll give you those 2 prod VMs at a fraction of the Azure price but again price really isn't much of a factor here so I think staying on Azure is the right move.

I think personally you should look at moving those 5 dev VM's locally i.e. not in the cloud. It's really strange to have that many dev VM's.

Probably 5 VMs in total, one for each dev since they are 5 devs.

> 5 small dedicated VMs for each dev

Make more sense for that to have been 1 per each dev I still read that as 5 for each dev and there are 5 devs.

In that case I would say definitely dump them as they are pretty much useless then you are down to 2 VM's which doesn't really matter how you are hosting to be honest.

There's also one argument which your CTO didn't make, but really should have: Hetzner is less reliable.

For instance, both my postgres master and replica went down one day the other weekend, because another machine in the same rack popped the breaker. In my specific application, I can cope with a once-in-a-blue-moon blip like that -- Hetzner saves me literally thousands of dollars per month in compute and bandwidth, I like doing [a limited amount of] sysadmin, and it's worth the trade-off. But, how do you think your CTO is going to feel about this move when all of your dev VMs go down just before a big customer deployment?

If you don't have a background in managing your own infrastructure, use the cloud. I say that as someone who prefers dedicated server hosting...but I've managed servers/infrastructure in a past life.

Your CTO is very right.

I use Hetzner dedicated servers for some non critical project, I find it fun, but it’s time consuming and requires some knowledge.

Just one example, having an encrypted storage on a dedicated Linux server is not trivial.

Not sure how the cloud helps here other than it essentially being pointless there since the VM hosting provider can just extract the crypto keys from memory (or already has the keys in the first place if one uses a managed product...).

Data being encrypted at rest is usually a security requirement. But if you only care about passing security audits, you can use the managed keys of your cloud provider.

If you don't trust your cloud provider enough to let them manage your encryption, you should probably not use their services. If you use an cloud provider from USA, you should trust the USA too as they can access your data without letting you know. It probably applies to other countries too, but the big cloud providers are from USA.

They are however the entity most capable of actually accessing the data for nefarious purposes. Any other entity would need to go through the hoster first, which means either mounting a physical attack on the actual data center or compromising their systems to gain access.

However with dedicated servers the next step is more complicated for both the hoster and the potential other attackers, since the only chance they have here is to catch the key material during reboots, which is quite a bit more noisy than silently extracting the key from ram of a running VM (especially since they would need to mirror the disks or other complications to have access at the same time as you are using the machine).

If it's a colocated machine (or if the hoster trusts you enough with their dedicated machines) one can use TPM/Secure Boot to make the MITM attack very very difficult (since one can use something like dropbear to have an encrypted and authenticated connection to enter the encryption credentials and TPM/Secureboot can prevent manipulation of the bootloader).

I trust in Azure much more than solutions like that. I'd imagine most of their customers do too.

Then we are back to the starting point. If you are already trusting Azure what's exactly the threat model the encryption is supposed to protect against?

I see two: someone physically stealing the storage devices, or Microsoft disposing of the storage devices without destroying them first.

Which both mean that you cannot really trust Azure. Although the latter is an easy mistake to make for hosting providers and thus indeed a valid threat model.

The theft scenario is exactly the reason why you wouldn't want to trust Azure to do the encryption for you, since they can just extract the plain text data from the virtual machine. Unless you are saying you don't trust Azure datacenter techs only?

And for preventing the leakage from old storage devices very non-fancy and simplistic encryption setups are good enough (most of the complexity comes from trying to prevent/detect MITM or manipulation by the hosting provider).

> Unless you are saying you don't trust Azure datacenter techs only?

Yes, and also the local datacenter people. I understood that Microsoft didn’t build a datacenter in every region and rents some space in existing datacenters managed by other companies.

> Right now we have around 5 small dedicated VMs for each dev (2 cores, 4gb ram) and 2 production VMs (4 cores, 16gb ram).

Everyone seems to be missing this part. This seems nuts. Why is each dev getting a dedicated VM?

This was my exact reaction!!! Why do dev need dedicated VMs? I can see a QA environment but devs should most definitely be using local dev environment… vagrant etc.

A few places give a personal VM to each dev so they can run multiple services on one machine without wrecking local performance.

I've been working at companies that didn't and people had to run N services, rabbitmq, mongo, postgres locally just to test some stuff.

Add this zoom on top and your CPU will hate you.

A half decent laptop would solve this problem.

It depends on how much crap you're running on it. At the above company everyone was on recent macbook pros.

The model I tend to favour in general is to have service being edited running locally and let it talk to services running in a dev environment.

This video provides a great lens for thinking about the cloud 'spectrum'. Using it as a lens, your product is living somewhere in the middle, and what you're proposing is at one end of the spectrum. Again siding with your CTO, your job is to deliver value, not infrastructure. Maybe you save a few dollars in app costs, but you'll more than eat through those savings in maintaining infrastructure.

https://youtu.be/k3bUvZf89k8?t=697

Azure is the absolute worst, consider moving to AWS once you run out of credits (and maybe stop at GCloud before that to pull the same trick).

Keep everything in docker + some cloud agnostic orchestration layer so it's easy to migrate to other clouds (or even to dedicated).

I have experience with all of the clouds and I'm never going to use one. They are ridiculously overpriced and they get way more expensive the bigger you are. What other arguments do you need? I've seen mediocre startups doing the same and ending up with a 1M/y AWS Contract while pulling very little money - and eventually scrambling to move everything to physical servers.

If you're going to be a unicorn, it may be worth to pay lots of money to the cloud because it will be a tiny fraction or your future profits - if you're unlikely to hit that, just roll something up on Hetzner. It's not that hard, and literally what people did for a decade before AWS. I've never been purely a devops guy but I've been rolling my own servers for 15+ years; I understand not wanting to deal with hardware failures (they happen and you'll waste lot of time) but a dedicated / VPS is as hard as running things on AWS EC2.

That said, it's a lost battle.

Most technical leaders are dead on the cloud and some even think they're saving money. It's the same with scrum, microservices, performance not mattering or other nowadays myths.

I would listen to the CTO. I would also recommend packaging your applications as containers already and run in Docker/Podman/K3s. That way once you hire someone to deal with infra they can easily bring the containers into a Kubernetes cluster and scale either way. And once it's on Kubernetes it's quite easy to migrate that wherever you want as well.

That's the beauty of containers right there.

Eventually you'll end up with a Kubernetes cluster and a bunch of Helm charts either way.

Are you sure that your applications is really not working or did you configure some arcane route within Azure wrongly?

Having a self-hosted server eliminates a lot of these questions. For hosting in production a managed cloud VM might be sensible as these services often provide additional functions like doing health checks, automatic renewal for tls certs, protection against attacks, etc..... Sure, you can get that on your personal server too, but it does need maintenance.

For development/testing/staging I prefer to have a non-managed server. That said, for most applications it is enough to keep the system of the host up to date and that is easier today than ever before and you might even be faster here than large cloud providers who have a bit of a momentum until problems get addressed.

Also cloud providers do change their infrastructure and might force you to adapt. That can mean extra work out of the blue. They tend to leave your systems as they are, but if you need to update certain components you have to read up on their docs again.

Current status sounds like Wild West. It's probably because you don't have any people who know about infrastructure on board, yet.

I'd challenge the idea of dedicated infrastructure per developer in the first place. There's little reason why this could not be done locally on a developer machine using Vagrant, local Kubernetes, Docker Compose, whatever.

> he doesn't want to have the extra burden of manually configuring infrastructure since none of us have any real background in systems administration.

It makes total sense to use the Cloud then, especially if you already have some experience there.

> Our use case does not really need instant scaling all that much since we're a B2B business and know ahead of time how much compute we will need.

If you had the expertise in the team, going with VMs + dedicated on Hetzner would have been a good idea, though.

I started with a dev team of three: One knew Linux admin stuff pretty well, I knew enough to keep the lights on, neither of us had proper cloud or container experience. We've done pretty fine with our Ansible managed Hetzner setup so far.

Especially if there are few moving parts (E.g. just HaProxy, a Postgres instance + your API) and your dev laptop has the same OS as your servers, I'd say it can be a lot easier to work with than some cloud provider.

I love fiddling around with servers but your CTO makes a perfectly good argument in the lack of other context, so I commend your being careful in constructing an argument of your own.

A lot of it depends on what you're actually doing, but for a B2B SaaS app you could potentially make an argument around vertical scaling. Nowadays you can spin up a 128 vCPU VM with 2TB memory on EC2 in minutes, so why bother having lots of separate smaller servers? I've seen people making this argument quite well recently, especially in relation to using things like SQLite instead of a networked RDBMS. But.. I can't see you making such an argument very convincingly unless you're heavily into ops or play with this sort of stuff on the regular, so I'd stick with your CTO's approach for now and keep alternative ideas in your pocket for hackathons, spare time, or prototyping sessions.

> Nowadays you can spin up a 128 vCPU VM with 2TB memory on EC2 in minutes, so why bother having lots of separate smaller servers?

For 3600$ [0] per month. Spending this kind of money makes sense if plan B is to spend man-years optimizing your app or if you have an insanely expensive per-machine license, but otherwise you are better off with multiple servers - not only for price, but especially for availability.

[0] Assuming a c6a.32xlarge in us-east1, with only 256GB of RAM. Dedicated machines are more expensive.

Yeah, but you wouldn't start there. You'd have a 8.. 16.. 32GB VM or whatever with fewer vCPUs until you have thousands of paying customers. It's just that you can get there so vertical scaling becomes a good possibility.

Neither of you is "right" because there is no general "right" answer, it will always depend on your specific situation. If the CTO has given this thought and decided against it then leave it be. If its an oversight or under-researched topic then it doesnt hurt to take some of that research load and become the advocate.

However, something I didn't learn/internalize until later on in my career was that not everything is your call. In this case, you absolutely should defer to the CTO's judgement because, you guessed it, it _is_ his call. He is the one ultimately responsible for these decisions both in authority and blame. If hes "wrong" and you are spending more than you should then thats on him. But being in his CTO position has earned him that right, you should respect it.

AWS, Azure and GCP provide services beyond just hosting. They address concerns like log mangagment, automated patching, secrets management, database hosting, encrypted volumes. to name a few. A growing company will need those very quickly and these issues can bog you down if not implemented correctly. Also, the big three deliver infrastructure-as-code, which may seem uneneccesary at first but I have found it incredibly easy to manage and govern them compared to ad-hoc management of servers and network firwalls.

Your CTO is smart.

While the "cloud" can look easier (and is indeed easier to start), there is quite big its own complexity around it - learning, tooling, specific problems, performance limits, costs and so on. Also most of this knowledge is not quite reusable.

But if you use the cloud carefully, selectively and avoid vendor lock-in as much as possible, you will be OK and it will be not hard to move to Hetzner (dedicated) incrementally later.

If managed mysql is enough for you then probably no need to change.

If you have more significant compute needs a dedicated server can start to make sense. Linux admin is becoming a bit of a lost art in this container-everything world but actually it's not that hard. I run a hardware server for a lab and once it's running there's very little work, super reliable, and very cost effective.

I am old sysadmin and understand your motivation. Technical people want to control their environment by having full control of their stack (which the cloud doesn't give you). Those paying the bills see you as the big cost and want to avoid over reliance on an individual/s.

If you aren't paying the bills use Azure. I do pay the bills for my product and use Hetzner.

For your sake I hope your salary isn't low enough that it makes moving a few small VMs to Hetzner even worth talking about.

With them being so small maybe it does make a difference. Small costs here and there add up quickly. That being said, this is probably not where they should be trying to save money as it will eat up their time instead.

Regarding dedicated dev VMs: You might be able to save a few dollars if the developers can run their VMs locally on their dev machine, e.g. with docker or docker-compose. Downside: Maybe not equivalent to production environment; creating the setup costs time (salary).

Regarding production VMs: How much do you expect to save by moving to self-managing everything? I’d expect a few dollars saved for the hardware, but a huge amount of time spent setting everything up and documenting it properly. Net loss.

Let me tell you, I host a B2B app on Hetzner because I cannot use typical cloud providers due to GDPR, and I hate it. So much fiddling around to get everything working and I still don’t have high availability because I don’t have the expertise in that area. It’s a huge time sink and takes significant time away from actual development.

Conclusion: Use your time and money wisely. Do what speeds up adding value to the business. Saving a few pennies but missing out on growth and recurring revenue is a bad deal.

I'm considering moving to Hetzner. Would love to know if this is Hetzner's fault i.e. downtime? Also, is this Hetzner Cloud US or Europe?

Downtime is not Hetzner’s fault. Any downtime is caused by me updating and rebooting the VMs. Hetzner Cloud in Europe.

i think the cto is right. is the saving significant? if not, just focus on the building.

You are wrong.

This debate is long since settled. Use the cloud, and devote that mental capacity to building things customers will pay for.

Hetzner and friends are on a lot of ddos lists these days - so a fair number of hacked systems in the mix.

If you are selling to enterprise / businesses AWS or Microsoft are much better places to be reputation wise I think.

https://aws.amazon.com/compliance/programs/ To get a feel for the paper pushing exercise they’ve spent time on.

BTW - I put a large memory instance on ECS anywhere, worked great - if you have very high memory needs something to consider - a basic dell gets you 3TB these days - very nice for development

Reply on news.ycombinator.com