As far as I am concerned, I just don't understand why people use cloud services.
Edit: Consider this article, and Geoff's statement about Azure credits.
https://www.theregister.com/2021/04/21/microsoft_revokes_mvp...
How is using cloudflare okay in this then? Cloudflare is also not Azure
(Edit: fix spelling)
Well, it might also come with contacts in the billing department.
> I'm going to miss the $13,000 USD (yes) a year in free azure credits. Just remember this amount of money when you are reading content about "how good azure is" and "what the latest and greatest is" from influencers and community leaders here on social media...
Next thing I see them being slapped with $700K bill and managers running like headless chickens all over development floor and yelling to turn off every VM, hard drive, database / whatever either resources.
From the users point of view, Cloudflare will frequently stop you from accessing things and introduces more single points of failure in the internet infrastructure. But on the good side, they have pretty good edge endpoints so your browsing might be a bit faster, when they allow you to browse.
They also offer Azure CDN, as a competing product. But I don't know if anybody takes it serious or not
= Azure has an integration to use cloudflare for the cdn.
I'd be suprised if his Microsoft Regional Director and MVP status isn't worth much more than 4 figures to him.
Those seeking to initiate engagements with Troy might care more about the fact that he pops up on HN and other high profile tech outlets frequently and the visibility of Have I Been Pwned, but the Regional Director status probably helps a lot with getting some of these engagements signed off.
He probably also receives significant subsidies from Microsoft as well.
"Firstly, I always knew bandwidth on Azure was expensive and I should have been monitoring it better, particularly on the storage account serving the most data."
...and he didn't have simple monitors in place to alert him of uncommon billing spikes.
I get your point, if he's not OK with using Hetzner how is Cloudflare any better? It's not. But the reality is Cloud operations are a fine dance of weaving services together to realize all of the heavily advertised savings. I'd argue that a lot of Troy's projects that use all of the cloud native functions could have also been implemented on much more standard stacks and, likely, been just as cost and performance effective. But that's not going to get him the advertising for Microsoft.
You want to waste money? Hire a car, with a driver, when you need it.
Want to save money. Learn to drive.
You always pay more for outsourcing stuff, a lot more, than doing it yourself.
You can buy 1000x the processing power, by buying baremetal. You can get 100,000x more bandwith for cost, when not using the cloud.
People think baremetal is hard. It isn't. It does take knowledge.
Hiring a personal car is more expensive because you are hiring a personal employee.
That said, I still argue for personal autonomy alone learning to do the thing is better in general, but I don't think it's because it's cheaper in all scenarios. And to your point some or maybe even most cloud services are more expensive relative to their self hosted versions.
Let's use your driving example (because car examples are always great!)...
>You want to waste money? Hire a car, with a driver, when you need it.
>Want to save money. Learn to drive.
This is true. You can save more money if you need to drive often if you own a car. But there are two scenarios that it still makes sense to rent.
1) What if you need a car in a different city? You just flew from JFK to SFO. You already have a car in NYC, but need one in SF. You're not going to buy a car in SF that you'll need to sell in a week. Sure, if you're going to be there longer, you might consider it, but then you're still carrying the costs of two cars.
2) Sometimes you need a truck. Maybe you have an IKEA run to make to get a bunch of desks, or stop at the hardware store for a few dozen bags of mulch, or ... But sometimes you just need a truck to get the job done. You could just buy a truck and be done with it. But trucks can be more expensive than a compact car, and they definitely have higher fuel costs. In this case, you'd probably be better off with a fuel efficient (or electric) compact car and rent a truck only when you need it.
This is how you save money with the cloud. But you definitely don't save money when you effectively rent a truck to drive to work everyday (even if you are in construction). There is a cost to renting -- it is more expensive on a per-use basis than it is if you buy. Cloud servers are more expensive than bare metal -- if you're constantly using them. It is only cheaper when you stop paying for the parts you don't need. And that also takes expertise.
Once, at a new job, I inherited a cloud server. It was costing us a ton of money per month and running 24/7 because the person who set it up never turned it off. After 3 months of those costs, they could have bought a new server with no other renting. They paid for a cloud server for three reasons: 1) they had no experience with hardware, 2) it was a pain to setup local hosting, and 3) it was faster to get running without waiting for a vendor to build a server, deliver it to the datacenter, etc... These were real impediments to the first person and the cloud server helped to get them moving. They just didn't have the longer term view of what their decision was going to cost in the long term.
The first thing I did was order a new server and make friends with our datacenter ops people. And now the only thing we really use the cloud for is archival (write-once, read-never) storage. If we ever really need these data, it will be super expensive. But, if that ends up happening, we'd be happy to pay the cloud tax.
Cloud was made for people who don't have the time, talent or desire to build and manage it in-house. You pay a premium for that convenience and that premium scales with your business growth via IT resource needs. I think that's what he was getting at in his analogy.
Isn't that exactly how companies use the cloud? Sure, there are contrived examples where the cloud is cheaper than self hosting. But the common case is that companies "use the cloud" by putting 100% of their infrastructure and hosted products in the cloud. That's what is meant when you say "X uses the cloud".
a - replication (within region / AZ at least) b - 0 software to maintain (no need to frantically patch apache / SSL / whatever) c - super quick set up / management / logs / etc
So, yes, bare metal is (on a cpu cycle to cpu cycle / GB RAM/HDD/Bandwidth) level cheaper, but TCO can be waaaaaayyy higher.
You have to do a lot of things right to get that Cloud Value, as the author of this blog post has shown. You have to do a lot of things right to get value out of on-prem bare metal as well, but those things are generally well-known, standardized, have less moving parts, and people with decades of experience and knowledge of best practices. The opposite of the current cloud landscape.
TCO is not a straight line.
This.
I always wonder how much of the "clouds" success (economic, that is) would have materialized, if the marketing term never got traction, and everyone just called it what it really is: "renting someone elses hardware without physical access, and less, if any, control over how the stack works from the metal up".
In the good 'ol days, when people wanted to put a service online, they rented the racks at a colo, and either stuffed their own hardware in or, worst case, used rented hardware.
Did that require some basic familiarity with hardware? Yes it did. Did people need to know how to setup, configure and administrate a LAMP stack? Sure. Was it guarded against sudden loadspikes by god-knows-how-many layers of abstraction? Nope.
But it worked, and surprise, in 99% of cases, it was perfectly fine if a website ran at sub-optimal speed for a few hours, or went down every now and then.
And the dirty little secret is: It still does, and it still is.
I delved into this pretty thoroughly last month - https://medium.com/@rykrk/everything-is-just-build-vs-buy-d7...
So is a LAMP stack on a dedicated machine.
> Plus, the more of the services you use,
The thing is, most webapps don't use a lot of services. Backend-Logic in whateverlanguage, a database, and a webserver. Maybe hooked up to some CRM system. That's it for 99/100 webservices.
Yes, the services cloud providers offer are amazing, they are complex, and it is natural for developers to be fascinated by complex things (I know it is for me). But it's important to realize when simple is simply enough.
Honestly, the question you need to ask in regards to cloud is a relatively simple one: Can I hire a sysadmin for cheaper than using the cloud?
The answer to that, once you start using enough resources, is more often than not yes.
Sure, it takes a while to get to that point, but eventually you will reach break even and it would be cheaper to do it yourself/have your employee do it.
I have been on both sides. Large media production companies with very large amounts of fast and redundant storage located on-prem. These range from local attached RAIDs to large shared SAN pools. Their clients also tend to be the types that sue the crap out of you if any of their content is seen by people outside their control. Switching to cloud solutions was (still is) a huge uphill battle. However, the cloud storage needs are no where near the same (not editing content from s3), but storing approved masters for distribution totally makes sense for cloud. Now that the content is in the cloud, why not perform actions on that content in the cloud. Faster deployment, better equipment, blah blah. Next thing you know your entire workflow past editorial is in the cloud. You start to analyze your expenses and compare them to on-prem amortized expenses and you see that it could be cheaper on-prem. Also, take into consideration how long it takes to bring up that new data center. You also have to look at bandwidth expenses. Bandwidth to a new site not directly on the backbone tends to be expensive for non-residential connections. The additional power expenses of that new equipment plus the cooling is also a new expense. Power redundancy you ask? $$$ Now, you need that sysadmin and possibly a small team. At that point, you go back to your cloud rep, and renegotiate fees. You have now created an entirely new department at your company on managing the on-prem.
> Want to save money. Learn to drive.
Oh please. As if learning to drive is the end of expenses. If you finance a car, you have monthly payments. If you don't, then you have periodic recurring maintenance bills. You always have fuel charges. You always have insurance charges. You periodically have parking charges.
I know how to drive, but do not own a car. From time to time, I hire a car, but it no where gets close to costing me the amount of owning a car would.
but i think where this analogy breaks down is that if i run a service, no matter how many users, peaky or not, at least 1 server always needs to be on, not "from time to time".
... I run a junker. That is to say a car that will go the dump as soon as it requires any significant expenditure, and the combination saving of not having to finance it, and most years minimal or no repairs, and only needing third party insurance makes it significantly cheaper than renting.
in practice I spend maybe 1k every year for cars. primarily for vacation. which owning a car wouldn't absolve me from spending.
In my bare-metal-hosted projects I can afford to do a lot of things that would be a major no-no in the cloud because I have so much hardware resources I can save development time doing things inefficiently and still come out ahead in terms of costs.
I haven't checked, but are the prices for Azure CDN relatively competitive with Cloudflare? I think you'd probably get similar savings going that route, and it would all be Azure.
Seriously, yeah, if he's an MVP, he'll be fine.
I really don't understand the cloud craze. Everything is more complex to debug, more expensive, and more shitty in all the possible ways you can imagine. I mean i was not exactly a fan of the VPS craze 10-15 years ago, but at least it wouldn't automatically ruin your bank account whenever you got a little traffic.
Kudos to the author for having so much money (thousands in one month?!) to waste. I wish i did too :)
Also, the cloud offers managed software as a service. You don't have to manage your own HA DB cluster or PubSub. It's all just there and it works. That can save you a lot on technical labor costs.
But yes, I do agree with your point. If you don't know what you're doing, you can nuke your budget super quick.
"This is the cost of scaling, this is the cost of owning our own infra, how does that fit into our budgeting and requirements?"
There are plenty of tools and systems that can present a sufficiently linear cost relationship to load and usage that, should your COGS versus revenue make sense, the marginal cost of increased cloud resources a no-brainer--especially versus always-paid-for hardware. If you don't have such a linear relationship you're as much in the position of deciding whether the project is viable as you are anything else.
And even if you do, which I think you’ll agree Troy Hunt does.
If I were building a new business, I would use both cloud and colo. But I do understand that everyone don't have that luxury.
The lead time for Hetzner or OVH is measured in minutes and is appropriate for the majority of use cases. Old-school providers like these used to run the entire internet before the cloud craze started.
Colocation & own datacenter is not a good choice for most startups. Seems like a lot of people here miss a step and only go from one extreme to the other. There's a middle-ground of renting bare-metal that gives you amazing price/performance with none of the drawbacks of colocation or running your own DC.
As to difficulty, they “solve” organizational problems by avoiding sticker shock when someone wants 100+k in equip that’s often a huge number of hoops to jump through and possibly months of delays, a giant bill every month and nobody a complains about the electric bill etc.
you can rest assured that even the largest company will come looking for the person responsible for increasing an expense by that large of a percentage. So maybe it doesn't come out of your personal checking account but you will certainly pay for it.
It’s easy to justify having a larger bill with more traffic. “A retail store isn’t going to complain that they need to buy more stock after selling more stuff that’s just a cost of doing business.” Meanwhile it can be hard to justify a capital expenditure just because traffic increased.
As a case in point, I worked in standing up a critical system in a large enterprise a few years ago. We spent about $12M on compute, storage, networking, etc. At operational state, it was about 40% cheaper than AWS. The problem is, it all sat there for 6-18 months filling up before we fully hit that state.
With a cloud provider, you pay a high unit cost but if you engineer intelligently your costs should move with utilization. Except for government, most entities generally want to see opex move with revenue and prefer to minimize capex where possible.
Coming from traditional infrastructure and development methods, you're mostly right. Part of the expectation of the cloud is that you do things _their way_. And even then each cloud provider does things a little differently. However, if you're willing to subscribe to the <insert provider> way of doing things it (and you'll have to trust me here) makes many things easier. Here's a short list:
* networking setup is free/cheap/doesn't require a Cisco cert. you can trust a developer to set things up.
* object storage is so much easier than any file hosting scheme you can come up with
* the path from container-on-a-host to container-in-a-cluster to container-in-{serverless,k8s} is extremely straightforward
* I turn all my dev/test servers off at night and they don't cost me a thing
* consumption based compute will result in a much cheaper solution than a VPS or colo (admittedly there are many assumptions baked into this)
* some core services (like sqs, sns on Amazon) are extremely cheap and have provably reduced development time because you're not having to build these abstractions yourself.
This all being said I'm not advocating an all-in approach without thinking it through, but to do so where it's easy and makes sense.
EDIT: clarity
Bare metal hosts set up the network for you. You may need to know how to configure a local network interface. Even if you actually rack and stack many colos will give you a drop with network set up. You don't need to do what you describe unless you are building your own DC.
> object storage is so much easier than any file hosting scheme you can come up with
That matters if your data volume is truly massive. Only a small percentage have this problem. Also AWS inbound is free so you could upload big data to AWS and warehouse it there if you wanted. Not using big cloud for everything doesn't mean you can't use it for anything.
> the path from container-on-a-host to container-in-a-cluster to container-in-{serverless,k8s} is extremely straightforward
This is the one spot where admittedly you will have to spend more in administration. You'll need to either run your own k8s or Nomad or adopt a different configuration, and you may have to think about it a bit more.
> I turn all my dev/test servers off at night and they don't cost me a thing
You could still do this. Just host live somewhere else. You could also test on a local VM, which is what we do. Obviously that depends on how big your app is.
> consumption based compute will result in a much cheaper solution than a VPS or colo (admittedly there are many assumptions baked into this)
You only see the savings if they are passed onto you. What we've seen is that Moore's Law savings have not been passed on by cloud providers. Look at what you can get at a bare metal host compared to how much the same compute costs in cloud. Years ago the difference would not have been so large.
Bandwidth costs in cloud are insane, and most use asymmetric pricing where inbound bandwidth is free. This is known as "roach motel pricing" for a reason. Data goes in, but it doesn't come out.
> some core services (like sqs, sns on Amazon) are extremely cheap and have provably reduced development time because you're not having to build these abstractions yourself.
Fair, but they make their money back elsewhere. Those are lures to get you locked in so you now have to pay their crazy compute and bandwidth egress charges.
Here's an example. There are more.
And yeah I agree about the "some services are super low cost so you get hooked" thing. Always been my impression of Amazon: they look for what they can apply scale savings on (usually object storage, it seems) and make it cheap and then over-charge for almost everything else.
The funny business in Amazon's pricing is their Egress Bandwidth, everything is rational.
You're looking at the pricing from a 'cost plus' perspective which is not generally how things are priced.
AWS core use case is IT departments being able to offload all of their infra.
It's a massive, massive advantage. It's so, so much easier and more flexible to use AWS that there is no comparison. It's a 'no brainer' from a cost perspective, which is why, cost usually isn't a barrier with AWS.
Cost only becomes a primary issue when the margin of AWS services is reflected in the cost of the product itself, i.e. when you are hosting a lot of content.
So if you are Phizer, and your IT department uses AWS, the cost is irrelevant.
If you are Dropbox, selling storage for $X/Gigabyte, and your competitors are reducing their prices and you're giving all of your margin to AWS, then you have to do something, i.e. 'make your own infra'.
Is that still cheaper? When you have 30+ very well-paid dedicated DevOps specialists? Maybe it is, I am just skeptical while looking at it as an outsider and without solid data.
I bet it still costs you more than my Hetzner one despite me not having to care about turning it on or off. I mean it's great that the cloud gives you this flexibility but you wouldn't need it to begin with if it wasn't so expensive.
Startups growing fast are the secondary audience.
The primary audience is large enterprises where their internal IT costs <<more>> than the cloud costs. Plus internal IT provides those resources after 6 months...
In what universe? This frictionless perfect vacuum where traffic comes in a wholly predictable consistent continuum?
Also, we may be taking the problem the wrong way around: do these multigig files need to be accessed by everyone from a web browser? No, it's a dump file used by specific people in specific circumstances. Then why are we using HTTP for this in the first place? In this case, only publishing over Bittorrent/IPFS makes sense and many people will happily seed, pushing costs toward 0 for the publisher (and very close to 0 if you only push to a first circle of mirrors who can then push to more mirrors, some of which can be declared webseeds in your torrent).
The opposite, I don't understand why anyone would ever put up a server if they didn't have to.
It's not 'processing power' that's going to be the 'big cost' for most projects.
It's headcount and salary.
If you can materially improve the operating ability of your company, then a few $K in cloud fees is dirt cheap.
I used to work at a 'tech company' that made a physical product and our IT was abysmal. We had to wait weeks for our sysadmins to order blades, get things set up, there were outages etc..
If a project is definitely going to be 'a few linux servers and never more' - even then it would be cheaper and more reasonable to use virtual instances.
The time to 'roll your own' is when the infra. operating costs are a material part of your business.
For example, 'Dropbox' invariably had to roll their own infra, that was inevitable.
Similarly others.
That said - as this article indicates, it's easy to 'over do it' and end up in ridiculous amounts of complexity.
The Amazon IAM security model has always been bizarre and confusing, and the number of AWS services is mind-boggling.
But the core case of EC2+S3 +Networking, and then maybe a couple of other enhanced services for special case works fine.
I also object to what I think is a vast overuse of Cloudflare, I just don't believe that in most scenarios needing to have content at the edge really changes the experience that much.
This only really applies to fully-managed services such as Heroku. Every other cloud still needs a DevOps person according to my experience in many companies.
Just security alone, in terms of managing access to all of those resources, various forms of backup, across regions ... it's just out of reach for most organizations.
This only happens when consumers fail to set budget alerts. Troy could have saved himself $10k with 15min worth of work.
Cloud providers love it when people do this and are famously easy to talk to when you get an unexpected invoice high enough to require remortgaging your house to even begin addressing it, but I think unless you're working on a side hustle that inherently will need to run in the cloud regardless of scale or are experimenting with cloud technologies in an explicitly time boxed toy project, using cloud services is the financial equivalent of handing a hobbyist craftsperson one of these chainsaw angle grinder attachments that even professionals find hard to keep from bouncing into your body.
If you do want to use cloud services for anything you pay out of your own pocket, the first consideration should be cost management and monitoring. Your employer might have big enough pockets to shrug off a runaway compute instance you forgot about for a month, but that can quickly translate into money that can be anything from inconvenient to life altering if it comes out of your personal budget.
Or just stick with the free tier and make sure everything simply shuts down if you run out. Sure, a "bandwidth exceeded" error page might not get you as many upvotes on HN, Reddit or social media, but it also won't impair your finances.
Of course, the delayed sticker shock is a problem.. I think Google cloud actually lets you create a budget that turns services off if they go over, so there's a solution here if you run a hobby project that you suspect might take off and cost you more than it's worth.
Hetzner has nice dedicated servers for €33/mo:
I'm on an older one that just got bumped up to €29/mo due to increasing electricity prices; it was €21/mo until now, and I can't blame them for that one. The specs are E3-1245 V2 / 16GB / 2x3T, there's over 45 vhosts on it across ~25 wwwroots plus other random services, and CPU usage is basically nothing. The cores are really there just to handle bursty stuff. Most random side projects and small websites don't need almost any resources on modern hardware.
Previously I was on a Scaleway Dedibox, which go as low as €15/mo right now. It was €10 at one point even.
But the ISPs I know do not bill $$$ if you use the max bandwidth (max bandwith they did advertize to you btw) for a sustained amount of time: they'll just start throttling you.
Anyway GP ain't asking about "cloud vs hosting at home" but about "cloud vs dedicated server(s)".
I'm sure there becomes a point where cost of (hardware + maintenance + staffing) > (cloud + staffing), in which case sure crack on. But like you, I'll stick to a rented server for my stuff.
I'm a major cloud skeptic, but there's a certain class of giant enterprisey companies that are never going to be able to attract good IT talent, and if they "just throw money" at the hiring problem they'll be innundated with slick imposters.
I think cloudy stuff lets those companies outsource a large chunk of something they'll never be good at. The cavalcade of Microsoft/Cisco certifications were an earlier decade's attempt at solving the same problem.
Even larger companies can work well with that model, traffic also tends to be cheap enough that you can spread across different vendors to avoid lock-in. And in that case, your sysadmins can sit wherever they want, no need to be physically close to the servers.
Also, as there's much less knowledge to be a dedicated server provider, competition is strong and prices are comparably low.
Also it is a major pain point getting anything done with IT operations.
Like the Oracle database server that half the department relies on stops responding on a Friday morning and it takes all day to determine the hard disk is full and fix it. I had never before worked at a company where this happened multiple times.
Or operations saying they were unable to restore a windows server hosting a database server and now everyone has to scramble to update their connection strings because operations somehow cannot use the same domain name for the new machine.
If you're Netflix, cloud is probably not that much more expensive than owning your own servers. Maybe even cheaper. But you're not getting Netflix prices.
But even if you're small fry you should however start regularly talking to your provider and go through a regular cost-cutting exercise and talk to them about how you're looking at provider Z and have been asked to cost out managed servers and on prem options.... You won't need to get very big before that starts paying off.
If your competition is doing this and you're not, and hosting costs starts becoming a big part of your cost base, you won't be able to compete.
Long term I think we're going to see disruption here to the point of startups failing because of competitors copying their idea but being better at driving down hosting costs by not being afraid of going to dedicated hosting or hybrid solutions (hybrids are my favourite - if your stack can be deployed semi-transparently both on dedicated servers and cloud you can go much closer to the wire on your dedicated servers by being prepared to spin up cloud instances to take care of spikes; ironically having the ability to spin up cloud instances makes relying on cloud services even less cost-effective)
I'd also expect to see more "hybrid" cloud offerings with companies offering you operations-as-a-service by giving you a virtual cloud type interface where they don't actually own a cloud service themselves but helps you abstract away cheaper hosting providers. You can already find plenty of people who'll e.g. run Kubernetes setups for you, so taking the step to do more cost-optimization on the backend is natural (and I'm sure there are people who'd do this for you today - if I was still doing contracting I certainly would be offering that - and maybe someone is already wrapping it up as a service offering; I haven't kept up on that market)
I used to spin up dedicated servers and then put an overlay network + a simple set of tools to spin up containers on them years before Kubernetes etc. was a thing, and we'd have a "global" (we had VMs in Asia, dedicated servers in Germany and colocated own servers in the UK) unified deployment mechanism that let me spin up containers wherever with a one-liner. Having a few extra dedicated servers with spare capacity standing by still made the whole system far cheaper than e.g. AWS, even if you attributed my entire salary towards it (I spent nowhere near all my time keeping that running).
It's easy enough to find consultants that can set up systems like this that abstracts away the dedicated hosting providers so you can mix and match and move with ease - especially today with options like Kubernetes.
If I was to go back to doing consulting I'd probably look at finding a way of packaging this kind of offering up behind lots of marketing speak and offer some sort of "abstract" hybrid private cloud layer on top of a choice of dedicated hosting providers to make that kind of hosting palatable to execs who refuse to believe the cost saving potential because they've never dived into the actual numbers (oh, the amount of time I've spent building out spreadsheets with precise cost models that'd get promptly ignored because someones had heard from a friend that company X swore vendor Y was cheap and believed it blindly)
Companies that have made a name for themselves by outsourcing to the cheapest IT contractor that will promise them the moon and fill the seats with barely warm bodies? I was one of those bodies so I know exactly why they can't attract talent - they don't bother, and don't reward it. They treat IT as a cost center and are surprised when they get disrupted. The only good options in those companies are to work on the business side or worst case as a project/product/program manager interfacing with the warm fungible contractor bodies.
Many Enterprises are only alive because of inertia and goodwill from earlier decades.
It’s just a shame they were permanently out of database servers with SSD storage, and for some reason couldn’t provision more for over a year.
There are smaller cloud providers, rented VMs, rented dedicated servers and rented colocation space.
Mid-sized companies can get cracking deals (like 10% cost) on major cloud providers.
The biggest issue, though, is how few people are aware they can negotiate with their cloud provider. I've seen so many places just pay the sticker price without even trying to get discounts.
(Conversely, I once got a contract to do zero-downtime migrations first from AWS to Google Cloud and then to Hetzner so a startup could launch on AWS and spend the huge amount of free credits they'd been given there, then migrate to Google Cloud to do the same, and then finally move to Hetzner once they had to actually start paying; relative to what they'd have to start paying if they'd stayed on either AWS or Google after their credits ran out the cost of having me do the extra setup to handle that was covered with ~2-3 months of their savings)
I kept being asked to price out a migration to AWS, and we kept coming up with 2x-3x the cost. Part of the reason was that we could pick and choose servers that fit our workload in a way we couldn't with AWS, and partly the absolutely insane bandwidth prices AWS offered.
I use AWS. I like AWS for the convenience. But it's a luxury that is ok when you're either small or really high margin, and you're paying massively over the odds for that luxury.
The reason these services get away with being so expensive is that people massively overestimate the complexity and don't bother actually getting quotes from people or companies to manage these services for them. When I was doing consulting my biggest challenge in offering up alternatives to AWS was that people were so convinced AWS was cheap that even when presenting them with hard data they often didn't believe it. For me it was a mixed bag - I tended to make more money off the clients who stayed on AWS as they usually needed more help to keep an AWS setup running than those I migrated to managed hosting setups, despite paying more for the hosting too.
Their monthly cost is something between 0 and a few cents.
Stuff like Hertzner is fine, but if you know your way around AWS you realize have massive cost savings. Prob the same for Azure.
Finally, in many places 40 EUR for a pet project is actually a lot of money.
For ~4€/month (depending on your country), Hetzner offers "Hetzner Cloud" servers with 2GB of RAM, see https://www.hetzner.com/cloud?country=us
Hetzner has such a VPS offering (2GB RAM/20GB nvme SSD/20TB bandwidth), netcup has one for ~3€, contabo has 8GB/50GB nvme/32TB for 6€/month and there are plenty of smaller companies around the world offering similar deals (usually somewhat less included bandwidth outside europe though).
EDIT: Personally I pay 9-10 EUR per month to Scaleway for a 2G RAM and 2 CPU VPS, private docker repo and S3-compatible storage which holds data and some backups, which run both my personal services and some toy projects when needed. I am not affiliated with them in any way
As for costs, setting up billing and usage alarms on AWS is absolutely trivial.
Finally, using stuff like S3 or dynamo for storage gives me a peace of mind I will never have when managing my own servers.
Doesn't change the equation, unless you set up all your PAYG cloud infrastructure and never use it.
Also, as you can see in a screenshot on TFA: Some services are simply dirt cheap. The storage account and its various “sub-services” is such a thing. It’s hard to compete with dedicated hardware here.
Depending on your dedicated hosting provider, the traffic cost trap exists, too. Hetzner is a bit of a special case.
Hetzner also offers managed servers where all this is taken care of, for relatively fair prices.
These things are now trivial enough that it doesn't make sense to pay 10x the cost of bare metal for a cloud provider to solve them for you unless you have a crazy amount of runway or absolutely no idea what you're doing.
I've been running something on AppEngine for 10 years and it costs me less than $1 a month. Not sure I could find a cheaper VPS.
On the other hand, I also manage a Mediawiki install, and a cheap Hetzner VPS works great for this.
apt install unattended-upgrades. And Hetzner's firewall.
Typical FUD. On modern servers and the type of software it occupies very little time. You'd spend more managing your cloud architecture.
He has a writeup here on how he gets costs down in a big way: https://www.troyhunt.com/serverless-to-the-max-doing-big-thi...
The sum of the GB figured shown in the OP doesn't even amount to 200GB AFAICT. But even if it's something like 10TB that's still not super expensive on many hosting providers.
($350 per day at .014 per GB)
> It's costing me 2.6c per day to support 141M monthly queries of 517M records.
Also, you might be able to store 1TB of data on a spinning disk with no problem but can you run the amount of queries he needs? Will you be able to run them as fast as you need? How much RAM would you need? etc.
Current workplace is considering a fully self-hosted stack as a unique selling point for the customers and segments we're in. That means, we have storage and linux admins available, as well as tooling and know-how how to run this securely and efficiently. Thus, placing large and often downloaded files on our file stores at hetzner is very much a no-brainer, because it adds very little workload to the teams maintaining these stores and it's cheap.
However, this can be a daunting thing if you don't have this skillset in the org. It can be learned, but that's time spent not working on the product (and it's not trivial to learn good administrative practices from the hell that google results can be). At such a point, a cloud service just costs you less man-hours. And again - it wouldn't be much time for me, but it would be a lot of time if you had to figure all of that out on the fly. That's essentially why the saying goes that cloud services save you time, but cost money.
1, when they need to adjust rapidly between different resource usage profiles, e.g. because they are growing rapidly and can't predict what the usage will be X days in advance
2. They have huge resource requirements and don't care to invest in their own infrastructure, but can negotiate lower rates with a cloud provider
3. When their resource usage is modest but profitability is high enough that cloud expenditure is a rounding error
One can add new servers in minutes, removing has a bit more latency to it, but I'd figure with the huge price difference between rented and cloud you'll come out on top with the former in most case. Also, just use a clustering or orchestration layer in between, they range from very simple to setup and use (e.g., Proxmox VE), to quite complex but also very capable (OpenShift, kubernetes, ...).
> 2. They have huge resource requirements and don't care to invest in their own infrastructure, but can negotiate lower rates with a cloud provider
Using hetzner or other providers is not investing in their own infra, that's using (= renting) the providers infra and ability (peering, fast uplinks, datacenter perks like utility redundancy and staff on site). The second sentence may be true but probably not for most use cases that aren't huge yet, like the post here.
> 3. When their resource usage is modest but profitability is high enough that cloud expenditure is a rounding error
IFF, yes, and often infra costs are relatively low compared to salary costs, so that's definitively some optimization problem one should go through when deciding such things. Chances are that for most projects the profitability can be good but not magic money printing and infra costs are a non-negligible part that eats on their revenue, and then it's definitively worthwhile to think about avoiding the high premium most of those cloud offerings ask for.
With Vercel I don't ever think about adding servers at all, huge win.
> infra costs are relatively low compared to salary costs
Enterprise SaaS here, this is it. Any second my team spends not caring about infra is well worth it.
A month later a ntp security vulnerability was discovered, soon the server was put offline, some 'patch your things asap' not so nice emails came in. From that time my take is one should spend some time probably daily on an own server if one wants to mantain it.
> A node is an Azure virtual machine (VM) or cloud service VM
> The terms node and VM are used interchangeably occasionally
> Azure Batch creates and manages a pool of compute nodes (virtual machines)
> In an Azure Batch workflow, a compute node (or node) is a virtual machine that processes a portion of your application's workload
So no, seems Azure Compute Nodes are VMs, not bare metal.
I’m a huge Hetzner fan, and their cloud offering is definitely growing but still isn’t as convenient and featureful as it could be (and they don’t share their roadmap currently so hard to tell what they’re working on next).
I’m trying to do something about it though, working on Nimbus Web Services[0]. In my mind all we need is something to bridge the managed services gap and make it very easy to set up the basic 3 tier app with some amount of scale/performance elasticity!
[0]: https://nimbusws.com
apt install nginxHonestly, they're not even connected/brain-dead-easy for a vendor like AWS particularly -- you still have to click around a lot or write a bit of terraform/aws-cdk/etc when all you really want to do is throw a folder or zip file over the wall and point it at a domain.
There are tools like Ness[0] out there which look like a breath of fresh air but there needs to be more tools like that where the supported backends include a cloud like Hetzner/Leaseweb/OVH.
I'm in no way a sysadmin and have set up these configurations manually in less than an hour for side projects. Cloudflare tunnel also allows you to lock down the server for everything but ssh with pubkey auth so the attack surface is really small.
Actually hosting files is super easy (Caddy is awesome, NGINX is awesome), but it's even better when you don't have to set up the server at all, for example just turn on "HTTP access" on a object storage bucket for example. So this is another place Hetzner kind of falls short though they do have hosting options[0], so basically the ideal solution here would be to deploy a simple Hetzner app (caddy/nginx or the hosted options hetzner has), set up a cheap CDN (Bunny, Cloudflare, etc) in front of it, and save money that way. If the bill is still too high just take the penalty or bias towards one geo (germany/US).
I was less talking about the difficulty of getting a server up and more about the CDN bit of the issue to make loads blazing fast!
[0]: https://www.hetzner.com/webhosting what you want is latency reduction. Usually what sites like Vercel and others give you is way faster loading time by putting stu
I agree! Cloudflare probably won't be this cheap forever but like I said I think that's the optimal solution, with the option to cut over and take the latency penalty if costs are out of control.
Latency isn't particularly relevant for this, and it probably isn't relevant for most hobby projects.
So that leads me to my question for HN. Have we completely abandoned non-HTTPS, particularly perhaps for the use-case of server-side caching of HTTP content? Also, isn't this a valid use-case to not use HTTPS and to re-enable that sort of functionality at the network/ISP level?
AWS is cool and all and has a bunch of interesting stuff, it's just expensive.
- Security Information and Event Management - exports, alerts, OS configuration
- OS/Application Hardening - Encryption, Password/keys rotation, CIS/other baselines, Drift Management
- Backup - Encryption, (don't forget your passwords/keys are changing), retention, data protection compliance, monitoring, alerting, test days
- High Availability - replication, synchronisation, monitoring, alerts, test days
This is just the tip of the ice berg, if you operate in an environment where Insurance, Reputation, Regulatory Compliance, certification, etc.. are important, then it's easy to see why PAAS solutions are desirable.
A lot of cloud cost objections would be solved if they defaulted to that instead of defaulted to just charging you the fees. That has its own tradeoffs, of course, but I find myself suspicious that the reason the clouds work this way isn't so much a cold and sober consideration of the aforementioned tradeoffs so much as "this way makes more money when we charge people lots of money they weren't expecting" and "this way makes lots of money when the people deploying the service are organizationally and fiscally disconnected from the people paying for it so they care and notice less".
This recurring question of "why AWS/Azure instead of Hetzner/OVH ?" keeps happening because people are incorrectly comparing higher-level PaaS to lower-level IaaS without realizing it.
PaaS != IaaS are not equivalent. IaaS is not a direct drop-in replacement for PaaS to save money if the workload is using PaaS features that IaaS does not include.
The author Troy Hunt is using the higher-level Azure services like Table Storage (like AWS DynamoDB/SimpleDB) and Azure Functions (like AWS Lambda), and others. E.g. One of the article's hyperlinks talks about using Azure Functions.[1]
If he used Hetzner, he'd have to reinvent the Azure services stack with open-source projects (some of which are buggy and immature) and expend extra sysadmin/programming work for something that's not as integrated. The Azure/AWS stack includes many desirable housekeeping tools such as provisioning, monitoring, routing, etc which he'd also have to re-invent.
TLDR: People choose Azure/AWS because it has more features out of the box. You just have to figure out on a case-by-case basis if the PaaS value-add makes financial sense for your particular workload.
EDIT to downvoters: if Hetzner actually has built-in equivalents to AWS Lambda and DynamoDB, please reply with a correction because I don't want to spread misinformation.
[1] https://www.troyhunt.com/serverless-to-the-max-doing-big-thi...
- My house is probably going to be build much faster if it's built by professional house builder (even more true for services since it's available immediately)
- I have better things to do than building houses
Hum, no. People are asking what kind of value that platform adds that can justify all that risk.
And nobody is giving any clear answer, so I'll stand with my previous answer of "none".
I use the credit card of my employer. For my own projects I use my own server for everything. Granted, it doesn't get much traffic.
Some offers from cloud providers are pretty good. If you want to scale to more (virtual) machines, it can be more easily done with the usual providers. I also expect Amazon to know more about firewall and reverse proxy configuration, it renews my certificates automatically and has rudimentary services for monitoring of server state. There is a certain convenience to it.
Would I recommend cloud based hosting? Absolutely not. You become dependent on the provider and prices are often steep. Even if you do not know much about server security, your unsecured s3 bucket will be far more exposed than your standard db installation on your own server. Better build expertise for systems you have full control over than to invest the time on the details of AWS which are more subjected to change.
For companies the benefits are the abiltiy to get new servers at a click of a button and get rid of a server. For example, asking the ops team to setup a snapshot of a database for a few hours while I do something is super useful.
There is also the ability to use autoscale and other stuff to automagically scale your system to handle traffic peaks. With dedcicated servers you need to always have those resources available. It's attractive to managers that they're only paying for resources when they're using it.
There are also managed services like DynamoDb, Lambda, S3, etc that can make things easier and reduce your sysadmin work. And allow you to get up and running very quickly.
Obivously, a major downside is that the pricing is extremely vulnerable to spikes like this. I think we see an article like this every 3 months or so. This one is rather tame compared to some others that were 10x as much for a 24-hour period.
I can snapshot a database disk with a click of a button and restore the snapshot with yet another few clicks.
I have 1.5 TB of highly available disk space, 40 cores of full CPU power, 160 GB of RAM, & dynamically provisioned IPs for metallb. For only $130USD a month. For the same price in Azure, I had 6 CPU cores & 8 GB RAM.
But let's say need 4x vCPU: 72, Memory (GiB): 144 for 4 hours. Or you need that 12 hours a day but for the rest of the time you need 2 cpu and 4 GiB of memory.
You need to handle traffic spikes such as TV traffic.
Yes you could self-host a cloud env but you can't scale your resources the way you can with cloud.
And that is literally the largest AWS Elasticsearch cluster option. So clearly that will be deployed for multiple orginsations. Otherwise they wouldn't have created a default node that size.
I would probably send you back to where ever you came from and tell you to re-engineer that. Cloud or no cloud.
Sometimes you need to spend lots of money on tech debt. I think it's nuts that was required but it was.
I wish they’d bring those same prices to some US data centers.
Well that's the first issue. Many people have automated large parts of their infrastructure in this way so that distributing one huge file becomes part of that whole mess. The goal is of course to keep costs down to a minimum. You can actually do a lot with little money using cloud services.
But the careful balance is that you can easily miss little details. But how does that differ from any systems administration? The details are just in new areas that didn't exist 5-10 years ago.
And the details you miss are more likely to increase cost. And when you process a lot of traffic, you're popular, that can go real fast.
20 years ago in hosting we might get a porn stash on a hacked NT4 server that would draw bandwidth. And back then a whole company might have 100Mbit fiber so you'd notice.
The reason to use cloud-style services is so you can focus on building the product quickly instead of building and maintaining architecture. But once the product is stable, a cost-reduction pass is in order.
>> why anyone would sign up
It happens more often than you think: people sign up for credit cards and forget to pay the monthly bill in full. Sign up for a cell phone plan and get charged with large bills of international roaming. People sign up for monthly subscriptions, and exceed the usage limits.
To handle that day of getting 1 million customers, which you've been forever optimising for.
Any.. day.. now...
From the article:
This was about AU$350 a day for a month… priced at AU$0.014 per GB
A company could not stay in business if every one of their “unlimited 1 Gbps” customers for €40 per month actually used that bandwidth.
Getting metrics on that is not a hard problem, there are various projects that are relatively simple to set up.
If you want to make it easier manage resources, metrics out of the box, and avoid (hoster) lock-in then I'd use a hyper visor distro like Proxmox VE (disclaimer, am a dev there) or the like, and you can migrate (or backup/restore) VMs or Containers easily to other providers. That gives you a (relatively) simple web-interface to manage most things and also opens the possibility to just add a second or third dedicated host down the line to scale out, if those new hosts are in the same DC or have a good interconnect (latency wise) you could even cluster the nodes.
I can set up Proxmox VE as hyper visor, some container for each DBs, load balancer in front and some app in about an hour max from scratch, with good testing and some bells and whistles, and here I really do not want to brag or the like, as such operations are not my job to do at all, I only know because I do that occasionally for some tests and for some private infra I just maintain out of interest - so I really want to say, if some operation-dork can do that, the engineer you hired should be able to do it at least as quick.
But yes you're right in the general point, upfront setup and frequent maintenance is naturally something you need to price in. I just think that if you have that many different parts with complex coupling to induce such a huge maintenance effort required to keep your product running, the cloud offer may not really be your salvation and just delay the fall while costing all the more.
Everyone does something for the first time once. Just because someone has not set up a hypervisor before doesn't mean they're inexperienced.
> I can set up Proxmox VE as hyper visor, some container for each DBs, load balancer in front and some app in about an hour max from scratch,
And I can spin up containers + load balancer on AWS in less than five minutes. That doesn't mean that it's just an easy thing to do. (although, this specific example is).
> upfront setup and frequent maintenance is naturally something you need to price in. <...> the cloud offer may not really be your salvation and just delay the fall while costing all the more.
Agreed 100% on both counts.
Wait, what? If you never did something, then you're unexperienced, ain't you?
Experienced != Knows 100% of things.
I'm sorry, but all that stuff you describe doesn't bring any business value. My customers don't care what hypervisor I'm running, so I don't care either. PaaS means someone else deals with it, forever. The last time I had to employ an ops (or devops) person was 2007.
Also, it doesn't take a week.
You're assuming that this is for recurringly set up infrastructure. Sometimes infra is set up once and maintained, othertimes it's set up and spun down. It's also not always automated. The time spent automating something like that might not be worth it in the medium to even long term.
> Also, it doesn't take a week.
The actual amount of time it takes doesn't matter; if it's a day or a month. what matters is costing the time spent on setting it up and maintaining it, and pricing that against AWS costs.
For a large company, it's not about $ cost, it's about risk management and avoiding cost centers.
I use rented dedicated servers for everything, and always travel by bicycle or transit. It's not as ridiculous as you make it seem.
You might like installing and configuring software, I don't. I'm more than capable of doing so myself, but I'd rather build things on top of other things. I'd rather use a battle tested Secrets Manager and have db replication set up for me. I'm grateful to people that like doing these things I don't and I'm expressing my gratitude by contributing to their paychecks via my cloud bill.
To go back to my initial reply, if you change the context, eg the context is driving a car, you can't possible crash the car you are not driving. If the context is "get home after a few too many pints at the pub", then riding a horse is much better than driving a car (and crashing it). Context.
Take a look at their datacenter in Germany: https://www.youtube.com/watch?v=5eo8nz_niiM
Love how they are totally not ashamed to kick off the video with their collection of 14,000 mini-tower desktop PCs. Not rackmounted. Mini-towers.
Also totally ultra-curious about the PS/2 kvm. All those machines are from an era when USB keyboards had been around a long time already. Wondering if this is a security measure...
I'd love to hear your reasoning why people who aren't European would prefer to avoid European service providers.
I'm generally a Hetzner fan as well for global services, but I can see the point in avoiding Hetzner (for example) if all of your users are in the US, since Hetzner only offers dedicated servers located in Europe (Germany and Finland if I'm not mistaken). Generally you want users to hit servers that are close to them, so something like Vultr would be better if the scenario mentioned before applies.
They also have dc's in the US
I didn't realize it was limited to their cloud offering. Nice one!