Leaving the Cloud
rework.fm
rework.fm
She also noted that when you search on these terms the main engines (all run by cloud service providers) return poor results not congruent with the scale of the phenomenon. They are dominated by the opposite message obviously heavily SEO'd up to the top, plus shill pieces pushing cloud services but presented as "critical". Dig deep if you want to find the real scale of the "anti-cloud" issues.
Her main conclusion was very interesting though. That the big issue is not finance, reliability or control - but de-skilling.
As companies move their operations out to the cloud it's not the disappearance of hardware from the premises but the loss of skill-sets. Later they don't even know who to hire or how to write the JD to bring people back in.
A good example was the broadcast industry. Entire branches of that industry doing post-production, colour, transcoding, and whatnot moved it all out to AWS. After price hikes, they wanted to go back to running their own services. But they can't. Because nobody knows about how to set-up and run that any longer - especially specialist things like buffers and transcoders. I mean, try finding a sys-admin who can just do simple tasks like set up and properly configure a mail server these days.
But you have to pay them enough or that is going to be like pulling teeth out.
And this is where the problems start.
That last sentence was something you never said to a boss back in '07-08. You would always know what was wrong and how bad it was within a few minutes of the issue, at least 90%+ of the time anyway. You could probably even guess how long it would be down for within ~10-15 minutes of discovering the issue.
FWIW, I think small/medium size businesses were/are probably the most affected by cloudification since those had the "most to gain" in the beginning. Then again, those were the types of businesses that I had the most interaction with until 2014-ish, I don't know what it looked like from a big-business perspective.
Cloud is to the IT department as big consulting companies are to HR.
And mainstream news will be either "company X offline for hours" if its just you, or "Amazon cloud outage takes multiple companies off line for hours" - the latter is better.
Average person hears on the news that company X or cloud Y had an outage, at most they have some anger towards the company for a day or so until the next thing in the news cycle comes along.
Switches are easy and cheap to put in a fail-over config. DB's are also running atleast in a primary / standby config everywhere. Besides if your running a single DB instance in a cloud VM the same thing can happen.
And don't get me started on the trouble you can get in with for example GCP SaaS PostgreSQL...
While I do look back with nostalgia on the days of editing files in Notepad++, FTPing them to a live server, then quickly switching to my browser to refresh the page and make sure it works...
Those days are in the past for a reason. No, git is not "fancy FTP". Yes, version control really does add much-needed structure to collaborative projects and change tracking.
You can still run a small business web server on a colocated server and FTP files to it if you want. You just won't find any modern engineers who want to touch that with a 10ft pole. The world has moved forward since then, and for good reason.
I was specifically referencing an old fad (might still be a thing but I haven't run into it in years) where production is a remote and a dev pushes their main branch to it to deploy. Yes, version control is amazing and it was something I used a bunch before my hiatus, but back then, it was usually SVN IIRC.
I think you're clinging a bit too heavily to the "good old days". It's ironic how this entire thread has turned into a mix of people either
1) complaining that modern technology is too complex and complicated
or
2) complaining that modern job applicants have lost the skills necessary to run things
There is much I love about today and I look forward to the future. I was just stating my observations. And yes, `git deploy prod` is a fancy FTP, but probably even worse than FTP if the build actually happens on the production server (which is something I saw more than once) and causes a production outage (also saw more than once).
Complexity is not a virtue. It is a dangerous resilience anti-pattern and should not be lionised.
Unless you're insinuating that things in fact remain simple but the complaints of complexity are down to the stupidity of commenters.
Its only innovative in an 'Apple released it and now everyone is on the bandwagon' way.
I've worked at companies that were using CVS for config file management in the 90s, the concept was already in play, it just wasn't as mainstream and didn't have a trendy infrastructure as code branding.
If you just add local machine testing, and a test server to that mix, what is the problem?
Good reasons for some people. But it turned out that those reasons (which you avoid explicating) were not so good for some people after all. Methods upon which you now "look back with nostalgia" had value which endures, namely control, simplicity, low cost at small scale, and a basis in standardised, accessible knowledge. These things are not valued much any more, but it's my opinion (and of others) that is a mistake.
I lived through part of that transition - I'll try and explicate through my own experiences.
The biggest thing automated deployments solved was human error. No more "oops, you forgot to FTP the JPEGs in binary mode" or "accidentally unploaded into a nested www/www folder". The transition was initially to tools like Ansible and Fabric which automated the transfer steps but had to be triggered manually. This also saved me a lot of time and pain.
However, this still left a gap where it was still possible to push broken code into production, so Continuous Integration (and/or Continuous Deployment) became fashionable. The benefits of these were you'd never push code that fails tests to production, the depth and breadth of tests being left up to you.
Incidentally, these developments coincided with my working with ever growing team sizes. There's no way I could have worked with 5 people with everyone S/FTP'ng stuff to prod. Neither would have Fabric + git been sufficient for four 5-engineer team working on the same software product. That's yet another benefit of the new way of doing things: it scales.
I think the additional benefits of checking-in your configuration and deployment steps into a version control system are self-evident.
Modern IT is swapped with CYA / No Accountablity
Cloud is not the only area this presents itself, everyone is more concerns about not being blamed for an issue than they are with resolving the issue. The more vendors you can point the blame to the better in their eyes...
AWS has never gone down across all regions at once. So we can do something about going down. You build the system to be multi-region and be able to evacuate traffic from one region to another, so when us-east-1 goes down again, you simply route around the damage and use resources in us-west2 until us-east-1 comes back up. People don't appreciate how much backend engineering goes into keeping something like Twitter up at all times.
Put it another way. Little guys go down when us-east-1 goes down. The big boys stay up no matter what.
Look back at all the big AWS outages of recent memory. Tons of the so-called big boys were down for the day along with AWS.
I still recall last year's AWS LSE when AWS went down globally, due to us-east-1
https://www.datacenterdynamics.com/en/news/aws-us-east-1-out...
Are you really suggesting that search engine providers are manipulating their own search results to bury some sort of hidden "truth" about a people leaving the cloud? Suggesting that people need to "dig deep" to find the real scale of the issue sounds a lot like the "do your own research" line that gets attached to dubious conspiracy theories.
The simple explanation is that your student simply anchored her personal beliefs to an idea that this was a large movement, but her light research didn't match her pre-conceived beliefs. This is how conspiracy theories are born.
Regardless, this claim doesn't hold up to even basic scrutiny. I just Googled for "cloud repatriation" and all of the results are think pieces about why moving out of the cloud is a good thing.
Try it for yourself: https://www.google.com/search?q=%22cloud+repatriation%22&oq=...
> A good example was the broadcast industry. Entire branches of that industry doing post-production, colour, transcoding, and whatnot moved it all out to AWS. After price hikes, they wanted to go back to running their own services. But they can't. Because nobody knows about how to set-up and run that any longer - especially specialist things like buffers and transcoders. I mean, try finding a sys-admin who can just do simple tasks like set up and properly configure a mail server these days.
If you can't find sysadmins with basic competencies or engineers who can build systems as simple as buffering and transcoding, I can almost guarantee that the real issue is that you're not paying enough for the position.
Either that, or the applicants can recognize from a mile away that your company is not the kind of place that skilled engineers actually want to work for.
Don't mistake your lack of candidates for a lack of skill in the market.
If the average engineer doesn't have a certain skill, I think it's fair to say that it's hard to hire for it, because it will be for the average company.
"Don't move to the cloud!" is hard to make money by selling to places not in the cloud.
"Move out of the cloud" is harder to sell to places in the cloud than "move to the cloud" is to sell to places not in the cloud, because (a) bigger orgs with big budgets to make money off of tend to be older and not in the cloud fully in the first place and (b) moving out of the cloud is less turn-key - every client is going to be different and likely lower-margin than pushing clients into the same set of existing packaged cloud services.
That said, I'm not aware of any companies from people in my network that are actively moving out of the cloud... so I also don't buy that it's a big trend.
I don't have any knowledge of the particular topic, but you're suggesting that firms maximizing profit is a lunatic conspiracy theory. May not be what you intended, but your toxic comment was a bit nutty.
I should note that these warnings aren't accompanied by much sympathy; companies and academic institutions both do a profoundly terrible job at incubating the skills they need, then bleat at their local governments about new graduates not having the skills to be employable.
More likely that they're just not paying enough, and those skilled sysadmins would rather go to any other company that will pay them appropriately for their skills.
This is just another take on the "nobody wants to work anymore" rant that happens when companies can't understand why nobody is accepting their open positions.
Companies are not paying enough, but we also do not have as many opportunities (for SysAdmins) to learn things.
Paid Internships, Jr Admin Roles, etc are fewer and far between, companies are looking for "experienced" people in a time where alot of the experience is retiring.
Dont get me wrong there are plenty of us Gen X's out there with lots of experience but most of us are also massively over worked with limited opportunities given for us to train the next generation, so pay is only 1 part of this problem, head count is the other
Even if you find a company that has good pay, they almost always under size their IT dept by 25-50%, and rarely promote internal training or continuing education, or offer entry level admin positions with the goal to skill up people.
I think if you want to go back into history and look at things, we had sysadmins who were not paid enough, but fell into "give a man a little power" trap and showed a side of themselves that we would rather not see, fed and reinforced by getting blamed for anything bad that happened 'on their watch'. The cumulative generational trauma on both sides of that relationship sets us up for a situation where outsourcing IT to someone who can't sass back or harass us about our deployment plans seems like a good deal. Yeah it costs about the same and it'll interrupt our roadmap but I get to 'stick it' to the IT department. Fuck those guys.
Now it's been a while and people are starting to feel nostalgia for the old days.
It was never, has never, and could never be about "power". I have always been a generalist, with one foot in both roles of dev and admin.
I am very good dev in 3 programming languages (not including scripting languages like powershell and bash.. php, C#, and JS are my strengths), and passable in about 8 or 9 others (go, python, learning rust now, etc). I hold multiple certs and years of experience in sysadmin roles.
There are very few of us out there, people with extensive experience in both. I can tell you, once you have seen both sides of that wall you come way with the clear view that neither understand the other.
DevOps was proposed as a solution to that, and if Devops would have been implemented as a Management sytle not a role it might have accomplished that goal but sadly most orgs just eliminated the valuable perspective of sysadmins, and as you said "outsourcing IT to someone who can't sass back or harass us about our deployment plans" which from my experience many devs need to have their plans pushed back on because often time they are widely inefficient, insecure, ill thought out, and expensive
I have to wonder, if many of the cyber security incidents we have seen over the last 10 years could be avoided if Dev's lost the chip on their shoulder, and listened to the sysadmin more, instead of just rejecting their point of view as a "god complex" like you seem to
You know why all internet traffic goes over port 80 and port 443? Entire generations of admins who learned that their job was to say “no” instead of “how”. By the time the fingers got peeled away from that issue the damage had already cemented. Similarly ORMs also got a healthy nudge from DBAs who decided it was their job to protect the business from ever doing anything new. Today the priesthood of the DB is all but gone. Devs are expected to do those roles because otherwise nothing gets done.
And similarly, the Cloud doesn’t say “no”, it says “how much”.
Are bad things happening now? Bad things always happen. I read Applied Cryptography while I was still in college, and have worked at a lot of startups and in other small projects, so I am biased toward thinking devs should not be making other teams babysit them. Advice and control are very different things.
Edit: I’ll note this isn’t my opinion, but my opinion plus decades of listening to devs vent about people out of earshot. Sometimes to the point of sedition. There are some big frustrations out there and creative people steer solutions away from frustrations. It’s not an accident we are where we are. People voted with their feet.
Now we have conflated network admins and sysadmin. Different things.
the reality is that was largely ISP's not sysadmins that caused that by throttling, shaping and in some cases out right blocking traffic.
Throw in some laziness by devs as well, it is widely inaccurate to lay that out the feet of sysadmins.
> Similarly ORMs also got a healthy nudge from DBAs who decided it was their job to protect the business from ever doing anything new.
Lots of ORM's were written by people that were DBA's because they were tired of devs that did not know how to construct a proper SQL Query, and thought that Select * was the always the correct way to return data.
This idea that devs had to ride in to save the industry from evil DBA attempting to block everything is a complete revision of history
>>my opinion plus decades of listening to devs vent
I am not sure how a circle jerk is proof of anything? devs in many cases may have gotten rid of sysadmins, but now a new beast is emerged that is making life hell for both sysadmins and devs alike. The "Cyber Security Analyst" and if the devs thought sysadmins saying no was a problem, you have not seen anything yet. Unlike the sysadmin cyberSecurity will be backed by compliance, insurance companies, and standards organizations. They will have actual teeth, and actual power something the sysadmin never had
Surely this calculation should include the cost of people with the required skills?
Something I've observed for some time as well. If Ops was truly outsourced to the cloud service providers then why is devops such a buzzword/trend, hmm?
Today, Intel (and probably also AMD et al.) offer them exclusive deals on and even exclusive models of their hardware, due to the insane volumes "hyper-scalers" procure. In parallel, Amazon offers their own CPU arch (Graviton) cheaper than x86 cores to users of their platform in order to make it more attractive - at the insane margin AWS operates at, it can make these instances a loss leader, no problem. As a result, those in the x86 market that do not have a cloud business attached will grow weaker and weaker in comparison, and most cloud-native developer mindshare will shift to the cheaper option (since most engineering work in cloud envs is about reducing cost, period), which is AWS/GCP/... custom CPUs. And you, the random shmuck with an appetite for powerful on-prem hardware, will be left with worse products to buy for a higher price.
Also, cloud environments are rarely cost reductions. They are very frequently more expensive than on-prem deployments, but they are used because they are easy to spin up and down and they allow you to engineer things faster.
Anecdata: I have 2 customers who have had open roles of this type for more than a year each. I'm pleased to report that they are edging their salary offers up into "reasonable" territory - but the fact remains that someone with that skillset would ordinarily be able to get paid twice as much at any of the 3 major clouds.
Also noone is doing this then eventually (deskilling is real) and as I really don't want to work for any of the evil empires...
And then they started taking away my Ops people.
Similarly, managing servers through Chef/Puppet used to be a thing just a decade ago. Now with the onset of serverless, Lambda etc., it's slowly fading away.
I will do that. Horrible work. I would need >$200K/year....
A dubious conspiracy theory, frankly.
I know HN comments bias heavily toward anti-cloud, but the suggestion that all of the search engines are conspiring together to bury some massive underground anti-cloud movement (that conveniently confirms anti-cloud beliefs but is unfortunately just hidden by evil search engines) is laughable.
Still it's probably just the general noise of the fully oligopolized internet :).
That's the yardstick to use here.
There are famous examples of large media companies screwing over their internal IT depts through out the 2000's and 2010's
Hell, for that matter, nobody knew how to set up a transcode server for live streaming 30 years ago - it's not some ancient art where we lost 50+ years of experience.
I don't believe that people who can learn to operate cloud services can't learn how to rack servers and install software. Ten to fifteen years ago I was in companies where ten percent of the software team was managing the datacenter. Now ten percent is managing kubernetes and AWS and all that. I've seen the cloud be better for smaller shops since you no longer have to have that server management if you only have one or two people, but I haven't seen it "deskill" things so much as "differently skill" them in larger orgs.
Was just thinking last night that I need to stand up a mail server to send/receive email from my self hosted products. Then I thought of the leg work involved and said words I’ve never though I would say: “I should just use SES and be done with it”
Who could have possibly predicted that the trap-shaped box was a trap?
The Trough Of Disillusionment is fast approaching. I'm surprised it's taken as long as it has. But then it always seems like the biggest, most expensive mistakes we make as an industry defend themselves, since nobody can bear to admit to themselves the magnitude of resources they set on fire in the process.
> As companies move their operations out to the cloud it's not the disappearance of hardware from the premises but the loss of skill-sets. Later they don't even know who to hire or how to write the JD to bring people back in.
So in that respect it's similar to outsourcing any other aspect of your business - there are many examples of hollowed-out suppliers of manufactured products that no longer have the ability to build things themselves.
They don't readily see that moving to the cloud increases entropy. You can't easily go back the other way, and so maybe companies should understand cloud as a one way process and think much more about the value they have and will be losing.
So that is where the next boom will be: a sysadmin-on-demand service. Quick, someone create a start-up which connects those with the required skills to those who need those skills, charge a 30% fee and become the next greatest thing since sliced bread.
Same thing with Jerusalem bugs. Wanted to check how careful I should be (I hike at night). First post a pest control company claiming that a bite can cause a fever. Further research into trusted sources mention no such thing.
I hate what Google has become. I miss the days where the first result in Google was valuable.
Pro-tip: if the post explains something and end with ‘conclusion:’ they’re pushing a product.
Moved to "devops" profiles since sys admins and regular ops guys stopped being valuable. Picked up puppet, ansible, aws/gcp. Moved to a big company, got bored out of my mind and decided to look for something else.
Questions from the first round of an interview(they call the round - * basics) in a small company - Advanced aws and k8s(no Linux or anything else in my resume). I cracked the round but was expecting it to include at least some Linux questions. Next was the scripting round which had nonsense questions involving a lot of calculation(because ops folks only know basic scripting). Cracked that too.
What I wanted to say is that there is no value in knowing Linux(to get a job, nothing beats knowledge on basics for debugging stuff once you land a job) and complex ops stuff since there is no market for these. Most of the people who were good at complex ops stuff moved into management roles seeing a lack of market for everything they knew.
I am planning to leetcode some more and move to a dev profile after a year or two since there is no upward career path for all the ops stuff. There is too much to prepare for devops interviews since there is no standardized interview template either(advanced Linux, advanced networking, advanced k8s, cloud providers, everything in my resume [a ton of stuff that I'm slowly removing one by one], ds and algos, leetcode].
It is not that there aren't people out there with the required knowledge, we are just removing all the seemingly useless stuff from our resumes.
Do you know what exactly increased in price? AWS usually doesn't increase prices, so I'd be interested if that was an exception or if that's a reason just given to cover another issue they had with AWS.
That's the case for a lot of different things, ranging from tech to politics to pretty much anything, and doubly so if people are selling something or otherwise have a strong interest in pushing something. It's just so much easier to extol the virtues of something rather than point out the problems with something, especially when the problems are not obvious from "common sense" at a glance.
"Searching for something" is often a very biased way of finding out about many things; it's not intentional, it's just that the "pro" and "anti" camps have vastly different levels of commitment to their cause, and you often need a lot of knowledge of context and nuances to make a fair judgement; any hack can say "X is brilliant!" but it's much harder to say "well, X is good in some cases, but [..]"
[1] https://files.ciokorea.com/archive/images/public-vs_-private...
Doing this correctly is mostly googling and using reference material to make sure you’ve set it up correctly, which means that anyone with decent Linux knowledge can do it.
I would not trust anyone who claimed they could set up a mail server with no external references.
Once I got into the thick of it, I actually started to have the thought "what would it look like to just run co-located boxes like the old days or do this on prem?" The answer was of course "waaaay simpler."
While I don't think we'll see an industry-wide shift back in this direction, I do see it happening on a non-trivial scale. Especially as things like economic instability and censorship wax and wane and wax in the coming decades.
Now, cloud providers offer a giant hairball of specialized services, built on top of other specialized services, built on top of yet other specialized services... such that no single person can comprehend all of them, let alone manage them cost-effectively. Moreover, as all these intertwined services become deeply entrenched in IT infrastructure, migrating away from them becomes incredibly disruptive.
The financial incentives of cloud providers are not aligned with those of customers. The more a customer is sucked and trapped inside those giant hairballs, the better for the cloud providers.
To paraphrase the Dark Knight, cloud providers may have been the heroes at first, but they have lived long enough to become the villains.
One big box can get you quite far. When it’s not enough: put the DB on another big box. Not enough. Put a caching/proxy server first.
You can use Cloudflare or others to offload even more static/caching stuff outside.
Still way simpler than to understand why AWS billed you something.
People with expertise in running private clouds and using public cloud services can judge each case pretty easily, by taking into account the organization's technical needs, plans, budget, abilities, etc. And for non-experts, the question is even simpler: use the public clouds if or until it becomes a technical or financial blocker. And in that case, hire an expert.
There's really not a lot of room for controversy. It seems most of the arguments come from people without the relevant expertise, who only know how to develop in public clouds, responding defensively. Like people with little programming experience flaming each other over which language is "best".
There's no single "best" cloud option, since they're just different tools for different things. Sometimes either way would work, in which case it comes down to personal preference, and sometimes there's a clearly better option for a specific organization's needs. Framed correctly, it's a "boring" question just like most technical questions should be.
They (think, they) have a niche use-case which is a bad fit for the cloud.
These niche use-cases exist, but they are niche, and most companies, who think their use-case is that niche so it doesn't fit the cloud,are probably wrong.
Chances are you belong to the vast number of companies that should simply put their workloads in the cloud and be done with it.
(0) https://mobile.twitter.com/swardley/status/15838141696564592...
Conversely: I find that the inverse is true very often.
People are very quick to throw their current systems away in favour of the cloud due to a myriad of reasons; be it sticker shock, promise of less headcount, the idea that you can hire people easier or simply because of a Gartner report that your CTO happened to read on a flight.
This causes migrations to the cloud that kinda don’t make sense, they’re borne out of ideology or “not being left behind” and sometimes it fails catastrophically.
We would do well not to cargo cult, but reasonably deduce and argue what our objectives truly are.
I have been on many teams that could’ve easily been in maintenance mode and not changed anything for years. The business would’ve hummed along stably and profitably. But it’s hard to justify your job in IT if you say your roadmap is to just keep doing what you’re doing. Nevermind the people in between business and IT who really contribute to neither.
So we get these pendulum swings and huge projects (move everything to cloud, move everything back, make engineering do ops, make a dedicated ops team, micro segment everything, flat network everything) just so IT management can convince the business they’re providing some value.
Gartner is the “independent” third party that allows IT folks and tech vendors to convince the business they are doing the right thing.
And yeah I’m sure some of these pendulum swings have come at great cost to the business, even halting it entirely (SAP)
There’s going to be mountain of companies going out of business in 5 years due to following Gartner’s low/no code advice.
I was big cloud enjoyer in like 2016-20 but then some hacker news post open my eyes to the truth. They are trying to milk everyone and hook every possible person on some free credits then bump prices and get fat profits and ofc make as hard as possible getting out of the platform.
I just don't care and move to smaller providers for VPS and database + cloudflare for dns and pseudo s3 (still in almost free tier).
The only places I’ve seen it work are:
- massive seasonal demand with negligible usage during off-season (think TurboTax if it weren’t part of Intuit)
- brand new product development where you don’t even know what you need yet or you are going through exponential growth that you can’t get ahead of. (This stage should only last like 6 months though)
I was hoping those lessons would be a bit more common by now.
But nowadays, it runs full load 24/7 on the cloud.
Turns out the after hours batch jobs are just as demanding. shrugs
It’s just a super lazy take from someone who likes to use serverless because he doesn’t like to maintain things. It’s essentially an argument to live in a hotel because they can do maintenance at scale better than you.
And with his maps he’ll have analysed things quite effectively
I might disagree with him on the EU but his technology analysis is usually bang on
No shit, taking advice from a car salesman on how often to buy cars and what cars to buy is about the dumbest thing you can do.
His entire livelihood and career was built on selling this story that you need to pay others to run all of your infrastructure for you. He’s shilling IT illiteracy rather than pushing for more sustainable on-prem solutions.
Most on-prem solutions I've seen aren't sustainable, massive Capex costs, teams of people that could be better used in other ways just to keep trivial things applications running
I've worked all sorts of cloud environments for about ten years now, and with metal servers before that, and I'm not all that convinced that the cloud is saving all that much in terms of engineering effort. Or if it does, I haven't experienced it.
Sure, no one gets woken up at 4AM because your server's disks are failing, but instead you get woken up at 4AM because all your cloud native infrastructure is doing $funky_stuff. And yeah, no need to install and maintain servers any more, but now you need to write a whole bunch of code to interface with all the cloud stuff and maintain that. Lots of open positions for k8s engineers "dev-ops" engineers to make all of that work.
I run my stuff on Linode VPS's, which is "the cloud" as in "renting servers", but it's not "the cloud" as in all the "cloud native" stuff we have now. My experience is that sometimes the cloud makes things easy, and it makes things harder. On balance, it seems we just swapped one set of problems for another set of problems, and we haven't really gained all that much in the end.
I don't think Hey is running a "niche edge case"; it seems fairly standard stuff to me. That Tweet comes off as quite dismissive.
You nailed it on the head. The problems have just moved from one camp to another. We make our life easier because we can talk to an API to get our network, compute, and storage but the complexity in doing so has a pretty high price as well. The current devops trends are mindbogglingly complex compared to most of software that is getting deployed with them.
Have you considered that sometimes (really, almost all the time) people run code that is more than 2 years old? Have you considered the costs of things like serverless when you have a reasonably constant load?
I am a huge fan of cloud as a platform for time-shared server rentals. Cloud for "value added" products like serverless and whatever new database they cook up starts to get really funky and weird, in addition to being extremely expensive.
He doesn't see it as a fad to use the cloud and serverless, but as the natural way of things, like you don't build your own power plant or highway.
I don't really like using analogies for this sort of thing as I think it muddles things more than it clarifies, and quickly leads to discussions about the analogy rather than the topic at hand, but if we must use one I'd say it's more analogous to a business deciding to purchase a company van or lorry. For a lot of businesses, this clearly isn't needed and renting one when you do is fine, but at some point it just becomes easier and more cost-effective to have your own (depending on what you do with it, nature of your business, etc.) in spite of the extra hassle of having to maintain, clean, etc. your vehicles.
We're just disagreeing when that point is reached.
In fact, there are very few companies over a certain size that haven't built a road. This kind of overhead is generally expected once you have scale, because the alternatives are far more expensive. Datacenters are similar.
If you're one of them, go get your datacenter.
By the same analogy, tech companies should not run from building server racks and/or datacenters. It may not be your company's core competency, but it should be one of your competencies to understand how to run a computer. It really doesn't take that much work to run 10 computers.
That is the crux right there. And some people need one set of problems fix (so do the cloud) and some need a different set (so do on prem).
I very much prefer on-prem, but for some sets of problems, the cloud does make sense.
Talk about broad, sweeping statements. Care to substantiate?
To use cloud efficiently you have to build for it. Startups are much more efficient in that regard in comparison to monolithic applications running on premise.
Or if you’re a really small company - sure. You should probably rely on SaaS mainly.
I would say many companies fall outside of these categories though and could potentially both save money and gain performance staying on-prem.
I know my current employer does, and we’re seriously looking at moving back to on-prem/co-lo.
My sentiment leans toward host your own for a number of reasons, but it's challenging to make good decisions. Can you trust the person who is telling you that you can do it? Their org chart goes away if they lose this argument, so of course you're going to hear from them how we can do this.
If the team you have has been doing a decent job, maybe you should listen to them. If they've been doing a poor job, then you have to ask why they're suddenly going to be able to provide a level of service you haven't been enjoying for years. My company went through one of these fishtails. I think everybody is happier that we ended up in the cloud, even those of us who will only begrudgingly admit it. As I said in another thread, I think the Hype Cycle for The Cloud will at least give some orgs with a dysfunctional IT department an opportunity to clean house and then start over in a couple of years as informed consumers. Or find a new kind of service provider that distances itself from The Cloud salespitch.
It’s reasonable to presume there’s now an entire generation of developers who simply don’t know what’s possible in state of the art hardware.
Most other things, I run in my homelab, which is just a few old computers repurposed to act as servers for whatever I might need (especially for storing larger amounts of data, e.g. backups), including things like acting as CI/CD nodes, or test environments etc.
I still write "cloud native" software (for example, 12 Factor Apps) because those make the portability of everything pretty well, as well as use something like Docker and some lightweight orchestration on top of it (like Docker Swarm or Hashicorp Nomad) because it's like having the ability to make configuration for what you need (ports, storage, memory/CPU limits, scheduling, restart policies etc.) rather easily and also version it, more easily than Ansible lets me (which I still use for configuration of the server nodes themselves). Basically like multi-node systemd, once could say.
I actually wrote about some of this a year back: https://blog.kronis.dev/articles/how-i-organize-my-docker-de...
Though since I've also moved over to using Apache as my ingress, for a few reasons: https://blog.kronis.dev/tutorials/how-and-why-to-use-apache-...
Either way, it's nice that we still have the ability to base our workloads on open source solutions and learn about all of this stuff on consumer hardware (though certain configurations might be badly supported on choice OSes, like some BSD versions with driver support).
A few days ago I actually published a blog series on making a file and metadata storage solution that's based on open source technologies, which easily handled millions of documents: https://blog.kronis.dev/tutorials/1-4-pidgeot-a-system-for-m...
What I want to see is what they are going to replace the cloud with, how the numbers compare, and what tradeoffs they have had to make. Preferably a year after they make their transitions. An honest comparison that makes it possible for other companies in a similar situation to clear their doubts (in some cases “clearing doubts” would mean “we're staying in the cloud”; I'd like this kind of honest).
Thus I'm staying tuned, with a grain of doubt. After all, it's possible that the transition off cloud will take time and may even fizz out without achieving much. Oh, and I'm wondering what's the cost of moving the data itself off cloud is going to be.
Once upon a time computer bureaus sold time on their main frames, and you used a terminal (on a 1200 baud modem) to communicate with it.
Computer programmers were all men in ties, and fine looking suits.....
History rhymes
Would this work on AWS or Azure? Theoretically yes, but I personally think their serverless offerings are hard to use and don't make things easier.
Guesss that’s why a JAR or APK is a ZIP file with another name.
https://manpages.ubuntu.com/manpages/impish/man1/httpfs2.1.h...
cost plane, whether it be heavy on bytes at rest, bytes in flight, or compute, could possibly be private cloud.
scale to zero is something not possible without public cloud. it opens interesting doors like services priced cost plus.
what if zero usage had zero cost? what is all usage was profitable?
from the perspective of an individual customer and assuming light egress, lambda, ec2 spot, and s3 are shockingly affordable.
heavy egress is where you maybe do something else with the cost plane. that could be private cloud, or cloudflare et al.
There is no one size fits all advice. You have to model your work load. Take scale into account too. Cloud often makes far more sense at a small scale than a very large scale since labor costs can amortize. There comes a point at which hiring admins may be cheaper than paying cloud costs.
I think the three best reasons to use the cloud are: (1) If you have a very bursty workload, (2) if you need compliance with specific government regulations that are expensive to certify, or (3) if your app is a hobby project or otherwise fits inside the free tier.
A lot of applications fit into one of these categories, and, anecdotally, people running them are very happy with the cloud. Most CRUD apps today do not fit into those categories.
I have also seen companies have a lot of success with hybrid cloud deployments, where they run a datacenter for their base load and scale out into the cloud when they need to do more work.
I could spin up a postgres docker container locally, in shorter time than it took me to write this comment. On the other hand that wouldn't provide readonly replicas, standby-replicas, cross regional, point in time backups where i can just click on a date and the database will restore, access controls connected to my company directory. All that would take months to engineer properly myself and after that continuous monitoring and replacing of disks.
Maybe you don't need all this, maybe you do, it's all a part of your workload and effort equation. It's not possible to just take the cost of a VM and multiply with number of hours in a year.
37signals Leaves the Cloud
https://news.ycombinator.com/item?id=33260061
(245 points, 78 comments)
https://twitter.com/albertgao/status/1582402385569525761?s=4...
Either way: You're not counting storage costs which are around 70% of the costs in my deployments.
The latest Xeons are Rocket Lake based.
But they do not run bleeding edge. It would be irresponsible of them.
- However their own employees are typically consumers of services, that can be hosted either on the cloud, or on premises (which turns the argument on its head)
- Even for businesses that do offer services to customers over the internet, the alternative to the cloud doesn't have to be servers on their premises for this. They can be bare metal servers, either collocated or leased on datacenters. This is just fine when their services don't anticipate explosive growth. It doesn't even mean they have to sacrifice fancy "infastructure-as-code" tooling, though it does mean that they have to think about scaling in more terms than just cost.
But I'm rather positive it won't.
What leads you to believe that paying a premium to someone else to use their computers is a solution that fits each and every single case?
Even AWS is clear in how their present their offering as something that allows projects to grow and scale quickly while avoiding upfront costs buying and hosting their own hardware.
Ease of scale? Ease of development cost? Or other marketing material?
AWS's shared responsibility model is quite clear in stating that end users are ultimately the ones responsible for security and compliance. So your points are moot.
"Flexibility" in this case is so vague that it's meaningless. I mean, is there anything more flexible than owning the entire infrastructure and software stack you chose all by yourself?
I feel that your comment is a red herring. Using someone else's computer does not mean taking advantage of computing power alone, and nothing more. In fact, the point of paying someone else to use their system was never computing power, specially nowadays.
The reality, though, is that the steady march of progress encourages us to outsource what we can to people who are better at the thing that is auxiliary to what we do. I don't grow food because I'm bad at growing. I don't repair my car because I'm bad at auto repair.
I do build software, and my company builds a very specific type of software to solve very specific problems. I'm happy to focus on that, because that's how we make money. Other people are much better at building infrastructure than I am, and so I let them do it for me. If there comes a time when the cloud offerings are either worse than what I can build or too expensive, then I trust someone else will come along and fill the gap in the market before it becomes worthwhile for me to do it.
Unfortunately, the competency you are outsourcing is rare and getting rarer, meaning that infrastructure products are getting very expensive (and profitable).
These analogies are really bad. It’s not as if this is binary. Nobody is suggesting you fab your own chips or build your own data centers.
But you do cook (presumably) most of your own food, and you drive your own car. You capture most of the benefit of economies of scale and specialization, and then you do the last 5%.
This is what people are talking about. AWS is like eating out for every meal or taking an Uber everywhere. Sure it’s convenient, and has its time and place, but it’s probably not the best default option.
you don't grow food because it s cheaper to buy, it's all about economics (which takes care of division of labour; the word literally means the division of things in a house). Free money essentially means people who do random shit remain unpunished
It's not nostalgia, it's a cynical wisdom of surviving the "move fast and break things" mentality with which so many businesses shoot themselves in the foot. The more you can do in house, the less impacted you are by externalities.
> The reality, though, is that the steady march of progress encourages us to outsource what we can to people who are better at the thing that is auxiliary to what we do.
Outsourcing isn't progress, it's a business strategy that involves shifting responsibilities to a third party. Done right, it's an effective way to build on previous work to achieve an otherwise intractable business goal. Done wrong, it devolves into a shitfest as your success lies at the mercy of some entity whose interests are not necessarily aligned with yours.
> I don't grow food because I'm bad at growing. I don't repair my car because I'm bad at auto repair.
Because those things require serious investments in time and money. But there's plenty of things you can quickly pick up that make no financial sense to outsource. You can buy your own groceries and cook your own meals. You can change your own tires when you get a flat, or change your own oil when the dashboard light comes on. Not everything that can be outsourced should be.