Hacker News Reader

Hacker News

Comment by colllectorof | Hacker News Reader

Parent Full thread

colllectorof

The tech churn cycle is getting more and more insane. It's the same process repeating endlessly.

1. Identify one problem you want to fix and ignore everything else.

2. Make a tool to manage the problem while still ignoring everything else.

3. Hype the tool up and shove it in every niche and domain possible.

4. Observer how "everything else" bites you in the ass.

5. Identify the worst problem from #4, use it to start the whole process again.

Microservices will save us! Oh no, they make things complicated. Well, I will solve that with containers! On no, managing containers is a pain. Container orchestration! Oh no, our clusters fail.

Meanwhile, the complexity of our technology goes up and its reliability in practice goes down. Plus, the concepts we operate with are less and less tethered to reality, which makes even talking about certain issues really hard.

View on HN

I've got a different read on this. It's always been complicated, it's just that each.. I don't want to say "generation", but roughly the same concept, grew up with and internalized and knew about the complexities of the tech stacks they learned, and so when something comes about that moves the abstraction one level higher than what people are used to, it's seen as unstable crap.

This isn't some ageist kids-these-days thing, I mean that the complexity has always existed, and it's always abstracted one level higher over time. Kubernetes doesn't exist because Google and Google-adjacent engineers wanted to foist off "ooh, shiny" on the world, it exists because it solves a problem with containers at scale. Containers solve a problem with resource usage at scale, which are really just an evolution of VMs and a response to their inefficient resource usage and brittleness at scale, and even they were once seen as the new hot fancy overly-complicated thing.

For context, I entered the IT workforce just as virtual machines were being introduced into the average enterprise, and a distressingly high amount of the complaints I hear about containers and k8s were being used against VMs as well.

I'm curious to see where we go from k8s. What the next level of abstraction up will be.

The fact that this is similar to something already happened reassures me a little. May Lambda and other FAAS frameworks be considered the next layer of abstraction? (even if cloud providers are using containers underneath)

Wow you must be old. Like probably retiring now?

VM/370 came out 49 years ago. So assuming you did a master's at university (I guess math at the time) I'd say you are definite retired by now.

https://en.m.wikipedia.org/wiki/VM_(operating_system)

(yes this is tongue in cheek sort of since you said average but you also said 'enterprise' which my cheek reads as large company :))

Bruv. You're catching downvotes for two reasons. A: ageism and B: also not realising that tools like the "old ones" used lasted decades. Release dates are a poor metric here.

to make this comparison funnier...

... the ancient equivalent of Kubernetes was released in 1973 (JES3)

I get it, nobody actually got the joke. Should've put the () first lol.

What I'm playing off of is that the poster probably meant when VMware and such took off in the enterprise. As one would see in my post history I know about what came before and that these things are not new in fact. And in many many cases still in use today which is your second point. What started 49 years ago is still in use on the IBM z series and all of that stuff has awesome backwards compatibility. My dad started his career with 360 assembler (as can also be seen in my post history).

So I guess it's ageism in the direction of _young_ people. I hope it's them down voting. Otherwise it's misjudging peoples sense of humour ;)

Imagine k8s but all the core features you need to run MMO-level apps are the equivalent of simple Unity Engine functions like Instantiate(gameObject). I want. Maybe I help create?

What's that quote about things you grew up with are seen as infrastructure, things invented during your early adulthood are amazing, and everything after is useless crap?

My complaint comes from the at scale part.

I don't do anything at scale. I don't have to scale. My core competency is as a library or tool builder, and often my primary deliverable is a tool or library.

In the past year I got a new JR engineer on my team who was all hot and bothered with docker and k8s and he spent a month changing out CI process to be docker based. It went from a 20 line shell script to a pile of garbage.

I disagreed with the decision at the time, and while a subject matter expert I'm not the team lead so I couldn't say no stop that's bad.

While I'm sure k8s solves problems for Google I'm not google. My company isn't google, and my team isn't solving that class of problems so docker is useless crap for me.

The person is no longer on the team. They left their docker mark, and ran off to dockerify some other project leaving a team with no container expertise and a CI pipeline that is hacks around docker bugs.

Sounds more like a people problem than a tech problem.

Everything is a people problem at the end of the day.

This is a thought-terminating cliché.

The existence of right tools for the job implies the existence of wrong tools for the job. The engineer in GP's story used the wrong tool for the job. That is a people problem. Had he used the right tool for the job, the problem would not have existed. GP wrote their CI stuff in shell to solve a tech problem, not a people problem.

Docker on its own is in most cases a great thing with a plethora of benefits. It's worth the upgrade from a 20 line shell script and I feel like you're part of the "I don't like new things gang". On the other hand it should stop there, k8s or swarm or whatever is not necessary for 90% of use cases or applications.

I'm with you for most of this, but I do think one element you are missing here is that the ever-increasing scale is at least in part to blame as well. Yes, software is vastly more complicated today, and perhaps suffers more errors, although I would want to see data on that. But, like, YouTube in 2021 is a vastly more difficult engineering challenge than YouTube circa 2008. The same can be said for any site with users numbering now in the millions. E.g., not only have Netflix's streaming user counts exploded since 2007 when it launched, the quality expected now of on-demand streaming has strongly increased as well.

Having said that, in reality that scale applies only to a small minority of products. I think an additional part of the real problem is the ever-present Cargo-Cult Oriented Programming model we've had for a while. I am sure container orchestration is important and maybe even required at places at Google scale. But I hear far too much about its use; it seems that nearly everyone is using it, and I really don't understand why. There seems to be a bit of a keeping-up-with-the-Jones effect, like people would be embarrassed to say that their startup relies on platforms that aren't the latest and greatest. Picking technologies because they sound cool on a resume or at a tech conference, instead of their use being appropriate for the circumstances, seems like a common issue.

I really don't understand why. There seems to be a bit of a keeping-up-with-the-Jones effect

The industry strongly incentivises individual engineers to make decisions that will ensure the latest buzzwords appear on their CVs - far more strongly than it incentivises making sound decisions for their current organisation.

It's true, nobody wants to be left behind with a bunch of skills that aren't attractive to desirable employers.

There's also very much a tendency to make things complicated in many workplaces as a form of job security and gatekeeping. But it doesn't last forever. Today's hot-shit is tomorrow's ball-and-chain.

10+ years from now, many k8s monsters will still be running and kept alive by stressed-out teams in India. Much like the "Oracle Enterprise Business Suite" giant-shit-ball from the late 90's is still working in many corporations doing absolute critical stuff with tentacles in every part of the org. Changes in these systems are nearly impossible because of the cost and nightmarish complexity, and _everyone_ is _forced_ to use it.

There's also very much a tendency to make things complicated in many workplaces as a form of job security and gatekeeping

It’s not only that it’s also boredom. Cranking out another cookie-cutter app can be much more fun if you do it in a novel way. Basically all the incentives are aligned towards exotic over-engineering rather than boring, safe but actually perfectly good choices. The danger will come if any savvy manager figures out what this costs them. So we’re probably all safe but you never know.

That's exactly what happens. But not out of ignorance and not only in IT. It's recipe for marketing success in any domain. Customers only know the problems of current solutions. Solve those and you get instant influx of customers. Sure, your solution brings it's own problems but it will take a very long time for them to surface and become common knowledge. First people will only check that your solution does fix the old problems in testing environments. And it does, so they move on. When they encounter implementation/production issues that's actually an opportunity for you to sell consulting. People who took the bait early on will mob on any naysayer: "Agile doesn't suck. it's just most implementations, but real Agile rocks!"

I think about it differently. The "problem" is that we are solving problems that didn't need to be solved before (or so we thought). The infrastructure I work on today is in many ways much more complex but it also handles a lot of things that we just didn't back in the day when everything was simpler.

> Meanwhile, the complexity of our technology goes up and its reliability in practice goes down.

I don’t agree with this at all. Reliability in practice is improving at a phenomenal pace. 10 years ago maintenance outages were a normal feature of every service, and unplanned outages were perfectly ordinary occurrences. Consumers today expect a much higher level of availability and reliability, which they receive rather consistently. Today it takes much less resource to produce a much more reliable system then you would have been able to produce in the rather recent past.

I don't know which industry you worked in, but outages in the telecom industry was strictly forbidden and came with severe financial penalties even 10 years ago. And those companies managed to adhere to really strict uptime SLAs even then.

It might take less resource today though to achieve the same, I agree with that.

Telcos have historically had availability regulations in many places because of how people rely on them for access to emergency services. So they’re a special case here, and the amount of resources they invested into optimizing for that is beyond the capacity for most organizations.

10 years ago I was working for a company that provided a financial OLTP service. We had to invest a huge amount of money to be able to provide a reasonable HA architecture, and to be able to meet 4 hour DR SLAs, and we still had weekly maintenance outages. The amount of effort required to accomplish those service levels today is comparatively trivial, and you could reasonably expect even a low-budget one person operation to be able to exceed them.

You’d expect a service outage to be a significant public controversy today for a lot of companies. It’s never been a good thing, but we’ve come a long way from it being a completely routine event for most services. Especially given the explosion in online services.

Depends on what you compare. If you compare mainframes vs k8s I would say reliability is not on par yet. If you compare commodity systems with simple monolithic apps vs k8s system maybe we just reached parity.

Reply on news.ycombinator.com