Hacker News Reader

Comment by anotherhue | Hacker News Reader

> It’s frustrating to hear the statements “you must use our infrastructure” and “you can’t do that with our infrastructure.” You waste time either working around the infrastructure or sitting in meetings where you attempt to convince the owners of the infrastructure to meet your needs.

Listen, I'm an infra guy, so maybe this hit a nerve, but 'othering' infra because it's inconvenient has got to stop. Infra serves many masters and you are probably the most flexible of them -- that's why there's so much push back.

View on HN

IMO... infra teams need to realize that folks "othering" their infrastructure means the infra team isn't meeting their needs. Infra teams are now competing against DIY and PaaS solutions, especially all of the fly by night YC startups. Infra teams need to prove their worth or get out of the way and let the devs go buy a solution that replaces the infra team. The business cares about time to market, not the perfect little homegrown PaaS the infra team built.

For the record, I'm an infrastructure person.

100% this. If your company can run on a managed platform then it should. The reasons the whole world isn't already include: GDPR, SOC2 (audit, role level responsibilities), customer firewalls, encryption requirements (hello FIPS).

Managed platforms are obscenely expensive.

I know, I know - "time to market", "scaling on demand", "open source isn't free", "not our core competency", etc.

Infra staffing, hardware, scaling, and upgrades are expensive. But it doesn't help the fact that cloud costs an arm and a leg too.

I can't believe we haven't seen more of a race to the bottom in terms of costs and offerings.

Amazon is gorging itself on your margin. I wish we lived in a world with 20 AWSes, all at the same scale and market penetration.

I don't agree. People are far more expensive than managed infrastructure and that is what the comparison should be... not against 'bare metal.' When you compare the managed infra costs to bare metal directly, you are comparing apples to oranges.

We run an 8 figure business on mid 5 figure managed infra costs. An infrastructure team would be several multiples more expensive than that... even if it was a single individual.

Just because you can do it yourself or more cheaply, doesn't mean the effective business costs are less.

Edited: for grammar

>People are far more expensive than managed infrastructure and that is what the comparison should be... not against 'bare metal.' When you compare the managed infra costs to bare metal directly, you are comparing apples to oranges.

Clearly, because "DevOps" people / skills are for free right?

Good software engineers who additionally have exp with Cloud and all its craziness definitely aren't asking for way higher salaries?

I've worked in small size data center and the amount of work performed by 2x minimal wage technicians was huge.

They were running on 5-10 engineers + 3 more skilled ppl at day and 1 person on night shift with 1 or 2 being "wakeable"

Just to be clear, I never mentioned costs as a factor. Those are about margin, what I'm discussing is about core product.

"Is a product made by grown ups familiar with your sector" is an actual selling point for most enterprise SaaS.

100% part 2. Infrastructure has SO many potholes and edge cases. Tackling it directly by 'rolling your own' is optimistic naivety at best.

> Infra teams need to prove their worth or get out of the way

This is an incredibly demeaning comment. Assuming your colleagues have inherent worth and that they're doing their jobs reasonably given the constraints and demands placed upon them is a far more humane way to go about work.

> The business cares about time to market, not the perfect little homegrown PaaS the infra team built.

Those "perfect little homegrown PaaS" solutions get built for a reason - because of what happens after you get to market: the business starts to care a whole lot about how much it costs to run the thing you just shipped.

One of the main reasons a "perfect little homegrown PaaS" gets built is because it's typically a lot cheaper to run in-house. Lots of downsides to it, for sure, but you can't neglect the immense cost of a PaaS when you're running a business.

> Assuming your colleagues have inherent worth

That sounds cute, but humans don't have inherent worth in a business. If you don't contribute to success (and aren't good enough at deceiving people that you are), you're going to get laid off.

I think that's what the parent comment is saying - infra is competing with potentially cheaper and more flexible solutions. Even if they do their jobs the best they can, put 100% effort in, yada yada yada - if there's a more efficient solution, infra as a whole will be left behind.

> infra is competing with potentially cheaper

For example? The managed or cloud or... solutions are normally magnitudes more expensive than in house. PaaS are the closest in cost, due to the admin savings etc. but even there, we know current prices are investor funded to gain share.

At the far end, compare what you can get for $150/month at Hetzner vs. AWS. At real scale, paying for an infra team to run it will save a lot of money (and I've yet to see a fully managed solution which doesn't have a team of YAMLers anyway.)

There are loads of startups offering platforms as a service for about half the cost per year of one devops engineer.

I think you overlooked (or perhaps just ignored) their point about that:

> even there, we know current prices are investor funded to gain share

Artisan t-shirt makers have inherent worth too... doesn't mean we employ an artisan t-shirt maker for every 4 employees.

> That sounds cute, but humans don't have inherent worth in a business. If you don't contribute to success (and aren't good enough at deceiving people that you are), you're going to get laid off.

Of course humans have inherent worth in a business - humans are the business. I refuse to emulate the hyper-capitalist mindset of "some people are worthless." I'm not sure if you're saying that this is a value you hold, but it is certainly a value I do not share. There's a lot of room to talk about what "value" is in a business (including how some "value" is more readily quantified than others, and how we tend to mistake that for the only "value" that exists). But we have to stop conflating that with the worth of the humans in the business - that's not a humane way to view your fellow man.

> infra is competing with potentially cheaper and more flexible solutions

[citation needed]

I've been in the field for awhile now, and I can't say I've ever seen a PaaS turn out to be more flexible or cheaper in the long run. Even my beloved Heroku, to whom I compare all present and past PaaS offerings on the market! There are some limited exceptions I recognize for things like internal applications, or niche applications that do not have a broad user base (and never will), but I don't think those exceptions are enough to support the statement.

Our industry chases fads in a circle. This isn't the first time we've all decried "infra is bad, long live hosted services!" - Heroku's heydey wasn't that long ago, and I fondly recall App Engine being a big deal. Then the bill comes due and people migrate. Hell, you could even consider mainframes a part of the cycle, if you squint at it.

> Of course humans have inherent worth in a business - humans are the business

OK, can you please send me paychecks for doing nothing in your company? I have inherent value in your business, afterall. Thanks.

I used to badmouth the MBAs at tech companies to no end, that they don't get it, just do management planning porn, aren't around for the consequences, favor sounding good over being substantive, and generally could all be fired.

I then realized later that leadership and getting a large group of humans to work on a common goal is very difficult.

The idea the other group is doing nothing is usually a perception (or a lack of perspective, lack of empathy, and maybe arrogance), usually.

Very true. I realize infra teams have their own priorities but all too often I meet infra folks that would rather tell me my requirements are wrong than work with me towards a solution. If you're not getting better customization from the inhouse team then it's not hard to see a big paas is probably a better offering than what the infra team deems necessary.

And I will nuke your off-books deployments from orbit.

I'm involved in implementing some big changes where I work and one of them is CI runner-only deployments.

Both infrastructure and applications will follow the same workflow - all changes via pull/merge requests, Jira integration, etc.

Developers won't have the ability to create resources outside of the CI platform.

I really hope you sold the devs on this instead of commanding them to use it, otherwise their director will use his credit card to sign up for their own AWS account.

And that director would be fired as soon as this was discovered. At least where I work, but we are heavily regulated.

This would likely be the same outcome. We're implementing changes because an outside entity states we require them.

It's a regulated environment which needed locking down. The developers still have access to the same resources albeit in a more controlled process.

It's not like we flicked a switch overnight - that would be insane - it's a large scale cloud migration project over many years.

I've seen many cases where someone signs up for a free trial at PaaS-du-jour and then starts pointing prod HTTP traffic at mycorp.magicpaas.com

I've also seen it plenty of times in the startup world.

Unfortunately I've also been involved in the emergency bandage fix once it was discovered...

Through the forcefield of a VP two levels above your boss who authorised it? Even if you know about it - which is a big "if" - you probably won't have either the authority or the practical means to shut it down.

It's become a classic problem. Organisations want to centralise infrastructure and services because consolidation saves on costs and increases visibility and control. Particularly if you're under specific regulations that can make things much easier.

But that only happens if the centralised provision actually works. It has to be adequately resourced and reasonably well managed and supportive of the people using the infrastructure or services it provides.

If the centralised provision is not good then it had one job and failed to do it. Whose fault it is doesn't really matter. It's not supporting the other parts of the organisation to do their jobs and inevitably a point will come where that infra gets "othered".

We've been seeing this phenomenon ever since we had central system administrators who could lock down the OS on individual employees' workstations. The admins argue that they need to make sure security updates are properly deployed and they need to prevent non-expert users from breaking things and they need to ensure consistency because they're also running the help desk and ultimately they're responsible for these things so they need the authority to control them. All very fair and reasonable. And all completely irrelevant to someone with actual revenue-generating work to do who is being blocked for days or weeks because some sysadmin whose opinions are not necessarily justified by their effectiveness hasn't enabled or authorised some essential facility.

Seems pretty silly. Why do they have permissions if it's not allowed?

Reply on news.ycombinator.com