Hacker News Reader

Comment by pflenker | Hacker News Reader

Not to step on the authors toes, but it’s not the job searching that is broken - it’s the market that has shifted to favor companies and to put applicants at a disadvantage, something that hasn’t happened in the IT field for a long time. People reject candidates without a college degree because they can afford to - they still have plenty of applicants to pick from after filtering out the rest. There is no need for them to pay time and money to filter for diamonds without a college degree - they will find someone suitable for the opening even with the crappy applicant experience outlined in the article.

The current system is not broken, it works just fine - for the companies.

View on HN

There's that but there's also this:

> However, after heading over to Y Combinator to check out startup jobs, I was again frustrated by the findings. YC Jobs has a category for Engineering, but it's all software. There's a category for Operations, but it's all CEO, CTO, management positions. After searching and searching I was unable to find any listings for any network or cybersecurity jobs along the sysadmin/engineer lines. How on earth are all these startups operating without a network person? You don't have someone running your server or your VPN?

These "operational" jobs where people manage servers without engineering knowledge are much rarer today. Most people run their services on cloud, you need ops people that can code and do engineering as well, the people that used to cable and connect servers are not as important anymore, as the market moves more and more towards cloud or managed providers.

If i was the OP i would start thinking seriously about pivoting careers to a devops kinda job instead, where they will find many more options. The "tinkerer" network jobs just aren't that available anymore, it has been heavily abstracted away and the little work that there is the engineers will do as well.

Startups are too cheap to pay for someone running a proper VPN. Some rather buy some proprietary appliance and call it a day, even, if it ruins workflows of employees for years. (source: experienced this)

Yeah I've only worked for one company that had a dedicated Network guy, and his work was being phased out as we moved everything towards AWS. Eventually the only work to do at the company was managing security for the company wifi network

I think there are still a fairly significant number of these jobs, but they just aren't at small companies, because you can get extremely far without even one person doing it full time. Like, into the thousands of employees in most cases, I think? Certainly into the hundreds.

So I think this role just isn't very compatible with startup job boards, specifically.

Even at large companies you see less of that, I don't think it's a particularly bright career path right now.

those "operational" jobs exists today. just not in the small startups, that still didn't figure out, that despite running in the cloud you need networking and cybersecurity people

The longer my career goes on, the less a lot of these cloud services make sense to me. I know why we have AWS deploy our production instances, but why are all of our staging and dev instances running on EC2? Can't we just buy an old PC and put it in the office and run dev builds on that? Sure it might be slow, but it's not meant to handle 100k QPS, it's just for QA

> Can't we just buy an old PC and put it in the office and run dev builds on that

It's cheaper to pay for those cloud resources than hiring 2-3 more people.

I think if you had a whole server rack, you might have to hire someone for that, but if you're just running one or two servers with containers to run your dev builds, you could get that set up on K8s and manage your deployments the way you would with AWS. I don't think it would require hiring new people, an existing devops guy should be able to do that

It's still additional operational overhead for marginal RoI.

If you're an organization that already has a SRE or Infra Eng hired, there's no point supporting a niche setup when every other team is using a prebuilt and troubleshooted environment.

Edit: Can't reply to OP atm, but he makes a good point. For a higher margins and less commodified segment like ML Compute, on-prem computing still makes financial sense.

That said, that's still managed by actual SWEs not a generic Sysadmin, due to intricacies of GPU Architecture and Model Development, and OP is extremely underskilled for this role.

I think if all you're running is web microservices, then it's probably not worth it. But my company builds machine learning models offline and I feel that if we bought a dedicated GPU machine (literally just build one and put it in the office, not a rack mounted thing), we would save money in the long run after building a custom training pipeline

Why would you have your QA environment unlike you prod environment? Even the infra setup should be as close as possible, so you can test infra changes in QA before promoting them to prod.

I can't see how it would make sense to deploy in two completely different environments that are not alike at all.

> I can't see how it would make sense to deploy in two completely different environments that are not alike at all.

Because you are not saving that much money by having multiple bespoke on-prem environments per team.

Troubleshooting, management, and security headaches grow.

It's just easier to have everyone use a pre-built and troubleshooted environment.

in higher management there is aversion to any spend on physical stuff. where i work, we as everywhere look to cut costs. one of ways allowed to save ~$1.5M on aws costs over 5 years, by one time investment of $150k into physical equipment. It fell through.

The other aspect it's that somebody needs to manage old PC. usually there needs to be a team managing this infra and it creates friction between developers and infra team. so devs run into cloud and try to manage everything on cloud themselves. but at some point arrives consolidation and new team dedicated to managing same infra in aws.

Sorry, this won't cut it anymore. You would loose all the fun instabilities when the cluster is upgraded again, the round trip times could not be optimized by distributing things further, the magic of all the different translation layers between you and the hardware would vanish. Worst of all is not all the money and headache you would save but you also would save time. Why would you do that?

Right, much better to deal with all of those issues in production only.

As a startup CTO, I agree you need networking And cybersecurity. I am not sure if you need fulltime people specializing in this though.

I used to work for large companies with big on-prem footprint. Networking and security in that world is a different game and warrants dedicated people.

But for a startup with two services running in the cloud, with so many out of the box tools? (IDS, WAF, log based monitoring, SDN and all the configurability that comes with it). That can go a long way, without dedicated people.

>As a startup CTO, I agree you need networking And cybersecurity. I am not sure if you need fulltime people specializing in this though.

highly depends on function of the service and it's scale

>I used to work for large companies with big on-prem footprint. Networking and security in that world is a different game and warrants dedicated people.

>But for a startup with two services running in the cloud, with so many out of the box tools? (IDS, WAF, log based monitoring, SDN and all the configurability that comes with it). That can go a long way, without dedicated people.

maybe for network. but in my experience most of engineering (starting with junior developers and ending with vp/cto level) doesn't understand cybersecurity specifically or security in general . so even if there is tooling available, people don't understand when, how or most important why to use them.

That's fair. It's a red flag in general if VP/CTO doesn't have the basics of security in place anyway. My experience in my peer group is that they are all fairly knowledgeable about security, if not experts.

Most startups don't have the scale, there are exceptions of course.

i worked in variety of companies that varied from development of security products and were very security oriented to companies where security were driven by client needs (telecom industry for example), to "other places" where management grew with the company and it's case of "you can't teach old dog new tricks". there are always security departments, but unfortunately most of the time their thinking is "slap crowdstrike everywhere, it will solve all problems" and "here is sdlc that engineering must to follow" (lol)

Indeed, I've seen that latter behavior in large companies back in the days. A security department that refuses approvals to upgrade operating systems because it's too risky, a full-blown ops team that doesn't know how to do it without killing all services for days on, doesn't have recommendations on security patches, doesn't know if a CVE is actually exploitable in that setup or not - the list goes on.

i work now on fedramp certification (essentially leading scoping and solutioning) and interaction with security department is both funny,sad and scary af. i discovered that they developed risk assessment policy for system components (in commercial environment) whose purpose to drop down risk level of components in order to remove need for security patching for SOC. and crowdstrike in monitoring mode (nobody knew that it's in monitoring mode) because they afraid of enforcing mode. and that temporary access from/to production network is actually permanent because there is no flow in ticketing system to remove it .

> I was unable to find any listings for any network or cybersecurity jobs along the sysadmin/engineer lines

OP has actually fallen behind skills wise.

This is why companies have been moving Cloud First - your DevOps/SRE or even your Dev team can also be your Networking and Security team.

The writing was on the wall for Network Engineers and old school Sysadmins almost a decade ago, and why so many transitioned into becoming Infra, DevOps, SRE, or Security SWEs or joining Security and Cloud companies as Sales or Support Engineers.

Edit: OP seems technical enough (assuming they know how the difference between Podman vs CRI-O, how to code a DFS from scratch, and what Netlink is), but isn't selling their skills right. They need to invest better and think about how to better tailor a resume.

Not having a LinkedIn is also career suicide at this point.

> Not having a LinkedIn is also career suicide at this point.

Not a single job I worked I got from LinkedIn. LinkedIn seems to be spam-only site to me. Thus not sure about that.

I wonder how much it matters been just having a LinkedIn account vs actually and actively using it. IE not having a presence on the platform ends up as a negative for whatever reason.

> After searching and searching I was unable to find any listings for any network or cybersecurity jobs along the sysadmin/engineer lines. How on earth are all these startups operating without a network person? You don't have someone running your server or your VPN?

For smaller companies the answer is: "VPN admin is just one of many hats that a given generalist might wear"

It would be very costly to pay a FTE just for that one thing, it wouldn't make economical sense until the company reached a certain size and could afford a dedicated IT dept.

> a given generalist might wear

That generalist is your SRE/DevOps/Cloud Engineer in most cases.

This is why companies try to be increasingly cloud first - you can have 1 person do 3 jobs at once.

In very early stage startups you won't even have one dedicated SRE/DevOps/Cloud Engineer, you'll have a few Software Engineers who wear the DevOps hat as needed.

Even once you get a little bit bigger, I lean towards having a few generalists who wear more hats over breaking out a hat and putting it on a team of one.

Yep, more cloud native companies don't need a dedicated network guy. How do I get networks and VPN? I declare them in Terraform and they pop out the other end. Done, no weird Cisco CLI commands required.

The two places I've worked with dedicated networking people have both been relatively small non-tech businesses. In one case a third-party logistics company and in the other a manufacturer. Roughly a few hundred people in both cases.

Just pointing out that these places exist but openings are rare and often not paying competitive salaries. I agree that a career pivot to "devops" or "site-reliability engineer" would probably be the way to go for a networking person of yesterday to make a competitive salary today while retaining the relevance of their expertise. (Of course, if the salary is less important, there you go!)

“the people that used to cable and connect servers are not as important anymore”

What an arrogant take. As someone who does this, and is also a self-taught, no-degree-having troglodyte, I am forced to completely disagree.

Your cloud servers are built by people. That you don’t value those people says WAY more about you than anything else. I understand the abstraction— but that isn’t gonna matter when the PDUs are offline or the fibers get cut.

Tinkerers still exist and still get hired. Just not at FAANG. There is an entire world outside FAANG.

You're free do disagree, doesn't change the reality that most infrastructure will be running on monopolized data centers that need less and less people to operate.

I'm not devaluing anyone here, just stating that, as running infrastructure becomes a more and more concentrated business, there will be less jobs because the small shops/ISPs/data centers are disappearing, and what will be left will be the large cloud providers and equinixes out there.

They're all centralized now, though. It used to be that every medium-to-large business needed in-house networking expertise and ran their own small data centers, or at least ran a server room that required a couple full-time people to manage. Now everything is cloud-hosted, and those kinds of jobs have all but vanished, unless you work at your regional AWS data center.

Offices don't even need simple cable runs and switches/routers anymore. A basic consumer-level network connection and ISP-provided WiFi AP are all the vast majority of businesses need. Everyone has a laptop and there's nothing to plug in or manage.

> thinking seriously about pivoting careers to a devops kinda job instead

I wouldn't even necessarily consider this a career pivot, it's just a matter of them embracing the evolution of their field. Most of the other "network people" saw the writing on the wall years ago and re-titled themselves and got whatever new training and certifications were needed to use the new tools. They're a bit late to the game, but they can take the same path that tens if not hundreds of thousands have already taken.

I think the key is being a generalist who can take on whatever systems the place uses. I think that no matter what, you still need someone who knows how all the pieces fit together and can troubleshoot it when it breaks. Obviously I'm a bit biased since that's how I see myself. I do actually still rack a server now and again and set up a switch or a firewall or whatever is needed that day, but it's not something that takes up much of my time. That stuff is pretty mature so it just doesn't need a lot of attention even for companies like mine who still have some on-prem gear as well as cloud.

I think the cloud's impact on the hiring market is steeply underrated from what I've seen in the discussions. More business seems (to me, anyway) to be ran by cobbling together these services rather than developing technology in house. Unfortunately this devalues more general skills like networking, security, DBA, SWE, etc. for proprietary stack specializations.

Somewhat of a cynical take, but a lot of programs called DevOps and SRE are actually the same old ops people armed with some scripts and better BI tooling.

It's rare to find definitional SWEs in these spaces and when you do you should cling to them. Even then, it's hard to resist the urge to revert back to ops because toil is somewhat addicting to companies.

From what I can tell, there's very little value in putting a SWE into an ops role, regardless if it's SRE. The open source (CNCF in particular) tooling is really good these days. Hiring an SWE for ops results in them building a bunch of custom tooling that needs to be thrown out when they leave.

I couldn't disagree more. The tooling that is Kubernetes was written by SWEs in ops roles and there's many more examples like that. Much of the CNCF donated tooling wouldn't exist without SWEs in ops roles.

"These "operational" jobs where people manage servers without engineering knowledge are much rarer today"

They never existed.

Arguing against this; the system doesn't work fine for the companies either. They're wasting huge amounts of capital on candidates that they're distrustful of as they ultimately leap away anyways for slightly higher returns, repeating the cycle. This leads into a whole lot of internal issues as well.

The system is broken. It's just that companies can weather the storm.

Also job searching on its own is broken even when you ignore the market favouring the employer, as online submissions are still disastrous from companies asking invasive questions, to demanding LinkedIn profiles so they can apply through there instead of their own network, to automatic resume apps most companies rely on not even doing their basic job properly, to forcing companies to have to invent weird ways to verify the person who is applying is even a person at all. It's all a huge mess nobody wants to tackle because again, companies can weather it fine, and eventually everyone gets a job somewhere that keeps them too busy to care.

The software business has been trying to break the back of its workforce for decades. There was a huge anti-trust lawsuit about it in like 2011 and there will likely be another about the mass coordinated layoffs and hiring freezes in 2023 (complete with tacky EPS beat on tacky EPS beat).

This isn’t unique to software of course, the attack on working people is broad spectrum in 2024, but it’s probably the last white collar profession to go down in terms of labor pricing power. Even five or ten years ago it was a seller’s market.

> there will likely be another about the mass coordinated layoffs and hiring freezes in 2023

Did you mean 2024?

> This isn’t unique to software of course, the attack on working people is broad spectrum in 2024, but it’s probably the last white collar profession to go down in terms of labor pricing power. Even five or ten years ago it was a seller’s market.

I'm curious of where we are on the curve. Is San Francisco 2024 anywhere near Detroit 2013? My guess would be that it's a "no": San Francisco is in a much more attractive natural location and white collar professions tend to have more flexible individuals than blue collar ones.

Companies still have to contend with wasting time interviewing candidates who look good on paper.

At least they save on the interview travel costs now since most interviews seemed to have moved to Zoom and ilk.

On /r/sysadmin and /r/itcareerquestions I have long been downvoted when I disagreed with the mantra of "just get a cert, college degrees are worthless". I do think that most valuable learning happens on the job and on your own. However, the job market is not always so hot for developers and IT, and like now it will have down turns. I always recommended younger people that even if they could get a job off of certs, they should strongly consider part time classes to get a degree.

While a lot of the IT jobs are cert and experience focused, more like a blue collar job. They are still in white collar companies, in offices, and play normal office politics. So eventually having a degree can be a filter to moving up, so its better to get one sooner rather than later.

The other side of this is what this person is facing. Operations jobs are becoming rarer than they used to be, and operations jobs that do not require coding are even rarer. Just having skills doing set ups is a lot less valuable if you can't script everything to happen automatically in a cloud environment.

At the end of the day you might be a good candidate, but on paper 5 years of experience vs 5 eyars of experience AND a degree, the degree wins.

This. There was a power shift and now that companies are sitting on their capital making interest, they are acting on a more selective pattern then years prior. There was also way too much hiring during COVID, so new people thought that was the market... but it was inflated and so now it is also more competitive for the all applicants.

Best advice as others have said- get out there and network, join side projects, contribute to open source.

Since 2009 only once has someone asked me about a college degree. Once you hit 30 (27?) I don't think it really ever comes up in conversation in a professional context. Your resume shows where you worked and how fast you progressed, which is a much more accurate description of your ability to work in that capacity.

Time to start a labor cartel.

only works if you have bargaining power, which is determined by supply/demand

Hmm, what if we gatekeep the profession through licensing requirements?

Isn't that what OP is complaining too? Cybersecurity have above average licensing requirements

P.Eng required for, checks notes safety rated javascript web app dev.

I am 100% opposed to BS licensing requirements. They are intended to protect the income of the few at the expense of productivity for the many.

Too many strike rats

Reply on news.ycombinator.com