Hacker News Reader

Comment by derefr | Hacker News Reader

Is it just me, or has the number of large hacks on “things you’d expect to be at least somewhat secure” picked up since the pandemic started? It seems like every week now there’s a source-code leak for some high-profile project.

My own assumption so far is that credentials that were previously being passed along through (essentially) in-person key-exchange parties, have now been forced to be passed along over channels like email/Slack/etc., making the ability to spin “mail/chat server admin creds” into “general system-level elevated creds” a lot more frequent.

View on HN

It wouldn't surprise me if the pandemic had an influence on some of these hacks. E.g. at our company there are some PCs still running Windows 7. For that reason they had no direct internet access, only to the internal network. But since about half the workforce, or at times more, had to work from home, and the licenses of our ERP software are bound to hardware, IT had to connect those PCs to the internet again, so the office workers could access them from home.

Of course the whole situation was less than ideal to begin with, but now it's even worse.

Just to point out this makes no sense: you can have connectivity between host A and host B without also enabling transitive connectivity to the world of hacker-contolled host N.

I believe this is assuming companies have segmented networks and jump hosts.

In my experience, most... don't.

Enterprise IT at non-tech companies is an absolute #&@_ show. The only rationale I've been able to come with is: if some portion of your company isn't developing / keeping up with tech trends, your IT shop isn't going to be pressured to either.

+1. Even when being told (for money!) to do the right thing most pay the consultant fee but don't act.

True story: A friend was pentesting a large company network in Germany, wrote his report and got paid. Years later, when being hired as head of IT security, he pulled out his old report and simply tried the default/easily crackable passwords he had discovered in the core infrastructure: They all still worked.

If his section about "use jump hosts" was ever useful for the company, it was because someone had that page open by accident, needed a place to put down his mug of coffee - et voila: His report even made a difference!

It's also the case that the perception of non-technical companies having garbage IT means they get overlooked by more savy admins in their job-hunt. Very few people want to drag a manufacturer kicking and screaming into modern IT, when it's way easier to get a job with a company that gets it.

This year is going to be huge for Google cloud and m365.

It's also been my experience that non-technical companies sometimes strongly undervalue IT or programming expertise. Few who have a choice ways to take a job at a rate well under market using outdated technologies with an employer who will not value them.

I had one of those jobs for a while. It was awful. The worst part was the ridiculous demands (example: all bugs should be fixable in 30 minutes or less) on top of the embarrassingly low pay.

> non-technical companies sometimes strongly undervalue IT or programming expertise

Half of this is actually the right way to do it. On one hand undervaluing IT in almost any company these days is a major problem waiting to blow up. IT isn't just an end, it's the means to do everything else. Using computers but ignoring any good IT practice under the excuse that it's not an IT company is like working from an office with asbestos, lead paint, and black mold because you're not a construction company.

But on the other hand non-technical companies, meaning ones without a strong IT culture and focus, should undervalue programming expertise. One of the worst things a non-tech company should do is deploy all kinds of custom IT solutions developed internally by their "programmers with expertise". Invariably (and I mean this in the most literal sense possible) they will end up with a patchwork of systems that nobody understands or maintains properly but which underpins all the core services the company needs or delivers.

"Undervaluing", by definition, means "valuing something under its actual value to you." It doesn't just mean "value the thing less than you do now," or "value the thing less than the average."

So...no, undervaluing programming expertise is never a good thing, whether your particular needs are for a lot of programming or only a little. Either you need exactly zero programming—in which case undervaluing it is impossible—or you need some—in which case you need to value it the right amount.

Well you're technically correct but I was hoping my point was clear beyond the vocabulary nit-picking. This being said I also suggested "the right amount" of value such companies should put on those skills is none, because it suggest they have wrong priorities and/or unrealistic expectations of what they can achieve. You can't undervalue 0. Nit-picking works both ways but just lowers the quality conversation.

In general precision is important but in this case it doesn't make that much of a difference besides a linguistic discussion. Let's try to look beyond it and more at the point I was trying to make: In the companies referenced above, as I understand them, valuing "programming expertise" at all is setting yourself up for disaster. You'll be tempted to use it but pretty much by definition in those companies you have no ability to support the outcome long term. Even tech focused companies have a hard time keeping up with the custom solutions they develop and are struggling with technical debt.

If you worked in these companies you know how this goes and have seen the story countless times. IT manager of small IT dept has a "great idea", hires some people to implement it, and they get it sort of done with the limited resources. Pretty soon both the techies and the manager move on to greener pastures, leaving the solutions in the hands of someone with little to no interest in it but who has another "great idea". The best solutions are manageable ones and custom stuff is hard to manage in the best of cases. Programming expertise is like a live grenade, only useful in capable hands (which "non-tech" companies are almost without exception not).

I think your first scenario is a good illustration of undervaluing IT, with attendant eventual disaster.

Perhaps your second is IT being overvalued? You've described a situation where technologies are being deployed inappropriately by an organization ill-equipped to handle the ongoing effort required. This is the sort of silver bullet thinking I would expect from leadership that does not understand IT beyond that it is powerful.

> You've described a situation where technologies are being deployed inappropriately by an organization ill-equipped to handle the ongoing effort required.

I took the archetypal "non-technical companies" you gave as an example earlier. If I understood your meaning correctly in the vast majority of those cases undervaluing "programming expertise" is probably the best thing to do. And by "undervaluing" I mean they such companies should not consider this as a skill they should rely on to build their IT around.

It's not that the skill is not useful in itself, just that it doesn't serve that type of company well, and valuing it suggests the companies are considering heading in waters that they're unlikely to successfully navigate.

Most of those companies will do things inappropriately because they are ill-equipped to handle this. For all intents and purposes "programming expertise" in such a company is like driving drunk. Sure you can get home safe anyway but that's not a reason to be proud of. Or you can crash but the real issue isn't that you couldn't cut it while driving drunk. You shouldn't be doing it to begin with.

I think there are two different kinds of IT value.

The first: I am a competent employee, who can do what needs to be done (in whatever technology).

The second: I am a forward-thinking planner, who surveys and keeps abreast of options and can identify, test, and deploy appropriate ones.

In my experience, it's the second that's lacking. Aka the "we can only deploy if Microsoft holds our hands through it" shops. Usually 1/2 because of lacking talent and 1/2 because of lacking / incorrect policies.

I've seen, but haven't seen too many, instances of "insert crazy state-of-the-art technology." Usually it's just institutional paralysis that prevents anything from getting done.

> non-technical companies sometimes strongly undervalue IT or programming expertise

They might also be unable to know if a job applicant is good at IT or not? And listen mostly to how he/she describes him/herself, and how confident he/she sounds?

Meaning, the company in effect hires IT people a bit randomly, and then mostly finds people who aren't that good at their job.

And if the company started paying more, then, more competent people, but also lots of more so-so competent people, would apply for the job? And I wonder if the company then still ends up with a random mediocre IT people who are unable to really secure the network? Just that the salaries are higher?

I wonder if the underlying problem is that 99% of the population is unable to know if someone is good at software or not

This goes back well over a decade, but a talented Linux admin at a tech shop had to spin up a fresh Windows VM instance to use some MSIE-dependedent intranet app. By the time they'd completed the task ten minutes later, the instance had been pwned, on the office LAN.

It's a sh*tshow everywhere.

Most non-tech companies don't have an IT department, they outsource.

Which means they rely on those companies to do the right thing, while not really prioritizing the work (i.e. not giving it much budget), or being able to critically evaluate the quality of the work being done.

So none of this is surprising.

Great point. + most of mgmt doesn’t see the real value in security until after the first (major) incident has occurred. (Has been my experience atleaste for < 50 employee orgs.)

You're absolutely correct. You can do that!

There may be some room to question the general technical expertise and competence of a shop that runs a bunch of Windows 7 systems, though. IT running beyond the limit of their competency may not be capable of doing what you so wisely and rightly point to.

There may be some room to question the general technical expertise and competence of a shop that runs a bunch of Windows 7 systems, though

Or they may simply not have a choice.

My company has a particular automated machine that I have to work with once or twice a month. It is controlled by a computer that runs Windows XP. It can only run Windows XP because the company that made the software went out of business years ago.

Because of that, the control computer is not permitted on the public internet. When I interface with it, it's using a dedicated laptop through a VPN to a remote session which then accesses the control machine on a dialup connection. It's slow as heck, but since I only have to do it once or twice a month, I deal.

Just before the pandemic, we looked into replacing the circa 1995 automated machine with a new one. Because of the nature of the machine, and the local government regulations about replacing it, the cost would have been little over one million dollars. Not going to happen.

Everyone hates it. But that's why my company's IT department, which tries hard to keep up with the times, has a single Windows XP computer in its stable.

which regs tells you what computers you can buy?

Sometimes medical devices and similar come with a computer as part of a whole certified system. The regulatory framework does not tell you which computer you can buy, precisely, but it does tell you what certifications your system must have to be used for a given purpose.

Changing out the part for an unapproved one, obviously, voids the certification.

which regs tells you what computers you can buy?

Not all machines are computers.

I should also mention that it isn't in-house IT, but a separate company, and the whole thing was set up in a rushed effort late on a Friday afternoon the week before the whole working from home business would start (our management slept on it until the last minute), with a lot of other companies requesting the same support from the IT company at the same time. Not that it completely excuses everything, and as you can tell from the Windows 7 PCs it was never the most secure setup, but simply to explain some of the reasons behind the mess.

This has all the hallmarks of an organization that views IT as a cost center rather than an enabler. Such an organization will tend to resent any spending on IT, so it's unfortunately not a surprise that it has likely scrimped and saved its way into ineffectual IT.

VPNs ?

Some? I've never seen anything newer than Windows 7 on a corporate PC. I'm sure they're out there somewhere but none of the companies that I've worked at in the last 9 years has ever had Windows 8 or newer.

Amazon was on Windows 10 in 2020 (and probably earlier; I don't remember when I upgraded while I was there).

The last Windows 7 I saw in production was on Wednesday. An attorney with a lot of secrets to protect had them on his notebook.

He isn't entirely oblivious about security, has his hard drive encrypted and encrypts some of his calls, but the non-updated system is a disaster waiting to happen.

I have to use Windows 7 daily at my current job at a bank (connecting via VDI). I forgot to wish the operating system happy tenth birthday a couple weeks ago.

> there are some PCs still running Windows 7.

A humble brag!

There are some very elderly machines out there.

I think the pandemic has a role to play in this for sure. I imagine may were required to use computers/tech as they've not done before. Often times those who are uneducated have to be accounted for so concessions are made.

For example, Sam needs to use a corporate network, doesn't really understand "apps" or what a "TOTP" is, so they optimize (weaken security) to allow them in.

Can we change this to "dumber than average" Sue? I guarantee when I'm 60 I'll still be able to safely operate a computer and understand what an "app" is, FFS. Ageism is really out of control these days.

Sure, but when you're 60 you might not be so hot on technology that you first encountered in your 50s.

I'm sure today's 60-year-olds are still perfectly good at the kind of things they've been doing since they were 20.

The implication that someone couldn't keep up with a relatively new things-- in your case vague example something they even encountered a few years earlier-- is pretty much the definition of ageism. Anyone who decides to keep their professional skills current will do just fine whether they're 5 years out of school or 40 years out of school. If someone doesn't keep current & is 5 years out of date on their skills, there's not a lot of difference between a 30 year old and a 60 year old that are equally 5 years out of date. Though the 60 year old might have decades more of experience working in their advantage, so maybe the two aren't equal.

25 years ago, the hot technology was dial-up modems. And PCMCIA cards to connect to WiFi.

Haven’t had much use of that lately.

You have a point about ageism, but you also make quite a condescending mistake of your own. You could have say "less tech savvy" than "dumber than average". It is not because you don't master something that you did not use before that you are dumb.

Absolutely agree on this. I've seen the entire spectrum by now: kids,who literally grew up with computers in their hands, not being able to operate simple software and people way past retirement age doing complex 'magic' on their PCs

I'm not impressed by the digital native, current generations.

I think a better descriptor would be "app-native" -- i.e. "I don't understand or care how anything computing works under the hood, and it'd better be tied off with a pretty bow and UI."

This is a topic we discussed with a colleague earlier this week: everyone's smart to click some buttons on an app,which, even a monkey with some basic training could do, but anything outside their comfort zone is a foreign language to them. Just a few days ago got asked what's the url to login to Office 365. There doesn't seem to be any willingness to think.

As someone who hasn't touched Microsoft office in a decade, what's the URL for Office 365 seems like a fair question. Isn't it competing with Google? I assume they offer an in browser client,

So yes, I have no idea how things I don't use work. This isn't that I'm unwilling to think, things are more open ended.

It's not the part whether one knows or doesn't know the url fascinates me: it's a professional setting, where people are quite used to solve problems much more complex than figuring out what's the url. For me it's the thought process: What's the url? I don't know.. error... error.. error. The equivalent of this is if I'd say to someone I've read a great article on Financial Times website,and they'd reply with: sorry, I can't read it,I don't know the url..

Isn't it also the kind of thing that large organizations self host? Sure they could have probably found it with a search engine, or some employee reference, but asking someone isn't a huge deal.

I don't understand your example. There is such a URL (https://www.office.com/). Or are you just suggesting that the colleague could have looked it up themselves?

I edited the comment, you make a good point. My bias is that in my experience the elderly tends to struggle the most as it wasn't common in their day, but point duly noted.

How about Pat? That’s ambiguous enough.

Or pretend you're in the southeastern US, where "Sam" and "Sue" are both non-gendered, and probably ageless, according to Mr. Cash, at least. You could also go with Courtney, Tracy, Billie, Casey, Drew...

Sort of: since 2017 the number of CVEs have basically gone up a steady 8% per year [1], but the fact that you only noticed it this year also suggests that this alone is not why you noticed.

The more likely reason you're noticing is that thanks to covid, and the accelerated death of real news services, combined with the echo chamber effect where we're all reminding each other of how bad things are, means you're getting more exposure to sensationalism, because that echos the best. And hacks certainly qualify as sensational, especially on slow news days, where news services desperately need clicks, and "X got hacked" gets those (even if it's a report on a hack that isn't actually one, like when someone walked into a data container with tens of servers, one of which happened to be rented by a password manager).

The real wtf is what happened in 2017, though.

[1]: https://nvd.nist.gov/vuln/search/statistics?form_type=Basic&...

I don't think major hacker sites getting hacked, or the MSE exploit from a few months ago qualifies as "sensationalism". These, especially the latter, are a Pretty Big Deal (tm).

What do you consider a big news item that isn't "sensationalism"? (Your link timed-out)

Something can be a pretty big deal in terms of security, and get dressed up in sensationalism by news outlets to get more clicks. They're not mutually exclusive categories. But it's the "we can get clicks by reporting more of these" that most folks notice, rather than the reporting being in lockstep with the number of hacks being perpetrated every month/week/day for the last 4 years.

I think people just have a ton of free time these days. I know I have.

Occam called, he wants his razor back :)

Pandemic has nothing to do with frequency of cyber security incidents at least not on a large scale. Every time when something global is happening spammers jump on the bandwagon and try to lure people into opening their email spam and buy something or download attachments filled with malware.

This global situation is specific because a lot of people are working remotely so they easier to target and compromise then before.

In the last decade or so a lot of businesses moved their presence and other ops online and lots of them practice poor cyber security that's why you see a lot of hacks happening.

Speaking of big companies they are always targets of state sponsored attacks and industrial espionage.

Generally the only people I've seen transfer credentials securely are IT and software engineers. You can count on every other department to send passwords over plaintext channels.

It’s not uncommon to test the resolve of a new administration soon after a changing of the guard.

>things you’d expect to be at least somewhat secure

I feel like with a lot of security or cyber-crime stuff there's a "physician who smokes" dynamic to it where the people who you expect to have great security actually often don't take a lot of precautions. How many hackers end up being exposed because of fairly trivial or random accidents is actually surprising.

I've heard it said that first day beginners on a wood saw will almost never hurt themselves. Their fear of it, combined with their known ignorance of it, makes them extra cautious.

Veterans of many years are the ones who get too comfortable, and too complacent. I guess the confidence of many years leads them to work a little too closely to the blade.

This might be a universal human trait, across any industry.

That reminds me that Qualys was just hacked.

I wasn't surprised, I've used their product...

This shouldn't be necessary.

Getting the SSH pubkeys of other developers is easy. Grab them from github or from a shared server or ask them once. Then use age/rage to asymmetrically encrypt the secret you want to share. No password has to travel in cleartext anywhere.

There was a US election.

And one in which many countries i.e. Russia, Israel, China, Saudi Arabia, North Korea did not want Biden to win and who have a record of state-sponsored hacking.

Solarwinds and the Microsoft compromises for example are clearly at a scale beyond your regular criminal hacker.

Reply on news.ycombinator.com