NATO Classified Cloud Platform Compromised | Hacker News Reader

NATO Classified Cloud Platform Compromised

ddosecrets.substack.com

238 pointsafrcnc116 comments

Everis is the typical meat grinder, and it is known for that in Spain.

Now, just as I'm writing this I'm sure someome from Everis will chime in to say he gets paid handsomely and works for amazing projects.

But everyone I've known working for Everis wants to die.

And if such project had to land in Spain for political reasons, there are plenty of companies capable on taking such project with way better prospects.

So it's akin to Capgemini, Infosys, Accenture, Deloitte, PwC, TCS and the like?

Don't forget Oracle.

Oracle still has a product though.

I wouldn't put Deloitte or PwC with the other ones. But yeah, you can add Atos techmindra etc for body shops. Accenture pretends to be like a big4 (like most consultancy) but they are much closer to an Atos/Infosys than PwC.

> wouldn't put Deloitte or PwC with the other ones.

I'm not convinced. I attended one of their recruitment events at university - lots of synergy going around. In fact I forgot about EY and a bunch of others too.

Yep. They take on some "advanced" projects too, but at the end of the day it's a meat grinder.

Article: The platform, known as NATO’s Service-Oriented Architecture and Identity Access Management (SOA & IdM) Project, is one of four core projects of NATO's IT modernization efforts.

So this expresses what "modernization" is - fob as much work as possible unto a job-shop/meat-grinder operation and watch things like this happen. I don't even know whether to laugh or be appalled, the Snowden leaks happened due to contracted-out sys admins, after all.

Nearly everyone who does real work like this for the government is a contractor anymore. The government employees manage the contracts.

> The government employees manage the contracts.

How long before that's outsourced to Accenture too?

No no, the companies executing the contracts are the experts, they know what’s best to put in them. It comes as free consult.

Edit: if only this was a joke.

"Industry can handle it better"

paying double if not triple if they would be internal worker.

Sure, the states of the world have decided that security isn't one of their core competencies - computer security, yes, but the point remains.

I find this mind-boggling but that's how it is.

It really is one of the dumbest things in government work. Politicians want to "shrink government" so they don't want many actual employees that would probably be well paid, stable, but someone needs to do the work. Thus, enter the I-too-want-to-shrink-the-government politicians who agree but then farm it out to contractors, some whom they might work for one day (or even come from in some cases in the US!). And so the actual government employees are few and far in between while everything else is farmed out to contractors, some who are good and many who do fly by night jobs on grants and have no incentive to do better otherwise.

It is also nice to have an entity to throw under the bus if necessary.

And that's actually a benefit for the tax payer!

It's notoriously hard to fire public servants, or shut down underperforming departments.

It's comparatively easier to shut down an outsourced project, or switch suppliers.

(Alas, not easy enough; and the organizational skills required to win government bids are different from those required to actually deliver quality.)

Do you think projects run and implemented by people directly employed by the government have a better track record?

https://en.wikipedia.org/wiki/Apollo_11

Not sure what you're trying to say with this.

The Apollo program was run by NASA, who had lots of government employees, but at the same time all the hardware and much of the software was built by contractors. The Apollo program was behind schedule and over budget for most of the project period.

NASA originally estimated the cost to somewhere between 7 and 12 billion USD; NASA Administrator increased this to 20 billion USD in 1961, but it ended up costing more than 25 billion USD in 1973 dollars.

Believe it or not, the Space Shuttle was an even worse disaster in terms of bang for buck.

I know, and they also failed on the primary goal of the project.

For the sake of debate, lets assume an equal level of incompetence between contractor run projects and government run projects.

In that case at least

a) There's no need to pay extra for the profit margins of the contractor companies (and the inevitable sub-contractors they use)

and

b) The incentives of the people doing the work are likely to be closer to the original project, in that the people doing the work are in the same organization as the project.

For me, outsourcing makes sense if the organization doesn't have enough of the specific type of work to have a fully staffed internal team, so specialist services, or if they are obliged to get an external opinion (e.g. auditing/pen testing). Otherwise you're just adding more layers of profit seeking middle-people...

Profit is a small part of costs. See https://slatestarcodex.com/2017/02/09/considerations-on-cost...

I'm not so sure about incentives. The incentives aren't necessarily great in either situation. See also https://en.wikipedia.org/wiki/Public_choice

"Overhead" is a large part of costs, and profit is often contractually limited in government contracts. But much of the profit is hidden in the markup, which can sometimes be 50%, meaning you pay $1.50 for $1.00 of supplies, labor, etc.

To add to this, the gov requires that all the markup is broken out, so it can be audited. I think usually contractors are allowed things like R&D budgets, salary (maybe not bonuses?) etc. I'd love someone more knowledgeable than I am to chime in.

Yes, though government departments also have overhead.

When allowed to do the job well, yes. The UK’s Government Digital Service led the way in this, and have massively improved the online experience of everything from finding information about the current Parliament to booking vaccinations. Their entire approach is focused on “what does the user need” as opposed to “what does the thousand page spec doc say”. The USDS was founded on the same basis and from what I’ve heard are similarly impressive.

That's heart-warming to hear, but alas cherry-picking examples won't sway anyone much?

Have a look at eg the outsourced Danish firefighters. Seems to work just fine, but would be unthinkable in most countries, including the US. (Btw, Americans have more firefighters per capita than about anywhere else.)

Dumb? Who did you think owned these companies that the work gets contracted out to?

"Shrink government" shouldnt be taken at face value.

Let me offer an alternative version of why people take the presumably-government roles and do them as private contractors: avoidance of byzantine requirements and cynical implementations of laws aimed at preventing fraud, waste, and abuse (FWA). People drastically overestimate FWA because they commonly describe or conceive as FWA government projects which they don't like. A couple examples:

It's relatively hard to get any work done at my job during the month of April, because about half the building is 85-88F (in a temperate mid-atlantic climate). That means that people will work in other cooler offices (problem when no one carries a cell phone) or just find reasons to leave early. I used to battle about this, but then I learned that it was for the unmovable ideal of economizing. Unfortunately, I'm a victim of economizing a 1970s building with huge plate glass windows that don't open.

Second example:office supplies. A supervisor asks me to have a room set up for a meeting. After determining that what I need is not kept on hand, I try to buy them through the government catalog as I'm supposed to. My internet connection fails, so I reboot my desktop (roughly 20-30 minutes). When it fails again, I lament that I probably have a dozen faster computers at home, ranging from 10 year old tablets to a raspberry pi, and with that I take the opportunity to step away from my office (low 80s in the summer, when this takes place, because the AC can't keep up.) I go down to my car where I can use my own air conditioning and my phone to look at the same website. The supplies aren't on the catalog, so I just order them for $10 on Amazon. Fast forward one week, when my supervisor asks how I have it ready so soon, and I get criticized for buying the supplies outside of the supply system with my own money. I place too high a value on my sanity, apparently.

Edit: I should mention that most people at this office are paid quite well relative to the area

This isn't an alternative version, this is what I mean about the horrible requirements which are aimed at FWA but just off-loading it to a contractor doesn't make it disappear.

Exactly. They also get the credit and retirement checks.

I'm a government employee. I work with a few dozen people across multiple locations doing actual work. Not one of us is a contractor. Zero contactors are involved in any real work. The few we have are in support positions. We have some contracted IT people who manage some of our computer systems but don't have any access to what those computers actually do. We have contacted security guards at the front gate. And I think the cleaners are contracted out but as they have NEVER cleaned my office I don't see them much.

I think there is some observational bias going on in this thread.

When I was a defense contractor it always struck me as weird that the guardhouse at USMC HQ at the Navy Annex was staffed by $12/hr private security guards.

On the other hand the commander of the group I was doing work for pointed out that a fully trained Marine private was far to costly an asset to waste checking visitor ids.

> the Snowden leaks happened due to contracted-out sys admins, after all

But that's a good thing, right? It's good that everybody found out that the government unlawfully collected their data etc.

Snowden was not an attacker asking for money, he was an honest guy with a conscience and a whole lot of courage.

It's good and bad. The bad news is they're evil, but the good news is they're incompetent. That's better than competent evil, but I'd prefer competent good.

Who can be good in a position of power? Arguably, "power corrupts" as the saying goes. I believe you could take the 500 best person on Earth to try and form the perfect parliament, and you'd still have an unjust system.

Abolishing the structures of domination/exploitation is the only way to have a good outcome, because it's the only way to ensure citizens will behave as decent neighbors, and not try to control and manipulate their peers. At least, that's what we anarchists believe and practice on a daily basis.

> The bad news is they're evil

Agreed, but this is really bad news, given the amount of power three letter agencies (and the deep state in general) have.

> the good news is they're incompetent

Here I disagree. I think the NSA are about as competent as such an agency is ever likely to get. We shouldn't be happy that they still have weaknesses, rather we should be horrified at how few of them there are.

Most of the spying programs had been in place years before Snowden blew the whistle. Countless other contractors and deep state employees could have done what Snowden did had they had either the courage or the moral clarity. Quite possibly many tried and were stopped before leaking to the public.

> the government unlawfully collected their data etc.

How did the ANT catalog leaks do that? Did you not notice Russia suddenly got more aggressive (annexing Crimea etc.) after the Snowden incedent?

In Germany this kind of task would land in the lap of T-Systems, SAP (who already do logistics / storage management and probably more for the Army) or one of these consultancies Delloite. You can't expect politicians to be competent.

I contracted for US government agencies for a long time. At every single job the contractors were doing the "work" (they had the knowledge) and the government employees were acting as managers. I have not met a government employee with an unusually high level of technical skill (though obviously many do exist). As far as I can tell (and I worked over a decade in this environment, in many countries) government employees exist to attend meetings and task contractors.

Technical government employee here. :waves:

Your observation is, in general, quite correct! Back in the 80s or thereabouts, the conservative administrations started a MASSIVE shift towards outsourcing of federal government work. This was for all of the usual, largely shortsighted and inaccurate “cost saving” and “efficiency” reasons.

The worst part of it is that they outsourced ALL of the technical expertise in many agencies, leaving them without the expertise needed to effectively manage the work being outsourced.

This is starting to shift, thanks to recent efforts like the USDS, 18F, and such but it’s the work of decades to really change. Come join us!

Former employee of a government contractor here.

Contracting worked on the Charlie Sheen principle: don't pay them to come around, pay them to leave.

Also, it is a much more efficient way to distribute patronage.

That would be the plan. If it is a government contract then the government is the customer. The customer manages the project and the hired contractors do the heavy lifting.

Look to the people who actually use these systems. Once it is up and running they will all be government employees. The real work, the thing the system is designed to do, starts after the IT infrastructure is up and running. The vast majority of people reading and handling classified informatiom are government employees ... most of them in uniform.

FYI, this system is apparently mean to protect "NATO Secret" material. That isnt a very high classification. I'd call it entry level, the sort of thing that nearly everyone in uniform is cleared to see.

The vast majority of people reading and handling classified information are government employees? That's not the case. I was cleared at TS/SCI + polygraph and many many contractors work in that environment.

In the office where you worked. In the office where I work there are zero. Classification levels don't really matter. It is about the nature of the information. Some is simply never shared with non-employees. Some isn't shared with people not wearing uniforms. Well, excepting one or two non-uniforms but they are still government employees.

I was military first then worked for defense contractors for a decade or so. In the areas I worked (several countries including war zones), at the levels I worked (secret, TS/SCI), in the agencies I worked with (DoD, special operations groups, intelligence organizations), I did not experience what you're claiming.

If you were American and in war zones then you were probably handling operational intelligence rather than strategic.

NATO Secret is one of the highest and you definitely need to have had extensive background checks and clearances to have access to this data. From my experience this is not the type of data your typical employee (uniformed or civilian) would be in contact with.

Per NATO's security guidance [0], NATO SECRET is the second-highest of ~5 classification categories It " is applied to information the unauthorized disclosure of which would cause serious damage to NATO."

In decreasing order of sensitivity, the categories are:

COSMIC TOP SECRET

NATO SECRET (NS)

NATO CONFIDENTIAL (NC)

NATO RESTRICTED (NR)

NATO UNCLASSIFIED (NU)

[0] https://www.act.nato.int/images/stories/structure/reserve/hq...

Those NATO levels are all just the start. Every NATO member runs many of their own layers. Over and above NATO come levels that are country-specific. In the US that means things like "five eyes" or US-only, which are technically secret (ie not TS) but are still above NATO. Remember that anything NATO is going to be seen by a large number of countries, including Turkey and Greece.

I'd just note there are problems besides information leakage when your secret information system is hacked - the system being used to attack more secret systems, information about the users being gotten, information possibly being changed, denial-of-service.

So you mean there are positive sides to it? Might also be true in this case since I believe they would host surveillance data on citizens.

Robert Hanssen was a government employee after all. It's incredibly naive to think that your employer makes any difference. https://en.wikipedia.org/wiki/Robert_Hanssen

I had a good LOL seeing this[1]. Those Land/Air/Maritime icons look very legit.

[1] https://imgur.com/Hcwayqq

I have that icon pack, it's from the early 2000's, if not earlier.

Pay is shit, there are a few interesting projects but that’s it. Good for people starting out young, but like all of these companies, full of lies and promotions that don’t mean anything except a title.

Forgot to mention: they don’t have an HR department anymore as they are too cool for that. They do have something they call “people” that manage contract, hiring etc. but according to them it’s not HR.

Add Indra to the list.

IMO this is the kind of project for GMV.

I worked as an outside consultant with a team at Everis. It was indeed a meat-grinder of a firm. The final week I was there we did a minimum of 15 hours a day, and I think the longest was 19 or something insane.

> "which basically tricks organizations into spending a ton of money for installing Docker into a CentOS image without any cryptographic signature to verify the integrity of that image."

Ouch

In the end, I think NATO was done a service by this.

Only if they learn something from the experience

Reading the start of this article reminded me of a somewhat unrelated thing I saw: I remember seeing in "tech influencer" youtube video on how "Japan hasn't kept up with the west" when it comes to IT. Not to be super orientalist or whatever and assume Japan is doing better the US in IT, but what should they do instead, go the US route and put every thing on the cloud? Is that better?

I couldn't help it, it's literally in the article that this was part of the "NATO modernization" efforts. Perhaps whatever they had before would have failed too but it's clear that these "modernization" efforts aren't always better.

Stop using fax will be enough to be considered modern by my standards

A type writer is more secure.

Maybe, until you need to send those docs somewhere.

never underestimate the avian carrier

EDIT: https://datatracker.ietf.org/doc/html/rfc2549

Beware:

https://www.manchestereveningnews.co.uk/news/greater-manches...

System error: Japan cybersecurity minister admits he has never used a computer[0]

And did not know what to do with a USB.

This person was 50 in the year 2000, has clearly given up being part of or informed about modern society, and is made minister of cyber security.

[0]https://www.theguardian.com/world/2018/nov/15/japan-cyber-se...

In line with the Persident of Nintendo who was reported to never play games. https://en.wikipedia.org/wiki/Hiroshi_Yamauchi

I don't have enough context on how well Yoshitaka Sakurada is doing in his job, but at a high enough level it's possible to be a good leader without skills required to do the job few levels below. (not sure it's a good idea to strive for, but still)

> at a high enough level it's possible to be a good leader without skills required to do the job few levels below

Sure. But being 68 and not being able to use a computer in 2018? Not recognizing what a USB stick is?

This is more about the fact that this person is not up-to-speed with society, doesn't really understand what has been happening around him the last 20 years. What dense Japanese cave did they find him in? Basic interest in society seems like a minimum to become a minister. The fact that he is minister for cyber security just adds irony to injury.

Agreed. One of my favourite manager types didn't have the first clue about the actual details of what we did. But he did listen to expert advice from the team beneath him and act on it. Bring the appropiate expert to meetings where it might be relevant and like you say, it can actually work. Obviously in an ideal world they'd know some basic bits, but if faced with the choice of a micromanager or a clueless person who listens to expert opinion i'll take the latter.

These counter-examples are what "prove the rule". They succeeded despite not playing with their own products. Good for them. But they are noteworthy because they are such outliers.

Counter-examples never prove the rule. Otherwise the same logic could also be used for the inept cyber-security minister, that he is noteworthy because he is an outlier. We have had education ministers who have never been teachers, defence ministers who have never served in the military, etc. and they are not noteworthy.

I'd say those people get the same noteworthiness treatment from the members of their respective professional fields.

well, that's the only way to never get pwned

I'm not sure how framing this in terms of IT migration to the cloud is helpful.

This could easily have happened if NATO contracted an equally-incompetent party to create and sysadmin a non-cloud data center.

I am pretty sure they use computers with internet on the government agencies. Unless the entire office uses typewriter or write with pen and the encryption being used by mechanical machines(does any mechanical machine exist provides the safety of AES?) your entire point seems void.

You joke about putting everything on the cloud, but it is one of the things that can massively accelerate innovation because you're no longer bound by paperwork to spin up a new server, or bound by server capacity for that matter.

And your Docker/k8s image provider is not bound by security policies to actually enable the firewall and/or have a default password at all for your database.

You can well have a government organization which embraces the cloud, yet keeps exactly the same level of paperwork for acquiring or activating resources. It is a cultural matter well before a technological one.

Yeah, you can no longer be bound by paperwork. But - having worked in digitalization projects for several megacorps - Bureaucracy is a beast, and it refuses to die - before long, you have now underemployed bureaucrats taking four weeks for registering an (internal) domain or a 14-page-project-outline with no less than 5 alignment meetings (involving ITSec, Compliance, the GDPR lawyer (even though you don't handle user-data), someone who wants to be promoted and 10 guys from Accenture for slidedeck-preparation) for a VM.

Favorite quote: "(...) so that the information security community and the general public can judge the quality of your work, which basically tricks organizations into spending a ton of money for installing Docker into a CentOS image without any cryptographic signature to verify the integrity of that image."

My only question is; why such project was connected to the internet at all ?

Do you know anyone who does not want to work from home?

Upper management. Everywhere they want the rank & file to stop WFH and come back to office.

More like people who can't. Working with sensitive documents sometimes requires physical engineering considerations (ie: viewing rooms). Not something that can be replicated in an employee's home. Contrary to what software devs like myself like to think, not every job can be done remotely!

A computing cloud that's not connected to the internet is completely useless. Besides, how else would the lowest bidder international contractor be able to work on it?

DoD has a parallel internet inaccessible from outside secure sites. They can follow the same approach with NATO computing needs.

Not if your private environment is large enough that you can create individual cloud platforms inside of it. One level more detached than AWS GovCloud / whatever Ali does.

The contractors would likely never touch the live system. Just deliver the project to deploy.

There's no NATO related data there - it's just a project for NATO.

If poisoned images were to be deployed into NATO datacenter, hackers would have no access to it as it's physically separated from internet.

Is this true? There are plenty of NATO systems that seem to be dual homed and accessible from internet either through port forwarding or VPN. That’s not “physical separation”.

> It will drive innovation and reduce operational costs by ensuring much greater reuse of capabilities.

I feel I have seen this vague promise on a lot of software projects that either failed or overran budget significantly.

It's because it practically boils down to: "We want all of the functionalities we are currently using in a variety of different systems in one system, but we're unwilling to make any compromises on the feature list. Everything has to be implemented exactly as it is in it's current system."

And then, obviously, the project fails spectacularly. The only projects like this I've seen succeed were ones where the top dog (CEO/president, whatever) basically forced everyone to compromise personally.

An interesting data point is that Everis is owned (2014 acquisition) by NTT Data Group [1], it provides consulting and outsourcing services and it doesn't have the greatest reputation

1. https://www.everis.com/global/en/about-us

Not that I think internal efforts are always success stories, but outsourcing your identity and access management to the lowest bidder sounds like a recipe for disaster.

Who thought this would be a good idea? And why was any of this on internet connected servers anyway?

Well, it's not like it's something new. Everything from roads to fighter jets get built like this. This is how we got the F-35s for example.

Does anyone know if there are other sources that can confirm the breach?

[Spanish] - https://escudodigital.com/defensa/hackeo-everis-otan-proyect... - https://www.everis.com/spain/es/news/newsroom/everis-disena-...

Distributed Denial of Secrets, you've done it again!

Oh well, shouldn't that be Centralized Distribution Secrets?

It depends! If they're compromised and put on a torrent, does that make them a Distributed Distribution of Secrets?

Would they still be secrets?

How was it breached and what’s the fall out?

How’s that Zero Trust working out for ya?

I really wish more companies would implement Zero Trust. It's just so nonsensical to trust any device at this point - No matter how "secure" someone tells you it is, or even you think it is.

Is there a clear definition of what ZTA even is? I read the O'Reilly book, and as best I can tell, the advice was fairly obvious recommendations and also 'replace your firewall as a single point of failure and make your access-control system your single point of failure'.

Better than trusting the perimeter. A single breach in a “trustful” intranet still results in full network compromise. Best to assume breach and work from there. Hating on zero trust is non-sensical to me, feels like the same as hating on security.

Given that, would zero trust plus perimeter have better security than each approach individually? Layered defense etc.

What does zero trust have to do with this?

The IdM box in the provided diagram [1] is the same single point of failure "policy engine" and "ID management" architecture that is advocated by the "zero trust" crowd [2]. It's the same type of flawed centralised architecture that allowed the Solarwinds Orion exploit to be used to such great effect [3]. At least the ZTA paper calls out this weakness [4], but unless I missed something the paper severely understates the risk that a central "policy engine" presents, and glosses over all the other components ("threat intelligence feeds", host-based agents, etc) which are centralised in this architecture and present a similar risk.

[1] https://imgur.com/Hcwayqq

[2] NIST SP 800-207, Figure 2, https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.S...

[3] https://en.wikipedia.org/wiki/2020_United_States_federal_gov...

[4] NIST SP 800-207, Section 5.1, https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.S...

Zero Trust is the future.

I wouldn't call running random Docker images from the Internet a "zero trust" environment.

Reply on news.ycombinator.com