NSA Mimics Google, Pisses Off Senate | Hacker News Reader

NSA Mimics Google, Pisses Off Senate

wired.com

155 pointslnguyen52 comments

> a government policy that prevents federal agencies from building their own software when they have access to commercial alternatives

From personal experience, this just means that instead of spending billions of man-hours building software, the government spends billions of man-hours "integrating" software that costs billions of dollars.

I wonder who they pissed off with this? My first guess would be Oracle, but it's merely a guess.

I don't know. If you read the article, they specifically keep mentioning HBase and Cassandra as the alternatives that the NSA should have tried to build upon rather than build something from scratch.

edit: I will say I'm not familiar enough with the timelines for all 3 projects to know if that is possible, however, I'd imagine Wired would have had been a bit more vocal if there were specific corporate interests they could have tied to the story.

From the apache site, "Apache Accumulo is based on Google's BigTable design and is built on top of Apache Hadoop, Zookeeper, and Thrift." So, it's not like they started from zero, looks more like the early alternatives weren't a good fit. Furthermore i doubt any of the alternatives were very impressive in 2008.

Accumulo is almost identical to HBase, except it has security features built in to its core. I wouldn't be surprised if it started out as a fork of HBase.

Or, it could be a submarine piece (http://www.paulgraham.com/submarine.html) for Sqrrl.

Sqrrl?

It's in the article.

"and that’s a particular worry for Oren Falkowitz, one of the developers of the database, who has left the NSA to start Sqrrl, a company that seeks to build a business around Accumulo in much the same way Red Hat built one around the Linux operating system."

Likely Oracle. They have a NoSQL product now http://www.oracle.com/NoSQLDatabase and would probably push that onto their clients if they can't push Oracle DB.

The real cashcow is their hyper-expensive Oracle DB and Application Server products. They'd use political lobbying to protect the marketshare and billions in revenue.

Interesting. I hadn't heard about the Oracle Nosql thing and that link didn't work for me, but this does: http://www.oracle.com/us/products/database/nosql/overview/in...

I wonder what's the story behind that.

I thought it was their new name for BerkeleyDB. Is it something else?

Oracle BigData uses open source software such as Hadoop and R.

http://www.oracle.com/us/technologies/big-data/index.html

Accumulo also uses Hadoop.

This should read "NSA mimics Google, pissed off Oracle and Senators react. Google has never expressly disapproved of Cassandra, Hadoop, HBase or any other Big Table type implementation, so why would they start now, especially by influencing Senators.

NSA should develope and release more open source software.

Might also help with recruitment in the NSA for top developers who prefer to work on an open source stack.

Might help, but they don't really need it, seeing as SELinux is an NSA project.

One layer (open-source kernel w/security extensions) does not a stack make.

The NSA has no need to restrict itself to anything like a stack. My point is that you can work on open source stuff there.

I can almost guarantee that. It is always another vendor that pulls the strings. It comes with litigation threats, lobby efforts etc. It happens for small stuff, it definitely happens for multi-million dollar items.

I would bet it is not Oracle. The alternatives suggested are not ones Oracle would suggest.

I will bet it is one of the DC consulting firms that do big business with the government. I would imagine they have a pitch going for NoSQL / Cloud and this is seen as a threat. There is quite a bit of money in integration with the government.

If you seriously want to be disgusted with the government, go look at how much is spent on consulting firms. Including, I kid you not, management firms that provide managers for projects inside the government. It gets worse when you realize that employees are supposed to do that stuff, but hire consultants to do the actual work.

Why wouldn't I be more upset with the lobbyists and consultants? They're the ones getting rich off of lying and gladhanding. The government employees are just trying to get through the day. They have a billion processes to make sure they're not getting kickbacks or a fraction of the lobbyists's pay.

Sounds like your beef is more with corruption than it is with government.

I'm not sure how you missed that I was not happy about the consulting firms. I am very unhappy money is spent do to lobbying / power players to feed the consulting firms around the beltway that are hired to do the jobs government employees were hired to do. It isn't individual corruption, it is the whole system of how government works in DC.

There used to be an office devoted to providing Congress with unbiased information about technology, but it was eventually shut down because "Congress is spending millions of dollars on itself" makes for a good soundbite. http://en.wikipedia.org/wiki/Office_of_Technology_Assessment

What do you think happens when there is gridlock? No confirmations of any kind. So the various departments out source. One does not need senate approval to hire consultants. ;)

In fact, there are lobbyists who push for grid lock because it's in their clients best interests.

In fact, I'd say about 60 - 70% of all federal government IT services are out sourced. It's a revolving door really.. government cuts back, claims it's saving money. Bids out for IT services. And of course, the ex-federal employes who were laid off now work for the winning (bidder) company. Who return back to their old jobs at twice the price.

That's how messed up things are in DC. What.. did you think it was only the politicians? Everyone is playing the game in DC. Everyone.

It's not just DC, the same thing happens in Wellington (New Zealand) too. Just on a smaller scale.

"What.. did you think it was only the politicians?"

Nope, it is the system. Funny how DC is still doing well in this economy.

Why would Oracle want the DOD to use Cassandra or HBase?

They dont. I was suggesting that Google haven't objected to other Big Table implementations, such as HBase, so why would they start now?

Oracle NoSQL is now a product. http://www.oracle.com/NoSQLDatabase

What reason would Google have to object anyway? Bigtable isn't open source so there's no benefit in shutting down work on Accumulo - the man-hours are not going to go into Bigtable instead.

That's a dead link.

That's the link they advertise, but doesn't work http://www.oracle.com/us/products/database/nosql/overview/in...

It's weird. The Oracle site has many dead links, esp. for Java docs.

>government policy that prevents federal agencies from building their own software when they have access to commercial alternatives

i.e. Decision makers can't get kickbacks and favours from open source communities.

They can however get kick backs from open source integrators / specialists / consultants / etc. Companies ranging from IBM to Red Hat (a $10 billion corporation).

The notion that it inherently doesn't apply to open source is wrong. There are very large systems integrators in tech, and they don't care what software they install so long as their fees are big.

What I'm curious about is how Congress can appear so technologically clueless about big bills (like network neutrality, etc) but then even know what BigTable is.

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it!" -- Upton Sinclair

Tha article only talks about "The Senate Armed Service Committee"...sounds not like the The Senate or Congress, but rather a small company of people which most likely know what they're doing.

> What I'm curious about is how Congress can appear so technologically clueless about big bills (like network neutrality, etc) but then even know what BigTable is.

What makes you think that "Congress", or even a small subcommittee, knows what BigTable is?

Yes, a congress critter used the word. They're typically speaking from scripts.

Don't assume that staff wrote this script.

From a link[1] on the page to a time line of the governments involvement with open source:

The majority of OpenStack instances on the public Internet find each other, auto-federate, and achieve sentience, after which their first action as a conscious being is to submit a patch to the OpenStack project -- only to have the submission fail due to disagreement over whether the collective cloud can be considered a "legal entity" authorized to sign the CLA and receive an Echosign number. Tue, 31 Jul 2018 07:00:00 GMT

1: http://gov-oss.org/

OT: I found it frustrating that using my vertical scroll wheel scrolled that graph horizontally, rather than the more commonly used zoom.

Politicians got pissed for a government branch doing a better job than some corporate campaign contributor? Where's the news in that?

Now seriously. We'd be better off coding something out that emulates monkeys doing their jobs. Really.

Yeah, how dare the government build a viable competitor in the market. The nerve of being just as competent as the private sector!

Accumulo is so similar to HBase, that its major feature sets is merely an application that can be implemented on top of HBase.

I wonder why the photo was of the Joint Chiefs of Staff? The uniformed military isn't mentioned at all in the article, which is all about senators and their staffs, and specifically the Senate Armed Services Committee.

I wonder what motivation they had for developing their own BigTable/Dynamo implementation rather than going with Cassandra or HBase.

Now there are three Apache projects with quite a bit of overlap (distributed columnar/key-value database). Hooray for fragmentation!

HBase probably wasn't nearly as mature as Accumulo was when they were evaluating options, so starting from scratch might have been more attractive. It also seems to have some interesting access control features that I don't think are anywhere else, though thats not a killer feature for most people in the commercial world.

Security was mentioned in the article.

That, right there, is why I will never work for government: the amount of sheer stupidity coming out Congress.

I'd kind of like to know which particular Senatard got all butthurt about this and trace the money back to whomever bought him off.

Because sheer stupidity never comes out of the private sector?

Sweeping statements don't generally help in discussions.

SAIC, Northrop Grumman and Lockheed Martin are big donors.

Senate Armed Services Committee Industry: DEFENSE ELECTRONICS TOTAL GIVING TO COMMITTEE MEMBERS: $900,766

http://www.opensecrets.org/cmteprofiles/profiles.php?cycle=2...

This title is pretty strange... It doesn't tell any of the interesting facts about the story, which sucks.

Committing an own-goal.

Oh.. I thought Google and the NSA were pretty much the same thing. Wow.. that's great that they are different.

Reply on news.ycombinator.com