Comment by thejosh | Hacker News Reader

Comment by thejosh | Hacker News Reader

Parent Full thread

from the page:

Identified - We have notified all users and recommended appropriate action:

"We are contacting you to inform you of an ongoing security incident affecting CircleCI customers, as a result of the compromise of our database (http://security.mongohq.com/notice).

We are taking aggressive action to protect your data and systems. At this time, we have suspended all CircleCI account access, and all builds & workers have been suspended. In addition we have revoked all access to Heroku and GitHub OAuth tokens and API keys uploaded to CircleCI.

We do not yet know the scope and impact of the intrusion and are therefore treating this event as if all data has been compromised. While we have no evidence that these credentials have been compromised, we urge you to revoke the following:

SSH keys that were uploaded to CircleCI API tokens added to CircleCI as environment variables secrets stored in GitHub repositories We will be keeping you informed at http://status.circleci.com and will update you at regular intervals as the situation progresses.

We deeply regret that this has happened and are working around the clock to resolve this incident and protect your data and systems." 22:25 PDT Update - We are still investigating the issue. The full team is engaged and we are working with upstream providers to diagnose and respond to the issue, and protect all of our users. We will keep you informed. 21:17 PDT Update - We are currently investigating an ongoing issue with our database service. At this time, we have suspended all account access to our service. All builds & workers have been suspended. We will have another update in the next 30 minutes. 20:20 PDT Investigating - "CircleCI is experiencing technical problems. We're investigating and should have an update within 30 minutes." 19:30 PDT

I'm curious why they mention uploaded SSH keys. I presume they mean cases where SSH keypairs have been uploaded? The public key is public, and in fact anyone can use the github API to pull the verified public keys for any user in the system, http://developer.github.com/v3/users/keys/. If there is code uploaded with private keys in it, then it's likely there are other security problems in the organization.

Because CircleCI stores private keys for the sake of deploying to your servers when your build passes.

It's separate from your actual codebase having any private keys, which I agree would be a Really Bad Thing.

> variables secrets stored in GitHub repositories

Does this mean our code is compromised too?

That would seem to be the implication, assuming the people who hacked mongohq started trawling through circleCI's data.

If they have your keys then they can get at your code.

Talk about domino effect.

Our stuff, well non test/develop, is gpg'd in the repo. So should be fine.

Guess: this + mongohq was a targeted attack, aimed at a single customer of theirs?

I do not think so. There is currently no evidence that CircleCI itself had its data directly compromised as part of the MongoHQ hack.

They are just being cautious and assuming the worst and recommending their clients do the same (quite right too).

However the fact that so many apps can be screwed by a breach in a "Database as a service" style setup will make me wonder "How is this SaaS storing my data? internal or oursourced?" When evaulating new ones.

[EDIT] - Just noticed that I did not actually give opinion on the guess. It seems a reasonable guess that this could have been part of an attack on a MongoHQ customer.

I don't think it's relevant that it's a database as a service over all the other hosted services we put our trust in - if someone hacked Heroku, RackSpace or whatever other service provider they would get some db access, aws keys, source code etc too.

Private repos on github and bitbucket are probably a goldmine of accidents waiting to happen considering how many api keys etc slip into public ones!

Not really sure what the takeaway should be for developers building on [anything] other than to do your best not to store stuff in a way that can hurt your users if a platform you use is compromised.

Reply on news.ycombinator.com