Continuous Intrusion – Why CI tools are an attacker's best friends (2015) [pdf]

For all these tools, there is documentation on how to lock down access to the server, and also how to run the agents under various users accounts. On my current engagement, for e.g., I ensure that the various GoCD agents run as low privileged users, with access granted via sudo.

Further, running commands directly is easy. But a better pattern is to push these commands into a script, to check in the script into version control, and to then check out and run that script. At least GoCD is opinionated that way by enforcing that users use "materials" (which could be a checkout from a revision control system).

GoCD enforces something called a "sandbox", where the file references are restricted to only the working directory for that particular job. If one needs to access files outside of this sandbox, then one needs to explicitly invoke a shell - after which point the admin/script owner is responsible for what happens. This is because the GoCD admin/script owner permitted either a command interpreter to be invoked directly, or the script owner did so within their script.

I now-a-days deliver scripts as Ansible playbooks using GPG signed RPMs, the execute the Ansible playbooks in local mode.

The non-salted SHA1 passwords is a good report. GoCD lets uses specify either an htpasswd generated file, or an LDAP source. On enterprise implementations, I always link GoCD to an LDAPs source and manage the credentials there.

In short, if one doesn't read the documentation and just implements distributed orchestrators with zero ops sensibilities, then such WTFs of being able to execute any command would of course be possible.

Good CI/CD processes increase not only security (less crufty servers lying around) but also recovery procedures because they force you to be able to deploy... to new, external, backup or refreshed infrastructure, if necessary.

For every project I've set up CI for, it doesn't matter if someone hacks my CI servers because they don't have access to anything other than dummy data. Sure; you could hack prod systems by building a trojan version of one of the binaries. But even then the binaries just get uploaded to a repository; so with proper code profiling and security testing you would notice the rogue binary before it ever hits a prod server.

It is important to keep CI as a dev process -- deployment should be handled by an orchestration system as it's much more of a tech ops process than a dev process. This is where config management and the like become very important; and those should be left to people who think like operations and security managers rather than hackers who just want to make shit work quickly (there's a place for both inside any good dev org).

The real trick is owning someone's CI system via the code under test. And it's even more versatile because it works even for projects which keep their executor scripts checked in to source.

1) Check out and gain an understanding of the target's CI scripts

2) Create a pull request (or whatever) which makes some reasonable code change but also introduces a vulnerability into the build/test scripts themselves. For example, curl'ing a domain you control.

3) Kick off pre-commit runs of all the tests to see if your patch should be accepted.

4) Own all the devices testing your patch.

Why would just anyone have access to production resources - which would be required to take advantage of any of the things he stated.

Also, dev/QA should be on a separate domain anyway.

It was dramatically obvious how bad this was from a security perspective, so perhaps this article isn't necessary, but certainly it's right on the money.

Nevertheless, it's a good reminder to consider build servers as part of the attack surface.

The way it works, Circle pushes the containers built to Google Container Registry and there is a super simple tool I built (https://github.com/crufter/puller) pulls those images. This introduces a couple of minutes of delay but we find that acceptable.

I guess we are still exposed to the possibility of someone breaking into Circle, using the keys there to push to our GCR repos, but at least they won't have full access to our servers...

My phrase is that it's the Nerve Center of the company. Don't let it get popped.