Exploiting CI / CD Pipelines for fun and profit
blog.razzsecurity.com
blog.razzsecurity.com
When I see things like those, they look so wrong to me. But sadly it's apparently uncommon nowadays: not only random bloggers, even my coworkers see nothing wrong with putting passwords or tokens into general config or source code files. "it's just for a quick test"1 they say and then they forget about it and the password is getting checked in, or shown at screenshare meeting.
Maybe that's why there are so many security problems in industry? /rant
(For those curious: for git specifically, use ssh with key auth. If for some reason you don't want this, you can set up git's credential helper to use your OS key store; or use plaintext git-crendetials, or even just good-old .netrc. For source code, something like "PASSWORD = open("/home/user/.config/mypass.txt").read().strip()" is barely longer than hardcoding it, but 100% eliminates chance of accidental secret checkin or upload)
You've never worked with humans, have you?
People & organisations tend to follow the path of least resistance. If it's easier to put passwords into a plaintext config file than not, passwords will invariably end up in plaintext config files in some projects. `PASSWORD = open("/home/user/.config/mypass.txt").read().strip()` will work right up until a colleague without `"/home/user/.config/mypass.txt"` attempts to run the project - at which point it'll be replaced with `PASSWORD = "the_password123"`.
The only pragmatic solution is to make it easier to handle passwords securely than to handle them insecurely.
Good security is expensive. Bad security is cheap (be it the example you mentions or a multitude of other ways). Management will favor the bad security done cheaply because the cost of bad security is extremely rare, and when it does happen, it rarely falls on the managers head. Either no one gets blamed (general blame the company, if at all these days), or the developer who made the choice to go with the cheap option gets blamed.
- an identifier for an environment variable that gives us the azure key vault scope (another identifier) - an identifier for the token to pull from that scope
Then the scope name and token name are used to pull the token secret value using the secrets api.
I am not experienced in how this is "supposed to be". Would it make sense to make both of these environment variables so neither identifier appears directly in code? (scope name and token name)
Thank you for the insight :)
Sometimes it's not "you":
> Pushing Malicious Changes to the Pipeline
mean that they already have full access to the repository in the first place? Normally I wouldn't expect an attacker to be able to push to master (or any branch for that matter). Without that, the exploit won't work. And with that access, there's so many other exploits one can do that it's really no longer about ci/cd vulns.
Other exploits might need more targeted steps to achieve. For example, embedding a malware into the source code might require language / framework fingerprinting.
I'm not saying that this is fine, just that access to master is probably protected, but it's still vulnerable.
> A surprising number of websites still expose their .git directories to the public. When scanning for such exposures on a target, I noticed that the .git folder was publicly accessible.
[...]
> With access to .git/config, I found credentials, which opened the door to further exploitation. I could just clone the entire repository using the URL found inside the config file.
The URL with credentials was found in the `.git/config` file, defined in the [remote "origin"] section. This is the way they won full access to the repo.
- they use that credentials to make a commit adding malicious code to the CI pipeline
- The rouge pipeline job adds their public SSH key to the `.allowed_keys` file in the production server
As the pipeline is run automatically on push, they get ssh access to the remote server.
That is the "CI / CD Pipelines" bit. That being said, it's a bit underwhelming, because given the title I though they were going to exploit a bug in the CI/CD software itself. I don't know if I'd call that "exploiting" CI/CD software.
It sure wasn’t a good decision to use git-config to store creds for CI though! I wonder if OP found a developer’s old cached creds in the history that weren’t used anymore but happened to still be valid?
This seems to be automatically mitigated in systems which might have a "build" / "compilation" phase, because for the application to work in the first place, you only need the compiled output to be deployed. For instance, Apache Tomcat.
Hidden files only attract the attention of pessimists.
For a Go service? Sure, that's easy and makes a ton of sense.
That's a bit of misconception. (Not to mention terminology mish-mash) what you probably call "interpreted" is languages like Python, JavaScript or Ruby. In all these cases, the projects are supposed to be compiled into an installable package, and then that package is supposed to be installed. So, compilation step is very similar to languages like C or Go.
Regrettably, developers rarely follow the due process and "deploy" what essentially amounts to the project's source code... This has a bunch of other problems, beside the security issues with potentially copying passwords into the deployment environment. This whole process reminds me of the early days of PHP where Web was full of examples of "guest books" which taught a generation of PHP programmers to interpolate values straight from the URL request into SQL queries. And then, PHP took the blame for "allowing it".
Ex. you use github pages and do homemade brew html or whatever that is not static-generator that outputs to a subfolder.
This is how all major players in the market recommend you set up your CI pipeline. The problem here lies in implicit trust of the pipeline configuration which is stored along with the code.
I was thinking maybe a better approach instead of CICD SSH into prod machine is to have the prod machine just listen to changes in git.