Curl to shell isn't so bad
arp242.net
arp242.net
Every decent Linux distro has a package manager that covers 99% of the software you want to install, and comparing to an apt-get install, pacman -S, yum install and so on - running is a script off some website is way more risky. My package manager verifies the checksum of every file it gets to make sure my mirror wasn't tempered with, and it works regardless of the state of the website of some random software. If I have to choose between a software that's packaged for my package manager and one I have to install with a script - I'll always choose the package manager. And we didn't even start to talk about updates - as if that isn't a security concern.
The reason we should discourage people from installing scripts of the internet is because it would be much better if that software would just be packaged correctly.
I wish this were true, but plenty of experience with Linux usage tells me that not having something packaged is a very common occurence.
Though of course this can be improved: More people should actually help working on their favorite Linux distro, so more software gets packaged. And upstreams should try better to collaborate with distros, which unfortunately they rarely do.
Certainly isn’t an easy problem.
It’s still probably a lot better than contributing to say, Debian. OTOH, the PR backlog is always daunting.
I've seen StackOverflow questions for "how do I install X" with thousands, sometimes hundreds of thousands of views and no-one has tried to contribute the 10 or so lines of code in the accepted answer as a package. Something is clearly too hard or not approachable.
Compare https://distrowatch.com/table.php?distribution=debian and https://distrowatch.com/table.php?distribution=ubuntu and you will realize that all the freshness of Ubuntu is build on top of what is available in Debian unstable.
I don't think this idea of "Debian packages are often very outdated" still applies nowadays. One can add "testing" or "backports" channels in your /etc/apt/sources.lists.d and get "upstream version" software. Even "stable" ships fresh enough software these days.
In the last case you can always get the source, update it and send a nmu back to Debian. Lets not forget that that is how open source works :)
This just isn't true. Ubuntu is typically quicker to update popular packages, such as desktop environments, kernel, etc. Debian unstable, even experimental, are often months behind on Gnome, for example.
I'm wondering if the periodic freeze-the-universe model that many distros use reflects a world that doesn't really exist anymore where distros came on DVDs (or CDs, or floppies). Whatever version you had on the disc, that's the version you're going to use.
I just started playing with FreeBSD in a VM, which has a frozen base system and constantly-updated packages separate from it. This works better for software you don't think of as an "OS component" but the question then becomes where you draw the line.
Or maybe it's just a fundamental disconnect between consumer-facing "move fast and break things" and enterprise-level "never break anything even if it means you can't move at all" and there's no way to make software that works for both.
Which with something relatively small and stable like sudo, is one thing. For big projects on rapid release cycles, like GNOME, it's got a much bigger impact.
Though I do see that Ubuntu does keep updating Firefox and Chromium to the latest versions in LTS, because they're so big and change so fast that backporting fixes has become practically impossible. That looks like a very rare exception to the rule; your typical Python library won't be getting that treatment.
But for Tor, I always use the Tor Project repository. Or for Docker.
And then there's stuff that won't even build in Debian, because it's been developed specifically for Ubuntu.
But yeah it's often that people don't understand what's Debian stable and its trade offs compared to Testing and end up unhappy with it or switching to Ubuntu (which is ~very~ similar to Debian Testing).
>there is security support for testing, but in general it cannot be expected to be of the same quality as for stable:
>Updates for testing-security usually get less testing than updates for stable-security.
>Updates for embargoed issues take longer because the testing security team does not have access to embargoed information.
>Testing is changing all the time which increases the likelyhood of problems with the build infrastructure. Such problems can delay security updates in testing.
How does it works? 1. Upstream release a new version, it goes to unstable. 2. Package is tested for some days in unstable and get promoted to testing.
So telling that testing doesn't get security updates is somewhat incorrect, since you are grabing recent software. But by the other hand having too recent software also has its downside ;)
https://www.debian.org/security/faq.en.html#testing
> there is a minimum two-day migration delay
Now you are making way to many assumptions with this phrase.
Do you really think that make sense to have critical security updates for stable having to pass through the normal release cycle? :)
I'm speaking from experience that when I was using Debian testing I would usually receive security updates days after they are available for Debian stable.
Obviously security updates for stable do not go through normal release cycle.
I wasn't commenting stable security updates, but lack of timely access to security updates on testing.
Also, this: https://www.debian.org/security/faq.en.html#testing
> there is a minimum two-day migration delay
Though the truly baffling bit of Debianese is "contrib" which means "this is free software but depends on non-free software." I can kinda see how it came to mean that, but it's very non-intuitive.
Stable basically means it won't change, and says nothing about freshness. Debian has recently adopted a policy of releasing on a time basis, so it's never very stale.
Personally, I don't trust Ubuntu LTS releases until they get their first point release, and even then I'm skeptical since they're a bit more loose with package versions on stable. I do trust Debian when it first releases because they're far more rigorous in their testing, though I usually wait a week or two before doing a release upgrade just in case.
I used to really like Debian testing, but I've since moved to OpenSUSE because they have a real rolling release (for my desktop) and a solid release based version (for servers). I like Debian, but testing gets a bit sketchy around release time (frozen, and then a ton of updates), and I honestly don't trust Sid aside from pulling in the odd package. I don't trust Ubuntu at all, since it has caused me far too many problems in the past.
Some people may say I'm stupid for not figuring it out, but the standard for usability of software has improved a lot since these systems were invented. The UX needs a serious overall.
Creating a Debian package is actually pretty straight forward:
1. download upstream tarball.
2. execute dh_make -f <path-to-tarball>
3. debuild -us -uc -b
That is it! There is no secret.
dh_make does the heavy lift of generating everything you need.
Your only job is to declare the dependencies (build and runtime ones) inside the "control" file, maybe change the "rules" file (it is a Makefile).
There are also several helpers that goes even further and automate 99% of the process.
I've been using Debian for 20 years and have never seen this advice. I currently use custom scripts for building local packages of things.
I think this comes down to Debian's weak documentation. The recommended intro guide from the wiki page [1] doesn't say this and looking through the maintainers guide I do see this [2] in there but it is buried in pages and pages of other docs.
Really, someone needs to write a new basic intro that doesn't bury the lede. It should start with this then expand on what to do for various issues.
Or maybe start with one of those 99% automation options. If they are even easier. As is you cannot find this information without some person telling you as you cannot find it by searching for it or reading the docs.
[1] https://wiki.debian.org/Packaging/Intro
[2] https://www.debian.org/doc/manuals/maint-guide/first.en.html...
Sure, in the most basic case it does this, but even in the most basic case it does more than that. At minimum it also verified the download matches a known good hash.
The important part (to me) is that Homebrew also ensures the package installed conforms to Homebrew’s standard. There’s a lot of standards (install in /usr, /usr/local, /opt, etc) on something as simple as where the files are put, let alone how it’s built and it’s dependencies are pulled in.
So to say there’s no value in installing from Homebrew over curl|sh is clearly misguided IMHO.
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Reminds me that there is still no official package manager on macOS. So https://nodejs.org/en/download/ has you comparing check sums.
2. Most of my post wasn’t addressing security, but actual real usability gains by using Homebrew.
As for ensuring that the package is well-behaved, could you elaborate on that? I'm not aware of Homebrew doing something like chrooting to /usr/local before running the install script. And the install script can do anything as your local user, same as curl-to-sh. Perhaps the Homebrew maintainers would catch something nefarious in the formula itself, but given that most formulas download code from the Internet and run it, that's not much help.
Yes it is, it’s ensuring that what the maintainer of the formula verified is still what’s being downloaded now. HTTPS does nothing to protect someone modifying the source URL, but the hash does that (assuming the maintainer actually inspected the initial download, which many if not most do).
> but given that most formulas download code from the Internet and run it, that's not much help.
1) the previously erroneously dismissed hash helps there.
2) the sandbox which you linked to later also helps too.
So yes, the hash prevents the upstream project from switching out the code at any time, but if they wanted to add some malicious code all they have to do is file a homebrew-core PR and hide it in a legitimate change.
By that logic, everything in any package manager should be treated with distrust then. Debian, RHEL, Arch, etc etc.
Not saying I disagree with distrusting, just making the point that risk exists everywhere, at some point you have to decide what you’re comfortable with.
That’s certainly true in some cases, but is definitely not the case in the majority. I think you are letting your personal biases color your judgement to much.
https://github.com/Homebrew/brew/blob/e2c76cce8e01fd80e0910d...
https://github.com/Homebrew/brew/blob/master/Library/Homebre...
https://github.com/pypa/packaging-problems/issues/74
Developer's website may change at any time, and even go back-and-forth between good and bad version. The pip software version won't. Put a version pin, and you can be sure you get a good package or an clear error.
Granted, you can record/verify the checksum of downloaded files as well, but many people don't. And crazy practices like 'curl | sh' make that impossible anyway.
Fixed that for you.
Compared to donwloading a binary and 10 other similar methods people use.
>Yeah, comparing to downloading a tar file from the website and running ./configure, make etc - right, it's probably quite a similar risk. But who does that?
Millions of people?
And even more just download binaries off of websites...
Install packages to allow apt to use a repository over HTTPS...
Add Docker’s official GPG key...
Use the following command to set up the stable repository...
> And we didn't even start to talk about updates - as if that isn't a security concern.
I think that's the worst part of it. Some software will nag you about updates, yarn and pipenv for example, but it's far more reliable to have one system that keeps everything up to date.
[1]: https://docs.docker.com/install/linux/docker-ce/ubuntu/
I have some confidence that at the least the Debian packages won't be changing rapidly, making it more likely that any problems will be discovered before they get to me. A script (or tarball) fetched and installed from the Internet can change literally from one second to the next. If a trusted site is compromised the next download could be tainted.
Not all "package managers" instill confidence. I've heard too many bad things about npm and the associated environment and won't have it on my systems, but I am not a web developer so the impact is the occasional utility I have to forgo.
For that reason alone I'm a huge fan of AppImage above snaps and flatpaks.
Many linux users? I get most of the software I need through package managers, but somewhat frequently I need to build it by source. Particularly if I want the most up-to-date version on debian.
Git cloning a repo is marginally better in the sense that, well, it's open. Theoretically if it were doing something nefarious, someone would've noticed. Is it perfect? No, of course not. I still think it's better than running random curl'd scripts.
are you asking who compiles and runs software? A lot of people, for example I every time a new emacs version comes out and it isn't in the package manager yet
After downloading you can at least do some sanity checks - even if you don’t checksum it if you are familiar with it does it look right? Is make doing anything weird? Whereas curl|sh doesn’t give you this opportunity.
Plenty of software projects put more care and focus into their software and not in their website, if you're running a vulnerable version of Wordpress or whatever CMS it'd be easy for someone to insert something malicious without being noticed whereas something that modified your code would show up in git, code reviews etc
For instance it would be typical in the past to sign the packages/software and publish the public key either to the site or somewhere else. Private keys used to sign the software would never touch the website infrastructure and would live on, typically, much more secure build or sign-only infrastructure.
The public keys used to verify the software could also be delivered through a separate channel, signed by a trusted third party, and etc. With Trust on First Use(TOFU) you'd trust the key when you first obtain it and be notified if the key ever changed unexpectedly.
I agree with the general point of this article. This is the weakest part of the arguments IMHO.
if I download latest music editor, and it wants root, I will be very suspicious.
Oh My Zsh use GitHub for their script so I trust it more than if they hosted it themselves for example
[0] https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b...
IMHO, "curl to shell" is uniquely dangerous, since all the other installation vectors mentioned don't support the bait-and-switch.
If I was already using curl to predownload and audit the script, I'd probably just execute the script I already downloaded which would be safe. Most of the people piping to bash directly do no auditing at all because they trust the source. If you're going to put a malicious payload in a script, you don't have to be that tricky about it.
Most people wouldn't know anything was up in any event until someone else discovered the attack and started raising a fuss on social media. I don't think serving the malicious script just to people who pipe it to bash (or really just download it slowly for any reason) would stop everyone from finding out. It would just make the malicious script more notable when found.
In this case, even "curl is dangerous" has at least two variations. The first is not knowing what the server is sending, the second is that the server can change what it is sending. My complaint is with the latter.
For example, a file in a repository somewhere or uploaded to a compromised web server is static. Everyone who downloads the file gets the same thing.
A file served by `curl | bash`, however, isn't. The server could send different files at different times of day, or only send malicious payloads to certain IPs (like known TOR exit nodes), or certain geographic locations, etc. which is something no repository I know of is even capable of.
Archives, packages, and installers downloaded from a server (instead of a repository or FTP server or S3 bucket where the attacker controls the file but not the server) share this weakness, so that alone doesn't make curl uniquely dangerous.
Where `curl | bash` differs from installers, however, is that it's interactive, so the server can alter its behavior on the fly. This is dangerous because, with installers, the attacker must commit to sending either a clean or infected payload before the installer can tell them if it's being run or not. In this way, even archives serve as a kind of a poor zero-knowledge proof of what the software is, since the attacker needs to commit to a version before knowing what the user intends to do. There's normally also a file left on disk as well.
With `curl | bash`, however, the server has the unique opportunity to get a callback from the installer before it has finished sending it, which means the server doesn't have to commit to sending malicious code blindly and hoping it's not being saved by someone who intends to audit it. Also, `curl | bash`, by default, leaves no trace, further frustrating auditing/reverse-engineering attempts. (Adding insult to injury, there's no way to check the malicious payload before running it, since running it is what causes it to appear. Even if run inside a VM, this can also be abused by an attacker to try to cover their tracks in real time)
In this way, `curl | bash` allows for obfuscation/anti-debugging techniques that no other method I know of offers. Hence, my opinion that `curl | bash` is "uniquely" dangerous.
Edit: Thinking about this more, this generalizes to any installer that interacts with the network, since all the attacker needs is a way to detect execution and some way to avoid leaving artifacts. In this way, curl is indeed not quite "uniquely" dangerous, since it's tied with other network-based installers. However, since the other popular installation methods don't have the ability to obfuscate their initial payload like this, I think the point still stands. (Obviously feel free to correct me if I overlooked something)
> you’re already trusting the vendor and site, and you’re already going to run the software that install.sh downloads.
I don't see how this makes sense? People do check what they run, and especially for sudo-calling commands.
To be clear, when I go to rust-lang.org, my goal is to download a large amount of extremely complex code that I never plan to audit myself and run it repeatedly on my computer, plus also trust it to download even more code that for the most part I plan to never read, and finally I'm going to trust it to take code and turn it into binaries which at least some of the time will run as root. In fact, it's very hard for me to imagine a scenario where an attacker is able to implement the timing attack in the grandparent post (which, to be clear, is very cool and clever and interesting), but is unable to pwn my computer in a huge number of ways that are both technically simpler and harder for me to detect.
The OP's point, as I understand it, isn't that it's impossible to pwn people via `curl | sh`, it's that in many cases, such an attack doesn't fit into a reasonable threat model.
I have full trust in Rust team, but even kernel.org was hacked once! And the worst part, experienced users won't likely to notice that installer does something weird -- because it is fully opaque, and because it
An alternative approach is a manual "git clone". This is way more secure, because the same endpoint and protocol is used by both new users and devs doing daily work.
Can someone compromise dev account and backdoor git repo? Sure. How long before this is detected? Not very long at all, I bet there are people who work on Rust and watch every incoming change.
by detecting the usage of `curl | bash` you can serve a different script only when someone does it, so someone doing `curl -O /tmp/some_script.sh` to audit the script wont see the harmful code.
It opens you up to a literally undetectable attack.
nonetheless, the point of the article author does have some truth. there is always a degree of trust involved when you're installing binaries from a third party. by using curl|bash you're just increasing the required trust a bit.
This is the crux of it for me. This is why it is dangerous. The author appears to have overlooked this attack vector entirely.
curl | bash -xThis has all been mentioned in the linked comment thread
Good point about -x being fallible to an adversarial script, even a simple set +x would be enough!
Where's the link where this has been mentioned? I missed it.
0: https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b...
Yep, this is why i hate piping curl to sh. Much prefer how e.g. go does this:
Tells you to just run
tar -C /usr/local -xzf go1.13.4.linux-amd64.tar.gz
Note that Qubes have some drawbacks, the main one being that it doesn't support GPU's, so not everybody are in the position to use it.
Maybe curl|bash is functionally equivalent to git clone && ./configure && make && make install, but my bet is on the one providing a standard install flow to be a better guest on my system.
In the curl|bash approach, all of this is lost.
The only system I've worked with that helps you truly deal with this is Qubes OS. Perhaps Fedora Silverblue will achieve this as well, once it comes out of beta.
There is a lot to be said about trust in software, but in general I agree that mitigations against untrusted software could be improved. The problem with this is that it's often hard to do this without affecting usability and, more importantly, in practical terms for a lot of people the status quo seems to be "secure enough", even though there are areas for improvement we (as an industry) should work on.
After the fact I realised I should have put if the account was root or not in the callback!
Alas all it needs is some trust and a sales pitch and someone will run it. At the time I didn’t think of the security consequences until after I had done it.
You could argue that by not having high quality independent third party review on a controlled market place (like the Apple App Store) has security implications. Because that would have checked for and vetted against abuse. But again this has nothing to do with curl.
But the OPAM install [0] deleted my $PATH, and it took me a while how to fix that one. They've since fixed what allowed the problem to happen [1], well it was a combination of that (normal shell problems) and a power-outage that killed the installer just before completion (in which case the script may just think it's complete).
But I'm sure similar catastrophic side-effects can occur in other install scripts out there.
Long story short, even when typing well intended shell commands, you can damage your system. (Even on Windows or macOS!) Directly piping curl into bash shows a lot of trust. It's amazing how well-intended the web is, must be at least 99,999999%
That said, I cannot count how many times following some tutorial blindly/some shell based installer made my carefully crafted *nix installation a bit worse. Nonetheless, most adware/spyware/malware I got through commercial download websites I think.
In a way it's a casting error. A type safety violation. You paste text into a privileged shell and coerce it to be sh, and when it goes wrong the sh input is rich in < and > characters.
Friends of mine have mentioned at least a) people accidentally pasting much more than the intended line into sh because they selected more than intended and b) sites that modify the cut buffer silently to add some "pasted from … blah … like us on facebook" or somesuch, I forget the details. The person who wrote the page intended one line to be castable to sh, another person who worked on the site added the script that transformed the cut/paste without realising that.
That environment variable was not set if the software hadn’t been installed and it wouldn’t run unless it was a root shell.
Guess who ran the update script instead of the install script and hosed the machine? I gave them a whole lifetime of bile over that.
The product turned out to be horrible as well.
Dbfile1 -> /path/to/dbfile1 ... Etc
Which needless to say hosed the entire box... over Christmas...
And this is why they don’t use putty anymore ;)
I know, badware ≠ malware
Not true: when you clone a repo with signed commits, you have forensic evidence that the repo signer provided the code you ran, while when you use curl you have … just the code itself.
That's not a lot, but it's not nothing.
The line you’ve quoted doesn’t say that there’s no fundamental difference between curl | sh and cloning a repo with signed commits, and I think it’s a stretch to think signed commits have enough usage among devs / users to make them a viable option.
But if github repo is compromised, anyone who pullls the repo can notice strange commits - and the evidence cannot disappear, as public head rebase will bring even more scrutiny.
The only software that has any right to rely on an ad-hoc install script on a Unix-like system is the package manager itself. It's awful enough that I have to do apt update and npm update separately. Please don't add even more ways to pollute my system.
The root cause of this is no universal packaging format for Linux in my opinion.
Which some have attempted to solve, so now we also have Flatpack, AppImage, and Snap -- each with their own little issues.
I wish there were a simple, modern, easily configurable tool that can take a declarative description of a project and spit out a ready-to-serve repository (just point nginx at it!) for most commonly used package formats. For Linux daemons this should be easier than ever before, now that systemd has gobbled up all the major distros.
Depends on who do you mean by "you". Software vendor can definitely write a script but they have no power over distributions to "move that logic to the packaging system".
This would have to be a collaborative work by different distros but from my casual look it's just not happening as everyone is happy with their own package manager that's "obviously the best".
I do agree on systemd. I didn't like it before I moved to Linux now I see a lot of value that it brings.
This may also be relevant: http://0pointer.net/blog/revisiting-how-we-put-together-linu...
I think there was a misunderstanding between us. I didn't mean anything so complicated.
By "moving that logic to the packaging system", all I meant is that instead of using a script to detect whether your app is being installed on Ubuntu or Fedora or whatever, you should just build and publish separate packages for each distro you wish to support. The logic for selecting the right package for itself is already built into every packaging system, ready for anyone to use.
Hopefully the process of building a dozen packages with each release can be easily automated once it is set up.
We have the freedom to use different package managers. It comes, likes everything else, with it's own drawbacks.
I can’t predict where a shell script will scribble or what other changes it will make to my system.
All these recipes and scripts are supposedly downloaded from GitHub. With supposedly anonymous user analytics: https://github.com/Homebrew/brew/blob/master/docs/Analytics....
IMO this is good advice for those that fall in the middle of these two categories, i.e. slightly technical people who run into problems and copy-paste solutions from Stack Overflow hoping that something will work.
> you’re not running some random shell script from a random author
This is exactly what is happening in the vast majority of these cases. These users are going to be vary if linked to an executable or installer, but "hey just run this simple line of code" sounds like a very appealing solution.
On the other hand, a solution on SO that would be a hidden attack would not gain upvotes and be an alternative for the one seeking advice there.
Verifying a hash that comes from the same server also doesn’t make that much sense. Verifying a PGP signature would be a compelling reason to not pipe to shell, and that’s really about it.
There are very few instances in which I've had to even use an installer on Arch. For many of those cases, the AUR provides a package that verifies the hash of the downloaded file anyway.
I've constantly been frustrated when using Ubuntu because something basic like having 'vim' not be months out of date requires a PPA.
The 'official' Rust installation method is a curl | sh. Or:
$ pacman -Q rustup && rustup -V
rustup 1.20.2-1
rustup 1.20.2 (2019-10-16)Rolling-release systems are awesome for personal machines where you can handle breaking updates or work around them. Usually I want the latest versions of everything when I'm doing exploratory stuff.
That said, modern software deployment is definitely moving away from "pick an LTS Linux distro and only change your application code", instead we mostly use containers now. A lot of production systems are probably still using the older technique though.
But no-one should be running this curl | sh nonsense in prod anyway, right. You at least want a defined version so you'd save the artifact instead of piping.
To me, the whole thing seems like a solution to a self imposed problem. It reminds me of the old "frankendebian" stuff in which people would be warned against having a system half-stable half-unstable.
it's too easy, and people with very scarce knowledge could develop a habit of doing this without asking questions and not even leaving any trace for a senior to inspect in case of a problem happening
I would say it depends. If the commits are signed by a key you know it's probably better. Even if it's not the case, cloning with SSH if you know the host key is also slightly better than downloading through HTTPS where any (compromised) trusted CA can MITM your connection :) (you can argue that those to use cases are rare in practice, and I would agree with you ;))
This is more like: not knowing what to do, when it doesn't work. And this is always the case until it works. Which is just a local Phenomenon and i can't expect things that work for me to work for others. So why don't write an expressive installation documentation with multiple steps instead of one-liners that either work or don't. There is just no in between.
Take the installation instruction of syncthing for example:
curl -s https://syncthing.net/release-key.txt | sudo apt-key add -
echo "deb https://apt.syncthing.net/ syncthing stable" | sudo tee /etc/apt/sources.list.d/syncthing.list
Same goes for the saltstack-boostrap-script. This script doesn't work on all platform equally good. This is not an reliable state. So in the end I'll stick with the normal way to install things which is very easy to automate.
Firstly, I made sure that the script told you what it would do before doing it.
Secondly, my instructions are two lines. Curl to a file, then run it through bash. A compromise, but if you mistrust the script, you can inspect it yourself before running it.
Well, yes. But the typical alternative is a tar-ball and a gpg signature - both via insecure transport, but verifiable (like with tls and a CA).
Git will typically be via ssh or https - so to a certain degree over a secure channel.
function main () {
# all code goes here
}
mainEven if your connection is TLS secured an MITM attack of causing a connection reset after X bytes could be a viable attack
To confirm unchangedness-in-transit I rely on TLS.
curl ... | less-and-maybe-cancel | shIf you want to cancel, just erase the file. Or `:cq` in vim probably works as well.
curl ... | vim -
:w !sh
If the app was a server instead of a cli, I’d start with a docker image. I ended up giving up on installing Erlang on my little embedded system and went with the docker image instead.
curl -fsSL get.docker.com | sh
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh