Hacker News Reader

Comment by xkxx | Hacker News Reader

Oh, yeah! Good old curl | sh via HTTP. Fucking awesome. I wonder how many people get unknowingly pwned while executing such curl | sh instructions. It's not hard at all to MitM HTTP, detect if shellcode is being transmitted, then add your own instructions to it. Or you can just compile a list of known sources that encourage to execute curl | sh and MitM only them if you want to make your exploits even more discreet. What a nice way to get yourself your own little botnet. Way to go! There's a catch though. You need access to infrastructure of ISP. But is it that hard to get such access? No.

View on HN

For me the insecurity isn't even the worst thing about curl | sh.

I like package managers, and I tolerate tarballs, because I know what they do and how to reverse it. I care about the organisation of my filesystem and suspect that the people who suggest I pipe their script into my shell do not care at all.

I have similar concerns. Also, these scripts pretty much never have an accompanying "uninstall" or "reset system state" scripts.

For parent: check out http://github.com/awalGarg/curl-tap-sh (disclaimer: I am the author, and it was on HN's frontpage already a while ago)

Will add uninstall func, and reset func. I will have a look at your project

I'm so exhausted with the top comment being some off topic rant that doesn't contribute anything to the actual discussion but attracts a lot of reactions because it's easy to say something about it...

And yes, I am aware of the irony of this being completely off topic. ;)

I really wish comments like the GP were removed. It's like reading Comic Book Guy's thoughts.

curl sh is no more dangerous than running any script you haven't read, or binary you haven't decompiled for that matter. The odds of mitm are exactly the same.

No, installing a package using my package manager will

a) ensure this exact binary/script is GPG signed by someone I ostensibly trust

b) ensure it's not been tampered with by a MitM between the hosting server and my computer

This reduces my risk exposure to "creator of software X (or distro packager Y) has turned rogue", which is many orders of magnitude less likely than "website of software X got pwned, or someone is MitM-ing me".

That does not invalidate what I said.

It absolutely does.

You trust the distro maintainer and have a security mechanism to ensure the scripts and binaries have not been tampered with.

So virtually no risk of MitM and very little risk of malicious scripts and binaries.

Reading script and decompiling binaries does not imply you understand every thing they do, but it does imply that you have an unlimited amount of time which is impractical at best.

No, it really doesn't. I was not comparing to distro packages (which are an excellent solution to the problem, like I said elsewhere) but to download links.

Assuming script being fetched by cURL is hosted using HTTPS that is reasonably well configured it basically provides the same guarantees.

Unlike a distro package you can also just -not- pipe it to the shell immediately and actually read it.

Even if you did unpack every distro package source before you installed it with the amount of crap and boiler plate you would almost never be able to work out -exactly- what it will do.

The way I see it the "curl | sh" people have found somewhat of a middle ground. It achieves similar convenience to "apt-get blah" but with simplicity level closer to binary tarball. It's almost as easy to work out what it's going to do as the tarball but it will also do it for you automatically.

> Unlike a distro package you can also just -not- pipe it to the shell immediately and actually read it.

IDK about other distros, but for the one I use (ArchLinux) it's trivial to inspect the install script, and they (PKGBUILDs) are usually well structured and easy to read.

Unfortunately the majority of package managers are bloated and awful. Pacman/PKGBUILD and emerge/ebuilds are notable exceptions, though there is also Nix and many super simple based tarball systems that are also very good.

But all of that is irrelevant when 99% of the Linux world is running RPM/APT.

Yes. But this site doesn't use https even...

There are 2 types of "any script you haven't read, or binary you haven't decompiled": 1. scripts and binaries you obtained from somebody you trust in a safe authenticated way; 2. everything else.

When I update my Debian distro, I get binaries via HTTP, that's true. But I also have the public keys of my distro's maintainers whom I trust. The authenticity of every binary I download is automatically checked using those public keys. It's an example of "any script you haven't read, or binary you haven't decompiled" type 1.

curl | sh via HTTP is an example of "any script you haven't read, or binary you haven't decompiled" type 2.

curl | sh via HTTPS is an example of something that is very-very close to "any script you haven't read, or binary you haven't decompiled" type 1.

The presence of https prevents most mitm vectors. It doesn't prevent the far more likely scenario of the website in question being broken into and serving malware or backdoorware.

Point is, don't run binaries you don't have some way of vetting. curl sh is no worse offender than "download link here".

Do you never install any software outside Debian's main apt repos? Never add a 3rd-party apt repo for anything that maybe you didn't trust as much? Never clone a git/hg/svn/etc. repo from somewhere and build & install the software yourself?

Ok, let's say you don't do any of that. The Debian developers are still fallible, and they certainly don't audit the source code of everything they package. Sure, you have to trust something, and I'll certainly agree that it's more reasonable to trust the Debian packagers (and their distribution infrastructure) than a lot of other things, but saying "curl | sh" (when you're curl'ing over https at least, from a website you can reasonably expect not to be compromised) is always bad is a bit extremist. At any rate, the shell script you download is there for you to inspect before running, and if you're not happy with it, you don't have to run it.

Personally, my main beef with that sort of thing is that I want to know if the script is going to accidentally clobber any existing files, or install things to a location where I might not prefer them to be installed.

Edit: I'm only now realizing that the OP has you curl from a non-TLS webserver, which I imagine is the first thing that got you upset. Ugh. Asking people to install something that way is just irresponsible.

> The Debian developers ... don't audit the source code of everything ...

Strawman. If an amount of critical code is audited (and it is) it's still much better than nothing.

> I want to know if the script is going to accidentally clobber any existing files

Packages are tested on various hosts and architectures. The packaging system check for files being overwritten and that the package can be removed cleanly Also suspicious things (e.g. unsecure file permissions) are checked. Sandboxing tools are often used to contain daemons.

Furthermore the package content is tracked, while "curl | sh" cannot guarantee that the same script will be received every time or by every user.

> Strawman. If an amount of critical code is audited (and it is) it's still much better than nothing.

Is it, though? I'm specifically thinking about the Debian fiasco a few years ago when a packager broke OpenSSL's key generation. Clearly less scrutiny is paid than one might think. I'm unable to find any evidence/documentation/anything that suggests that code audits by Debian packagers of critical packages are done regularly (or ever).

I did find a few links to some Debian-specific tools to aid code auditing, but nothing to suggest where they're used, how often, and on what packages. Regardless, they look more like linters and static analyzers -- nothing that would help you discover backdoors or just flat-out malicious behavior.

> The packaging system check for files being overwritten...

Yes, I'm well aware, not sure why you're bringing this up. I was merely pointing out (regardless of any other argument being made) that file-clobbering is a reason why "curl | sh"-style installation bothers me, personally, much more than possible security considerations, which I consider to be overblown.

> Do you never install any software outside Debian's main apt repos?

Only the official repos, yes, because anything else would be insecure.

> Never add a 3rd-party apt repo for anything that maybe you didn't trust as much?

Heck no, never, ever. Not even once. That's insanely foolish. Frankly, I view, 'please add my PPA/repo to install' as a different way of saying, 'I don't know enough about security for the software I write to be installed on your computer.'

> Never clone a git/hg/svn/etc. repo from somewhere and build & install the software yourself?

I view that as somewhat different, given that the source is there and has a weakly-cryptographically-secure history (weak because it's SHA1), so that if someone ever did something bad then it'd be easy to prove it.

Cool, then you actually are "paranoid" enough (sorry, not trying to use that word in a pejorative manner). I frankly don't think most people are, and I don't necessarily think that's as dangerous as some might make it out to be.

How do you end up dealing with things that aren't packaged for your distro? Do you end up downloading source, doing some checks to whatever level makes you comfortable, and package yourself? Or do you mostly either not find yourself in that situation, or just take an "oh well, I'll deal without it" attitude.

I use a few 3rd-party repos, though ones that I would consider more trustworthy than a random PPA (for example, Google's Chrome apt repo). I certainly don't trust them as much as Debian's official repos (but, again, I'm not sure how much that trust is actually rational!), but I'd consider it unlikely that Google would get compromised or slip something nasty in (and even if they did, it's not like the Chromium source in the Debian repo has been audited).

> I view that as somewhat different, given that the source is there and has a weakly-cryptographically-secure history (weak because it's SHA1), so that if someone ever did something bad then it'd be easy to prove it.

That's all well and good, but that sounds like an after-the-fact reactionary thing. It's little comfort to be able to prove something bad happened after you've been owned.

I just feel like most of desktop security, even on Linux is on pretty shaky ground, and we greatly overestimate the care we take when installing software. I think you're probably ahead of most people by using only official repos, but it's like the classic analogy of a chain with a single weak link: all it takes is one "git clone" of something that does something malicious, and that's it. Sure, the probability of a successful attack is reduced by avoiding 3rd-party repos, etc., but attack possibilities are still very much there, and it feels like people don't seem to see that.

But, overall, yeah, the state of desktop security wrt software installations is so much better on nearly any Linux distro than common practice on Windows or macOS, it's crazy.

It's not exactly the same, and http injection by ISP's is definitely a thing.

https://www.djm.org.uk/posts/protect-yourself-from-non-obvio...

I know of the attack. It's very unlikely to matter in practice.

There's a ton of potential issues when you're curl-piping; all those issues are present in some form when you're simply doing curl first then sh.

It's far more egregious to download a hard-to-inspect binary then run it, than it is to download a shell script then run it. Although it's harder to inject code on the fly, if you're at the stage where you're worried about these kinds of attacks it makes very little difference.

I don't see people bitching about https://www.terraform.io/ offering download links for example. So yeah, it gets tiresome to always see "oh! curl|sh! boo! bad!" - there's actually no better way of sharing such scripts short of making and vetting a package with a high-entry-bar distribution. And a two-step curl + sh instead of a pipe is security theater.

https would certainly be appreciated on that site but that's a general issue, regardless of the download script.

if you believe shell scripts are hard enough to write that systemd can take over. Then you should also believe that they're hard enough to inspect that you want some level of auditing.

RPM's are inspectable, signed by default and integrate better with your system. (the same is true of apt). And if they're in the package repository then there is a maintainer who is ultimately responsible for ensuring the quality of the code.

For the second time here, I'm not dissing package managers... I even explicitly said the solution is "vetting a package with a high-entry-bar distribution".

But the whole point of high-entry-bar distros is that it's highly curated. Some random script from the web will not be in there. How do you serve those that need to casually distribute trivial software?

There's some answers to that question - they're not widespread at all. It's one of the major failures of package management on Linux. macOS gets it mostly right.

Are you claiming that macOS has better vetting of its packages than Linux? And what do you mean by "Linux" anyway? -- lumping RHEL in with ADistroIMadeInMyGarage is a bit odd.

I was talking about adhoc packages.

I am interested. How does ad-hoc package management on macOS improve over "Linux" ad-hoc package management?

Well, linux doesn't have adhoc package management.

Because there is no standard packaging format between distributions, the best people can do is release a deb that may or may not work on either debian or ubuntu. Or an RPM that may or may not work on RH/Fedora. Or a tar.gz that will contain sometimes source code, sometimes a binary, and if it has source code there's no clear way to compile it, if it has a binary there's no clear way to install it.

So instead, curl sh.

It's pretty ridiculous that it's still a problem, tbh. What we need is somebody with the right political clout at either RH or Debian to sit down with devs of the other, come up with a decently generic solution that fits both systems and a backwards compatibility policy. It doesn't have to be perfect nor the best, it just has to be good enough to support the use cases of most Linux distros.

Once both Debian and Fedora support it, Ubuntu (and its many flavors) and Red Hat will follow. And if it's a generic enough system, with a pluggable-enough backwards compatibility policy, it's not unlikely for it to be adopted in more minor distros. If it's sold correctly, distros will gladly adopt it - we've all been waiting for this a long time.

I guess I do not understand the manner in which you use "ad hoc" to refer to package management.

It seems as though you are now talking about cross-distribution package management. It is further confusing that you are contrasting all "Linux" to the single distribution macOS.

To me the term is "ad hoc" current w.r.t. NixOS , GNU Guix and other purely functional systems, and there the distinction is made between:

1) declarative -- in which a rebuild of the system will ensure the package is present and configured as expected; and

2) ad-hoc -- in which the user can affect all of the system with side effects and rebuilds will not result in the same state.

Why would you want to allow users to randomly and aribitrarily affect all of the system and end up in an unknown state? Isn't that exactly what curling shell scripts achieves? And what method does macOS implement in order to avoid this?

If all you're talking about is cross-distro package management then you and your users can either give up on this and standardize on one OS and just pony up the cash for RedHat (same as you do for macOS) or else start writing Flatpaks.

I read OP's comment as ".... via HTTP!!!"

No.

https://jordaneldredge.com/blog/one-way-curl-pipe-sh-install...

A two-letter word is not a good rebuttal to a point I addressed in one of the replies you did not bother reading.

I'm guessing almost nobody.

What you have is an acute case of righteous rage against a flaming straw-man. The only medicine against this affliction is girding your loins and guarding your mind against those seemingly plausible but ultimately horseshit security theater tropes, so popular among the impressionable today.

Attacking curl yielding nerds is not going to get you a botnet.

I have replacs http with https now.

Even before looking at curl | sh over http, simple copy and paste of code from a web page is a very real security concern.

https://news.ycombinator.com/item?id=5508225

http://www.ush.it/team/ascii/hack-tricks_253C_CCC2008/wysinw...

> It's not hard at all to MitM HTTP, detect if shellcode is being transmitted, then add your own instructions to it.

Is there any evidence of this ever happening, anywhere?

The -f option cleans up the security concern of executing an incomplete download at least:

> -f, --fail > (HTTP) Fail silently (no output at all) on server errors.

You need access to infrastructure of ISP.

You don't even need that. You could spoof a wifi network with a PineAP or similar and MitM at that level.

The first thing I looked for is an installer that actually does something else than curl | sh.

Relevant: https://curlpipesh.tumblr.com/

Reply on news.ycombinator.com