Hacker News Reader

Comment by TeeWEE | Hacker News Reader

Note that dowloading an app (.exe for windows, or an apple app) and running it, is just as bad. Downloading an installer, and running it, is also insecure. So the question is... What is the proper alternative?

I think the best alternatives are appstores such as found on ios and android, right?

However that doesnt really fit for open source. Is the way to install open source software, to download the source and compile it from scratch? Well, thats also risky, you should check the source code first...? But thats undoable.

All in all, it comes down to "trust". Do you trust the website? You dont care about security too much, well, imho then it makes sense to curl | sh.

View on HN

I've written down some general principles we should follow, but any reasonable implementation of them seems pretty far off:

https://defuse.ca/triangle-of-secure-code-delivery.htm

tl;dr: (1) Reproducible builds, (2) Make sure everyone is getting the same thing (to detect targeted attacks) and (3) Cryptographic signing.

Package managers and appstores are the best we have right now, but they're missing (1) and (2). In the meantime, offering a pgp-signed installer file is a lot better than curl | sh.

I'm really surprised this hasn't gotten more attention from the commenters here. :(

^~ To the people scrolling by at 70 mph: READ THIS

`curl | sh` is protected by a ssl certificate issued by CA that my system trusts.

"pgp-signed installer file" is protected by a key from a stranger.

Both are insecure.

Could you enumerate several points that show "pgp-signed installer file is __a lot better__ than curl | sh" ?

Say we're trying to download the PHP source code from php.net. If all that's protecting us is SSL, then if an adversary compromises the php.net servers (happens all the time, and actually has happened to PHP), they can immediately replace all the downloads with backdoored copies. Whereas with a PGP signature, the key can be stored off-line (even on an air-gapped system), so that even if the web server is compromised, the adversary can't make me believe the file is legit.

PGP can also be used in a trust-on-first-use manner. Get the public key once over an insecure channel, and if the attacker missed that single opportunity, you're safe until the key changes. With SSL, on the other hand, you're at risk every single time you make a connection, because any of hundreds of CAs has the power to sign that certificate, and as above, you have to assume the web server isn't compromised.

Another reason PGP is important is mirroring. Big F/OSS projects let others' volunteer mirrors. Even if those mirrors support SSL and the transport from the author to the mirror is encrypted, there's absolutely no guarantee that the mirrors themselves are not malicious. The mirrors could be backdooring their own files. The fact that you have an SSL connection to the mirror doesn't do anything to prevent this. But with PGP signatures, you're assured the files come from the software's developer, and haven't been tampered with by the mirror.

So the difference is: SSL secures the connection between your browser and the web server. PGP ensures you're getting the file the software developer intended you to get. It's a semantic difference.

I'd also argue hard against `curl | sh` for (assuming it exists) the psychological effect of teaching users that it's OK to pipe random things from the web into sh.

If an adversary compromises php.net server then naturally the files will be signed by adversary's keys. In both cases, all you need is the ability to replace files (the package files, signature files, html instructions), nothing else. I don't see how gpg is __a lot more__ secure here.

You need some other channel to communicate what keys should sign what files. You need some other channel to import the keys. Catch 22.

"trust-on-first-use" do you mean something like the certificate pinning?

Let's consider https://www.torproject.org It uses GPG signatures for its packages.

If SSL is compromised as you say then all I need to fool you is to give you files that are signed using my keys (unless you know that you should use 0x416F061063FEE659 key (magic secure channel) and you've already imported it (again the magic channel) and tor never changes the key.

Where should I go to check that 0x416F061063FEE659 and pool.sks-keyservers.net are correct values (google?) if we assume that the key and the key server (and the connection to it) are not compromised?

Ask yourself: when was the last time that you've tried to check that the instructions that show the key fingerprint, the key server to use are genuine?

Also, If you have paper walls I wouldn't try too hard to make the door impenetrable. It is a trade-off: if you are downloading code from stranger's repository then you won't get much from replacing `curl https://github.com/... | sh` with a gpg-signed (by the same stranger) download.

Security is like onion rings: there are layers but it is only as strong as its weakest link. We know that a real adversary will just hack your machine if necessary.

No it's not because MSI packages and EXEs are signed so there is at least a chain of trust. SmartScreen kicks you in the face if it's not "known" by various metrics and it's hard to get around that these days (as someone who just had to get EV certs for their MSI). If you click through that, and you don't know what the source was then you're a muppet.

curl + bash is suck and blam the machine. Hope you didn't do a sudo in the last few seconds...

http://www.nastybastard.com/funky-super-installer.sh:

   #!/bin/bash
   echo "mwuhahahaha"
   sudo rm -rf /boot

> No it's not because MSI packages and EXEs are signed

Sure, some are. But most software packages that a user is going to download aren't signed.

Right. And really, who checks signatures?

I'm a developer and I don't think I've every checked an md5 signature of a jarfile/gem/package I've downloaded. Nor have I ever been in an environment where that was ever mentioned. (Have mostly worked in small to medium businesses--I imagine that bigger orgs or the defense department might do this.)

You might be surprised. I just pulled up Process Explorer, which makes it easy to see which running apps are signed, and the only unsigned things I'm currently running are VLC, Evernote, and some Brother Printer utilities.

Things from Google, Intel, MS, VMWare, Spotify, Github, Dropbox, and even f.lux are all signed. Of course YMMV but the trend has been positive.

(A little worrisome is that there are two running Broadcom bluetooth apps that have explicitly revoked signatures...I wonder what that's about.)

> No it's not because MSI packages and EXEs are signed

Is it more difficult to provide your own fake exe installer than to middle-man https that curl examples use?

The question is, is it easier to place your code on that curl site. If there is some web-layer vulnerability, you can put your own code there.

With signed MSIs and EXEs, you'd need to get your code signed, which is probably more difficult than the web layer.

SSL certificate signing provides basically the same level of trust as a signed installer. As long as you curl over https from the domain you trust, you're good.

Actually no as that signs the communications and not the software. If the target server is compromised then you are screwed. Also for example there is no guarantee that github.com isn't serving malicious traffic from one user under a legitimate request I.e. the poisoned sharecrop problem.

EV signed software is usually done off the internet. In our case we use a physical key to sign it offline and then upload.

I get a little bit of heartburn over running arbitrary scripts downloaded from internet websites.

The easiest alternative is to run "curl <blah>", download the script, take a peek at the source, and only THEN follow up with the sh.

I mean, you're probably going to see a bunch of "download and install" commands, so you don't get much reassurance if you're worried that one of THOSE might be compromised too. But that's a problem inherent to unsigned internet installer scripts. At least you've somewhat mitigated the possibility that it's dumping your private SSH key to a server somewhere or some other blatantly malicious command.

On the other hand, the script might also include appropriate SHA1 verification commands, or be setting up a repository in a package manager, in which case it's pretty reasonable to trust the script.

My preferred hypothetical installer Trojan method is to make it append a value to ~/.ssh/authorized_keys and then ping a C&C server with the current username and IP ;)

You would still have to hope that sshd is enabled and accepting connections (which by default it shouldn't, at least on end-user distros).

True. I've mostly seen PHP devs using this to deploy composer onto staging and production servers, so it would work on them. ;)

The proper alternative is some sort of lightweight virtualization like Docker, with a proper security policy in place. You can mostly do this today

    curl <blah> | docker run --rm -t -i ubuntu sh

but the containerizing naturally locks down/virtualizes some stuff you might want the script to be doing, like installing stuff locally.

I think cleaning this up and making it friendly enough is a solvable UX problem, but I haven't quite solved it yet.

I can't tell if you're trolling, but "put some Windex on it" (or in this case, containerization) is hardly a solution, only a bad workaround. The root cause of curl | sh being a broken paradigm to begin with is left unaddressed.

I'm not sure that curl | sh is a "broken paradigm" -- no more broken than than running JavaScript in a browser. The only difference between the two is in the sandboxing.

The solution I'm proposing isn't to add Windex, but to cleanly wrap whatever curlpipesh is doing. Make it equivalent to opening a browser tab: interact with it safely, keep what you like, and throw everything away if you don't like the way things look.

Curlpipesh can be just like loading a site. We have the technology to do it; we just need to figure out the UX.

I think one of the larger issues with curl | sh is what could happen in the event of a network outage or early termination on the connection.

For example, if you're downloading a script that has a line like this

  rm -rf ~/.tmp/foo/bar

But the HTTP connection was lost before the entire file was downloaded and `rm -rf ~` was the end of one packet and `/.tmp/foo/bar` was the contents of the other (lost) packet, you're screwed. The incomplete script will still be piped to sh and it's game over.

In at least some of those cases, I would expect curl to exit with an error and the pipeline to abort.

Some scripts also detect this and are written so that there is no code executed before the file is not complete. Of course, that's a minority.

Definitely including a checksum and validating that before executing would be ideal.

To elaborate, this is quite easy: you wrap the entire contents of the script into a function definition, then call the function as the last line.

All bash sees are bytes coming in on stdin, and eventually an EOF. It neither knows nor cares what caused the EOF.

Sure, but if the server response is not pipelined (as is probably often the case), then bash should never see anything.

HTTP pipelining is about reusing a TCP connection for multiple requests. It doesn't influence when curl outputs data and wouldn't apply here anyway. I don't think there's any mode which would cause curl to buffer the entire response before writing any of it.

Yes, I'm aware of how HTTP pipelining works. It was a poor choice of terminology. My point is that by default curl does buffer some of the response. And if the connection was terminated before the first buffer was output, then I would expect this to result in an error which would abort the shell pipeline.

Yes, in some cases curl won't produce any output, like if the web server is down, or the connection fails before anything is returned. And yes, it would also happen if curl buffers some of the response and then dies. I don't really see why that's interesting.

The default buffer size is typically the page size, which is typically 4096 bytes. I would expect a large number of these scripts to be less than 4096 bytes meaning curl would output nothing before producing an error and the partial script would never be evaluated.

That's the default buffer size for pipes, which won't matter here. When curl terminates, whatever's buffered in the pipe will be flushed. The only thing that could prevent downloaded data from being received by the shell would be internal buffering in curl, if it does any.

Good point. curl doesn't do any internal buffering. I was thinking that the pipeline should be aborted if the curl exits with a non-zero status, but of course this is not the case.

Yeah, it would be nice if there were a way for a part of the pipeline to signal that something bad happened and everything should stop. Ideally, some sort of transaction system so the script is guaranteed to run fully or not at all. But instead we have this crazy thing.

Why don't shells NOT execute a command if there's no trailing EOL character, to mitigate this very problem?

Why should shells encourage this behaviour (piping unknown stuff to them?)

You should at least download, verify the size and checksum (if available), take a peek at it, and only then run it.

Because you might be piping known stuff to them. Receiving a partial input stream is not necessarily reliant on networking.

OK, but the detection of partial vs complete script is not as simple as 'does last line have EOL'? There are various builtins that require an end token, like case/esac, if/fi, etc. Do these work properly when truncated at an arbitrary line?

Those control structures do work properly - the shell reads ahead until it finds the end token, and fails if it's absent.

It is true that there remains the problem of potential truncation exactly at the end of a top-level line, but I contend that "it stops running here" is a much easier thing to reason about (and, strictly, could always happen if hit with a SIGKILL anyway) than "does the meaning of this line change if we cut it off in a weird place".

This can be mitigated by wrapping your whole script in parens, as in:

  #!/usr/bin/env bash
  (
    # real work
  )

I had something like that happen once, but it worked out in my favor. I had misconfigured BIND, which I had intended to run just as a caching name server for my network, and it was listening for outside connections too. Some variant of the Lion worm found it and used a BIND bug to get onto my system.

It sent my password file to someone in China, started a scanner to look for other systems to infect, downloaded a .tar.gz file that contained a root kit to hide itself, unpacked the .tar.gz, and ran the install script contained therein to install the root kit.

Or rather, it tried to. I had ISDN at the time, and had noticed the modem lights heavily blinking even though I was not doing any internet activity. This confused me, and I pulled the plug on the modem. Turns out I pulled the plug while it was downloading the .tar.gz. It got most of the file, but not quite all of it. It lost the last file inside the archive--which happened to be the install script!

Without the install script, it could not install the root kit, and that made getting rid of the worm a heck of a lot easier.

If you're really paranoid, download the MD5 and PGP keys of the application release's source code tarball from the author's website using SSL. Then unpack the source (being careful to use a sandbox, because an exploit in gzip/tar/etc might be bad) and examine the source code for any unexplained or dangerous behavior. Then compile and install it using a non-root user.

Personally, i'm totally comfortable with the idea of someone owning my box. The files I care about are backed up offline, I don't have many secrets, and the only accounts that would affect my life in general have passphrases, pins and multiple-factor auth tied to them. The worst thing someone could maybe do is impersonate me and cause havoc using my accounts, but I really don't see anyone having cause to do that. Anyway, I don't sweat it.

If you trust your distro, just verify the PGP keys of packages you download before installing, which is mostly automatic for most modern OSes.

""" Personally, i'm totally comfortable with the idea of someone owning my box. The files I care about are backed up offline, I don't have many secrets, and the only accounts that would affect my life in general have passphrases, pins and multiple-factor auth tied to them. The worst thing someone could maybe do is impersonate me and cause havoc using my accounts, but I really don't see anyone having cause to do that. Anyway, I don't sweat it. """

Are you trolling? This sounds like the same sort of argument from the "I don't have anything to hide. NSA/GCHQ can spy on me all they want" camp.

Uh, it's not an argument, it's my personal feelings and my personal opinions. It's OK if you don't feel the same way.

I look forward to you getting 0wned then. Just my personal feelings and personal opinions. :)

Who do you enter your passphrases, pins, etc if you box is controlled by someone else? How do you know there's no keylogger running? How do you use PGP if you don't care about the security of your private key?

I do assume there could be a keylogger. I'd enter a password into the computer, but passphrases would be spoken over the phone to a representative that I call, and pins are entered either over the phone or once I receive them from a 2FA source. I avoid using sensitive accounts online if I can do it over the phone or in person.

(part of why I want 2FA providers to verify an identity using two or more devices is so that if your workstation is compromised, it still couldn't complete the authentication without a secondary device, making mitm much more difficult; though obviously they could still hijack an existing connection on either device... a benefit would be that for example, an attacker couldn't use a hijacked connection to initiate a money transfer behind your back without having you confirm it on your mobile device as well)

If you are depending on PGP, and only install software from verified sources using secure connections, your private key is still secure. If you don't depend on PGP, you wouldn't care about the security of your private key. You could also do private key operations on an airgapped machine, which wouldn't depend on the security of your main workstation.

It's interesting to note that we essentially do the same with javascript in the browser, except the user doesn't even know there's some code that is automatically run.

The big difference, of course, being that javascript code runs in a sandboxed environment. Maybe we should create some kind of sandboxed sh ? (I actually hardly believe it will ever take on, seeing that so many sites tell you to run not `sh` but `sudo sh`... yeah, sure, why have unprivileged users, we should do everything as root anyway)

Well, the Unix approach to this is a sandbox application and a shell application, not a sandboxed shell application. Docker attempts many of those tasks.

On windows there's Sandboxie, which does more or less what you want as well. It creates a sandbox whose filesystem and internals can be modified by sandboxed applications without affecting the state of the REAL filesystem. Very nice utility for mitigating bad programming - I often use it to run multiple copies of applications that crash or refuse to launch multiple instances.

Trust is really the only solution, we as developers might be able to inspect some code before we run it but if it is sufficiently complex we could spend months auditing it. Even worse for the non-technical users that can't tell a hello world from a keylogger, there really is no solution other than some trust based system.

Trust is a partial solution. You need to be able to make sure that you're only trusting those who are trustworthy. A part of that is assessment of trustworthiness, and a part is making sure you're aware who you're trusting (probably allowing delegation).

I agree, it is difficult to find a proper alternative.

App stores do have some advantages here, today. It's also true that they don't well fit the model for open source. App stores define some root of trust for signing, but just like the CA model for HTTPS certificates, there are issues with that. App stores also bring a host of issues with user freedom. (Even the free ones can have this effect unintentionally: what percent of your coworkers know how to manage apt pinning? If one person on ubuntu 10.04 has an working, installed package, and wants to hand that exact (known working) version of it to a friend on ubuntu 12.04, can they do it?)

PGP signed binaries are one major step in the right direction. More friendly tools and popularized workflows for this would help a lot of people avoid major moments of vulnerability.

PGP (or any other signing system) also leave an interesting challenge, which is what to do in the case of key compromise. Revocation is hard. And say you want an audit log of all prior releases: what can you do?

We're all used to git having an immutable log of history by hash chaining, at this point. Is it time to start using systems like that to keep an auditable log of software releases, as well?

I've been working on something called "mdm" [1] that attempts to do an auditable, immutable chain of releases like this. It was originally designed to be a dependency manager for (arbitrary, language-agnostic, any binaries) software development needs, but I wonder if something like this is what the software distribution world needs to start evolving towards as well. Imagine if your end users could all verify that they have the same hash... and the same picture of historic releases from the same author, and the same picture of if the releaser's signing key changed, and so on.

I'd love to hear other thoughts on how hash chains and signatures can be made to intersect to tell the broadest story. The challenges are very real here.

---

[1] https://github.com/polydawn/mdm/

"appstores such as found on ios and android" are actually simpler versions of Linux package management, which fits very well with open-source. Packages are retrieved over HTTPS and signed by maintainers/authors. For distros like Lunar or Gentoo, the source is retrieved in a similar fashion and compiled locally.

Reply on news.ycombinator.com