On piping Curl to apt-key | Hacker News Reader

On piping Curl to apt-key

tablix.org

51 pointsgandalfar46 comments

I hate this practice, no idea how it became commonplace. Of course lots of times, installation procedures can be long and tedious, but it takes one popular project's script server to be compromised, and tons of people are suddenly running malicious commands.

I would go through manually installing dependencies and setting up my system, adding repos, etc. over running some script any day. But then again some projects wouldn't be that popular if they were hard to install.

Some of npm's installation instructions ask you to pipe curl into bash, to run a lovely script [0] which makes things easier for you, but not by much. Is it really necessary? Would developers give up trying to get npm and node just because installing not as easy as "curl https://some.script.com/that-script.sh | sudo -E bash -; sudo apt-get install npm"?

Other than building/installing programs, adding GPG/SSH keys like in the blog post can be as dangerous, and while not simple, there could be some method built to make things easier without having to run commands you don't even check.

Anyways, hope projects grow out of this habit.

[0] https://deb.nodesource.com/setup_6.x

People are lazy, developers are people, most developers are not interested in process of system maintenance, so if something could be done fast and easy, they could care less about it. My colleague uses Windows 8 preinstalled on his laptop 5 years ago, it's slow as hell and ridden with all kinds of toolbars, but he doesn't want to make an effort of full reinstall. He would happily pipe curl to bash, actually I'm not even sure if he understands what curl or bash is, he knows some Java and JavaScript and that's enough to be paid.

I like tinkering with system, I reinstalled my home server may be 20 times, I'm always reinstalling macos from scratch when new version released, but honestly it's not very productive time spent, so I understand people who just want to get things done.

I think it's failure of current operating systems: installing software is still too hard and tedious. Opening terminal and copy-pasting strings is not a trivial activity. If one could install npm by pushing button in their website, they would do that instead.

This oh so much include the upsurge in containerization and the quagmire that is Linux distro package dependencies.

First of all, containerization allows for static linking via other means. Meaning that as the container holds everything a program needs to run in exactly the right versions, it produce what is practically the same as a static binary with the linked libs compiled in.

Similarly, package dependencies are a quagmire because package A may want lib B, while C wants B+1. And B+1 introduce some subtle (or overt) change that makes it unsuitable for A. This then result in a game of telephone with package and file names to try to get both B and B+1 to exist on a system at the same time. But there are no real conversion for how to settle these conflicts so each distro have their own rules and procedures.

But really the distros are trying to make the best of a bad situation brought on by lib and program developer laziness. Laziness whereby they do not properly check breakages before shipping, or simply grab the latest and shiniest that gets their job done rather than track down the lowest common denominator that is suitable.

Containers are more than static linking. You can have multiple binaries, which is not really possible with static linking.

Your A and C package example are actually an argument for containers.

It may be used, by developers, as an argument for containers. But that is just the same old lazy devs being their same old lazy self. Containers are not fixing the underlying problem.

Look at it this way: whenever you are running a program you didn't write yourself, you're running a bunch of commands you never checked. This is no different to, say, downloading a precompiled executable and running it, with all the same problems and tradeoffs.

It is different. While it is obviously true that I haven't checked all of the binaries I'm running, I at least can, through the various signatures involved, rely on the fact that it was created by a particular individual or group, whom I may trust.

Would you really assign the same level of trust to, eg, a sudo(8) binary downloaded somewhere of the internet as you would to the one provided by your distribution?

There are various levels of trust, of course. The packages in Debian or RedHat are more trustworthy (there is a process) than those in NPM or Maven (free-for-all, even if you have some assurance that the package you're downloading is the very same the developer uploaded).

But installing a random NPM package is no more dangerous than curl-piping a script from Github to bash over HTTPS (without -k). You're still sure that what you're downloading and running is what whoever is in control of that repo intended.

What IS more dangerous is training a generation of developers to solve problems by quickly copy-pasting random strangers' magic incantations from random blogs or Stackoverflow into their terminals. You could probably infect a large number of machines very quickly by stalking certain categories on Stackoverflow for "noob" questions and giving a good answer in the form of a GitHub gist curl-pipe to sudo that fixes the problem, but that also discreetly backdoors the target.

That's not the comparison being made. It's between piping curl to bash, or just downloading a script and running it with sudo, without inspecting.

Yes, you "could inspect". But this is about the instructions. And instructions to pipe curl to bash are no more or less harmful than instructions to download a binary from a "random" server and run it verbatim.

"Piping curl to bash" is a red herring. It's "running unverified code" that's the problem. Piping curl to bash just makes it viscerally obvious how dangerous that is.

I've wasted several days trying to get programs I write turned into debs and rpms, I gave up. It's a single executable you can download and put wherever you like, or download the source and './configure.py; make'.

Also, I release new versions regularly, so now being in the official repositories is no good as they will get out of date, I have to run my own repositories, for several versions of ubuntu and redhat. No chance.

I have succeeded in turning my programs into DEBs and RPMs that also properly comply with all the distribution packaging guidelines, but it took me two months. I am now onboarding two other developers but they are struggling with making changes to DEBs and RPMs. All because of the complexity, archaic toolchain and poor documentation.

The only reason why we made DEBs and RPMs was to please users, but it's not an experience I want to recommend anyone.

did fpm not work for you?

> I've wasted several days trying to get programs I write turned into debs and rpms, I gave up.

The formats themselves are pretty easy to create; for a .deb you just:

- Make the folder hierarchy you want to include, e.g. things like `myBuildFolder/usr/bin/myProgram`

- Make a `myBuildFolder/DEBIAN/debian-binary` file containing `2.0`

- Make a `myBuildFolder/DEBIAN/control` file which contains e.g. the name of the package and its dependencies

- Run `dpkg-deb --build myBuildFolder`

This gives a .deb package which will be tracked by dpkg/apt, allowing clean removals/upgrades/etc. unlike `sudo make install` which will spray cruft all over the system. When I used Debian, I would install all software like this.

I've never made a .deb that's compliant with Debian's packaging policies though, since that does take a lot of effort.

Is checkinstall still available? I had great success with it.

> But then again some projects wouldn't be that popular if they were hard to install.

Slackware was my first distro back in 2007 and I still wish it had at the very least a decent installer and package manager, it is so well built but upgrading, and installing is just not as trivial when compared to Debian. I guess openSUSE (afaik) is the only remnant of Slackware that's usable - really loved openSUSE but just like every other distro in the world I have to dance to get my Wi-Fi working properly.

I like very much this formalism to installation procedure description. Being curious and informed about the risks, I will not do a copy paste of curl ... | sudo .... I will do a curl to a temporary file, edit the file and if there is nothing fishy, I will copy the commands (from the file, not a browser) to a terminal.

If the content of the file is a very messy (or too long) script, I consider that the software will also be messy and does not deserve my time.

This formalism flatter my ego, proves transparency and makes me spare time. Full benefit. It is dangerous for noob, but is also a good opportunity to educate them.

Isn't the solution to publish the key on a keyserver and fetch it with apt-key? Something like

    apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys <fingerprint>

Try doing that behind a corporate proxy. If `apt-key` just fails there is no way to enable logging.

It also apparently works by generating a bash script in /tmp/ and executing it...

hkp literally stands for HTTP keyserver protocol. Does your corporate proxy really mess up HTTP connections?

Why does it matter how apt-key is implemented? Its purpose is key management, and whether it uses bash for the job or perl or C is completely irrelevant. It's been in use for over a decade. Do you have any reason to suspect deficiencies in it just because it uses shell scripts?

Your example fetches the key from the keyserver without https. Fetching the key from the project's own site over https using curl is better.

Edited to add: Fetching from a keyserver is OKish if a) you use the long form of key id and b) your gpg is new enough that it checks that it got the key for the id it requested. Still, the Web page you copy the key id from is as vulnerable to an attack on the server as the server serving the key directly.

Right, sorry, it should be using hkps as protocol and leave out the port.

Especially when copying and pasting things anyway, the long form should always be preferred. I think there was an article on here several months ago on the dangers of using abbreviated fingerprints.

Manipulation of the fingerprint on the web page could be easier to detect using the archive.org wayback machine, which might not index the keyfile. Doesn't prevent manipulation but might make it easier to detect if you're suspicious.

In the past GnuPG didn't verify that the keys it receieved were the keys it asked for.

Is this bug fixed now?

> Consider the case where download.docker.com starts serving an evil key file

At that point I can't trust the key ID in the docker documentation either. Since Docker doesn't use web of trust (who does honestly?) there is no way that I can verify the key ID in any way in the provided key file. So I don't know how it does any good inspecting the key file before adding it to the apt keyring.

On a related note, don't copy commands from web and paste it to your shell (or any terminal program):

http://thejh.net/misc/website-terminal-copy-paste

On piping anything from the Internet directly to your system for execution: Don't be lazy. Don't be an ass.

When I am working in a persona that responsible for managing a server or a service, I insist on knowing everything I need to know about how to keep that service and the environment in which it operates safe, alive, and providing usable performance.

I require good, clean and coherent instructions for deploying something at production level, where all required components and their preferred method of interaction are clearly explained and documented by the developer, and can be repeated in a predictable manner by me.

If all I have to work with is "pipe this to the shell, alternatively read the code" I'm going to go with "nah, I'll find something professional".

Time spent installing a system should be only a minuscule fraction of time spent actually operating the system. Spending a few extra hours doing it right shouldn't make a difference.

[edit: added "...and another thing" argument]

Wouldn't it already help if apt-key printed the list of keys it imported instead of the absolutely superfluous "OK"?

Yes. But that would break all existing tools wrapping around apt-key.

That's the downside of everything-is-just-strings in Unix

apt-key could detect terminal output and modify its output.

Or just an extra flag like --list-after-import.

Writing any script would be hell if the commands you call suddenly start modifying their behaviour. This is bad practice, and the GNU coding standards (for example) agree: https://www.gnu.org/prep/standards/standards.html#User-Inter...

"Likewise, please don’t make the behavior of a command-line program depend on the type of output device it gets as standard output or standard input."

Wouldn't it already help if apt-key printed the list of keys it imported instead of the absolutely superfluous "OK"?

What would really help: Publishers providing the key in a clear text copy-paste format, and providing instructions on adding the key to apt-key.

> providing instructions on adding the key to apt-key

    curl | apt-key - # that works

No really, they shouldn't tell you how to add the key to your store. If you don't know how to do that yourself, you shouldn't be admin/superuser. (Also, `man`)

How would this help? They're still pretty unreadable and as the post describes:

> However, two public key blocks could easily have also been exported into a single ASCII-armor block.

Or even better yet: `--match-fingerprint "9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88"`

Actually, ideally GPG public keys would be small enough that they could be inlined with the script. Why is it that a SSH public key are on a single line but GPG have to be a page-full?

A GPG public key does not only consist of the integer number that forms the cryptographic public key, but usually you have multiple user IDs so other can recognize to whom this key belongs. In order to prove that these user ID belongs to this public key, each of them is signed. Obviously, that will take a few more bytes for the additional data and signature.

The exported file size also depends on the way you export the GPG public key. By default, you will also export all signatures made by others, but you can use `gpg --export --export-options=export-minimal` to strip everything except the last self-signature on each user ID.

Thinking about this I came up with the following. I tried getting the fingerprint in full but only got the short version. Not being a gpg-ninja it would suffice to make an offline version of the add command.

  key="$(echo "-----BEGIN PGP PUBLIC KEY BLOCK-----||mQINBFit2ioBEADhWpZ8/wvZ6hUTiXOwQHXMAlaFHcPH9hAtr4F1y2+OYdbtMuth|lqqwp028AqyY+PRfVMtSYMbjuQuu5byyKR01BbqYhuS3jtqQmljZ/bJvXqnmiVXh|38UuLa+z077PxyxQhu5BbqntTPQMfiyqEiU+BKbq2WmANUKQf+1AmZY/IruOXbnq|L4C1+gJ8vfmXQt99npCaxEjaNRVYfOS8QcixNzHUYnb6emjlANyEVlZzeqo7XKl7|UrwV5inawTSzWNvtjEjj4nJL8NsLwscpLPQUhTQ+7BbQXAwAmeHCUTQIvvWXqw0N|cmhh4HgeQscQHYgOJjjDVfoY5MucvglbIgCqfzAHW9jxmRL4qbMZj+b1XoePEtht|ku4bIQN1X5P07fNWzlgaRL5Z4POXDDZTlIQ/El58j9kp4bnWRCJW0lya+f8ocodo|vZZ+Doi+fy4D5ZGrL4XEcIQP/Lv5uFyf+kQtl/94VFYVJOleAv8W92KdgDkhTcTD|G7c0tIkVEKNUq48b3aQ64NOZQW7fVjfoKwEZdOqPE72Pa45jrZzvUFxSpdiNk2tZ|XYukHjlxxEgBdC/J3cMMNRE1F4NCA3ApfV1Y7/hTeOnmDuDYwr9/obA8t016Yljj|q5rdkywPf4JF8mXUW5eCN1vAFHxeg9ZWemhBtQmGxXnw9M+z6hWwc6ahmwARAQAB|tCtEb2NrZXIgUmVsZWFzZSAoQ0UgZGViKSA8ZG9ja2VyQGRvY2tlci5jb20+iQI3|BBMBCgAhBQJYrefAAhsvBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJEI2BgDwO|v82IsskP/iQZo68flDQmNvn8X5XTd6RRaUH33kXYXquT6NkHJciS7E2gTJmqvMqd|tI4mNYHCSEYxI5qrcYV5YqX9P6+Ko+vozo4nseUQLPH/ATQ4qL0Zok+1jkag3Lgk|jonyUf9bwtWxFp05HC3GMHPhhcUSexCxQLQvnFWXD2sWLKivHp2fT8QbRGeZ+d3m|6fqcd5Fu7pxsqm0EUDK5NL+nPIgYhN+auTrhgzhK1CShfGccM/wfRlei9Utz6p9P|XRKIlWnXtT4qNGZNTN0tR+NLG/6Bqd8OYBaFAUcue/w1VW6JQ2VGYZHnZu9S8LMc|FYBa5Ig9PxwGQOgq6RDKDbV+PqTQT5EFMeR1mrjckk4DQJjbxeMZbiNMG5kGECA8|g383P3elhn03WGbEEa4MNc3Z4+7c236QI3xWJfNPdUbXRaAwhy/6rTSFbzwKB0Jm|ebwzQfwjQY6f55MiI/RqDCyuPj3r3jyVRkK86pQKBAJwFHyqj9KaKXMZjfVnowLh|9svIGfNbGHpucATqREvUHuQbNnqkCx8VVhtYkhDb9fEP2xBu5VvHbR+3nfVhMut5|G34Ct5RS7Jt6LIfFdtcn8CaSas/l1HbiGeRgc70X/9aYx/V/CEJv0lIe8gP6uDoW|FPIZ7d6vH+Vro6xuWEGiuMaiznap2KhZmpkgfupyFmplh0s6knymuQINBFit2ioB|EADneL9S9m4vhU3blaRjVUUyJ7b/qTjcSylvCH5XUE6R2k+ckEZjfAMZPLpO+/tF|M2JIJMD4SifKuS3xck9KtZGCufGmcwiLQRzeHF7vJUKrLD5RTkNi23ydvWZgPjtx|Q+DTT1Zcn7BrQFY6FgnRoUVIxwtdw1bMY/89rsFgS5wwuMESd3Q2RYgb7EOFOpnu|w6da7WakWf4IhnF5nsNYGDVaIHzpiqCl+uTbf1epCjrOlIzkZ3Z3Yk5CM/TiFzPk|z2lLz89cpD8U+NtCsfagWWfjd2U3jDapgH+7nQnCEWpROtzaKHG6lA3pXdix5zG8|eRc6/0IbUSWvfjKxLLPfNeCS2pCL3IeEI5nothEEYdQH6szpLog79xB9dVnJyKJb|VfxXnseoYqVrRz2VVbUI5Blwm6B40E3eGVfUQWiux54DspyVMMk41Mx7QJ3iynIa|1N4ZAqVMAEruyXTRTxc9XW0tYhDMA/1GYvz0EmFpm8LzTHA6sFVtPm/ZlNCX6P1X|zJwrv7DSQKD6GGlBQUX+OeEJ8tTkkf8QTJSPUdh8P8YxDFS5EOGAvhhpMBYD42kQ|pqXjEC+XcycTvGI7impgv9PDY1RCC1zkBjKPa120rNhv/hkVk/YhuGoajoHyy4h7|ZQopdcMtpN2dgmhEegny9JCSwxfQmQ0zK0g7m6SHiKMwjwARAQABiQQ+BBgBCAAJ|BQJYrdoqAhsCAikJEI2BgDwOv82IwV0gBBkBCAAGBQJYrdoqAAoJEH6gqcPyc/zY|1WAP/2wJ+R0gE6qsce3rjaIz58PJmc8goKrir5hnElWhPgbq7cYIsW5qiFyLhkdp|YcMmhD9mRiPpQn6Ya2w3e3B8zfIVKipbMBnke/ytZ9M7qHmDCcjoiSmwEXN3wKYI|mD9VHONsl/CG1rU9Isw1jtB5g1YxuBA7M/m36XN6x2u+NtNMDB9P56yc4gfsZVES|KA9v+yY2/l45L8d/WUkUi0YXomn6hyBGI7JrBLq0CX37GEYP6O9rrKipfz73XfO7|JIGzOKZlljb/D9RX/g7nRbCn+3EtH7xnk+TK/50euEKw8SMUg147sJTcpQmv6UzZ|cM4JgL0HbHVCojV4C/plELwMddALOFeYQzTif6sMRPf+3DSj8frbInjChC3yOLy0|6br92KFom17EIj2CAcoeq7UPhi2oouYBwPxh5ytdehJkoo+sN7RIWua6P2WSmon5|U888cSylXC0+ADFdgLX9K2zrDVYUG1vo8CX0vzxFBaHwN6Px26fhIT1/hYUHQR1z|VfNDcyQmXqkOnZvvoMfz/Q0s9BhFJ/zU6AgQbIZE/hm1spsfgvtsD1frZfygXJ9f|irP+MSAI80xHSf91qSRZOj4Pl3ZJNbq4yYxv0b1pkMqeGdjdCYhLU+LZ4wbQmpCk|SVe2prlLureigXtmZfkqevRz7FrIZiu9ky8wnCAPwC7/zmS18rgP/17bOtL4/iIz|QhxAAoAMWVrGyJivSkjhSGx1uCojsWfsTAm11P7jsruIL61ZzMUVE2aM3Pmj5G+W|9AcZ58Em+1WsVnAXdUR//bMmhyr8wL/G1YO1V3JEJTRdxsSxdYa4deGBBY/Adpsw|24jxhOJR+lsJpqIUeb999+R8euDhRHG9eFO7DRu6weatUJ6suupoDTRWtr/4yGqe|dKxV3qQhNLSnaAzqW/1nA3iUB4k7kCaKZxhdhDbClf9P37qaRW467BLCVO/coL3y|Vm50dwdrNtKpMBh3ZpbB1uJvgi9mXtyBOMJ3v8RZeDzFiG8HdCtg9RvIt/AIFoHR|H3S+U79NT6i0KPzLImDfs8T7RlpyuMc4Ufs8ggyg9v3Ae6cN3eQyxcK3w0cbBwsh|/nQNfsA6uu+9H7NhbehBMhYnpNZyrHzCmzyXkauwRAqoCbGCNykTRwsur9gS41TQ|M8ssD1jFheOJf3hODnkKU+HKjvMROl1DK7zdmLdNzA1cvtZH/nCC9KPj1z8QC47S|xx+dTZSx4ONAhwbS/LN3PoKtn8LPjY9NP9uDWI+TWYquS2U+KHDrBDlsgozDbs/O|jCxcpDzNmXpWQHEtHU7649OXHP7UeNST1mCUCH5qdank0V1iejF6/CfTFU4MfcrG|YT90qFF93M3v01BbxP+EIY2/9tiIPbrd|=0YYh|-----END PGP PUBLIC KEY BLOCK-----|" | tr '|' '\n')";shasum="c0c3cb1cf3161419daabebaa8632dd714b9a8dda"; [ "$shasum" == "$(echo "$key" | gpg --with-colons | shasum | cut -d ' ' -f 1)" ] && echo "$key" | apt-key add -

Turns out that I could not compress the public key to make it shorter either.

Please don't dump that much text in long lines to HN, you are breaking the layout of the page and make reading extremely annoying.

Please remove or edit your post.

Seems that I can't do much about it now. Sorry!

... and then you have to check the fingerprints manually, and delete the ones you don't want manually.

No, what would really solve this specific issue is allowing apt-key to add only a single key, and give it the expected fingerprint (as zimbatm explains)

Apt-key should just have a built-in way of importing keys from HTTP(s) URLs, preferably in interactive mode so you can confirm the keys are legitimate before adding them.

Would it be possible to write a "confirm" tool that writes stdin to stderr then waits for the user to press enter before writing to stdout?

    curl ... | confirm | apt-key ...

Of course there's no such thing as a stderr input stream, so I'm not sure this would even be possible.

That would only help if it were possible for a user to visually identify if a stream is not malicious.

Yes, but the proper thing to do is never curl to your terminal, especially a privileged terminal. And really, you shouldn't paste either.

Of my first steps into the world of Linux this year, this sort of procedure has been one of the most glaringly disturbing. Another similar was packages being downloaded over HTTP.

Debian packages are signed, they are safe to transmit over http. See https://wiki.debian.org/SecureApt (which appears to have been written around the time of the transition, so it's out of date, e.g. SHA1 signatures are no longer trusted etc)

Reply on news.ycombinator.com