Show HN: Crypt – Secure Configuration Storage in Etcd or Consul | Hacker News Reader

Show HN: Crypt – Secure Configuration Storage in Etcd or Consul

xordataexchange.github.io

69 pointsbketelsen24 comments

Open articleView on HN

"After encryption it is gzipped" is a red flag. After encryption it should be noise, why try to compress it?

Sorry, if I said that in the video. What we do is base64(gpg(gzip(data)))

We gzip the data first, then encrypt it, finally base64 encode it.

The article says you encrypt, then compress. You might want to clarify that.

My bad - fixed.

In the next paragraph after the one you fixed

> crypt encrypts, compresses, then encodes your value for storage

dammit.

In this thread: just talking or writing _about_ encryption can be difficult. To say nothing of correctly implementing it in a project.

How can you possibly compress encrypted data? An ideal cipher should produce output indistinguishable from randomness, which is inherently difficult (read: near impossible) to compress.

Hopefully we corrected the docs and the comments here help, but we don't encrypt before we compress. We compress before we encrypt.

Don't compress then encrypt. It doesn't matter in this use case, but it matters very much in others. When attackers have chosen plaintext (which is virtually always), they can exploit compression to create a traffic-analytic side channel. This is the CRIME and BREACH attack on TLS, for instance.

Schneier, in Applied Cryptography (which: avoid) recommended compressing before encrypting. That was bad advice. Compression and encryption interact in treacherous ways. Get of out the habit of combining them.

Are you saying that compressed data should not be encrypted? We are trying to limit the amount of data we have to store in the backend K/V store. Can you provide links so I can read up on this. Thanks.

Exactly, the wikipedia articles for BREACH and CRIME are good starting points if nothing else.

[1] http://en.wikipedia.org/wiki/BREACH_(security_exploit)

[2] http://en.wikipedia.org/wiki/CRIME

Do not compress chosen and/or known plaintext. The compression ratio alone reveals information about the data. i.e., how similar the unknown data is to the known data.

> compressed data should not be encrypted?

That's really tricky to answer simply. "No" might be taken to mean you should store compressed data unencrypted, which would be much worse. Nor do you have to decompress already compressed data before encrypting. But if somebody hands you data to encrypt and store, don't compress it as part of the encrypting phase.

This is interesting and counter to my intuitions. Care to elaborate or point to references so I can educate myself? =)

As another point of reference an HTTP load balancer that mailgun built and uses, called vulcan[1], uses secretbox[2] to encrypt secrets into etcd. There are no good docs on how to use this in practice with vulcanctl so I will need need to ask them to document that :)

[1] https://github.com/mailgun/vulcand

[2] http://godoc.org/code.google.com/p/go.crypto/nacl/secretbox

Hi Brandon! - secretbox is another great way to encrypt data, and would have worked for this project too. Ultimately we chose openpgp because it allows us to encrypt the same data with multiple public keys all at once, for multiple consumers.

Awesome project, thanks for sharing.

Looks like it takes a similar approach as the hiera eyaml project (it also encrypts on a per-key basis using gpg) which I've found to be really nice to work with in the past (as opposed to other tools that use symmetric encryption or encrypt the entire blob of all secret keys together). Glad to see a tool that does this with etcd and consul, gives the same benefits without a centralized puppetmaster.

Any plans for clients in other languages? Or if you're not planning to build would you accept PR's for them?

Yeah, I'm open working with anyone on other client libs. Ideally we can track them in an official doc in the repo. To help others get started I'll document the data we store in etcd/consul in more detail.

Ideally clients only need to store values with the following encoding base64(gpg(gzip(data))) and do the reverse when retrieving data. As long as the public/private keys are available, everything should work.

I had always wondered how these service discovery tools handled the encryption of data you put in them. I guess now I know! :)

Before this was created were people just doing an encrypt/decrypt on in/out in their application code?

or, perhaps they weren't encrypting.

That was my thought but I was trying to be nice :p

Another great and needed product written in Go. Nice work!!

It's particularly useful in clustering environments like CoreOS and Kubernetes where it's a little harder to pass configs securely. Instead, just bind a local volume to your docker container with your private keys, and load your configs securely from etcd/consul.

This is interesting, I'm happy that the tool be available as a library.

Reply on news.ycombinator.com