Hacker News Reader

Hacker News

Comment by Zamicol | Hacker News Reader

Full thread

Zamicol

This is one of the reasons why Go has its own versioning system. From a project's `go.sum`:

example.com/example v0.0.0-20171218180944-5ea4d0ddac55 h1:jbGlDKdzAZ92NzK65hUP98ri0/r50vVVvmZsFP/nIqo=

Where "h1" is an upgradeable hash (h1 is SHA-256). If there's ever a problem with h1, the hash can be simply upgraded.

Git's documentation describes how to sign a git commit:

$ git commit -a -S -m 'signed commit'

When signing a git commit using the built in gpg function the project is not rehashed with a secure hash function, like SHA-256 or SHA3-256. Instead gpg signs the SHA-1 commit digest directly. It's not signing the result of a secure hash algorithm.

SHA-1 has been considered weak for a long time (about 17 years). Bruce Schneier warned in February 2005 that SHA-1 needed to be replaced. Git development didn't start until April 2005. Before git started development, SHA-1 was identified as needing deprecation.

View on HN

> Instead gpg signs the SHA-1 commit digest directly

A minor correction: when signing a commit, gpg does not sign the SHA-1 digest of that commit. This is impossible since the signature becomes part of the commit header which is one of the inputs to the hash function that produces the oid.

Instead, GPG signs the serialized data (parents,headers,tree,message) which would otherwise be the input to SHA-1. Then the sig is inserted into the buffer at the end of the header and the string is digested to produce an oid.

Source: https://github.com/git/git/blob/39c15e485575089eb77c769f6da0...

Thank you for pointing this out, and thank you for the link the the relevant code section. Excellent comment.

Lmao how were you that confident in your original comment which basically claimed git signatures are only as secure as SHA-1.

Perhaps you misinterpreted my reply. I didn’t intend to dispute parent’s claim, merely to correct a mechanical detail.

Git signatures are more or less only as secure as SHA-1, although the properties it relies on are not yet compromised and several factors mitigate the real-world risk.

A practical preimage attack on SHA-1 would seriously undermine the security of git signatures since the string being signed includes two or more SHA-1 hashes representing a content snapshot and the commit ancestry. Arbitrary preimage attacks would make it possible to modify a repo’s contents or history without invalidating oids or signatures.

In practice only collision attacks have been found, all of which have a detectable signature that git has been modified to detect.

Disclosure: I work at GitHub, but am speaking for myself.

Also check out multihash from the IPFS folks: https://github.com/multiformats/multihash

It's a more robust, well-specified, interoperable version of this concept.

Though it's probably overkill if you control both the consumer and producer side (i.e. don't need the interoperability) and are just looking to make hash upgrades smoother, in that case a simple version prefix like Go's approach described above has lower overhead.

There's no need to explicitly version your first version of this though. Those first-version values are easy to identify: they don't contain versioning information :)

E.g. say you have `5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8`. What version is that?

Well. It's exactly as long as a SHA1 hash. It doesn't start with "sha256:" or "md5:" or "h1:" or "rot13:". So it's SHA1. Easy and totally unambiguous.

Versioning can almost always begin with version 2.

Me, sowing: "Each record begins with a 4 octet BE value indicating the record length."

Me, reaping: "Each record begins with a single byte indicating the record format version. In version 0, this is followed by a 3 octet BE value indicating the record length."

if you're storing the raw binary rather than hex or base64: yeah. there are often no illegal values, so there's no way to safely extend it, unless you can differentiate on length.

for those, you have to leave versioning room up-front. even 1 bit is enough, since a `1` can imply "following data describes the version", if a bit wastefully in the long run.

sow then reap

Well this fucking sucks. What the fuck.

What are you guys even talking about

https://twitter.com/screaminbutcalm/status/11055778456428789...

i deserved it, i guess, lol

That's not applicable to Grox's example. The initial version uses only hexadecimal digits for the SHA256.

If you had: "each record begins with an 8 character record length, in hexadecimal, giving 32 bits", you have no problems. The new version has a 'V' character in byte 0, which is rejected as invalid by the old implementation.

Love too put another branch in the decoder I need to run a billion times.

I beg you: please clone git, do the changes, and benchmark them. I bet you won't be able to obtain a statistically significant result from this single branch.

It’s “almost always” though, not “in git for this one ID we already messed up”.

versioning also allows you to change the inputs on the future to include/exclude more info

inconsistencies in how data is presented (optional version number) is a pain to deal with in code

I think the implications for Go are a bit different, though. It's a very simple matter to change the hash algorithm used for go.mod. Even if there was no hash version prefix, it's trivial to add one after the fact, though older tools would probably give a confusing error message without foreknowledge of the concept of an unrecognized hash algorithm. And adding a new hash algorithm is just a matter of writing a relatively small amount of code, and then probably waiting a few Go releases before making it the default and assuming most people will have it.

Git's entire foundation relies on SHA1 hashes. Each commit is its own hash, and contains a list of the hashes of all files that are a part of it. Branches have hashes, tags have hashes. Everything has a hash. A repository that uses a different hash algorithm is a completely different repository, even if the contents and commits are otherwise identical. You can't even store your code on someone else's server (well, aside from manually copying the repository data over, though that won't be too useful) unless that server has upgraded their git version.

The counterpoint: Fossil did it, it was easy, no big deal.

Well, Fossil's database is much better designed, you reply.

That it is!

I think the argument that gp is trying to make is that it's really hard for git to implement this in a backwards compatible way. You may be right (I don't know anything about Fossil, will take a look!) that Fossil allowed for this by making good design decisions in the past. This is not something that git maintainers can do right now without a time machine though. Old versions are in use out there and will need to keep working if the goal is to make the transition easier for users.

Just to nit on your portion of signing: wouldn't you need to rehash all prior commits as well so that they used the better hash function? Otherwise someone could find a collision for a prior commit hashed with sha-1, slip that in, and the final commit being hashed with sha256 wouldn't matter.

This then makes the signing code use its own form of hashing that is different from the rest of git's commmit hashing, and seems like a novel way to introduce tooling issues / bugs / etc.

> and the final commit being hashed with sha256 wouldn't matter.

Git stores content, not diffs. So the signature verifies all content stored in that commit. It does t verify anything that came before it, unless those are specifically signed as well.

> Git stores content, not diffs.

But the "contents" is just pointers to tree roots with a trusted hash. If the hash is no longer secure, you can't garantee that any such trees are your content, or safe.

The assumption in this context is that all those have been rehashed to SHA-256 too. The point was about whether that rehashing needed to be extended to previous commits.

Whenever the word "upgrade" rears its head, beware.

The intent behind it is obsolescence and phasing out, resulting in an endless make-work treadmill for the users.

If there is ever a "problem with h1", and you neglect to upgrade your data right there and then, five to ten years, it will be unreadable.

What in the world are you talking about? Generally, systems with upgradeable hashes will remain backwards-compatible with old ones forever.

or you know, find the version that handles transition and run it to upgrade

stepping through required versions a common operation

Versioning hashes is definitely not a new idea with go - just look at how unix stores password hashes.

The author of the comment did not imply this.

Reply on news.ycombinator.com