HMAC in Go, Python, Ruby, PHP, and Node.js | Hacker News Reader

HMAC in Go, Python, Ruby, PHP, and Node.js

blog.turret.io

27 pointswilsonfiifi34 comments

If some of these dev environments required you to implement HMAC yourself from the SHA2 primitive, I'd see some of the value of a "Rosetta Code"-style page for it. But all of these are just examples of calling into the environment's library interface for HMAC. In fact, except for Golang, all of these examples are essentially just calling into OpenSSL's HMAC.

Also:

* Do not use string/buffer comparison functions to verify HMAC. In incredibly common error.

* Do not use human-intelligible strings as HMAC keys. HMAC keys are like AES keys: fully random; they should come from /dev/urandom.

* Do not embed HMAC keys in your source code!

Ultimately: I'm not sure I understand the impulse behind pages like this. It's clear that the authors aren't domain experts (that's fine; they're probably better programmers than the crypto domain experts are). They know they aren't getting everything perfect. In fact: they know some of the examples on their own page have flaws, because they were corrected about timing attacks and only fixed Python.

Crypto bugs in your own code are one thing, but propagating crypto bugs to other people's code seems much worse.

> Do not embed HMAC keys in your source code!

Where else would you put it? A separate configuration file is still your code (or is that what you meant?)

But don't check the key into source control.

In a large Node.js project, I have a file secrets.js, which is full of lines like:

exports.passwdHmacKey = 'awaibaeSeingoo4phahjeekee6Ufetev';

Put secrets.js in .gitignore, and copy it where needed manually.

Things like this are fine. Big sites sometimes have key management servers, provisioned sort of like Zookeeper is.

(Just using your comment to riff now:)

As long as you're not (a) checking it into VCS and (b) having every developer hold a copy on their laptop, whatever you're doing is fine.

What this site does, though, means that any time someone loses their laptop, or really just leaves their laptop unattended somewhere unsafe, everything needs to be reprovisioned. In fact: if you don't have audit controls (who's used which key when), technically, any time that happens, you need to disclose that it happened to your users. A nightmare.

Development and production should not use the same keys!

Losing a development laptop should not also lose any important keys.

What I do is check in a throwaway key into source control, then modify the file with the real key on production and never check it in. Since you never do checkins from production this works well.

I agree of course, but in practice, the dev/prod distinction is very unreliable, because people will stand up dev servers that run with dev keys, for which a dev key compromise pivots to a data center (or PII) compromise.

> HMAC keys are like AES keys: fully random; they should come from /dev/urandom.

I've been using pwgen -s: based on /dev/urandom, but typeable. My understanding was that it first took a hash of the key, so as long as the total entropy was high it was fine for it to be ASCII.

Sure. ASCII isn't the problem; structure is. You already know the attack: take a sample MAC (built with a very, very fast MAC function) and aim a permutating dictionary at it.

Gah, Go pet peave, they could just use a map literal (good for short low-perf code).

Or even better perf/safety wise:

  type Payload struct {
    Name string `json:"name"`
    Category string `json:"category"`
    Action string `json:"action"`
    Where string `json:"where"`
    Timestamp int64 `json:"timestamp,string"`
  }
  payload := Payload{
    Name: "joe smith",
    Category: "people",
    Action: "transport"
    Where: "pluto",
    Timestamp: time.Now().Unix(),
  }
  jsonBytes, err := json.Marshal(&payload)

This code doesn't allocate heap for payload, is much faster and typesafe...

P.S. And could be shorter, but I was trying to be clear.

Original author here -- thanks to everyone for the feedback. Some time has passed since this was originally written and the remaining languages were never updated to improve the safety of the signature verifications.

The original reasoning behind this post was to provide a single reference for signature generation and verification in some common languages -- something I struggled to locate myself. Admittedly, I should've provided warnings about using simple and hardcoded keys in the examples, which were done that way for readability.

While there is still a lot of debate about the ability to perform true constant-time comparisons in many of these languages (https://bugs.python.org/issue15061, https://github.com/joyent/node/issues/8560) I agree that for those who would be otherwise using the unsafe string comparisons, the benefits certainly outweigh the slightly added complexity.

The updated gists are available in the post to anyone with comments or improvements.

Cheers!

There might be a chance of timing attacks against such "security". A secure system should never depend on obscurity.

It's also pretty sad that they didn't use a constant time comparison function. Whether it exists in the target language standard library or not, it's easy enough to write one.

Wanted to add that as of 3.3, Python has a nice constant-time compare built in: https://docs.python.org/3/library/hmac.html#hmac.compare_dig...

2.7 actually

https://docs.python.org/2.7/library/hmac.html#hmac.compare_d...

It looks like they're including and checking a timestamp with the payload. Is this not sufficient?

No. A timing attack is used to determine the shared key that creates the HMAC. Because == and === (depending on the language) use memcmp(), the time of the comparison varies. EG:

36D5F4EA999342FED17D7488CB260FC92926 36D5F4EA999342FED18D7488CB260FC92926

The code would compare only up to the difference and return, indicating through the time spent analyzing the HMAC how many characters are the same. The attacker can then work the HMAC like they would a combination lock, till they reproduce the key used.

That's why a constant time comparison is so important: it leaks far less information.

This is a forgery attack, not a key-recovery attack.

I am not a cryptographer. I don't quite understand the distinction you are drawing with so little information. Care to explain what I misunderstood?

Given a message M and a secret key K, we have:

    MAC(K, M) -> T

To validate a pair (M, T), we verify:

   T = MAC(K, M)

Ideally, the execution time in this verification is independent of T. But many languages use string-comparison algorithms that exit immediately on failure. If Eve can detect this difference with high granularity, she has an oracle telling her how many leading bytes of a guessed tag T' are valid:

    O(M, T') -> n

She can use this to recover the first byte of a valid tag for M:

    for k in [0, 255]:
        if O(M, k || 0000...) > 0:
            return k

She can extend this to recover the second byte, the third, and so on. Eventually, Eve will recover T such that MAC(K, M) = T. In other words, Eve is able to forge an authentication tag T for an arbitrary message M.

What she won't do is recover K. So while she can forge tags for arbitrary messages, each forgery will require a fresh, online interaction with the verifying party. She cannot work backwards from (M, T) to K.

To put it another way: You (slowly and painstakingly) defeat the authentication without having to brute force, e.g. 2^128, possible keys.

However, the key is still unknown to you. It just doesn't matter, because you can forge messages without it.

Thank you for the in-depth explanation!

Thanks for the explanation.

Node.js example:

    return new Buffer(sig).toString('base64') == signature;

PHP example:

    return hash_hmac("sha512", $string_to_verify, $shared_secret) == $signature;

Yeah, like fdomig said: timing attacks. The Python example included a mitigation. PHP includes hash_equals() and a constant time comparison in Javascript isn't difficult to write.

Possibly also, the Ruby example:

    return OpenSSL::HMAC.digest('sha512', shared_secret, string_to_verify) == signature

(I don't write Ruby so I don't know if this is overloadable somewhere.)

> (I don't write Ruby so I don't know if this is overloadable somewhere.)

Ruby allows overriding the == operator[1], but OpenSSL::HMAC.digest returns an instance of String[2], rather than returning a special subclass of string or some other kind of special HMAC-representing class overloading ==.

[1] http://docs.ruby-lang.org/en/2.2.0/syntax/methods_rdoc.html#...

[2] http://ruby-doc.org/stdlib-2.0.0/libdoc/openssl/rdoc/OpenSSL...

To be pedantic, in Ruby, the fact that an object is of class String is not sufficient evidence that a method has not been overloaded (they can be overloaded via the objects eigenclass), though you're probably right.

Even if it did (and it doesn't, I checked), you're probably still better off doing it explicitly since it's easier to audit, less likely to differ across implementations, and less error prone (e.g. swap the order of the comparison and you're back to plain old String#==).

For Ruby you should be using ActiveSupport::SecurityUtils.secure_compare or similar.

In theory openssl could return a String-like object with an overridden ==, but it doesn't.

Yes, that Ruby version is also insecure.

Okay, so only the Python version was acceptable. :(

(And maybe Golang?)

The Golang example looks fine but I didn't read it carefully and do not vouch for it.

> The Python example included a mitigation.

Unnecessarily so too, Python provides hmac.compare_digests since Python 3.3 and it was backported to Python 2.7.7, which was released almost a year ago.

All crypto libraries should provide and developers should use constant time comparisons for exactly this purpose. A good example from the same page is the Go crypto/hmac package includes mac.Equal to check for equality without introducing timing weaknesses.

Adding a nonce to the request and storing it is a good idea to add to prevent replay attacks. Especially in the case of OAuth where some implementations hash the query string but don't hash the request body - which could allow all kinds of mischief without even cracking the secret key.

In this scenario a replay attack is limited to a 30s window during which the timestamp is valid. Depending on the implementation, this may or may not suffice, in which case an nonce would definitely be an improvement albeit with the added complexity.

Reply on news.ycombinator.com