Hacker News Reader

Comment by p-e-w | Hacker News Reader

If you want unique identifiers, use version 4 (random) UUIDs. Problem solved.

The probability of a collision is roughly the same as the probability of a fully grown dinosaur spontaneously manifesting in your bedroom due to quantum fluctuations.

View on HN

I will take my chance :-)

More seriously, If you can use them, good old increments are probably best. They are fast and cheap. Especially in a database. They can have privacy/security issues (you could guess things by the values of ids of stuff). UUIDs are better in those case or when you deal with a distributed system.

> They can have privacy/security issues (you could guess things by the values of ids of stuff).

Push them through a secure hash function, and that problem is solved too (assuming you can keep the base counter private).

If you're going to do that then you might as well just use UUID, since you effectively reintroduce the negative aspects of that (infinitesimally miniscule chance of collisions, computation involved in the calculation, etc.)

The difference is that you can still use sequential IDs internally, while exposing hashed IDs to the outside. This protects your database from collisions under all circumstances, while in the absolute worst case, a single user might experience bugs because two external IDs collide.

This is a weird proposal. If you're using non-hashed IDs internally and exposing hashed IDs externally, you are going to need to map those (securely hashed) ids back to internal ids when the client hands them to you.

I guess you could do this with complete table scans, hashing the ids and looking for matches, but that would be horribly inefficient. You could maintain your own internal reverse index of hash -> id but now I have to ask what's the point? You aren't saving any storage and you're adding a lot of complexity.

Seems like if you want random unguessable external ids, you're always better off just generating them and using them as primary keys.

Also, you aren't protecting your database "from collisions under all circumstances" - there's no guarantee your hash won't collide even if the input is small.

Yes, it is more reasonable to use encrypted IDs externally from structured/sequential IDs internally, not hashed IDs. Recovering the internal ID from the external ID is computationally trivial since it will fit in a single AES block and you don't have to worry about collisions.

Yes, I tend to like this philosophy in database design, of internal sequential ids which are used for joins between tables etc. and an exposed "external reference". But I typically would use a UUID for my external reference rather than a hash of the internal id.

Doesn't that just add a whole lot of unnecessary complexity? If elements have multiple IDs, one of which should not be leaked to the outside, that's just asking for trouble in my opinion.

Is generating UUIDv4 or UUIDv7 really too much effort? I'd assume that writing the row to the database takes longer than generating the UUID.

It also means once your hash function leaks for whatever reason or gets brute forced because of whatever weird weakness in your system, it's game over and everybody will forever be able to predict any future ids, guess neighboring ids, etc., unless you're willing to change the hash and invalidate all links to any content on your site.

If I'm in a scenario where I think I need consecutive ids internally and random ones externally, I'll just have two fields in my tables.

You need 2 fields anyway, unless you want to have to brute force your hash function when you need to invert it.

Store just the sequential id, compute the hash on the edge.

This keeps your database simple and performant, and pushes complexity and work to the backend servers. This can be nice because developers are typically more at home at that layer, and scaling the backend can be a lot easier than scaling your database. But it also comes with the downsides listed in this thread.

That's fine, but when a request comes in referencing only a hash and not an id (because you're not leaking ids to clients), how do you get the id?

Good point. Back when we did that we just used a reversible hash function (some would call it encryption). There are some simple algorithms meant for encrypting single integers with a reasonable key.

I might be misremembering, but didn't YouTube do this in the early days? So yeah, that was what I was thinking of when replying, not a traditional hash function.

If you're hashing for security reasons, I think you should still maintain a salt.

Hum, no. You can easily hash numbers from 1 up to the value you see and guess the next value.

If you want a secure identifier, make a random 64 or 128 bits number (a UUID type 4). And do not use this number as an internal identifier, because identifiers performance is all about predictability and low entropy.

If you use a robust salt, such a randomly generated long salt, attacker won't be able to guess the next hash.

A block cipher is better, as it's guaranteed to not have collisions.

Unless the RNG itself is the problem, there's no reason not to. All kinds of civilization-ending catastrophes are vastly more likely than a collision in a space of size 2^122.

v7 looks nicer since it solves v4's locality issue and you're still a gazillion times more likely to win the lottery than generate a collision.

Unfortunately, many libraries don't implement it yet.

It is still in draft and I've watched the spec changed at least once.

> The probability of a collision is roughly the same as the probability of a fully grown dinosaur spontaneously manifesting in your bedroom due to quantum fluctuations.

I'd love to see the math for the probability of the second option.

> The probability of a collision is roughly the same as the probability of a fully grown dinosaur spontaneously manifesting in your bedroom due to quantum fluctuations.

But now the probability of bad things happening increased by about a factor of two, which is not acceptable.

I don't see how sudden dinosaur appearance is a bad thing.

How will you sleep?

(Anyways, comparing the probabilities does not necessarily mean that the things being compared are both bad or both good.)

Perhaps it is made of anti-matter.

the dinosaur is now on my bed, what next?

I would just call Randall Munroe, he will know what to do

Close the door and immediately switch to v5.

Enjoy the non heat-death of the universe

Knowing nothing about UUID v4 generation, I have likely a stupid question. What makes you so confident that all implementations and their entropy sources are flawless enough to make actual collision probability close enough to theory?

What makes us so confident that our database implements ACID correctly, the RAM stores bins correctly, and the disk drivers store the data correctly?

In the end we have to make some assumptions about the correctness of (some of the) components.

We are not confident in any of that. Which is why we mitigate it with checksums, ECC etc.

So depending on the consequences you might opt to reduce the risk or help mitigate the consequences.

To just state that it is about as likely as my coffee maker being a portal to the future isn't very helpful. Poor entropy sources or bugs are not uncommon.

but is it higher or lower if the dinosaur is not fully grown?

I would guess slightly higher probability for a not fully grown dinosaur because the mass will be less, I think.

Reply on news.ycombinator.com