TIL: Versions of UUID and when to use them

> Only no known details if the only document you're reading is the notoriously poorly-specified RFC

To be fair, in RFC9562 I did cite two documents that are UUID Version 2 specifications. But RFC4122 was too cryptic for my taste.

As for the historical UUID types specified by the 0-7 Variant space: We are starting work on an informational RFC that will help folks understand those.

See https://github.com/yocto/draft-yocto-uuid if you want to add to some discussions and/or review text. We are still a bit early into the stages but I hope to have some progress soon.

haha I had the precise same thought process and immediately didn't finish the article since they didn't have much attention to detail.

i enjoyed reading the appendix though as a snapshot of time.

Probably written by a language model

I mean, there’s two things that one might learn about. One is reading a specification for what a v2 uuid is. The other is learning information about the circumstances of a UUID’s generation from the uuid. Obviously the sentence was about the latter, as could be determined from the context as the other versions are described as being generated from specific things.

I agree the sentence is a bit unclear, but I don’t think it’s misleading or whatever.

just use creation timedate plus auto increment int.

and then a small hash with base64 or 37 or whatever is in vogue these days.

thats what old timers used before uuid 1.

guess we should guerilla standardize something like this as uuid-0 or uuid-deprecated-2.0 for keeping up with the spirit.

The closest that comes to minds is ULID[0]. It is short (26 character base32), 128 bit and lexicographically sortable.

I think the reason there's no other popular standard is you give up something. 128 bit gives a pretty low risk of collisions in almost all uses, but as you go smaller you start having to consider the specific scenario and impact, etc, which doesn't work well for a standard.

You could use another encoding (eg base64 or base85) to get it shorter, but you start sacrificing other things (case sensitivity, url-safeness) - again, not great for a standard.

[0] https://github.com/ulid/spec

Not a standard per se but nanoid seems to fit the bill. Widely implemented.

Git uses SHA and then dynamically set the number of characters to use based on repository size. You could do something like this.

Sqids[1] might fit the bill for you - the IDs it produces are much shorter than UUIDs, however they're not universally unique - they're generated from an integer sequence.

1. https://sqids.org/

I was just today wanting shorter UUIDs so if you like a more compact/short UUID you can convert them like so to url safe base64.

It's the same UUID just in 22 character form and can be converted back. It's n ot really a conversion because a UUID is just a 128 bit value so its an alternative representation.

  483971cf-aad7-4c84-abf1-4a94c9d72f99 -> SDlxz6rXTISr8UqUydcvmQ (length: 22)
  fb67926f-3cfb-486c-a7da-30662147a20b -> A2eSbzz7SGyn2jBmIUeiCw (length: 22)
  799069a9-b32a-415f-b689-a8cc3f51bfa4 -> eZBpqbMqQVA2iajMP1GBpA (length: 22)
  8161ee0b-f7a5-4b32-95ea-9b9efe94e5f2 -> gWHuCBelSzKV6pueBpTl8g (length: 22)
  b1ea416c-f209-43cb-bfaf-d9cf6229459e -> sepBbPIJQ8uBr9nPYilFng (length: 22)
  ee70989a-b614-4665-9881-41054544c313 -> 7nCYmrYURmWYgUEFRUTDEw (length: 22)
  cce06fe2-b64f-47bc-a91a-d3dfd343e1e5 -> zOBv4rZPR7ypGtPf00Ph5Q (length: 22)
  aea3de6e-e769-4c8d-ba2d-77922d227176 -> rqPebudpTI26LXeSLSJxdg (length: 22)

    import uuid
    import base64
    
    def make_short_uuid(data):
        encoded = base64.urlsafe_b64encode(data).rstrip(b'=').decode('utf-8')
        return encoded.replace('-', 'A').replace('_', 'B')
    
    def generate_and_print_uuids():
        for _ in range(8):
            uuid_obj = uuid.uuid4()
            uuid_bytes = uuid_obj.bytes
            print(f'{uuid_obj} -> {make_short_uuid(uuid_bytes)} (length: {len(make_short_uuid(uuid_bytes))})')
    
    generate_and_print_uuids()

I usually generate N bits of randomness and base58 encode it. Choose N to your liking. You loose the benefits of monotonic sorting that is present in some UUID versions. Base58 is url safe and does not contain any special characters. And you can still store values as binary (eg. bytea in Postgres instead of a text column).

> combination of random and short

IMO we need to be clear on the distinction between (A) the UUID bit-generation scheme versus (B) the way it is encoded for human use/reading/transcription.

They are mostly-separate problems.

For example, you could have a very secure mathematical scheme, but it gets ruined by a horrible representation where each bit is written as either a capital-I, a lowercase-l, or the number 1.

Conversely, could have a deeply insecure scheme that uses a nice compact serialization where everything is grouped into chunks and "1Il" confusion is not possible and there's a check-digit, etc.

Nanoid? https://github.com/ai/nanoid

We are working on standardizing an alternate encoding technique for the 128 bit UUID so it can have a shorter text form.

You can the discussions here: https://github.com/uuid6/new-uuid-encoding-techniques-ietf-d...

It’s also great for key-value store like DynamoDB. Much cleaner than compound keys prefixed with a timestamp or iso date (or worse: wasting a secondary index on timestamps).

Interesting. I don’t think I’d ever really consider using uuids to look something up by time (I’d probably want an actual timestamp field instead). To me, the big advantage of the uuids being sorted by time is that You get better locality – inserting new entries into an index is probably a mostly-append operation which could be cheaper than a random insert (but may have more contention too; worth considering a hybrid with some random bits before the timestamp to have more sorted ‘shards’) and in many places reads tend to be for more recent data so it can be nice to have the more recent data in one place (and eg pulled into more caches).

Their original purpose was to identify messages in Apollo's distributed computing architecture. UID and later UUIDs were a reversible way to mark an intersection point between two dimensions. Any two machines would generate the same UID/UUID for the same two inputs, and a recipient of an identified message could reverse the identifier back into the original components. They were designed as labels for ephemeral messages so the two dimensions were time and hardware ID (originally Apollo serial number, later Ethernet hwaddress etc).

I think a lot of the confusion can be traced to the very earliest AEGIS implementation where the Apollo engineers started using “canned” (their term, i.e. static or well-known) UIDs to identify filesystems. Over time the popular usage of UUID fully shifted from ephemeral identifiers where duplicates were intentional toward canned identifiers where duplicates were unwanted and the two dimensions were random-and-also-random.

The history gets even more complicated because Microsoft hired one of the top Apollo guys to do MSRPC for Windows NT, so there is also “GUID” which differs from UUID in the layout of the fields and is not mixed-endian despite what a lot of sources will tell you. In addition to ephemeral RPC message-identifying GUIDs Microsoft are also in love with canned GUIDs for identifying COM classes, media codecs, and almost anything else that would ever need a well-known identifier. See https://gix.github.io/media-types/ for example.

Apologies for linking my own repo twice in the same comment section but I started (and need to get back to) compiling the history of all this in the README of my UUID library. Apollo started in 1980 and the Leach/Salz UUID RFC draft didn't happen until 1998 so there is a huge amount unsaid by the modern standards: https://github.com/okeeblow/DistorteD/blob/NEW%E2%80%85SENSA...

True but the appeal for most developers is it's simple to implement. Virtually every language has a UUID library that works in one line or code.

Like in Go, it's just uuid.New().String() vs using crypto/rand to read random data, convert it into Base64 of hex... which will take more lines and effort.

UUIDs are 128-bit numbers and the hyphenated-string representation is only one of many ways to represent that number, sort of like how an IPv4 address is a 32-bit number of which the “dotted-quad” is only one representation. If you are thinking of UUID as a string format then your most fundamental concept of UUID is flawed.

Even if you do just want a random identifier (not really the original point of UUID but has become their most popular form) I still think it's cool how random UUIDs have a little flag bit to tell you that it's intended to be random. Useful when one runs across a lone identifier with zero context.

UUID 4 also sets 4 bits to fixed values to indicate it's version 4. You can argue whether creating different namespaces between the different methods of creating UUIDs is useful. But your plain random number generator has only a 1/16 chance of generating a valid UUIDv4. (setting the bits correctly is however trivial if you do want to roll your own uuid generator)

MD5 is “broken” as a cryptographic hash function. It still is perfectly fine as a non-cryptographic hash function.

Like V3 but with a specifiable hash algorithm.

If you really need it, consider UUIDv4 as encoding, and use your cryptographic algorithm to create a 122-bit output and encode it as a UUID.

Otherwise, you'd want longer outputs.

Use v4 if creation date could conceivably be sensitive information of if you depend on your uuids being completely unguessable. Otherwise use v7

An issue I've always had with UUIDs and ULIDs is there isn't a great way to generate one deterministically, as far as I can tell: for a lot of use-cases, being able to reprocess data and generate identical IDs is really useful and there isn't a standard way that I know of to achieve this.

Why are you dismissive of security-related issues?

One advantage of uuids is they can be generated on several distributed systems without having to check with each other that they are unique. Only long ids make this reliable. Youtube ids are random and short, but youtube has to check they are unique when generating them.

Maybe one way is to split up a random assignment space and assign to each distributed node, but that would be more complex.

Even UUIDs are not guaranteed to not have duplicates. It's just extremely unlikely, largely due to their length.

If you are fine with creating IDs in a centralized way (as you would do in 99% of cases anyways) you can just use a normal incrementing integer primary key. Then encrypt it with XTEA (either at your API boundary or in the database) to get non-sequential unguessable 64 bit keys. [1] has example code for postgres. If the original key don't have duplicates then the XTEA encrypted keys don't have duplicates either.

Then just encode it in a format of your choosing. Youtube uses a modified base64 encoding (no padding, and + and / are replaced by - and _). And youtube video ids seem to also be 64 bits, just like xtea output.

1: https://wiki.postgresql.org/wiki/XTEA_(crypt_64_bits)

An atomic counter of some sort solves the problem of UUIDs with the cost of a synchronization step (although this can sort of be minimized). UUID and its variants are long to avoid duplicates without having to synchronize.

If you're not distributed, use an incrementing integer.

If you're distributed, look into vector clocks[1] or snowflake[2][3]

[1] https://en.wikipedia.org/wiki/Vector_clock

[2] https://github.com/twitter-archive/snowflake/tree/snowflake-...

[3] https://en.wikipedia.org/wiki/Snowflake_ID

Immagine a world where every framework / API / database had its own incompatible UUID format. Without a standard specification that's where we would end up. Do you want to live in such a world?

Indeed. Also, add many other unique identifier formats (ULID, nanoid, bson ids ...) — in contrast to compatibility complaints here — and you'll see that people just love tinkering with trivial things (me too).

What was important during the times when we didn't know how to generate random numbers on computers, perhaps shouldn't be as important today?