Hacker News Reader

Comment by traceroute66 | Hacker News Reader

Honestly that's a poor blog post.

Randomly concludes "the best time-based ID seems to be xid" without saying why or comparing to others e.g. ksuid, UUIDv7 etc ("xid" is only mentioned twice in the entire blog, first in the above statement and second a link to the reference implementation). Equally unfortunate that they picked "xid" as their supposed "best" because Postgres has an internal identifier that is also called "xid" and is very much NOT to be used as a primary key !

Downplays the many issues with "serial", including somehow thinking the word might "might" has a place next to the words "not want to expose them to the world though" .... you DON'T, full stop. Exposing predictable identifiers to the world is never a good thing.

I'm not really sure what that blog post is supposed to be achieving really. I didn't learn anything.

View on HN

I really hate this trend away from basic IDs. I feel like it's driven by folks who've never actually worked in the real world. I got account paperwork recently where the company ID account ID and invoice ID were all uuids. 100% this company if I call them will not use this BS to lookup my account and will instead use something easier try to guess like a company phone number.

I also had to do some support tickets recently and because of the issues copying uuids from the screen they insisted on screenshots that included the url bar showing uuid. 100% trying to tell them the ID over try the phone or even rekey would run into issues.

Give me back my sequential invoice ids!

At some point those sequential integer ID's will become so long they might as well be GUIDs. Of course you could "compress" the integer using some kind of encoding scheme similar to base64. Then your long integer becomes a few characters.

... Maybe though. Even ID's in the millions are probably easier to read than a long ass GUID.

> At some point those sequential integer ID's will become so long they might as well be GUIDs

Worth bearing in mind that an unsigned 32-bit integer sequence can uniquely identify 4,294,967,295 records. If you're really storing that many records (let alone enough to exhaust an even bigger integer sequence), the length of the identifier is probably the least of your worries :)

Standard accounting practices require that invoices/receipts have sequential numbers. So it has to be integer, or some kid of sequential alphanumeric.

Even if one uses UUID as primary key, it should have separate invoice_no (in whatever formatting they require such as 2023/001, or 1001,1002...) which is the human readable and referenced number.

This is especially important if you are developing a multi-tenant system where the invoice number, say 2023/001, may exist for more than one tenant.

Are sequential id’s a security risk?

In one of our systems we’ve seen customer guess at other accounts by just incrementing the sequence.

The rule of thumb I used to use is if an Id is going to be used for lookups or being exposed externally use uuid otherwise us sequential.

The hard thing about the above rule is that it’s hard to tell when you are designing the db if the id will be used externally/for lookups or not. Requirements change later.

"Protecting" records by making IDs hard-to-guess just seems like putting the responsibility in the wrong place. If you're so worried about people getting their hands on the wrong records, I'd be more worried about your lack of trust in the application that queries that database in the first place: remember, even if you do make the IDs hard to guess, your "untrusted application" might at some point decide to simply leak the entire table instead.

There are much better solutions for this, like server-side prepared queries that do not simply return on an ID but rather as the result of a join, or just proper security practices in general, rather than reactively making something that was once guessed simply harder to guess.

Also, something like an ID is in my experience something that will be referred to orally between two human beings when discussing a problem, like referring to user 6201 or discussing invoice 540567. When you switch to UUIDs you're basically also saying "no human will ever have to say this out loud".

> "Protecting" records by making IDs hard-to-guess just seems like putting the responsibility in the wrong place.

It's a pretty powerful implementation of capability based security.

I would hope that a capability-based security system entails considerably more than just knowledge of an ID.

Many actual capability systems explicitly construct their capabilities to be unforgeable by not making them be mere secret byte strings, but objects protected by the kernel or other environment. For example, https://www.cs.cornell.edu/courses/cs513/2000SP/L08.html

And then people naively copy operating system design methodology into web APIs... Unrevokable leakable know-it-and-you're-in secrets are a bad idea.

> In one of our systems we’ve seen customer guess at other accounts by just incrementing the sequence.

What can they do with that guess? A sensible access control would not even let them see if the account is active, cancelled, or invalid.

UUID does not protect your records, that is, it is not a security measure against what you describe.

UUID is a part of the protection. If your access controls fail then preventing an attacker from enumerating through data by incrementing an integer is protection from Insecure Direct Object Reference. [1]

Even if there are no vulnerabilities in access controls it can also prevent competitors from knowing how busy your platform is. If I register a new account on your system and I get ID 57854 and a week later I register another and get ID 57978 then I know 124 new users have signed up in that time.

1 - https://cheatsheetseries.owasp.org/cheatsheets/Insecure_Dire...

Thank you so much for this.

It is if you don't expose a 'list' api, only a 'get'.

integer/year works almost flawlessly as invoice ID. It is unique and readable.

If the company has more than one unit that creates invoices than just add unit ID, so it becomes invoice_id/unit_id/year. Still unique and very much readable.

To avoid the German Tank Problem and leaking your business vitals to competition, I'd recommend making invoices be scoped to the client, e.g. <client_id>-<number>. And then client_id is either random or for small business e.g. unique slug derived from client name, for example you could get PERAK-0001.

https://en.wikipedia.org/wiki/German_tank_problem#Historical...

This is a very worthwhile point. You definitely need to prepare for readability if these ids will be customer facing.

While not a perfect solution to it, we use UUIDs throughout our application as IDs. But when we expose them in URLs or other places provide them as an encoded id (https://docs.crunchybridge.com/api-concepts/eid/). It at least makes them a little more compact, easier to copy/paste than UUIDs, and generally a cleaner look.

As it happens in at least some jurisdictions invoice numbers have to be sequential by law.

That's orthogonal to having a programatic ID for them

Collaborative databases (Wikidata, TheMovieDB, VNDB, etc.) all use serial identifiers. What is the problem with this? These websites don't want to hide how many entries they have (they tend to promote them), and it doesn't really matter if you iterate through all the numbers – the data is available through open licences anyway.

I think there are many situations where you don't want to expose predictable identifiers, but there are also examples where predictable identifiers may actually be beneficial.

If your data is public and can be scraped anyway, obviously a serial identifier doesn't matter. If your data is sequestered between accounts or has tenants all sharing the same database or API, that's just one accidental permission error away from being able to scrape every single record. If your customer's data is meant to be private, simply easier to generate unique IDs for each record. Plus it makes it incredibly easy for competitors to see the size of your business simply by signing up for an account and looking at the IDs.

> What is the problem with this?

It causes meetings with people who think all predictable identifiers are a problem.

Yes to me xid has some pretty significant downsides. It is composed of a timestamp, a machine identifier, a process id, and then a sequence that starts from a random value. It can be a bit faster because it doesn't have to generate any random values to make a new ID, but I don't see how it would ever work client-side. I much prefer the simplicity of UUIDv6 and ULID, and they both perform nearly as well. UUIDv6 basically is ULID but without the standard base32 representation, which I think is unfortunate.

>> Exposing predictable identifiers to the world is never a good thing.

Sometimes it doesn't matter. Example below:

https://news.ycombinator.com/item?id=34451344

> Sometimes it doesn't matter. Example below:

There is a saying for the examples you and others are posting ...."The exception rather than the rule"

Posting contrived examples in order to attempt to prove a point. For the majority of cases, a random ID remains the better option.

But unfortunately developers still treat security as an afterthought. They continue to use "serial" because of what can only be described as sheer ignorance, i.e. demonstrably false misunderstandings about database technology.

Case in point, I placed an order on an e-commerce site a couple of weeks ago. I was not best pleased to be given a tracking URL that read: "https://example.com/order/WEB-nnnnn", where nnnn was clearly an incrementing number. Such a thing is inexcusable in 2023 !

It took a lot of strength to resist the temptation !

Changing the nnnnn should not magically give you an invoice.

If your argument That a guid is better used here then you’re wrong. If the endpoint is not secure it doesn’t matter if your used a sequential id or a random id/guid. You could brute force a discovery.

You should always validate the input and verify the accessed invoice belongs to the person requesting it.

For sensitive information it should not be trivially guessable. Naturally, we need encryption and auth, but using not guessable identifiers multiplies the (already small) probability of successful attack by 1/2^128 so it is a good idea.

There's always going to be valid reasons to have non-guessable identifiers. But it shouldn't be used for security. It's not a replacement for not checking resource access.

Sometimes it's not 'sensitive', and even when it is, it doesn't really matter.

An invoice id, doesn't matter. You need to be logged in to access it, and when logged in, you can only access your own invoice. If you're going to try discover other invoices, well you know the user who is trying to hit random urls to find an invoice that doesn't belong to him. If you want to prevent random signups from doing it, block users who don't have => 1 paid order from accessing the page entirely.

Claiming "Exposing predictable identifiers to the world is never a good thing." tho is just FUD. There are use cases. But it's not 'never' a good thing.

>> It's not a replacement for not checking resource access. It isn't a replacement but it does compliment security. Imagine that your service accidentally hands out the wrong jwt for a short period of time. The attacker has the jwt and a known url but not the url to access another user's information. It will take the attacker a long time to find the url if it is a 128bit uuid. By that time the jwt has expired. I'm not suggesting that this will always work but it does reduce the probability of a successful attack - and security is 100% about probability.

>> Claiming "Exposing predictable identifiers to the world is never a good thing." tho is just FUD

Fully agree, this post is a great example: I can increment the id in the url and see the next post - no harm done.

> It's not a replacement for not checking resource access.

If I had a hundred bucks every time I've seen a website fail to do this access check... I'd be a thousand-aire.

Look i’m not taking a stance on this argument in general, but your statement that you can brute force a guid seems misguided. If you can brute force a guid url parameter you might as well just brute force the guid session token.

Just write a test case that "user 2" can't access "/order/WEB-nnnn" from "user 1" and that works too. You should have this test case anyway, even with random IDs. They can provide an extra "defence in depth" bonus, but they're of course no replacement for authentication checks.

You're going to need some type of readable relatively small ID anyway, because things like "Hi there, I have a question about order b1a354c5-ac2b-4990-a189-1f2b4f537b09 I placed on your website" doesn't really work.

The canonical solution for that is to encode the ID appropriately. Your example with common choices:

as base64url (RFC 4648 §5): saNUxawrSZChiR8rT1N7CQ (22 octets)

as base85 (RFC 1924 §4): p&bTO@+boru-j3)#beDJ (20 octets)

as QR code: data:text/plain;charset=utf-8;base64,4paI4paA4paA4paA4paA4paA4paIIOKWiOKWhOKWiCDilogg4paIDQrilogg4paI4paI4paIIOKWiCAg4paE4paA4paI4paA4paI4paADQrilogg4paA4paA4paAIOKWiCAg4paI4paI4paA4paA4paI4paIDQriloDiloDiloDiloDiloDiloDiloAg4paA4paA4paE4paA4paIIOKWgA0K4paA4paI4paA4paA4paAIOKWhOKWiOKWhCDiloDiloDiloTiloDiloQNCuKWgCDiloDilogg4paI4paAICDiloQgIOKWgOKWiOKWhA0K4paA4paA4paI4paA4paIIOKWiOKWhOKWhOKWiCDilojiloDiloDiloANCuKWgOKWgOKWgCDiloDiloDiloAg4paA4paAICDiloAg4paA

The problem with base64 is that it's still long and ugly, and also case-sensitive. I wouldn't want to say "order saNUxawrSZChiR8rT1N7CQ" over the telephone. Is it better than "order b1a354c5-ac2b-4990-a189-1f2b4f537b09"? Practically speaking, not really: both only really work when copy/pasted on a computer.

I described another scheme I've used in the past in another comment: https://news.ycombinator.com/item?id=34454430

My example is not contrived, it is literally this exact post (and HN isn't the only site with publicly available information).

However, I agree with you - sensitive information should not be easily guessable even if other security mechanisms are in place (and they absolutely should be of course).

What I am really reacting to is rules of thumb - "always do X and you will be ok". The issue is, it is often easier (and far more valuable) to understand the real reason behind things than to remember the rule of thumb.

> For the majority of cases, a random ID remains the better option.

If your ID is going to be exposed to a human you have to admit that random UUID's are kinda clumsy.

One other pro to UUID's that I don't see discussed is you can generate them client side and be 99.9999999% assured that they won't collide when stored in the DB. I kinda forget our use case for this but I think it was to make "create" and "update" REST calls a lot easier.

But if you’re handing out a random ID, you still need to have an index over that ID to have efficient lookups of random ID -> internal ID right?

One advantage is that you only need to resolve it once at the edge and then internally you use the external facing value. Are there any others I’m missing?

I agree, I would have loved a deeper dive in xid since it seemed to clearly outperform everyone else.

Not necessarily. The idea here is that the id can be exposed publically because it is random. The problem it solves is someone sees a page /accountdetails?id=123 and can easily look for /accountdetails?id=124 and assume it is likely to be valid.

If you use a random id, you cannot quickly know what other ids exist which makes looking for unauthorized access to objectids much harder.

If there needs to be an auth check for other IDs, then it shouldn't matter whether the IDs are random or not.

Depending on how highly aesthetic URLs are valued, it's not unlikely that after being in business for a while, the density of your keyspace will mean that even random IDs are found.

A better rationale for disconnecting public and private IDs is to make certain types of database migration a little bit easier. I don't think it's a huge win though, it's a chunk of work to maintain multiple IDs, do the required inderictions all the time, ensure public IDs are always sent to front end, and so on, while adding a translation layer as part of a database migration delays this kind of work until it's actually needed, if ever.

https://en.m.wikipedia.org/wiki/German_tank_problem

Preventing competitors from estimating the size of your business (or of your customer's businesses, if generating sequential IDs on their behalf) is one big reason for having unguessable public IDs.

This of course assumes that all the records backed by these IDs are not public information. In a lot of cases, this may not be true.

Unauthorized access is dealt with by implementing correct authorization procedures, not by obfuscating IDs, which valid ones can still be discovered through brute force or other means.

In your example those simple, sequential IDs should have no impact on security.

You really care about exposing a serial number scheme for a list of books your company publishes, or the identifier for each of the various hotels you own, or the cities you have an office in or something?

It's not only about predictability.

Exposing a serial number can also be competitive intelligence. Number of users, transactions, etc.

Sometimes this matters.

Yes of course. Which is why I picked a few examples where it doesn’t matter. Stuff you’d just always have a public catalog of.

Absolutely (once your business reaches a certain scale).

It can give competitors the ability to estimate the size/success/etc of various aspects of your business.

This was a major motivation for a certain online retailer to generate non-sequential IDs.

Some other interesting examples in an old HN thread: https://news.ycombinator.com/item?id=7278198

Probably not, but you definitely should care about not issuing sequential credit card numbers. Probably not the best example, but I don't think it's hard to imagine some scenario in between the two that still presents a concern.

You can't just use sequential numbers for credit cards because they have a builtin checksum validation called "Luhn's algorithm"

https://www.creditcardvalidator.org/articles/luhn-algorithm

Sure, that wasn't really my point though. My point was that there are some cases where sequential identifiers would cause security concerns and some cases where they would not.

Reply on news.ycombinator.com