Hacker News Reader

Comment by jrockway | Hacker News Reader

Kind of pointless. Just use a UUID for the ID and a date for the thing you sort by. Twitter stores a "posted at" time, so why not sort by that? Using an ID column for a time-based sort when you are storing the time anyway is silly.

Oh well, at least it was a fun hack.

View on HN

Just use a UUID for the ID and a date for the thing you sort by.

Sure, except for the stated design goal of being backwards compatible with code which expects a 64-bit ID and uses that value for sorting posts.

Remember that this is for Cassandra. Cassandra only allows you to sort records by the primary key, so if you want to sort by date, the time has to be part of the primary key while also being unique. So vanilla UUID doesn't work. Some UUIDs can take a time argument, but as the post says, they couldn't use UUIDs anyway, because they are all 128-bit and Twitter is limited for historical reasons to 64.

> Using an ID column for a time-based sort when you are storing the time anyway is silly.

Your solution violates the listed requirements that the ID is 64 bits and the IDs need to be roughly sortable.

Creating an index on the posted_at time may be harder than it sounds, with "tens of thousands of tweets per second" and a distributed database. The "within a second" approach (ie. loosening up the constraint of ordre) sounds pretty briliant to me.

But you don't need to index on time; you can sort on the client. This is what twitter clients already do, except they sort an integer instead of a datetime.

Remember: Twitter is nothing new. We have done massively distributed messaging since the 70s. It's called e-mail.

Because the APIs fetch recent tweets, the service needs an index on time.

Although it should be easy for client applications to sort on time instead of id, that's not what all applications do. Twitter chose not to break clients that sort on id.

You could just fake the id later. Why ruin your architecture because your users are lazy?

Except in email everyone has a separate mailbox. On twitter everyone's mailbox overlaps and there are an infinite number of possible public views of the tweets (searches, lists, etc.)

You don't need a separate column if you are using Version 1 UUIDs

Version 1 UUIDs require 128 bits.

right, so it doesn't fit the needs of the OP, but for other users without the 64-bit requirement, version 1 UUIDs can be k-sorted without an additional column.

Reply on news.ycombinator.com