Sometimes this is very hard to do. This is in effect Twitter's core use case as well, with the list of accounts you follow.

And it's a hard problem. Twitter spawned a huge wave of NoSQL to deal with this, and it's still not really dealt with.