Hacker News Reader

Comment by Nican | Hacker News Reader

FoundationDB has been growing as my favorite database lately. Even though it is only key-value store.

Out of curiosity: what are the scale limits of FoundationDB? What kind of issues would it start to have? For example, being able to store all of Discord messages on it?

I see blog posts of Discord moving to Scylla and ElasticSearch, but I wonder if there would be any difficulties here.

View on HN

Note that FDB can support other paradigms on top of KV

https://foundationdb.github.io/fdb-record-layer/SQL_Referenc...

Also IIRC Apple uses FDB at tremendous scale:

https://read.engineerscodex.com/p/how-apple-built-icloud-to-...

There are a lot of strict limits so AFAIK everyone uses FoundationDB for fast, consistent, highly-available metadata while doing replication/storage of actual data elsewhere (such as in S3).

https://apple.github.io/foundationdb/known-limitations.html

That is to say it's more like part of your solution and not the entire stack on its own.

Yeah. I read over that page, and everything seems sensible.

The only part that is not well explained is that "FoundationDB has been tested with databases up to 100 TB".

In their data modelling pages they mention you should break up rows into separate keys per column. Or separate keys per field in a document. This is indeed how many databases model rows on a distributed kv store. So this might be how they achieved 100TB.

However you still have the issue of any single key-value needing to be in their limit. (But it's not like people typically store enormous blobs in Postgres or MySQL either I think?)

The documentation is woefully out of date, sadly. Despite the code being in active development no one is touching the public docs. Though I don’t know for sure, that limitation was probably written something like 10 years ago.

ScyllaDB discontinued it's free and open source version, so I personally wouldn't build anything new on it.

Reply on news.ycombinator.com