Secondary indexes in "distributed strongly consistent" systems is what ruins performance: because each index is +1 write to another "index table" (... +1 raft quorum check!).
I don't think FoundationDB has "secondary indexes" to begin with, so one may never run into the +1 write per index issue.. it's just a layer on top of TiKV (equivalent to RocksDB in CockroachDB).
Calling FoundationDB "just a layer on top of TiKV" is… well… :-)
Can't secondary indexes just be implemented in your state machine?
I.e. your state machine handles the insert and also writes a secondary index at the same time. Every state machine could do this identically off the log.
Secondary indexes are implemented as an additional smaller table in most databases (Index -> Primary Key), which requires a write, too.
Don't take my word for it- run some benchmarks yourself to see the performance cliffs appear in these "distributed strongly consistent" databases when you have a handful of indexes.
It's why many require 10x the hardware to scale past the performance of a single node postgres / mysql.
> Calling FoundationDB "just a layer on top of TiKV" is… well… :-)
FoundationDB recommends running TiDB ontop to get an SQL layer...
You might be mixing up projects here - TiDB/TiKV are completely separate projects from FoundationDB. I don't think FoundationDB has a supported SQL layer.
> Don't take my word for it- run some benchmarks yourself to see the performance cliffs appear in these "distributed strongly consistent" databases when you have a handful of indexes.
I'm not saying secondary indexes aren't a performance cliff. I'm saying I can't think of a reason they must require additional quorum on top of the initial message that would require a secondary index update.
Since you said:
> Secondary indexes in "distributed strongly consistent" systems is what ruins performance: because each index is +1 write to another "index table" (... +1 raft quorum check!).
This depends on the specifics of how your database works, of course. I’m not sure this would involve multiple consensus quorums in FDB for instance, given that I believe it instead relies on a centralised timestamp service.
There are some well supported document interfaces like the official record layer or Tigris though.