That's how it's done in the rust stdlib alternative https://github.com/brurucy/indexset

Faster reads, slower inserts, but then you get the capability of indexing by position in (almost) O(1). In regular B-Trees this can only happen in O(n).