Got it! One more question if I may: in https://arxiv.org/pdf/2607.26356, all the JOB queries are around a second or less, if I read correctly.
My understanding is the query:
movie
.with(keyword.eq("my-kw"))
.select(title)
for movie in 0..movie_count {
let lo = keyword_offsets[movie];
let hi = keyword_offsets[movie + 1];
for pos in lo..hi {
let kw = keyword_ids[pos];
if keyword_text[kw] == "my-kw" {
emit(movie, movie_title[movie]);
break;
}
}
So there's no index lookup on keyword, right? E.g., to use a hash index on keyword to find the resulting movie rows. If I wanted to do so, would I re-normalize the data in some way? This is what surprised me: that even without the secondary indexes, it is still the same (or more) performant than DuckDB.
Perhaps the index-lookup version would use something like
let movies_by_keyword: HashIdx<_, _> =
keyword.text().inv().collect();