Hacker News Reader

Comment by macjc | Hacker News Reader

This is probably the slowest commonly used hash table library.

https://github.com/tezc/sc/tree/master/map

For those who are interested in faster hashmaps, I tried bunch of hashmaps and this one performs better than others. This is for C. Maybe C++ has better hashmaps.

std::unordered_map is in the similar ballpark. linked lists, oom checks, integer key support.

In a "hold my beer"-moment, I've ported GNU Trove's hash map to use memory mapped files instead of allocated memory. With good reason, granted, and it's not commonly used, but compared to almost anything else, it is fairly slow.

That sounds pretty interesting (in a sick way ;) ) Are you able to elaborate a bit more what you were doing?

I'm using it for the lexicon in my search engine.

I basically need a mapping from arbitrary short strings to unique integer identifiers, with a cardinality in the billions. Too much to keep in memory, most db solutions keel over as well.

for this I would use CHM, ordered minimal perfect hashes with external mmap'ed keys. still constant lookup, just the position needs to be stored in the value also.

Problem is I don't know the items beforehand, instead I need to build the hash as I encounter new pages. I don't know any way of reconcile that with a minimal perfect hash.

ok, with trillions keys creating a CHM MPH is risky, and needs a few secs. then your external hash is good enough.

And std::unordered_map is then the slow version in C++ compared to absl::flat_hash_map or folly::F14FastMap

Care to elaborate a bit?

Reply on news.ycombinator.com