Honest question: isn't Go a bit of a weird choice for a database given the GC? Why not C/C++/Rust?
Go GC creates problems and they are non-trivial (as all memory management) but it's not so twisted as in JVM.
Examples:
- DGraph: migrated from RocksDB (C++) to Badger (pure go, developed themselves, based on jemalloc)
- CocroachDB - pure go for storage engine
- go 1.20 proposed feature: package `arena` for region-based memory management
Seems like this: https://www.cockroachlabs.com/blog/why-go-was-the-right-choi... and more specifically this: https://www.cockroachlabs.com/community/tech-talks/challenge... would be good to explore for how cockroach has made it work w/ Go+GC.
Sure, but premature optimization is the root of all evil right? Get something functional, then make it performant by handling the problematic cases. GC doesn't prevent this progression, it's just a significant help to getting something functional.
GC can sometimes lead you into designs that can't be optimized without a change in abstraction, but it's not at all obvious that you would have landed on the right abstraction if you didn't have the GC to begin with anyway. I think getting something working as fast as possible lets you gather the data you need to make a performant design, and having to handle all of that complexity up front without a GC just delays the data gathering stage.
It is not premature optimization but an intentional design choice, and a valid one to make.
This is mostly a myth.
Making sure they're aren't a bunch of alloc's or GC spent in the hot path of a high performance library is hardly a premature optimization.
If you're tackling problems in DB land changes are you already have a fairly clear picture of what needs to happen and have a list of shortcomings you're trying to avoid. Memory layout, buffers, wals, etc are all things that should be accounted for upfront.
You often don't know the what the hot paths are.