Hacker News Reader

Comment by gavinray | Hacker News Reader

One of the biggest things preventing software like SQL DB's from being written in TypeScript is the lack of proper threading.

I genuinely think you could write a competitively-performant multi-threaded DB in Bun + TS if you had shared-heap threads and fast atomics/locking primitives.

View on HN

"I genuinely think you could write a competitively-performant multi-threaded DB in Bun + TS if you had shared-heap threads and fast atomics/locking primitives."

Not likely. Databases that attain any significant use in the field end up getting optimized to the n'th degree because they're the bottleneck of the entire system of every system they get put into. Javascript runs on the "5-10x slower than C" language tier. Personally I think even picking Go, in the "2x slower than C" tier, is a huge mistake, though a few people seem to be doing OK with it. I don't think you can call it "competitive" when your C++ or Rust competition is consuming a factor of magnitude less resources.

WASM DBs, maybe, especially as it continues to mature. Not Javascript.

Something compiled to WASM still gives a fair amount of control over memory layout, something that AFAIK is not possible in JS without building effectively a new embedded language on top of an array (Emscripten being an existence proof that you can do more or less anything that way).

One place where an interpreter + JIT language could be interesting is if it were sufficiently safe to allow user code into the query execution engine, such that the JIT could optimize it all together.

Yeah I kinda agree about Go. CockroachDB didn't catch on nearly as much as it should, and I wonder how much it has to do with the fact that optimising code in Go (both in terms of memory usage and performance) is just much harder than C

You have web workers, and for shared memory and synchronisation respectively SharedArrayBuffer and the Atomics namespace.

Exactly. Nothing stops your writing a high-performance parallel database in TypeScript today. Given that runtimes and tooling are actually pretty good, I think TypeScript is actually a fine choice of language for the task.

The only thing you can't do with JS today is share a heap across threads. You have SharedArrayBuffer. You have atomics. You don't need a shared address space.

There's a high performance database called "PostgreSQL" you may have heard about. It doesn't use threads. It uses separate processes and shared memory: just like standard JavaScript, with its service workers and SharedArrayBuffer.

If not sharing an address space is good enough for PostgreSQL, it's good enough for your TypeScript database.

The problem with shared-everything, unmarked, preemptive-parallel concurrency is that 90% of the time it gets used by people who don't know they shouldn't.

SharedArrayBuffer gives you a byte array, and asks you to re-implement the entire JavaScript universe on top of that. String -> use TextEncoder to copy in / out of a byte buffer. Object/class? Sure, devise your own protocol to serialize + copy instances in/out of a byte array. Even basic arithmetic operations on a TypedArray over a SharedArrayBuffer is slow AF in v8 compared to a "native" JS Array<number> for small-medium integers, because our current JIT compilers can jit the fuck out of regular arrays but struggle with TypedArray stuff. It's so sad.

> Even basic arithmetic operations on a TypedArray over a SharedArrayBuffer is slow AF in v8 compared to a "native" JS Array<number> for small-medium integers, because our current JIT compilers can jit the fuck out of regular arrays but struggle with TypedArray stuff. It's so sad.

Yeah, that sucks, but at least it's just an implementation problem. Java's analogous ByteBuffer methods have been fast compiler intrinsics for a long time, so there's an existence proof of this stuff done right.

This sounds pretty insane if you wrote any database code that needs to care about memory management at all

Are you hoping to, like, run postgres in nodejs or something?

You can get parallelism with web workers and shove sqlite over there if you like, e.g. for running more intensive queries. Beyond that I kinda don't see much of a reason to use JS for databases, except maybe for isolation (e.g. via wasm).