Hacker News Reader

Comment by eatonphil | Hacker News Reader

Well for one, if it's in Go, it wouldn't need Cgo.

The same author ("cznic") has also built a C-to-Go compiler/transpiler and used it to convert SQLite to pure Go source, so that it doesn't require CGo: https://gitlab.com/cznic/sqlite ... the converted Go code is pretty much unreadable and it's somewhat slower than SQLite via CGo. But a pretty neat idea! I use it in one of my projects and it works well.

I saw that recently. I would be interested in using it but it wasn't clear what level of compatibility it has or how many tests it's passing/not passing.

Yeah, fair concern. I saw a while back (looks like Aug 2020) that he got all 928,000 SQLite tests passing -- search for "0 errors out of" on this page: https://pkg.go.dev/modernc.org/sqlite#section-readme

Wow! I'd almost trust it more if only 99% of tests were passing rather than 100% of tests passing. :)

Hehe. Well, if you look at the previous release it shows 99.7% percent of (a smaller number of) the tests passing. So you've got the best of both worlds. :-)

Nice. I'll have to give it a try.

I think there are many tests used for testing sqlite that are not public. Hipp said so in a podcast I recall.

There's TH3[1], and you indeed need to purchase a license for that. The price is listed as "call"[2], so I guess it's not cheap.

[1]: https://sqlite.org/th3.html

[2]: http://www.hwaci.com/sw/sqlite/prosupport.html

Nice. I tried to cross-compile a Go app using SQLite for darwin/arm64 a couple months back and couldn’t figure it out for the life of me. Ended up just shipping a darwin/amd64 binary and letting Rosetta handle it. Will give this a spin when I have time.

I should write a blog post this but here’s what I do in GitHub Actions for a Go binary that uses SQLite: https://github.com/crossjoin-io/crossjoin/blob/main/.github/...

Not sure why this is downvoted. SQLite is great and all, but not depending on FFI is really nice for a lot of things (e.g., cross-compiling is a lot simpler). I'll trade quite a bit to avoid even the very highest quality C dependencies.

It's pretty easy to statically link SQLite[0]. Are there other disadvantages to CGO?

[0]: https://www.arp242.net/static-go.html

Yes. It doesn’t play nice with goroutines and Go’s concurrency model, from what I understand.

Yes, function calls across the boundary are slow. It's fine if the functions do lots of work, it's very expensive for function calls that should be cheap.

For sqlite it might not matter too much, since any query does quite a bit of work.

Agreed. I wouldn't be surprised if a pure-Go alternative would be slower. Someone remarked upthread that the same author wrote a C-to-Go transpiler specifically to make a pure-Go sqlite and it was (somewhat unsurprisingly) considerably slower than the CGo version. It's possible (and to an extent, likely) that the transpolar was generating suboptimal Go code, but I would wager that a hand-written version would also be slower.

A query may do a lot of work but iterating over many results could be very slow.

Yes, the API is probably suboptimal for Go. It's better to make a wrapper that does the query and fetches many results in one call, into a Go buffer.

Without additional configuration cgo breaks cross-compilation.

Oh right how could I forget that.

There is pure go port of SQLite

There are a few go libs now that offer SQLite without cgo.

https://github.com/crawshaw/sqlite https://github.com/zombiezen/go-sqlite

crawshaw/sqlite actually uses CGo -- it's a wrapper around the C version of SQLite. For example, see https://github.com/crawshaw/sqlite/blob/23d646f8ac00d9dd2390...

zombiezen/go-sqlite uses cznic's pure Go converted version of SQLite, so avoids CGo. It's explicitly stated to be "a fork of crawshaw.io/sqlite that uses modernc.org/sqlite, a CGo-free SQLite package. It aims to be a mostly drop-in replacement for crawshaw.io/sqlite."

Ah yes my bad

Reply on news.ycombinator.com