Hacker News Reader

Comment by carbocation | Hacker News Reader

Basically a throwaway comment that is somewhat off topic, but your current Github README is very "inside baseball." I think it would be worth a one-sentence introduction to what your library can do (allows an app to incorporate a database, and it also has compatibility with a very popular library called SQLite, etc), then to tout the benefits of not requiring CGO, etc.

View on HN

I think when you create a website (write a document, write a README) it is important to imagine the top 3 reader/visitor personas. What do they want from the README (website)?

  1. Go+SQLite user: No CGO
  2. Go user
  3. Wanting to use the library
  4. Someone following a link from HN
  5. ...

Agreed, although I think a brief one-line intro to what this thing does wouldn't really interrupt the flow for any of the above use-cases. Right now, the top of the README is a list of sub-libraries, listing the library first.

Personally, I would add ~2 sentences in the order I outlined, and then I would reorient the list (currently library-first) in a use-case first arrangement like you mentioned.

I mean, realistically my git repos are a mess, so who am I to talk? But this one looks promising enough such that it's worth offering a bit of feedback.

Agree,

4. Someone following a link from HN

I'm often confused when clicking through from HN and can't figure out from the website or README what the project is doing.

Thank you for the feedback! I tried to improve the copy a bit, but this is clearly not my thing…

Sometimes you want people who don’t get it to bounce.

Keeps the signal to noise ratio higher.

I'd like to read what the actual benefits of not linking to C sqlite implementation are. I can imagine using wazero.io for libraries, but I fail to understand _why_ would anybody prefer to run sqlite that way from their own go program, except if it's in cases where execution times are clearly not important (which explains to me the background story).

The major one is (cross) compilation and portability.

You can (e.g.) easily whip up a small CLI that embeds an SQLite database in the binary [1], cross compile this to all the platforms Go supports [2], copy one file, and be reasonably sure it'll run in any of them.

This is an example of embedding a database in a binary (with go:embed), and opening it for reading/writing in memory. You can then save the modified copy to disk using the backup API:

[1]: https://pkg.go.dev/github.com/ncruces/go-sqlite3/vfs/memdb#e...

[2]: https://gist.github.com/asukakenji/f15ba7e588ac42795f421b48b...

I'm still missing some details to understand that story, as I don't believe that there is any platform that Go supports on which C sqlite would not run, more the opposite, the memory requirements of wazero.io way must be clearly higher than of the C sqlite library?

It seems to me it's not about would C sqlite run or not on that platform but that there is still some other convenience, and that's what I'd still like to learn, as I'm sure ncruces does solve some specific problem.

To compile C for a given platform, you need a C compiler for that platform. Which definitely exists but you may not have it at hand.

To cross compile from your platform A to that platform B, you need a cross compiler, which may be even harder to come by.

If you're working on a Go project, you already have a Go compiler that can do this (cross compilation) very easily, out of the box.

But if you add a C dependency, you also need a C cross compiler. SQLite is easy to configure, and very portable, but this still adds some friction.

Then if you want to setup CI, you need all this infra in CI. If you want to automate releases for a bunch of platforms, you need all of it in your release process.

I'm not saying it's a huge pain but it's enough that some projects avoid cgo as a matter of principle.

Many thanks for this good answer, that level of detail is exactly what I have been missing up to now.

I'm surely aware of the fact that C cross-compilation is often very clumsy implemented. But I also have to mention that there is a project that also tries to solve that too, while also creating a whole new language:

https://zig.news/kristoff/building-sqlite-with-cgo-for-every...

it's also worth noting that even on the same platform, compiling with CGO enabled creates a dynamic link against `glibc`, which creates a dependency on having the matching version in your deployment target.

We ran into this recently as our CI system is running a newer OS than we currently deploy to - and disabling CGO was an easy way to sidestep this requirement

Is this something inherent in CGO implementation? If your dependency happened because the compiled library depended on it, do you know that that is probably avoidable (I haven't tried your use case, so I could be wrong) by using zig as a compiler as it can "cross compile" for lower glibc version and also to musl, avoiding glibc altogether?

With zig I think you can avoid libc deps on Linux, at least, by statically linking musl instead.

Not sure SQLite is tested in this configuration, for the fans of “this didn't pass TH3 so we can't trust it at all.”

AFAIK: you also link to a different (lower!) glibc version from the one on which you do the compilation, probably solving the problem you had without having an excuse that you linked an "untested" library.

I’m not 100% sure why you would like to run SQLite in your go program this way (using WASM) but I think avoiding cgo in the go community is a common thing for all the reasons mentioned here [1] (or perhaps even more directly, because that article exists :-p).

—-

1: https://dave.cheney.net/2016/01/18/cgo-is-not-go

Reply on news.ycombinator.com