To cross compile from your platform A to that platform B, you need a cross compiler, which may be even harder to come by.
If you're working on a Go project, you already have a Go compiler that can do this (cross compilation) very easily, out of the box.
But if you add a C dependency, you also need a C cross compiler. SQLite is easy to configure, and very portable, but this still adds some friction.
Then if you want to setup CI, you need all this infra in CI. If you want to automate releases for a bunch of platforms, you need all of it in your release process.
I'm not saying it's a huge pain but it's enough that some projects avoid cgo as a matter of principle.