It's not so much a secret as we've been rather busy, and I haven't had time to clean up the repositories to the extent that I'm comfortable making them public. Hopefully real soon now, and absolutely before we ship the hardware to anybody.
> different "distros" of illumos would have different kernel ABIs
We do have strong guarantees around various kernel API/ABIs, which one can absolutely use to produce out of tree modules that will work over the long term and across distributions. We call this the DDI, or Device Driver Interface, and it's documented as public and stable in our manual.
The vnode layer bits that FUSE uses are ostensibly sort of public but not documented. This is of questionable utility, to be honest, but it's apparently there. A mistake was likely made in adding support for inotify to a particular fork of illumos, and a struct member was added in the (again, wishy-washy) "public" preamble of the vnode struct.
> The biggest question to me is why Helios is using the FUSE kernel module from OmniOS
This is my fault. Before a distribution can be self-hosting, it needs to exist at all. I am a big fan of OmniOS so I had bootstrapped the first Helios bits by building modified OmniOS bits on OmniOS, and then massaging the packaging metadata until it looked how I wanted it for Helios. I then installed a Helios machine from those packages and everything was subsequently built on Helios from then on.
Regrettably I didn't realise the business with the vnode stuff, or even that we had a FUSE package. It has not been rebuilt since that original bootstrap build, so it was built with the wrong vnode.h. In my defence this is the first time in two years anybody has tried to use FUSE on Helios so nobody else noticed either!