Hacker News Reader

Comment by rwmj | Hacker News Reader

I'm slightly concerned if this means changing C extensions (of which we have rather a lot).

We only have a few places that use "naked" pointers, which are in any case deprecated, and we should be able to fix those easily enough.

Mostly the rest are fairly ordinary extensions that just call C functions and use the usual CAMLparam stuff.

We do have a few places that register global roots. And several packages that do callbacks from C back to OCaml. We also use @@noalloc a lot.

Is there anything else? Since there's so much code, what should I be grepping for to find code that might be of concern?

Edit: Some examples of simple stuff:

https://github.com/libguestfs/libguestfs-common/blob/master/... https://github.com/libguestfs/libguestfs-common/blob/master/... https://github.com/libguestfs/libguestfs-common/blob/master/...

More complex stuff:

https://gitlab.com/nbdkit/nbdkit/-/tree/master/plugins/ocaml http://oirase.annexia.org/tmp/ocaml/

View on HN

You may find the “nnpchecker” configure option in 4.13.0 useful. This will print a detected use of naked pointers to stderr, which can hopefully be triggered by test suites.

Noalloc and registering roots should all be the same, as are callbacks (for sequential code, which all existing code will be)

To echo what avsm said, our intention is to preserve the C API. With the exception of naked pointers, if you follow the rules around the existing C API then your extensions should continue to work in sequential code running on 5.0.

We have a scheduled build and test of every package in opam with multicore: http://check.ocamllabs.io:8082/ to try to shake out C API incompatibilities and that's proved fruitful.

If you do find things that don't work on 5.0, please let us know (and if you can, get it in to opam so we test it automatically!).

I think the challenge will be how to convert existing C FFI/stubs to work correctly with code that wants to take advantage of multicore(domain) capabilities.

i.e. previously some code could've in theory assumed it only ever gets called by one thread at a time due to the global lock (and that lock would get dropped in a controlled manner by the usual released/acquire). I don't necessarily mean just global state in the C stubs (I don't recall seeing many of those), but implicit global/shared state in the C functions called. Reading manpages will usually tell whether a given function is safe to be called (or when not there are sometimes _r variants that are). This is no different than when writing regular multithreaded C code, one has to be more careful about what functions they can safely call. (and in fact current OCaml code can already be multithreaded, so if the C stub was meant to work correctly with multithreaded OCaml code with C->OCaml callbacks then it should've already taken necessary precautions).

I like the approach taken wrt to backward compatibility of sequential code though: only code that wants to take advantage of multicore will need to audit its C stubs for safety, if you don't use multicore everything will work as before.

How do you recommend to find these kinds of multicore-specific bugs in existing (or newly written) OCaml-C stubs? Would tools like parafuzz help here?

Reply on news.ycombinator.com