There are a few libraries out there; Async_parallel is one: https://blogs.janestreet.com/async-parallel/
Lwt also has the Release framework which does multiprocess: https://github.com/andrenth/release
and more if you search through OPAM.
1,549 karma · joined September 17, 2010
Lwt also has the Release framework which does multiprocess: https://github.com/andrenth/release
and more if you search through OPAM.
Mark Shinwell is working on just this; he has a very impressive branch of OCaml which can do OCaml-level source backtraces in gdb; WIP tree is here for anyone who wants to glance at the patches: https://github.com/mshinwell/ocaml/compare/ocaml:4.02...4.02...
Also a nice talk from a couple of years ago by him on some tips and tricks for using gdb/OCaml: https://www.youtube.com/watch?v=NF2WpWnB-nk
While I obviously can't comment on the details of the course you took a decade ago, I do have to disagree vehemently with this. OCaml is close to as ideal a systems programming language as I've ever used since:
- it has a simple, stable and predictable runtime with strict evaluation semantics, so you can identify what statements allocate memory by inspecting code.
- the compilation to native code is fast and Unix-like, with standard object files generated that can be linked to and from C programs easily.
- the memory representation of values is uniform and native code debugging with gdb/lldb works great, thanks to the DWARF symbols emitted.
- OCaml's basic language features such as algebraic data types and exhaustive pattern matching are fantastic for both building low-level programs, and refactoring them in the future as they inevitably get more complex.
> not for simple tasks which are primarily IO and need to handle many cases.
And this is exactly where libraries like Async or Lwt are so useful, since you can juggle millions of concurrent threads and still do high-level programming without getting lost in the noise. Error monads, failure monitors, function wrappers to ensure resources get freed, can all be built directly within the language itself using the usual ML abstractions, and the compiled code is still lightweight and natively compiled.
For those interested in reading more about the runtime system in OCaml, the entire third part of Real World OCaml is dedicated to this topic: https://realworldocaml.org/v1/en/html/pt03.html
The current thrust of the effort in the TCP stack is to make sure that we cover all the corner cases, and build a functional testing framework to check regressions and protocol traces versus other implementations. It's also quite remarkable how thin on the ground test suites are for TCP...
Once all this is done, then I have an alpha-grade multipath TCP implementation to merge in, and defences like SYN cookies will be parameterised options that can be activated in a unikernel in response to traffic surges.
[1] https://www.schneier.com/crypto-gram/archives/1998/1215.html...
This works pretty well in practise, and is quite similar to Erlang's monitor model. It's verbose and undesirable to handle _all_ exceptional cases in a distributed system explicitly via checked exceptions, and using monitors to catch exceptions that we're not expecting and do a process kill/log/restart is a good model in my mind.
OCaml gives the language flexibility to migrate to this as well, which is nice (since, as you note, we'll never get all the third-party code we depend on to be exception-free, and nor do I think that is a desirable goal). Core and Async from Jane Street also exhibit this split: Core tends to use On_error to explicitly handle errors, and Async (being used for distributed programming) instead adopts a "raise-exception-and-catch-it-in-a-monitor" model. Theres more on that in the Async chapter in RWO: https://realworldocaml.org/v1/en/html/concurrent-programming...
https://github.com/mirage/mirage/blob/master/types/V1.mli#L2...
type error = [
| `Unknown of string (** an undiagnosed error *)
| `Unimplemented (** operation not yet implemented in the code *)
| `Is_read_only (** you cannot write to a read/only instance *)
| `Disconnected (** the device has been previously disconnected *)
]
val read: t -> int64 -> page_aligned_buffer list ->
[ `Error of error | `Ok of unit ] io
The core is as simple as:
type _ typ =
| Type: 'a -> 'a typ
| Function: 'a typ * 'b typ -> ('a -> 'b) typ
I talked about this usecase in the ML Workshop in ICFP this year; video here: https://www.youtube.com/watch?v=oKJ8Sxqr4r8
Some blog posts here: http://openmirage.org/blog/intro-tcpip
It's worth bearing in mind that OCaml is really two distinct languages: the core ML-style language (which is concise due to type inference, and I think what the above poster is referring to by "pretty functional-programmey"), and the module language (which is verbose and signatures explicitly written down with no inference).
The latter is really the key to constructing large scale systems like Mirage, since they allow fragments of OCaml code to be mapped onto everything from Xen unikernels to JavaScript. Modules can be higher-order (that is, depend on other modules). You can read more about it at https://realworldocaml.org/v1/en/html/functors.html
It's also not yet clear how well Rust's ownership types will scale in bigger systems, but I'm sure we'll find out as the language stabilises. OCaml is a stable, fast and mature implementation of ML that has been around for years.
Jerome Vouillon's Android repository looks like it does most of the hardwork to get a userspace version running at a minimum... https://github.com/vouillon/opam-android-repository
See the related discussion on packaging it in the OPAM pull request: https://github.com/ocaml/opam-repository/pull/3083
- http://andrewray.github.io/iocamljs/oh261.html (H.261 video decoding)
- http://www.ujamjar.com/hardcaml/ (HardCaml RTL hardware description language, similar to Chisel in Scala)
- http://gazagnaire.org/fuconf14/ (building a 2048 game implementation)
One of the nice things about OPAM (the OCaml package manager) is that it can recompile upstream library dependencies if some new optional features gets inserted downstream. In this case, Thomas has put together a "lwt.profiling" library that adds profiling hooks, and then every upstream Mirage library gets systematically recompiled against this new functionality.
This way we can have profiling-free production code, and then just "opam install mirage-profiling" and have a coffee while everything gets instrumented. This is going to be incredible as we hook up more complex distributed unikernel-based systems.
We've also started putting Windows support in place via Appveyor: https://github.com/ocaml/opam/blob/master/appveyor.yml
It just can't be activated by default since GitHub doesn't support multiple commit updates yet, so we have to chose between Travis and Appveyor. When GitHub sorts that out, we can have Linux, OSX and Windows tests for all new packages. In general, help porting packages to Windows is very welcome from interested parties.
For problems where objects and classes are a good fit (e.g. something requiring open recursion), then OCaml objects are fine and sound. They are used quite nicely in CIL for implementing the visitor pattern, and I like them in the Lablgtk OCaml bindings as well. They also provide a nice bridge to foreign code in OCamlJava and js_of_ocaml when compiling to platforms that make more use of objects.
It's worth noting that class types and object types are distinct in OCaml, so you can often use objects without involving classes at all. This is explained in Real World OCaml: https://realworldocaml.org/v1/en/html/objects.html
Not using OCaml because it possesses an optional object and class extension is a little strange. It never really gets in the way...
http://opam.ocaml.org/blog/opam-1-2-pin/
(How to pin a development is central to the day-to-day development workflow of OCaml/OPAM users and quite annoying to change after-the-fact, so we're eager for feedback on this iteration before we bake it into the 1.2.0 release).
The OPAM blog is only about 2 weeks old, so there'll are quite a few more posts coming up as our developers discover there's quite a lot to write about :)
We've taken a pretty good shot at this in the OCaml ecosystem the via OPAM package manager (https://opam.ocaml.org).
* OPAM composes its package universe from a collection of remotes, which can be fetched either via HTTP(S), Git, Hg or Darcs. The resulting package sets are combined locally into one view, but can be separated easily. For instance, getting a view into the latest XenAPI development trees just requires "opam remote add xapi-dev git://github.com/xapi-project/opam-repo-dev".
* The same feature applies to pinning packages ("opam pin add cohttp git://github.com/avsm/ocaml-cohttp#v0.6"). This supports local trees and remote Git/Hg/Darcs remotes (including branches).
* OCaml, like Haskell, is statically typed, and so recompiles all the upstream dependencies of a package once its updated. This lets me work on core OCaml libraries that are widely used, and just do an "opam update -u" to recompile all dependencies to check for any upstream breakage. We did not go for the very pure NixOS model due to the amount of time it takes to compile distinct packages everywhere. This is a design choice to balance composability vs responsiveness, and Nix or 0install are fine choices if you want truely isolated namespaces.
* By far the most important feature in OPAM is the package solver core, which resolves version constraints into a sensible user-facing solution. Rather than reinvent the (rather NP-hard) solver from scratch, OPAM provides a built-in simple version and also a CUDF-compatible interface to plug into external tools like aspcud, which are used by other huge repositories such as Debian to handle their constraints.
This use of CUDF leads to some cool knobs and utilities, such as the OPAM weather service to test for coinstallability conflicts: http://ows.irill.org/ and the solver preferences that provide apt-like preferences: https://opam.ocaml.org/doc/Specifying_Solver_Preferences.htm...
* Testing in a decentralized system is really, really easy by using Git as a workflow engine. We use Travis to test all incoming pull requests to OPAM, much like Homebrew does, and can also grab a snapshot of a bunch of remotes and do bulk builds, whose logs are then pushed into a GitHub repo for further analysis: https://github.com/ocaml/opam-bulk-logs (we install external dependencies for bulk builds by using Docker for Linux, and Xen for *BSD: https://github.com/avsm/docker-opam).
All in all, I'm very pleased with how OPAM is coming along. We use it extensively for the Mirage OS unikernel that's written in OCaml (after all, it makes sense for a library operating system to demand top-notch package management).
If anyone's curious and wants to give OPAM a spin, we'd love feedback on the 1.2beta that's due out in a couple of weeks: http://opam.ocaml.org/blog/opam-1-2-0-beta4/
It's all good fun in library OS land :)
Interestingly, it looks like the easiest way to support Azure...