State of Multicore OCaml [pdf]
kcsrk.info
kcsrk.info
We're going to phase it into 3 distinct phases. First, we will upstream the multicore runtime so that it co-exists with the current runtime as a non-default option. This will give a chance to test and tune the new GC in the wild. Once we are happy with it, it will be made the default.
The second PR would be support for fibers -- linear delimited continuations without the syntax extensions (exposed as primitives in the Obj module). This will help us test the runtime support for fibers and prototype programs.
The last PR would be effect handlers with an effect system. Any unhandled user-defined effects would be caught statically by the type system. The effect system is particularly useful not just for writing correct programs, but to target JavaScript using the js_of_ocaml compiler where we can selectively CPS translate the effectful bits à la Koka. As a result, we will retain the performance of direct-style code and pay extra cost only for the code that uses effect handlers. In the future, we will extend it to track built-in OCaml effects such as IO and mutable refs. At that point, we can statically distinguish pure (as Haskell defines it) functions from impure ones without having to use Monad transformers for the code that mixes multiple effects. I'd recommend Leo White's talk [0] on effect handlers for a preview of how this system would look like.
[0] https://www.janestreet.com/tech-talks/effective-programming/
Or perhaps port or create a system similar to OTP using the effects, fibers, and threads.
Rust is beginning to understand that they need more stability and LTS versions, and libraries are blossoming nicely.
Ocaml already has a very mature module ecosystem and is now becoming safe and modern.
I think rust will still have an edge in adoption due to its portrayal as C++ unfucked, but ocaml is definitely the easier tool to work with, imo. And maybe that will change.
I don't think even linear types and multicore would be enough for ocaml to make any significant dent in the systems programming world. Rist and C/C++/D/Zig all do memory management too conveniently, and it opens doors too close to the bottom for ocaml to keep up.
Any ocaml hackers: would you want to write system drivers in ocaml? Why/ why not?
What should I learn if I want to develop desktop apps and I like functional programming? I feel like most of the cool 'new' programming languages are OOP (rust, go, scala).
really looking forward to seeing where they go with it!
If you like windows but also want cross platform, C#/F# and Xamarin.Forms or UI or whatever its called. I've heard very positive things about dotnet gui work these days.
I'm spending time with rust to try my hand at game development. As an older developer, it's all a bit foreign to me so I don't really know what's good and what's not.
That having been said, desktop development is something the rust community at large is dedicated to ant interested in. There's already a fee libraries, like Conrod for Piston, gtk.rs, etc but I don't know how these options stack up against industry standards.
C# 7 is a lot more functional and pleasant than the C# 4/5 everybody dismissed years ago.
F# can compile to wherever you need it and you can write functional first and just include it in your nuget built just like you normally would.
If you're adamant about functional, consider dotnet. Of course C# is still very much OOP, that's just what it is. But I find the primary benefits to FP in my own code to be more related to correctness, testability (and therefore more correct) and a convenience in modelling data structures quicker than subclassing. Even with generics, having unions and record types makes it just that much more satisfying.
F# actually has a little micro-discipline called domain-driven programming. Search for that term and you can see how different researchers have used F#'s incredibly flexible type modelling to adapt to their domain and make their code easier for people in that domain easier to understand.
Hope this helps!
That's not a very good way to characterize functional programming languages. Obviously you can do functional programming without type inference (see Lisp), and type inference is similarly applicable to non-functional languages (see OCaml's purely imperative sublanguage).
Also, as far as I can tell, Rust uses (a modification of) the Hindley-Milner algorithm to infer lifetimes, not types. As languages in the functional camp typically don't have lifetimes, this argument is a bit strange.
True, I had forgotten that you can just write let foo = bar. (Not a regular Rust user.)
For what it's worth rust and go are not OOP languages. Scala and Swift (to add another new, hot language) have inheritance and OOP features for ffi with their platform langagues java and objective-c respectively. However, the OOP features are largely discouraged and their type systems have more in common with haskell and other strongly typed functional languages. Rust has basically the same type system as haskell and swift with its famed borrow checker on top. Scala, Rust, Swift are all mixed paradigm with their community's' preference for functional design in roughly that order. Go has a totally different type system from the others and is almost entirely an imperative langauge.
I think Go works best for applications that don't need any "internal plumbing". For example, one of my projects is a query engine. It takes an AST and plans a pipeline of operators that compose together (map, filter, join etc.). It's pretty damn awkward to write in Go, and type safety often goes out the window. It's something that in other languages (Haskell, OCaml, Rust, Scala and Swift would all qualify) could be expressed with an elegant smattering of union types, generic functions and monadic chaining.
But that awkwardness is because the app is 99% framework (various types and functions being used to compose stuff together), most of which is a big, generic factory to produce a small engine. If your app is all imperative "meat", with little framework, then Go is a much better fit.
I found myself in that space for a while, but everything kind of made sense when the "Erlang is the most object-oriented language in the market" meme clicked into place: a (micro-)service, an Erlang-style actor and an object are almost the exact same unit of abstraction, applied in different ways. The model is always the same: you communicate by message passing — and, for the most part, you want your objects to be given orders rather than be asked questions.
When I went back to Java, it became evident how damaging the "everything is an object" mentality of Java really is — it really pushes people towards a whole bunch of anti-patterns. One obvious example is that structs are not objects, and you shouldn't try to make them be. Getters/setters are a clear sign that your entity is a struct rather than an object — if you're at that stage, stop trying to apply OOP principles around its design. Another recurring problem is abusing inheritance, and underusing interfaces. I've written maybe 2-3 abstract classes over the last few years, and, when I code review a PR with any abstract classes (or inheritance of any sort) in it, I can almost always get the original author to factor it away while agreeing that it was a problem in the design.
I suggest giving Kotlin a try. Data classes are as close to structs as you'll ever get in the JVM (until Valhalla ships), delegation is crazy easy to achieve, inheritance is heavily discouraged (by having everything be final by default), bare functions and extension functions help keep interfaces clean, and the lambda situation is much better than Java's (receiver lambdas are the language's killer feature, IMO).
If you're thinking of functions operating on immutable objects and returning new objects then that is not typically what we refer to as a OOP pattern though technically it may fit the definition.
More universities should spend time teaching BETA, Self, CLOS, Smalltak, component based architectures.
Yea but don't forget that closures are a poor man's objects!
-- Guy Steele
Or F# with WPF, though F# is also OOP, but for some reason I feel its less so then Scala, and forces more functional constructs on you.
ClojureScript with Electron can also be an option if you're okay with Electron apps.
Finally Haskell and reactive bananas is an option too, that's pretty strongly functional.
I'll also mention Red, red-lang.org, its still in alpha, but is pretty sweet for simple GUI app on desktop.
If you don't mind your app working through Electron or some other variation of a web-view, there's a lot of resources for those.
Since it's basically OCaml, it has objects, but it's mostly a FP language.
Because significant portion of your learning/cognitive effort you will be spending on understanding the UI paradigms offered by a given GUI toolkit.
So take a look at the languages, in language bindings available for
1) Qt UI toolkit http://wiki.qt.io/Language_Bindings
2) at the languages, in the language bindings available for wxWidgets UI toolkit
https://en.wikipedia.org/wiki/List_of_language_bindings_for_...
2) And same for FLTk http://www.fltk.org/wiki.php?LC+P139+TC+Q
Perhaps, also, you can also investigate if you can use F# (equivalent somewhat to ocaml), to develop Windows Apps with https://github.com/Microsoft/react-native-windows
Same for Nim https://github.com/andreaferretti/react.nim (there is probably a way to package it as Electron App ).
(depending on your desire for adventure/early/small community/small ecosystem interest)
Some links:
https://fsprojects.github.io/FsXaml/
https://github.com/Prolucid/Elmish.WPF
https://github.com/GtkSharp/GtkSharp
Starting a new project in WPF at this time doesn’t make sense.
So they are getting some newly found love on their way to the store.
Microsoft has realised that the best way to force devs into the shop is to merge Win32 and UWP worlds.
I code mainly in C# but I as F# for its REPL for some data manipulation tasks.
You can also check out Racket.
Ocaml is of course already fast. INRIA has been using ocaml for compiler hacking for several decades. The speed comes as no surprise, but eventually rust will have inline assembly support, it's already nurturing some basic SIMD features, it is already being turned for lower level dominance.
I think ocaml could compete, but not the way it is now.
With linear types in the future, perhaps ocaml could follow rust's lead and implement a memory model around move semantics.
I'm not an expert in this area, but I do perceive a nontrivial gap here. Ocaml is not built for writing system drivers. And a good system driver is written in something that was designed to be used for doing so, imo.
https://svn.inf.ethz.ch/svn/lecturers/a2/trunk/
Mirage TCP/IP drivers
In terms of throughput, there's nothing wrong with a garbage collector, but in terms of latency (particularly 95+ percentile) you're never going to be able touch 'manually' (I'm including Rust here) managing the heap. Even Azul's 'pauseless' gc only guarantees something like 10ms max pauses. That's an eternity for a lot of use cases.
(The above assumes that GC is only triggered by allocations. Many GCs are like that, but others trigger periodically even if there isn't anything to do. I don't know which group OCaml is in.)
And even if the GC is only triggered by allocations, they'll still pause the other contexts in order to scan their stacks for live references. So, in the case of a unikernel, you can end up with critical paths being paused by allocations outside the critical path.
If the GC is only triggered on allocations when there isn't enough free heap space, then why would stack scans happen at other times?
What I'm saying is that the necessary heap allocations happening in other contexts will block progress of your non allocating critical path, irq code on a GC based unikernel because of the need to discover liveness information. There are potential schemes to fix this, but they aren't implemented by MirageOS/OCaml, or most other managed unikernel environments I've seen.
Fair enough. I was thinking of low-level things like "shovel a few bytes from some hardware component into a user-provided buffer" without allocating anything, you seem to be working on more high-level drivers.
Computing history has lots of examples, all the way back to Mesa/Cedar workstations at Xerox PARC.
Android Things user space drivers are mostly written in Java.
shared_ptr is rare in idiomatic C++ code, and is only used when you actually have a shared object, or need a cyclic data structure with no clearly identified root. I wrote plenty of modern C++ with not a single shared_ptr in it.
Do you have any resources on this? I am learning Ocaml and am interested to see what is changing.
"ocaml has a healthy module system and lots of module authors and maintainers and recent developments suggest ocaml the language continues to move towards more safety and more modern, convenient features."
Linear types are HUGE imo. I'll be playing with ocaml a lot when those comes out. I can't wait to see how far ocaml wants to chase other languages into the memory safety/ correctness rabbit hole. It's great.
Wercker only just got the ability to run jobs as a non root user. It's been painful.
Not sure what you are speaking about. I'm using opam as root in containers everyday. It prints a warning, but apart from that works OK. What version are you using?
In some ways, OCaml's problem mirror that of Erlang. The gnarly syntax is largely being addressed by Reason (much like Elixir does for Erlang), but I don't see it catching on as much as I'd like, and it still has some warts, I think.
As for killer apps, distributed data processing is something that OCaml could be great at, given that it marries Erlang's functional style with a rich type system, and there was a minor wave of libraries (for concurrent, distributed actor programming) way back in the 2010 or so, but those libraries are now dead and nothing really happened. Meanwhile, Scala largely leads that story (Akka is very popular, and Scala also seems to be the language of choice for Spark, Beam, etc.) and Haskell now has Cloud Haskell, which is modeled on Erlang. Also, we kind of need multicore for this.
(Distributed data is an area where I hope the Java/Scala world gets competition soon. A lot of people, I suspect, would like this, so there's an opportunity to rapidly gain mindshare. I don't see much happening there. Pony is promising (e.g. Wallaroo), and some people have had success with Go (Pachyderm). I don't know Pony, but without genetics, Go is a pretty awkward fit for data pipelines; witness the number of hoops jumped, and resulting limitations, in the Apache Beam SDK for Go. Spark et al rely on distributing bytecode to worker nodes, something you just can't do in Go. Not sure about OCaml.)
Between Opam [1], Dune[2] and utop[3] the "new comer experience" has been really good so far. The editor integration with vim/emacs has been top notch, and visual studio code has turned into a really nice option as well for people who don't like vim/emacs. I'm sure there is still room for improvement, but i'd say its atleast heading in the right direction!
[1] https://github.com/ocaml/opam
[2] https://github.com/ocaml/dune
I don't remeber anyone on those assignments ever complaining about syntax.
Also, nesting pattern matches is a bit ugly:
match x with
| Foo y ->
match y with
| Yes -> ok ()
| No -> not_ok ()
| Bar z -> whatever()
match x with
| Foo y ->
begin match y with
| Yes -> ok ()
| No -> not_ok ()
end
| Bar z -> whatever()
Some people would also really like to write
let x = foo
let y = bar
x + y
let x = foo in
let y = bar in
x + y
A bunch of these things look a lot cleaner in Haskell. Although it does it by being indentation-sensitive, which other people love hating.
The other criticisms are the ones worth attending.
i though rust has been stable since 1.0. am i wrong?
It's not that rust code is breaking, that's not it. It's that with new things happening every six weeks, if I wanted to write something that 100 developers will work on, how would I do that? Where I would begin? What version should I choose to ensure everybody is on the same page?
Once 2018 edition is polished up, this will be mostly a solved issue, so kudos to rust team.
But yeah. This is an ocaml thread anyways.
Actually there was a long thread on Rust internals forum where they worried people may mistake Edition as LTS, but conclusion was that confusion can be prevented by explicit messaging. Looking at your post, I think worry was justified.
It just needs a cargo equivalent. The existing tools weren't sufficient last I checked.
(Haven't used cargo, and only briefly touched OPAM. Don't know the feature gap, but am curious to learn more)