Leaving OCaml | Hacker News Reader

Leaving OCaml

blog.darklang.com

237 pointsrbanffy182 comments

> it's a high-level language with static types, so easy to make large scale changes as we figure out what the language/product was

> you mostly model data with sum types, which in my mind are the best way to model data

>it's very similar to the language I wanted to build (in particular, we could reuse built-in immutable data structures for Dark's values)

> it had a reputation for being high-performance, which meant that we could write an interpreter for Dark and not have it be terribly slow (vs writing an interpreter in python, which might be too slow)

> We have struggled to make editor tooling work for us.

> Lack of libraries

> Multicore is coming Any Day Now™, and while this wasn't a huge deal for us, it was annoying.

> we also use ReasonML

> lazyness and pure functional-ness I felt were not great

> I plan to leave keep the frontend in ReasonML

You want a statically typed functional (but not pure) language with sum types, immutable data structures, good performance, editor/IDE support, a broad library ecosystem, multithreading, a C-like syntax (I assume, since you're using ReasonML), and a nice compile-to-Javascript experience to run in the browser.

I don't want to sound too much like a fanboy, but all this really sounds like Scala. You want Scala.

If only there was a recently published high quality book that could walk them through Scala with some hands on examples

this? https://www.handsonscala.com/table-of-contents.html

You the author or something? I'm actually learning F# and OCaml and am interested in Scala as well.

I would discourage you to try Scala, AFAIK it doesn't default to immutability.

I'm sorry but that statement is simply false. Scala does default to, and encourage, immutability.

Ok sorry I might have misread it from somewhere[1], but I remember this talk: https://twitter.com/migueldeicaza/status/466209065698590720

[1] just looked at it again and while it doesn't default to mutability, it doesn't default to immutability either (i.e. to compare it with F#, it's 'val vs var' versus 'let vs let mutable', so the latter being much longer means the default/lazy thing to do by devs is immutability, in the F# case).

Honestly I think the "val vs var" debate is kind of a misdirection. The real difficulty with mutability comes from interior mutability. The scala standard library has collection.mutable.{List, Map, Set, etc.} and collection.immutable.{List, Map, Set, etc.} with collection.immutable being the one in the prelude (imported by default). In scala like any ML language that supports mutability, you can do

    val immutableValue = mutable.Map.empty
    immutableValue.insert(key, value)

In this case the distinction between "val" and "var" is at best a misdirection. The only language that I know of that explicits interior mutability is rust. Where your map must have a "type marker" `&mut` to be able to use the `insert` method. The only way to have that type marker is to define the map with a `let mut`:

    let clearlyImmutable: Map<String, Int> = Map::new()
    clearlyImmutable.insert("hello", 10) // Does not compile

while

    let mut clearlyMutable: Map<String, Int> = Map::new()
    clearlyMutable.insert("hello", 10) // Compiles :)

The conclusion here is that if you are not working with rust, immutable data structures (where no method mutates the object it refers to) are what guarantees you you won't run into spooky mutability at a distance.

However, it would be disingenuous to not mention that you can run very quickly in the wild into scala code that is just "java without semicolons". You can very deliberately break null safety in scala. For example `val x: (Int, Double) = null` does compile. You won't normally run into `null` unless you are interacting with java libs or with code from programmers who don't understand type safety.

I want to point out: scala let you do all those nasty things, but this will be a deliberate choice. The mutable data structure from the standard library are clearly marked. There is no methods in the standard library that returns null (outside of regex methods which are thin wrappers around java regexes). Writing mutable code requires a completely different architecture and design choices. If you are in an immutable team, it will be very hard to justify that kind of code. While in a mutable team, the reverse might be true.

Your rust example isn't considered interior mutability, since it needs `&mut`. Since `&mut` references are exclusive, they're quite similar to immutable values in functional languages, despite the API appearing mutable.

In Rust the 'interor mutability' term is used for types like `Mutex`/`RefCell`, `Cell`, or atomics, which are mutable even through shared references. These come with the same problems as interor mutability in scala.

OP is (Li Haoyi). I am looking forward to read his book, whenever I find some time.

Scala.js generates significantly larger bundles than Rescript — because Ocaml's semantics map more closely to those of JS, the runtime dependencies are smaller and there's a lot of tricks to minimize indirection/overhead. Additionally, interop with npm modules is much less painful with Rescript and bindings for popular third party libraries are more mature. Ignoring the merits of Scala-the-language, there are very good reasons to pick Rescript over the Scala.js toolchain.

Compare https://rescript-lang.org/docs/manual/latest/shared-data-typ... with how Scala.js represents arrays and records.

In the small, Rescript generates smaller code than Scala.js, that is true. It is mostly a question of the interdependencies in the standard Scala collection library, though, not so much the semantics of the language. And Scala.js also has its share of "tricks" (aka optimizations) to reduce indirections and overhead.

Regarding interop and the link you posted, here is the equivalent documentation page in Scala.js: https://www.scala-js.org/doc/interoperability/types.html

Yes, Rescript has more built-in types that map straightforwardly, but that is at the cost of some correctness. For example, using a JavaScript array means that JavaScript can resize it under your feet. Using `undefined` to represent `None` means that you cannot tell the difference between `None` and `Some(None)`. It's a fine trade-off to make. Scala.js happens to make the other trade-off, and offers separate types for JavaScript interop.

I like the concepts and design of Rescript, really. It's very interesting because they have all the essential requirements right (IMO) like comprehensive JavaScript interop, while making all the opposite design decisions on nonessential trade-offs compared to Scala.js.

Hi,some minor corrections.

ReScript is more correct than scala, ReScript's type system(essentially borrowed from OCaml) is sound while scala is unsound.

> Using `undefined` to represent `None` means that you cannot tell the difference between `None` and `Some(None)`

I don't know where you get the impression, of course they are different in ReScript.

One thing missed is that ReScript compiler may be 10-100 times faster than Scala.js (I am not kidding)

> ReScript is more correct than scala, ReScript's type system(essentially borrowed from OCaml) is sound while scala is unsound.

Ah, I did not make it clear what I meant by "correct" in this context. I meant "faithful to the original language" (OCaml natively compiled and Scala on the JVM).

You are of course right about the type systems. (I believe many soundness holes are fixed in Scala 3.)

> > Using `undefined` to represent `None` means that you cannot tell the difference between `None` and `Some(None)` > > I don't know where you get the impression, of course they are different in ReScript.

I get that impression from the document linked above, which says that `None` is represented as `undefined` and `Some(x)` is represented as `x`, so `Some(None)` must be represented as `undefined` as well. Oh and `()` too. Do at run-time you can't tell which is which.

> One thing missed is that ReScript compiler may be 100 times faster than Scala.js (I am not kidding)

Yes, I believe that's true. It is often mentioned as the biggest weakness is Scala, by advocates and detractors alike.

I'da wagered dollars to doughnuts you were going to say Rust. I believe Rust hits each of those topics head-on, as well.

Rust doesn't work well as a purely functional language. You'd be constantly fighting the type checker for memory issues. (Or otherwise, why doesn't Haskell take the same route as Rust and ditch its garbage collector?)

Nah, just .clone() gleefully the 99% of the time that you're not in the hot path.

Most of the other cases, you can use typed_arena or bumpalo.

The other 0.1% of the time, you'd have to be pedantic about allocation anyway and might appreciate the compiler's help.

> You'd be constantly fighting the type checker for memory issues.

I think you mean borrow checker for ownership issues...

> why doesn't Haskell take the same route as Rust and ditch its garbage collector?

The Rust language was designed in a way (with ownership) that doesn't need a GC, while Haskell have many different properties (such as lazy evaluate everything) that I don't think would work well without a GC.

The borrow checker is usually a nuisance when you're writing highly mutable code, which isn't a problem for functional code. If you need to store the same data in multiple structures, the sibling is right, .clone() and .to_owned() will usually fix your problem with a very small overhead.

Rather why not F#, which has all of that but with a fast compiler and is just a superset of OCaml

And, indeed, F#, per the followup article: https://blog.darklang.com/new-backend-fsharp/

Doesn't it have fewer features?

It doesn't have a real module system.

“Fewer” is a relative term - F#’s type system is noticeably less fancy (basically generics are “weaker” than in OCaml) but having the entire .NET System library is very useful for practical programming, along with many 3rd party options. The author mentioned a lack of GCP support in OCaml as a major stumbling point, and also mentioned how nice it was that F# had a built-in immutable map. There is more to computer science than type theory - System.Collections.Concurrent is particularly cool, and hard to do in OCaml. If you know C#/Java/C++/etc some of that code is also very well-written and informative.

There are use cases where OCaml’s type system leads to elegant type-safe code that, if expressed in F#, involve inelegant unsafe boilerplate. It’s usually not a whole lot and it’s typically routine but it certainly happens. For some teams and products OCaml is a better choice.

But there’s also a reason why many developers abandoned Lisp for Python: some theoretical fanciness and robustness only infrequently outweighs ease-of-use and quality-of-life.

There is a library called py.ml [0] that lets you call Python code from OCaml. That should let you use Google Cloud's Python SDK from OCaml, although things could get unwieldy if you have to call back and forth constantly. I think it's at least worth a try, especially if the alternative is rewriting your whole codebase in Python.

[0] - https://opam.ocaml.org/packages/pyml/

> When you're searching for product-market fit, you do the simplest, easiest thing. If you lack a good SDK for your cloud provider, the simplest, easiest thing is often a terrible architectural choice.

I understand his point here. Spending week(s) writing a good SDK is hard to justify, but if you're building new infrastructure to support a project in a more obscure language, you kinda have to sacrifice time building said infrastructure well. When Naughty Dog programmed their games in GOOL/GOAL they built their own compilers, debuggers etc. - not a trivial task.

They are already writing their own compiler anyway, albeit with a limited focus (initially). Perhaps they’ll eventually get Dark on Dark.

The first negative sentiment OCaml post on HN that I see. Nice to see some perspective every now and then.

Content on HN about the holy trinity of programming languages, text editors, and operating systems generally favors less popular options with more mystique. Anything about Java is probably going to be largely neutral or negative, while a piece on Common Lisp might be much more positive, even though Java is a far more successful programming language in most ways.

I don't think that this bias is a bad thing. It's nice to read about cool technologies that you might not use in your day job. But anyone using HN to decide on the right tool for a project should carefully consider other sources of information as well.

Assume for the moment that all languages are equal. (They aren't, but ignore that for the moment.) If you say something good about Java, there are a lot of people here who have used Java, and some of them have bad impressions that they came to through some pain.

But if you say that Common Lisp is great, there are fewer people who have really used it, so fewer people who have used it and didn't like it. And since fewer people got into it unless they really wanted to, it's easier to say "You didn't really understand it, then, or you would have liked it." The number of people who can say "No, I really got it, and I still didn't like it" is relatively small.

So far, that's just demographics. But then you get into the language itself. Java isn't an exciting language. It's not fun. It's kind of corporate and bland. There aren't many people who are really enthusiastic about Java - nor does Java give them reason to be.

That's true, I also think Java is most people's job, and your job is never as exciting as your side project. That said, I am quite excited about Java and maybe especially advancements in the JVM even though part of my job is in Java.

GraalVM is doing some amazing things with polyglot support and native compilation.

Java's project Loom is trying to bring fibers in a really nice way to the language.

It is slowly working on bringing value types and inline types as well.

And it's got a few cool GCs being experimented on, as well as a nice foreign memory API.

Java recently got really nice improvements, multi-line strings, text blocks, local type inference, a shell, running code from source scripts and switch expressions.

My favorite thing about Java (well, I guess more of the JVM ecosystem) is that there is a battle tested, production ready canonical package for just about anything that you could imagine. And that there is a battle tested, canonical way to get that package onto your system and into your build .

But leaving OCaml for F# is a bit like leaving Scheme for Common Lisp. Most of the underlying ideas are largely the same.

Moving from OCaml to C++ or Go would be a drastic and surprising change, but it's not nearly like that.

Not really sure that's a close comparison. From a language semantics perspective, yes, OCaml and F# are very similar. But F# has access to all of .NET, so you aren't going to be at a loss for libraries -- which is probably the biggest challenge in using OCaml in production, as discussed in the OP.

It's (way) more like leaving Scheme for Clojure.

Isn’t it more like leaving Scheme for Clojure?

OP here. I don't mean this to be negative, this is just my ongoing developer blog to keep the Dark community appraised about what's going on.

There's some really great stuff about OCaml. I don't think we could have gotten through the first 2 years without OCaml, we repeatedly made very large scale changes to what our product was and how it worked, and that would have been 10x harder in something else.

I actually think OCaml stands to be in a great place in the next few years with the work being done by core community members. See https://twitter.com/patricoferris/status/1323330884515356672.

thanks for sharing, I find this really informative.

so if you already had experience with Rust/etc, why did you decide to choose OCaml?

Also why not to choose F#/Mono (and now .NET core) for example?

I wouldn't say I had experience with Rust, tbh (1 day in 2010 doesn't count!)

I don't remember considering F#, so probably just never occurred to me.

Expecting open source developers to write a well-tested proprietary SDK for a vendor is going to be hard ask. If you need that and unwilling to do/fund the work yourself, just stick with vendor supported languages.

I would turn this around. As a guy who was doing OSS here and there (and still got burned out by doing only a little) I completely sympathize that nobody should ever expect people to do important work for free. Absolutely.

At the same time, it's a good response when somebody muses in a saddened manner "I wish my favourite language was more popular". Then I can spring out of the bush and tell them why it isn't. This doesn't mean I expect the OSS community around the language to serve me -- of course not! But it's a good explanation on why language X isn't more commercially popular.

I want to see more negative sentiment rust and go posts.

I used OCaml for several years professionally. It's a phenomenal language in many ways, but there's no way I'd seriously consider using it for a new project at this point.

I don't really agree on the "Learnability" and "Minor annoyances" sections here. In terms of overly academic discourse, well, everyone's experiences differ but that's simply not a problem I've personally seen in the community (and I'd happily tell you other languages have that problem in spades). In terms of learnability: Ocaml, (the good parts) is an extremely straightforward language in most respects. The tutorials on the Ocaml website are enough to get you up and running, and Real World Ocaml is freely available and it's great. It'd be nice if v2 ever came out, but...well. Between those two, if you're still struggling, give me a call and I'd be happy to tutor you :)

The rest of this article hits most of the nails right on the head. The library support just isn't there, the community is too fragmented (why is stdlib vs. Core vs. Batteries and Async vs. Lwt still an issue?), the tooling is still too rough. OCaml's lack of multicore isn't actually a problem for most use-cases, but the fact that it's been in limbo for a decade despite the endless clamouring for it tells you something about what you're hitching your wagon to.

Twenty years ago OCaml would've been a no-brainer. At this point if I wanted a practical garbage-collected language I'd choose Scala and get a large and active community, as many Java libraries as there are stars in the sky, and the most sophisticated runtime environment mankind has ever dreamed of (for better or worse). If I truly need a compiled language, well...tragically, I've been bitten by the Rust bug (to the extent that I'm starting to believe its rigid approach to ownership helps engineer better code - that the ability to freely and thoughtlessly sharing references to objects the way garbage collection grants makes it a mistake, but that's a story for another post). And if I need 100% of the power and safety types can give me, Haskell is there waiting, with a better ecosystem to boot. Everything OCaml does well other languages do better, and then some. It's an also-ran now.

> I'd choose Scala and get a large and active community, as many Java libraries as there are stars in the sky, and the most sophisticated runtime environment mankind has ever dreamed of (for better or worse).

Hahahaha oh wow. You never ever used Scala, aren't you? I did, I moved from OCaml to Scala.

Oh, good luck joining Scala community then, we don't have aforementioned problems, don't we? Because Akka vs Zio vs Cats vs Scalaz is not a thing. Because ensime vs metals debacle never happened. Because we don't rewrite the complete ecosystem each couple of years.

> most sophisticated runtime environment mankind has ever dreamed of

Which you need to tune quite a lot to fit Scala into it (hopefully, it's very tuneable). Oh, and this "most sophisticated runtime" doesn't support TCO btw.

Oh, and any AnyRef value (so anything but basic types) can be null, thanks, runtime.

I love Scala, but all these "I wouldn't choose OCaml, I would choose F#/Clojure/Scala/other obscure language" usually mean "I've tried ocaml but didn't try other obscure language, so maybe it's better"

I love OCaml is well, and most of your points could be reduced to simply "it's lacking manpower" (Scala is lacking it too).

Because all of the "why is stdlib vs. Core vs. Batteries and Async vs. Lwt still an issue" we have in C, C++ and many other popular languages too, but it's not a big issue. People complain on meson/cmake/autotools/boost/glib/qt but not leave for some reason.

Maybe people don't use OCaml that much rather because there are simply too few jobs and too few programmers and it's a simple chicken and egg problem. And all these technical rationalization is not the reason because it's applicable to quite a lot of languages (safe for lack of libraries)

Yep, I completely get the "the grass is NOT greener on the other side" sentiment. I've been a victim of it many times and I am much more careful these days.

> I love OCaml is well, and most of your points could be reduced to simply "it's lacking manpower" (Scala is lacking it too).

For the better or the worse, this is literally the #1 priority one has to have before trying to technically evaluate a language / framework these days. Otherwise you get sucked on a joyful ride at the end of which you find lack of employability. :(

I personally much prefer OCaml's syntax but since Rust has much more mindshare and is mostly serving the same niche (with GC being the apples-to-oranges comparison here), I preferred to work with Rust. I still would like to work with OCaml but the odds are stacked against the financial incentives of doing so.

> Because all of the "why is stdlib vs. Core vs. Batteries and Async vs. Lwt still an issue" we have in C, C++ and many other popular languages too, but it's not a big issue. People complain on meson/cmake/autotools/boost/glib/qt but not leave for some reason.

Sure, many other languages suffer from that but it's still a very good ideal to strive for -- namely have very few (ideally one) ways of doing things. I personally look for such languages / frameworks and sadly they are very few and far between (Elixir and Clojure come to mind as refreshing exceptions and even they don't follow it everywhere).

---

In the end, we all do this for money. If I could work without charging for it then I'd likely know more languages and even work more overall, but it's not the reality we are living in currently.

> For the better or the worse, this is literally the #1 priority one has to have before trying to technically evaluate a language / framework these days.

Well, people use OCaml, I've introduced and used it in a couple of workplaces. OCaml shines in domains like system programming, and it's worth trying to introduce it for your team and see how it will end (for small projects if OCaml fits the domain).

I did that couple of times and people were glad not only because it's a delightful language to work with, but also because you can learn some stuff while working with it.

> Sure, many other languages suffer from that but it's still a very good ideal to strive for

I think that it's an unachievable ideal. There are two approach: batteries included and community driven.

Former leads to bloated slow half-baked or even straight incorrect implementations of every algorithm or parser or server people need in the standard library (python way).

Latter leads to fragmentation. Neither is perfect, and I haven't seen a language without such troubles.

> I think that it's an unachievable ideal. There are two approach: batteries included and community driven.

You know, there might be a somewhat nice middle ground: after several community driven libraries have materialized, maybe include some interfaces in the standard library, that allow users to choose which community library the want without having to hardcode too much to that library.

It's not perfect and wouldn't work for every situation, but might be a way to soften the downsides of the community-driven approach.

Well many OCaml libraries do abstract from Lwt/Async using Functors allowing you to choose library you want to go with.

But it's still not perfect since sometimes third party libraries still need to write concrete implementation of something diving into lower level, hence breaking the abstraction.

This "interface" approach wouldn't work in all cases.

> after several community driven libraries have materialized, maybe include some interfaces in the standard library, that allow users to choose which community library the want without having to hardcode too much to that library.

That's what Elixir did with calendars and timezones. Worked great.

So...I think a decent choice to make here is to switch from OCaml to F#. You'll get almost all of the benefits and most of the drawbacks go away. And for the most part, you can directly translate the code from OCaml to F#.

The big reason to not use F# vs OCaml tends to come down to the lack of functors... but it looks like your current codebase might not be using them much.

> The big reason to not use F# vs OCaml tends to come down to the lack of functors...

What do you mean by that? F# has support for functors, or are you referring to something else? https://fsprojects.github.io/FSharpPlus/applicative-functors...

Those are an implementation of functors as they exist in Haskell. The functors I'm talking about are actually a language level construct that is a primary abstraction mechanism in OCaml/SML. They are essentially higher-level modules: modules that are parameterized by other modules.

https://stackoverflow.com/a/16353711

Can someone explain to me like I'm 5 what the big deal about functors is?

Probably not, as it is a very abstract generalization for what is essentially a simple concept that for most purposes doesn't need a math-y abstraction. And it would depend on whether you're talking about Haskell, or OCaml, or Category theory.

The best I could describe the general concept is that a functor allows you to take something of some type and maps it to another type via some unspecified function.

If you mean ML (i.e. OCaml) functors, then to put it simply: they are the primary code abstraction mechanism in ML. ML doesn't have inheritance and runtime dispatch, instead it uses functors which do static dispatch.

Simple example: in an OOP language you might have a Map<K, V> class for any type K, V where K must be an instance of an Ordered interface so that the map implementation can do efficient lookup.

In OCaml, you have a Map.Make(Ord : Ordered) functor which explicitly takes a module Ord as its parameter. The Ord module implements the key data type, which must support the Ordered interface. When you (statically) apply the functor, you get back a new module which specializes a map data type to work for the specific key type. E.g.,

    module StringMap = Map.Make(String)

The String module implements the Ordered interface, so it can be used as the argument here. Now you can create and manipulate maps of strings to any values.

You might be wondering, what's the payoff? It's very similar to the payoff for interfaces--code abstraction. It's just that it's all statically resolved and highly efficient.

Sounds like this is covered by F# already, simply by the use of genercs (similar to what you described for OOP, which in that particular case doesn't really have any OOP downsides, i.e.: no inheritance, no state).

>simply by the use of genercs

It depends on what you mean by "generics". Functors are generics, but very powerful ones, akin to packages in Ada.

They are way more generic than what parametric polymorphism allows you to do in F#, they support Higher Kinded Types, module can have many type variables within it etc etc.

In F# you have List<'a> but you cant have a single map operation abstracting over Lists, Arrays, Options..... so they have to be implemented separately Option.map, Array.map.....

Not just with generics, but also with dynamic dispatch on the runtime class of the K type which implements Ordered interface. ML traditionally doesn't have dynamic dispatch, so it uses the static mechanism of building a map type specialized to a given key type. The value type is completely generic, though.

I said 'traditionally' above because OCaml actually has dynamic dispatch now; it has a powerful and feature-complete OOP implementation, and a way to pass around modules at runtime as first-class objects. And in fact people take advantage of these capabilities to build more 'modern' APIs. But I would argue that the functor approach is one of, if not the best, overall.

He's talking about functor from OCaml and SML which are basically a function in the module level. So a module which take other modules as arguments and produce a new module.

In OCaml, "functors" are parametrized modules—mostly unrelated to functors and applicatives in the Haskell sense. Think about it as a module that can take another module as an input.

OCaml functors are not the same as Haskell/F# functors/applicatives (which are essentially one and the same in OCaml if you use named arguments).

Instead, they're roughly "functions from modules to modules": https://dev.realworldocaml.org/functors.html. Functors allow you to replicate much of what classes/objects in an OO language offer, but with static instead of dynamic dispatch.

I am an F# person rather than OCaml and am not familiar with the compiler, but this seems to be overstating the difference: I thought the OCaml functors are “real” mathematical functors that implemented a structured type polymorphism over the “input type” - they look pretty similar to C++ templates superficially but they work similarly to Haskell type classes. They are “functions from modules to modules” but those functions take types as values.

Real experts should let me know if I am mistaken: I don’t think OCaml and Haskell have strictly the same type system, but they are both similarly more expressive than F# because of this specific type -> type polymorphism that in particular lets you do type-checked monads, etc.

Neither Haskell's nor OCaml's functors are the same as the mathematical one I believe, but both are similar in some way that the word makes sense to be used for their respective feature.

In OCaml, functors map between an abstract module and some concrete one. And in Haskell functors are structures that can have a function that maps over their elements. In math a functor is a map between categories. So you can see how all three kinda make sense as functors, but also are different in exactly what they are.

Both Haskell's and OCaml's functors are special cases of the general mathematical one.

How? What are the categories that OCaml's Hashtbl.Make maps between? What are the morphisms in those categories and how are they mapped?

It maps between a category with HashedType implementations as objects and a category with Make implementations as objects. Unfortunately it's not a very interesting functor because to satisfy the functor laws I think the categories have to be discrete. It's basically a function! (And functions are indeed a special case of functors.)

https://caml.inria.fr/pub/docs/manual-ocaml/libref/Hashtbl.M...

Right, in other words it is just a function. I don't think the categories have to be discrete just to satisfy the functor laws; even ignoring that, what would the morphisms be in principle? I don't think there is an obvious choice, at least.

I think in OCaml, "functor" basically just means "like a function but for modules". They might technically be categorical functors, but it seems quite different to the meaning in Haskell where they are plainly categorical functors (modulo Hask technically not being a category).

Agreed, I'm not sure it's a terribly interesting functor.

Here is one way one might implement a categorical Functor (https://www.reddit.com/r/haskell/comments/eoo16m/base_catego...).

A function S -> T maps a S(ource) type to a T(arget) type, like FunctorOf (-S>) (-T>) does between the source category (-S>) and target category (-T>)

    type  Functor :: forall (s :: Type) (t :: Type). (s -> t) -> Constraint
    class (Category (Src f), Category (Tgt f)) => Functor (f :: s -> t) where
      type Src (f :: s -> t) :: Cat s
      type Tgt (f :: s -> t) :: Cat t
      fmap :: Src f a1 a2 -> Tgt f (f a1) (f a2)

    type FunctorOf :: forall (s :: Type) (t :: Type). Cat s -> Cat t -> (s -> t) -> Constraint
    type FunctorOf src tgt f = (Functor f, Src f ~ src, Tgt f ~ tgt)

The usual endofunctor

    type EndofunctorOf :: forall (ob :: Type). Cat ob -> (ob -> ob) -> Constraint
    type EndofunctorOf @ob cat f = FuntorOf @ob @ob cat cat f

we have in Haskell can be defined as FunctorOf @Type @Type (->) (->), or

    type OldFunctor :: (Type -> Type) -> Constraint
    type OldFunctor f = EndofunctorOf @Type (->) f

For simplicity written with a type synonym, but it cannot be partially applied. So I would use constraint synonym encoding (https://gist.github.com/Icelandjack/5afdaa32f41adf3204ef9025...)

    type FunctorOf :: forall (s :: Type) (t :: Type). Cat s -> Cat t -> Constraint

    class    (Functor f, Src f ~ src, Tgt f ~ tgt) => FunctorOf src tgt f
    instance (Functor f, Src f ~ src, Tgt f ~ tgt) => FunctorOf src tgt f

The other definitions can be eta reduced

    type EndofunctorOf cat = FunctorOf cat cat
    type OldFunctor        = EndofunctorOf (->)

I was thinking the same thing. Using multicores and having a library ecosystem (through interop rather than conventional F#) should be a win. I don't have an idea what performance difference to expect but certainly closer to OCaml than Python even for single-core.

At least as of a couple years ago, OCaml wins on performance for anything single core, but not by much. F# is usually within a 5-10% performance gap. Absolutely better than Python.

I don't think that's accurate. The Benchmark game [1] shows F# winning more and by larger margins. My own benchmark [2], which was reimplementing Dark in three languages, had F# with 2x the throughput.

(standard caveat that benchmarks suck)

[1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

[2] https://github.com/darklang/fizzboom/issues/13

Interesting, they have definitely closed a lot of gaps since I last used it.

I think .NET Core (esp v5) has had lots of good performance work: https://devblogs.microsoft.com/dotnet/performance-improvemen...

I enjoy f# a lot, but it doesn't necessarily help with all of their problems.

One of the biggest problems they mention is with postgres. F# has a postgres problem as well.

Your options in f# for postgres usage include the following high-level categories:

• Dapper/RepoDb/Other community libs

• Typeprovider based like Fsharp.Data.Npgsql

• EF core (where you build the fsharp design-time support yourself)

• Using C# in another project as a database interaction abstraction

I've had to mix and match for the projects I've worked on because I've had trouble finding a single library where:

1) Joins are supported in a sane way

2) Batch inserts work

3) Interacting with native features (views, functions, SET variables) works

4) It's still in fsharp

It's not bad, but when I work on $DAYJOB projects in Typescript/etc where database access is smooth, I wonder how long it might take to rewrite for the database heavy projects.

Also should mention that I'm very thankful for the libraries that _do_ exist, and the effort put into making them as feature as they are. It's just one of the areas that I don't think is a strength of F# right now.

But those are all database abstraction mechanisms or ORMs. Regular SQL access to postgres databases is pretty much a fully solved problem in F#, even if the ORMs are not.

This is true (with the caveat of some interesting edge-cases with npgsql and F#), however the next paragraph in the article mentions "a lack of a high-level, production web stack".

Definitely a personal take here, but since I've worked for a while with postgres, a "high-level, production web stack" for me is something that has an established, good DX way to work with this popular database.

Managing ADO.NET connections, working around the type systems and writing a lot of mapping/connecting code are all things that don't solve my business problem.

While not prolific, I've written a handful of production web applications in F# (Suave, Giraffe, Saturn, and Razor Pages) as well as other stacks, and have found a hybrid approach that works for my concerns, but I've found a comprehensive solution not as simple as other ecosystems.

I'd love to read through some production Saturn and Giraffe setups - is any of your stuff public?

Unfortunately almost all of my work is not public. I've always envied folks with lots of public code to show and share. I had created an F# starter-kit repo (somewhere between hackathonstarter and bullet-train saas kit) along with some blog posts, but I just haven't had free time to wrap it up and publish the repo/blog posts.

I see questions frequently about data access and serialization/validation in F# web apps, which I had specific sections about. Hopefully I'll have some time in the near future to continue the work.

In the mean time feel free to DM me any questions. There's nothing too fancy about the code. If you looked at other production code, and swapped out what your best guess for the F# translation would be, you're probably most of the way there.

are you freelancing by any chance? dm me on twitter at same username

We're using Giraffe and it's great. The nice thing IMO is that is sits on top of ASP.NET core. So it's fast and if you need something like SignalR, ... it integrate flawlessly.

Have you looked into Linq2Db [0]? (I'm a contributor to the project) I know a few F# users have had success with a couple of minor caveats. I can say it handles most of the points you mentioned fairly well however (Great join syntax, lots of bulk insert options, easy wire-in for native features.)

[0] - https://github.com/linq2db/linq2db/

It's been a while since I looked at linq2db, but it looks like a great project. I'll probably carve out some time this weekend to play around with it!

I've done quite a bit of F# Postgres in a previous life and I didn't find it much different than other programming languages. However I tend to just use the Npgsql library directly with ADO.NET for performance which I often needed for my cases with Postgres. These days you can insert array objects, insert whole objects graphs, do batch inserts and create little helpers to read/write sets with Npgsql/PSQL directly; often much faster than using a higher level ORM. In my experience Postgres often would produce query plans that required a lot of SQL tuning/rewriting in a different way to get right (e.g. loose index scan SQL to get latest entry per group). I just wrap the solution/build with DB tests to make sure the SQL is covered well vs using the type provider. Tbh the amount of code (in lines) is mostly the same as well to do it vs a higher level framework - we did the comparison.

maybe you should give this one a try: https://github.com/rspeele/Rezoom.SQL

We've been using https://github.com/Zaid-Ajaj/Npgsql.FSharp and it's great. Uses well supported lib behind the covers and just provides a small layer on top.

Same thing exists from MS SQL https://github.com/Zaid-Ajaj/DustyTables , also great.

Used both in prod. No regrets and wouldn't wan't something else. Also involves zero magic - just raw SQL and a nice functional API.

I also had a good experience with Npgsql.FSharp. I love that it is just an idiomatic F# layer on top of raw SQL. I don't think that ORM is a good fit for functional languages anyway.

I believe there is some work with F# analyzers to provide compile-time checking of raw SQL, although I haven't tried it yet.

agreed. Often dealing with an ORM is more complex than just to write a bit of SQL and reading code.

Yep, never tried it either. https://github.com/Zaid-Ajaj/Npgsql.FSharp.Analyzer

If you're writing a while platform with a new ide concept, built in deployment and your own framework on top. (darklang) I think an ORM is the least of their problems.

They'll not going to pick F# because they're in competition with Azure.

They don't want ocaml because of marketability. It's technically the right choice and direction but the learning curve for developers is turning them off the platform as a whole.

I tried doing that for my program and failed miserably, mostly because CLR is not optimised the same way the OCaml runtime is optimised to minimise copying of memory. As a result the (direct translation of the) program became more than 5 times slower, which was a deal breaker for this use case.

Yes, there are ways to write it in a .NET-y way in F#, but then it requires a "complete" rewrite and F# gives no real benefits with "being similar".

In contrast, after that I migrated to Rust and I am delighted. I wrote about it here: https://news.ycombinator.com/item?id=24893285

Of course, this is also a full-rewrite without using the same idioms as in OCaml (mostly because I mutate objects directly when needed) but I still think it's much better than F# as the language (Rust) is miles ahead in the practicality domain. Indeed, it's much better than anything I've seen so far. And still so very much "functional" and "elegant". I just miss the transparent currying from OCaml.

> You'll get almost all of the benefits and most of the drawbacks go away.

That's a bit overstatement. You can call .Net code from F#, just like you can call Java from Clojure or Python/Rust/C from OCaml.

It just would be foreign and unidiomatic (hello, null pointers, methods, OOP).

And you can call C/Python/Rust from OCaml, we made quite a lot of wrappers for Gstreamer and some Rust libs in OCaml and it was quite painless.

So it's a bit of simplification, you can call foreign code in OCaml, it's not that much harder than F# considering there are higher level FFIs for C, Rust and Python available. Author could just use these as well.

Writing an F# wrapper for a C# API is considerably less involved than writing an OCaml wrapper for a C API.

This is because F# and C# share:

* build tools

* package management

* garbage collector

* base types (string, int, etc).

The F# type-checker understands C# types, so you know that you are consuming the C# API correctly (at least at the type-level). F# can be be used as a new syntax for C#, if you wanted to go that way.

More interestingly you can also go the other way if you use only the lower level F# features (F# being mostly a superset of C# features) and a C# developer won't realise that they are calling F# code - it's all just .NET IL in the end. Did it often when creating libraries at work; it isn't like invoking another language's binary code at all. One of the pro's I guess of working with a VM (JVM/.NET) based language - interop between languages is usually much easier and so niche languages can have a much larger ecosystem than they otherwise would have.

I was thinking the same all throughout the reading. I've been using F# on GCP in production for 3 years now and it's fantastic and only getting better. You can leverage existing .NET libraries (for example, you get official GCP libraries from google) and if you use them enough it's easy enough to write a functional wrapper around them. I recently rewrote the frontend using Fable.React so that I can share code between backend and frontend.

I wonder if anyone has built a list of functions, libraries and toolings that are considered as essential among mature Programming Languages. Not the core features but ecosystem around it.

So that when people are evaluating a PL, they can known immediately what is missing. And people wanting to make a new PL knows what is missing in their ecosystem, the community can jump in to help.

"OCaml folks talk about Fancy Type System problems, instead of how to actually build products and applications"

I had the impression that's a general problem with FP.

Save for Erlang and Elixir, this tends to be the case with ML-family FP languages, not so much with LISPs, in my opinion of course.

agreed. Clojure is probably the most pragmatic FP language in the space.

The problem is there is no middle ground. FP people forget about the product because of FP. Alot of non-FP people forget also forget about their product because their are swamped with unecessary maintanence. There is a middle road in there somewhere where you use just enough FP to avoid the problems you would face without it, without getting totally sucked into making it perfectly consistent.

From my experience in Clojure projects and watching Erlang/Elixir projects, it's not like this at all in those cultures.

Mostly due to interest in FP being driven by intrinsic desire to use it, since it's so rare in enterprise. Anyway, a lot of that discussion filters down into more popular languages, so it's pretty incorrect to think of it as all academic junk or something.

It is much less of a problem with OCaml than most non-OCaml users would expect. OCaml isn't Haskell; if you want for loops and similar generic imperative features you can have them!

I know and it could very well be that most of the FP crowd I talked to were Haskell/PureScript/Nix/Dhall proponents.

OCaml/ReScript/Clojure people seem to be a bit more chill.

You've been misimpressed :) Clojure/ClojureScript, Elm, Erlang/Elixir are examples of pragmatic communities and cultures.

I had very similar problems with Crystal early on; I remember having to write my own S3 library, which meant having to write the signing, which meant having to patch the core OpenSSL lib to add the correct SHA. At that point, I realized it was "early". Things change though, excited to see where you end up Paul - not Rust?

Did you end up using Crystal in production?

We do, but only for a few smaller services. It's great, but most of the things we do aren't CPU bound, so we use Ruby. Personally, it's become my default for side projects.

Very cool. Have you written anything about your experience with it in prod and on side projects? I'd be interested to learn more.

Actually, no - but we/I probably should

Not OP but I've used Crystal in production at 3 companies: Qualtrics, InVision, and my own side-company (~40 employees) which I briefly mention in this podcast[0] from a few months ago.

Been using Crystal for about 5 years.

[0]: https://www.indiehackers.com/podcast/166-sam-eaton-of-crave-...

I don't know anything about OCaml, but if it's anything like other slightly-unpopular languages, GCP probably was much less supported by the ecosystem than AWS. The double-whammy of an also-ran cloud platform with an also-ran language will really get you.

This really shows how much we depend on (and exploit) open source software. The amount of infrastructure needed for a very simple product is tremendous.

I guess hindsight is 20/20, but even though I think OCaml is a really endearing language, I would not build my startup on it, for the reasons everybody else is saying! I don't have time to shave yaks with respect to database access, cloud storage, etc.

The savings in just going over to GCP or AWS and filtering your "potential languages" by the list of supported languages seems impossible to overestimate.

If you want something with more safety machinery than Go (and I get it, so do I), I think your real choices for a language that needs to interact with cloud providers are something on .NET, or something on the JVM - you pick one, and you go solve your actual problems instead of having to write your own stdlib.

I'd be way more open to experimenting on the front-end with buckle->rescript or similar, where I'm leveraging existing JS libraries and don't have to worry about high quality database drivers.

It seems really questionable to take on ANY new language if you're trying to build a business. If you're building a gaming business, you had better use C++, C, or C#. If you're building a business that might need to use the cloud, look at the supported languages where your chosen cloud has an sdk and pick one of those. If you're building something for Windows avoid programming languages that only have Unix implementations. Etc. Other choices are just onboarding unnecessary risk.

Yeah definitely feeling on the same page. I've seen a lot of writing on the web about "novelty" budgets, etc. I know people argue that really expressive languages are a secret weapon (and I definitely think that Java as I was writing it years ago wasted a lot of my time), but I think haven't a great ecosystem is also a (not secret, but) secret weapon.

I'm sure that by new you meant obscure, but OCaml has been around for 24 years.

Right, I did mean obscure/non-mainstream. Thanks for the correction.

A mature take, I applaud the author for overcoming "commitment bias" and being willing to make a change. The lack of a cloud SDK for thier provider seems crippling in hindsight.

Wish them well with whatever the replacement language becomes!

From reading the threads here I can see most people agree the JVM is a great language in terms of capitalizing on the existing ecosystem.

Some people mention Scala as an alternative to OCaml but honestly the language seems a lot less elegant (at least at a distance...) and the community seems to be surrounded in controversy and confusion ([1] ?)

I think a SML inspired language on the JVM platform would be really popular. [2] lists a few but none of them seem to have gained momentum. [3] looked promising but it seems like development has stalled.

1: https://github.com/lampepfl/dotty

2: https://smlfamily.github.io/

3: https://eta-lang.org/

It was interesting to hear that the Ocaml community has something of that “talking about category theory and functors” vibe that Haskell has. The Haskell posts/community vibe that I see is very heavy on the “ideas of computing research” and I’ve always thought Ocaml was a more “get things done” language because I only ever really hear about it from JaneStreet or ReasonML angle which is very much on the side of “get things done”. That’s a pity.

Actually I think there is a distinction to be made. Both Haskell and OCaml are strongly typed ML-family languages, and to some extent you'll find some overlap in community interests, trending toward the academic side. But Haskell tends to be used more for type theory research, whereas OCaml is far more popular for programming language research due to its popularity for writing compilers.

While I don't have internal insight to Jane Street, some of their talks and public information seems to support the idea that they are actually doing a lot of similar work to compiler development. They have stated that they tend to do a lot of symbolic logic and analysis, and they make it sound like they create DSLs that traders can more easily use to convey trading strategies.

So OCaml does really well at getting things done, but historically it has been within very narrow definition of the types of things to be done.

Understandable how you'd get that impression, but it's off the mark. We do lots of "ordinary" programming that looks nothing like waiting a compiler. Data analysis, high performance packet processing, log analysis, UI work, you name it.

We use OCaml as a general purpose tool, and it really excels in that role.

I think the disconnect here is that for much of the world these days, "ordinary" programming is a backend for a web or mobile app, using REST (or maybe GraphQL if you're avant garde). I'm sure Jane Street does lots of web stuff too, but I haven't see a bunch of Core libraries to do, for example, a CRUD app on Postgres in the cloud somewhere.

Interesting. I've always wanted to dig deeper into OCaml but haven't really had the chance to do so yet. I really enjoy a lot of what I've read about your work with OCaml and it looks like a great place to work for someone like me (4 years of automated trading experience in futures markets and a love of strongly-typed functional programming), but alas I'm not in a position to relocate from Seattle.

Hmm. All programs are either compilers or repls.

Log analysis compiles logs into an analysis. A GUI repls user interactions.

It all has to do with how you define the abstractions.

I would say OCaml’s most famous use (or at least its most successful) is in the compiler/runtime for Coq, the proof assistant. Which means that a huge bunch of its user base is in academia and theoretical computer science , and whose jobs it is to worry about fancy type stuff :)

It also does seem that OCaml is more popular in Europe than North America but I don’t have any data on that.

The Haskell community, in my experience, is far more academic. A recent post to the Haskell libraries mailing list began with:

"It was pointed out to me in a private communication that the tuple function \x->(x,x) is actually a special case of a diagonalization for biapplicative and some related structures monadicially."

It received 39 pretty enthusiast replies.

I don't expect this sort of discussion on Ocaml mailing lists (please correct me, Ocaml mailing list subscribers!) And I think the reason is that Haskell makes this sort of abstraction very very easy, and makes it pay off, while Ocaml does not. In Ocaml, it's not even useful to abstractly define functors, which are the most basic data of category theory. In Haskell, functors are a typeclass that comes with the prelude and are hard to avoid. And there's no reason why you wouldn't use a biapplicative either:

https://hackage.haskell.org/package/bifunctors-5.5.7/docs/Da...

Do you have a link to that mailing list post? Search engines are not turning anything up for me.

https://mail.haskell.org/pipermail/libraries/2020-September/...

Why not use Ocaml for the compiler part only. When you need to go to the real world, use python or go.

While I agree with most of the points, the conclusion sounds kind of dishonest. "the best choice of those we had at the time." really? Despite the lack of libraries? of tooling? of multi-core support? If you needed a strong, staticly-typed language why not Scala? Why not Haskell? They have multicore support, a lot of libraries and good tools. While the author is honest about the shortcomings of the language they pick, they don't fully own what looks like a bad choice.

OP here. I don't know Scala or Haskell well enough to make a "correct choice" either now or then, but basically I had some experience with ML type systems (from Elm and Haskell) and thought sum types were great.

Scala seemed like a mess at the time (from a distance - no idea whether I'm right).

My attempts at Haskell had honestly not gone all that well, the lazyness and pure functional-ness I felt were not great. Basically, I wanted server-side Elm, and OCaml was pretty close.

Tried it, didn't hate it, figured could always do a rewrite later. And here we are.

> Scala seemed like a mess at the time (from a distance - no idea whether I'm right).

I work on and use the .NET Port of Akka from the Scala/JVM world. In those travels it has included reading a pretty decent amount of Scala code.

I wouldn't call it a mess so much as a very -expressive- language. But the cost of that is there are a lot of different ways to do things, and some parts of Scala that are very powerful can be very confusing until you understand them.

My favorite analogy for .NET developers; if they remember their confusion the first time they had to comprehend => delegate syntax, there's lots of little things in the same vein in Scala; ways for the compiler to generate sugar. There's just so -many- of them it can be overwhelming, doubly so when in some cases the wrong choice can in fact have pretty big performance implications.

It sounds like you've already made a choice, but on the off chance you haven't I would suggest Kotlin.

I would recommend Scala in second, but you really need at least one person who knows Scala well to navigate the wildly divergent styles of Scala programs out there.

Kotlin gives you sum types via sealed classes. In fact it's basically the bare minimum of Java + sum types + some syntactic sugar.

It has rock solid tooling due to Jetbrains support. It has access to the entire JVM ecosystem. It has a huge amount of learning materials out there. And it's got all the resources on how to deal with web CRUD problems you could ever imagine.

It's not server-side Elm by a long shot, but to be fair, Elm would suffer from all the same problems you listed.

Eh, for someone who likes OCaml and sum types, I don't think they would be wild about Kotlin. It doesn't have any generalized pattern matching, for example.

Yeah, and its ecosystem loves mutability, but with discipline in your own codebase you can get 80% of OCaml with access to an amazing ecosystem and tooling. After all nested when expressions aren't the end of the world...

'but with discipline in your own codebase'

If I had that (and, more importantly, trusted every other developer I work with to also have that) I wouldn't value immutability enough to look for it in my language choice.

It's a bit more subtle than that. Almost every language has a sliding scale of discipline rather than a binary one.

E.g. you could decide to write extremely mutable code in OCaml, but it's highly discouraged and the happy path is generally to default to immutability.

So you need comparatively little, although not none, discipline to have a pervasively immutable codebase, since there's nothing preventing you at a language level from introducing crazy mutation everywhere.

Haskell goes further, where you have to use functions named "unsafe*" to introduce mutability. Something like Elm goes further still, where, barring bugs in low-level libraries or heavy abuse of the JavaScript runtime (e.g. redefining prototypes of basic types), you essentially have to fork the compiler to introduce mutability (Dark moved away from Elm because, as I understand it, it actually became too restrictive on this front).

On the other hand, in something like Java, you need a lot of discipline to have pervasive immutability and you have to go out of your way to avoid using otherwise common constructs (e.g. if statements since those are statements rather than expressions) that require very kludgey workarounds (encoding an algebraic data type as a fold and using that in place of if statements).

Kotlin is in the middle. If you don't use certain parts of the standard library and don't use things like var, you have pervasive immutability. At a language level this is quite similar to Scala (Scala has a more immutably focused standard library than Kotlin does).

I'm currently working on a backend system written in Kotlin and the Reactor framework, and came to the following conclusions:

- Kotlin's biggest problem is how it handles nullability (that is, it simply enables you to leave nulls in your program). This plays extremely badly with classical FP, and Reactor specifically, because 1. the Reactor "monad" (Mono) can't wrap null and 2. flatmapping an empty monad is not the same as flatmapping on Some(null) for example.

- Kotlin has no syntactical support for flatmap/map, like Haskell or Scala.

- Kotlin has no pattern matching, inline types are still experimental, and it's type system is limited, it is clearly targeted at the UI/Android folks who are not interested in expressing their data in a static way. Making sure that people don't shoot themselves in the foot is great for the average Joe developer, but not so great for library writers.

Having said all that, it's still better than Java, but it's 2020, ZIO is past 1.0 and Scala 3 is around the corner. I'm really looking forward to 2021...

"It's better than Java" - I've said that about numerous languages older than Java. It's definitely a lowest common denominator.

1. Isn't that just a reflection that union types don't form monads? I find it's not a big deal most of the time and when it is just convert so that you have legitimate maybe types, for example like the Option type provided by Kotlin Arrow. You don't lose type safety either way since the type system won't let you use one in place of another.

2. I assume you're talking about `do` notation (bind/return + ap/pure if ApplicativeDo is turned on) and `for` comprehensions (flatMap and map + filter/filterMap if you use `if` statements) respectively? It's a minor pain that they don't exist, but ultimately not something that I find is a huge deal. Besides I'm not actually a huge fan of those syntaxes. They force you to rewrite code in A-normal form, which means that wrapping monadic types can cause large rewrites to your original code.

I much prefer an approach like Scala's monadless (https://github.com/monadless/monadless), perhaps with additional syntactic markers indicating that this is a syntactic macro rather than a function, which does not force users to rewrite their code entirely in A-normal form. Kotlin has something similar to this (coroutines) which is what Arrow Fx uses, but again, like many other things in Kotlin, it's hard-coded to be only one thing and cannot be generalized to other monads. However, it's a nice middle ground for marking side effectful code and syntactically much nicer from my point of view than either do notation or for comprehensions.

3. Honestly pattern matching isn't a huge deal for me. Destructuring gets me a lot of what I want and the much bigger deal is exhaustive sum type checks, which Kotlin provides in its `when` declarations. Experimental inline types are fine with me. They're stable enough and I've never really regarded `AnyVal` in Scala to be any more stable (it fails in a bunch of corner cases to the point that some codebases entirely eschew it in their style guides). Opaque types in Scala 3 will hopefully fix this and Haskell's newtypes are great. The two main differences in type systems for day-to-day code are lack of higher-kinded types and lack of typeclasses. As Elm and F# demonstrate I don't think either are fatal blows.

Speaking of Scala 3 I really hope it goes well. I'm still a tad nervous about introducing indentation-based syntax since it seems like a needlessly divisive choice for little benefit, but we'll see how it goes.

Yeah, I wasn't saying it wasn't a sliding scale, or rather, a series of granular features that move you further one way or the other. Just, I value having more of the ones that move you toward immutability.

> Scala seemed like a mess at the time (from a distance - no idea whether I'm right).

I love Scala, and it's getting better year over year from all angles. It's also getting a little more opinionated. The upcoming version 3 removes long-standing warts and adds great features including sum types, intersection types, enums, and more. A huge thing is that it addresses binary compatibility issues, by allowing Scala 2 and 3 to use each other's libraries. Also, Scala has wonderful, super stable JavaScript support with Scala.js, and under development native support Scala Native.

I'd be curious what you think about Scala 3. Interoperability with Scala 2 and Java solves the library problem. And the language itself seems to really take on learnings from its own history and other modern languages.

I loved the ML based emphasis. I'm a big fan of f# unfortunately as the article details .net has a shadow of the libraries that Java has. Clojure and Scala have much broader access to popular libraries. I've found most developers are ok with the hum drum c based languages thus c#, java, golang are good enough for most people with no interest to jump the chasm to an ML inspired language.

While it is true that .NET has less libraries than the JVM ecosystem, I'm not sure it remains sensible to still call it "a shadow of". In certain domains the .NET ecosystem is better. Either way, if a team was almost able to put up with the OCaml ecosystem, I believe the .NET ecosystem would be no problem at all.

That's unfortunate, but hindsight is 20/20. If you wish to build a tool that will be used by the world, it's best to bet on a boring language like Java, Go, Python or you will eventually run into something like this. For pure tools attacking a specific domain, exotic languages like Ocaml, Lisp, and Haskell might shine.

Don't bet on boring languages. Bet on community and ecosystem. If those are vibrant and a good match for your domain, go for it. If multiple languages meet that criteria, choose the one you believe in.

Conversely, if you are developing a new language, focus on building a community and ecosystem around certain problems, even niche problems. That will give a consistent group of users that can build out the rest of the ecosystem slowly.

I always face this dilemma when starting projects. I want to use a language with nice types that's fun to write, but damn it's so hard to not use Rails or JavaScript.

My dream would be for Rails to get good types or for a language with good types to get a Rails style framework. Maybe Sorbet is the solution, although I've seen enough confusing Ruby metaprogramming to be skeptical of that. Maybe someday Rust, Swift or Kotlin will get a mature web stack.

Mature in what way? In terms of features/completeness, or just...age and battle testing? I'm not really in the Rust world any, but I feel like https://www.arewewebyet.org/topics/frameworks/ is a fair amount of choice to start with? A couple of those are post 1.0.0, but to be honest, I wrote an Erlang web service using Cowboy, before it hit 1.0, back in the day, and it was fine.

Everything just takes too damn long. You gotta trawl through tutorials and connect a bunch of libraries just to get a simple CRUD app with user auth set up. In a couple years, hopefully we'll have a mature default library for most usecases and they'll play nice with each other.

For instance, there's exactly one JWT library. To get a simple auth setup going, I'd need to connect this JWT library to my password setup, which I'm gonna need to design too, since I'm using a barebones password hashing lib. Now I need to write the token exchange and validation logic. Then manually put the token in the header/response, etc.

Compare that to Rails where I can install devise_token_auth and just have tokens out of the box. No fuss.

Okay, so you're looking for batteries included.

What Javascript framework are you using? Or did you just mean on the frontend? I figured from your including it you meant something backend, like Express, which is probably the most popular in that space, and fairly minimal, but super easy to grab the middleware pieces you want.

As, yeah, if you're looking for one giant "does all the things" framework, I don't know of any. Even the ones that started that way, reconfigured their projects to keep them optional (i.e., https://github.com/actix/actix-extras ) ...and honestly, I feel like most frameworks are moving in that direction. Even ones that want to be like Rails (i.e., Phoenix for Elixir) tend to be very intentional in making the included pieces fit the middleware pattern so you can swap them out easily.

So you may be staying with Rails for a while if that's important to you. I will say, comparing when I had to learn enough Rails to get something done, and having worked with Express, and Phoenix, and Gin, it's generally no harder to learn the pieces you need than it is picking up any new batteries included framework. Oftentimes the same web search that would turn up the one bit of syntax you need for the batteries included framework (i.e., "actix-web auth token") will turn up the most obvious choice of middleware you'd need for the middleware approach. And the usage will be of similar effort, once you've installed one.

Yeah, so far Rust is proving to be that language for me. See here: https://news.ycombinator.com/item?id=24893285

> We've built our own queue on top of our database rather than using the production-quality cloud queues available on GCP

Cloud Pubsub has HTTP and gRPC APIs, wonder why they didn’t use those? https://cloud.google.com/pubsub/docs/apis

> Cloud Pubsub has HTTP and gRPC APIs

Followup post: https://blog.darklang.com/new-backend-fsharp/

Are there any programs coded in OCaml in common use by non OCaml programmers? E.g., in Haskell I found only Pandoc, plus a personal finance and a parametric 3D CAD system. I know the FFTW C library is generated by an OCaml program, but that is just for programmers.

"In common use" might mean commonly packaged for Linux distributions and not targetted for programmers, or driving multiple public websites that are not programming-related.

It seems that most of the problems the OP has could be solved with minimal distruption by switching to F#. The .NET ecosystem is large enough to provide the libraries required, multi-threading already works and the language is very close to OCaml. You can also leverage Fable to run F# on the front-end too. However, you do lose some fancy type-system features and (arguably) some single-core performance vs OCaml.

Ecosystems matter a whole lot when you're just trying to GSD. That's why Rails, for all its flaws, still beats the pants off many other solutions (for the right problems).

And then, when you have a ton of code, you have a lot of embedded knowledge and throwing it all away is tough.

Looking forward to the next installment to hear what is next.

> I’ll write in a few days about what we chose instead

My bet is Python - it has some types but more importantly a massive ecosystem, and I think Dark’s target market probably has a little Python knowledge (which would come in handy if they’re looking for open-source contributions down the line).

Python is a bad choice for writing reliable software. If you need the reliability of type checking, just use an early-bound language, not a late-bound language with checking grafted on. And very few Python libraries have type annotations.

And while 3 fixed some footguns, e.g. 'a' < 5 now throws a TypeError and there's a clear distinction between str and bytes, others remain, e.g. if you reuse a generator it will silently be empty.

Looking at the repo[1] the languages tab shows:

OCaml 78.6% F# 12.2%

The F# hits are in a directory "fsharp-backend", and all in the last 3 weeks.

1: https://github.com/darklang/dark

Haha, also checked the repo and thought the same. We're using F# in prod (about 200K LOC) and it has been great so far.

We also use F# for two react front ends with fable (F# to JS compiler). More new alternatives for using F# in the browser (FsBolero/Blazer) via web-assembly or running it server side like Phoenix live view also exist.

One issue we had with Fabel is how .NET DateTimeOffsets are handled on the Client side.

Honestly surprised I was able to keep this any kind of mystery, since the repo is open and it's plain as day what I've been working on. But congrats on being the only person that actually looked!

There's a more to reliable software than static typing. See eg Erlang and its stellar record in building reliable systems.

Static types aren't necessarily the best fit for distirbuted systems, network protocols, asynchrony etc.

huh this is all extremely relevant information to me, as someone who has always been more on the "use the cool language on the marginal side of the fence" side, and i can't wait to see what the next step for this project will be. i know that the project itself has hit some headwinds, but i gotta say i have actually played around with dark quite a lot and i really, really enjoyed it. so i very much look forward to what's coming next (especially if it makes it easier for me to hook dark stuff into my own systems more directly-- the kinds of hacks i underwent (ssh port forwarding and the like)... don't quite cut it for stuff any fancier than what i've already written).

Coincidentally, I chose to look into using OCaml for a project last week. I was pumped and then I was disappointed at the lack of interest in supporting Windows. That just tells me they're not a serious player.

I do OCaml programming on Windows and I found that it was a bit confusing at first with too many different ports and install options. However once I settled on https://github.com/fdopen/opam-repository-mingw I was fine. To my surprise I was able to extend existing C bindings to use Win32 APIs fairly painlessly (for example https://github.com/mirage/mirage-block-unix/commit/7cf658f8a... ) . I did have problems with I/O scalability at first but I fixed these by using libuv via https://github.com/fdopen/uwt . The core compiler and runtime are rock solid on Windows. Docker (where I work) ships OCaml/Windows binaries to lots and lots of desktops with no problem.

Apart from the too-many-ports problem, I think the main remaining problem is that too many 3rd party libraries require Unix-isms to build, like shell scripts. This necessitates the presence of cygwin for build (but not at runtime). However the ongoing "dune-ification" of the OCaml universe should help fix this since dune can do everything directly from OCaml code. I'm really looking forward to being able to open a powershell window and type "git clone"; "dune build" and have everything just work.

Use bucklescript so you can use JS libraries on the server.

>learnability

I feel this is the single most important causative factor for language popularity.

Javascript may have much more bad parts to the language than say Haskell but it's far easier for a programmer to learn all the bad parts of javascript and avoid those bad parts then for him to learn all the good parts of haskell and use those good parts.

I find it curious that this kind of thing still happens "oh we wrote it in the wrong language". Shouldn't every mainstream language by now be good enough to build most (if not all) applications?

Most languages are good enough, but they rarely have good enough ecosystems.

I think that's a great criterion to use when choosing a language

Probably yes, but OCaml isn't mainstream.

The author stated many reasons this just wasn't the case. Lots of missing libraries to common services or libraries that weren't mature enough like the saddeningly reported Postges driver. That seems particularly damning, if I can't reasonably access one of the most popular and general purpose DBs then how am I going to expect a litany of other software I might want to use and expect after having worked with Java or Ruby or Puthon. Personally this drives me to Clojure more, which they said they used at CircleCI and had success with given it's interoperability with the large Java ecosystem. That meshes well with my experience too.

It's perhaps worth pointing out that the author of this blog post explicitly decided against using Clojure a second time for Dark after cofounding CircleCI, mainly due to a lack of sum types.

Right, Clojure is dynamically typed, and I'm not interested in dealing with the challenges of that again (esp nulls).

I, for one, find nil-punning useful in Clojure.

There’s a big gap between “can theoretically build this application” and “can efficiently and reliably build and maintain this application.”

Maybe that happens, but the author has stated that wasn't the case here--that OCaml was absolutely the right choice to write Dark in, given the constraints they faced at the time.

Reminds me of Joel Spolsky's 2006 post on languages. And it even mentions OCaml !!

https://www.joelonsoftware.com/2006/09/01/language-wars/

FWIW I can see this igniting more arguments. And it is for sure interesting that he mentioned as Python the 0.5 that's on the edge, and Ruby on Rails that's a little further on the edge.

And Ruby on Rails turned out to be phenomentally successful for many billion dollar companies since 2006.

https://news.ycombinator.com/item?id=24279611

But that doesn't make Joel wrong. If you adjust 2006 to 2020, the same point can be made with different languages/platforms.

I know that typically on new projects there’s a long evaluation period where you decide what technology to use, along with lots of debates that include some crazy person actually wasting quite a lot of time evaluating Squeak and Lisp and OCaml and lots of other languages which are totally, truly brilliant programming languages worthy of great praise, but just don’t have the gigantic ecosystem you need around them if you want to develop web software.

These debates are enormously fun and a total and utter waste of time, because the bottom line is that there are three and a half platforms (C#, Java, PHP, and a half Python) that are all equally likely to make you successful, an infinity of platforms where you’re pretty much guaranteed to fail spectacularly when it’s too late to change anything (Lisp, ISAPI DLLs written in C, Perl), and a handful of platforms where The Jury Is Not In, So Why Take The Risk When Your Job Is On The Line? (Ruby on Rails).

Reply on news.ycombinator.com