Comment by tombert | Hacker News Reader

Comment by tombert | Hacker News Reader

That makes me sad, but it's not surprising.

I was part of Jet.com before and during the acquisition, and the main reason I joined was because I was (and am) a big functional programming geek, had a fair bit of Haskell and Erlang experience, and F# seemed like a neat language.

It turned out that it absolutely was a neat language, and I consider the two years I was at Jet to be a fun highlight of my career. I got to see where functional programming shined in real-world distributed applications, where it didn't shine, and how it integrated with other libraries.

I quit in 2018 to work for Apple, stayed there for about two years, and went back to Walmart in 2021 for another year. In that time, I worked on the chatbot; one of the very last things that was still F# in the Walmart ecosystems. I had less fun the second time around (not really because of anything Walmart did and not because of F#), so I only stayed for about a year, but I still found some enjoyment doing professional functional programming in a pure-ish language.

It doesn't surprise me that they're changing to JVM stuff. Even when I was there, there was a lot of external pressure to move our stuff to Kotlin. We stayed with F# because of a reliance on the Microsoft Bot Framework, which was predominantly .NET, but once ChatGPT started getting traction I figured it was only a matter of time before they migrated to a Transformers model and rewrite it into JVM.

It makes me sad. F# is a language that I really do not think gets its fair shake. It's a language that really does push functional purity, but manages to still be pragmatic. I found the integration with the core .NET libraries to actually be very good, and even when I had to do OOP, F#'s implementation was exceptionally pleasant compared to C#. Honestly, if I had any say in running a business, I'd consider F# for at least a few of the services.

Out of curiosity - what were the rough edges in the distributed application space?

Performance.

Generally speaking, F# was actually very fast, and had nice concurrency support, but there were times that wasn't the case.

For example, in 2016 I was part of the initiative to rewrite the ad feed. We had to read in several Kafka topics, do some joining on our end, and emit to a separate Kafka topic. This isn't terribly hard to write, but we were dealing on the order of about ~100gb of data being pushed into memory. This is hardly "big data" stuff, but it's enough to highlight some issues.

Specifically, the built F# persistent map structure was simply too slow to get the performance we wanted. I really like that structure, it's really handy and nice, but I ended up having to make heavy use of the ConcurrentDictionary that was built into .NET. This wasn't that hard or anything, but it made me a little sad that I had to move to a mutable store to get the performance I needed.

There was also the fact that the `async` monad, while generally very good and useful, had bizarre bottlenecks that were hard to measure. It was difficult to know when the async task was actually started, and when you tried to measure performance bottlenecks you were really only measuring the scheduler, not the actual performance. This isn't really F#'s fault, this is an issue with any kind of cooperative scheduling system, but occasionally to get the performance we needed we'd have to move to lower level threads instead of the pretty monadic stuff. Microsoft eventually released the Task monad which generally performed a bit better.

There were other things here and there; the Kafka client libraries for .NET simply aren't as good as the Java ones. Jet actually open-sourced their own (https://github.com/jet/kafunk) which did make it a bit more functional and nice, but it had performance issues as well, so a lot of us ended up using Confluent.

There were little annoyances specific to F# as well; there's no real concept of a monad transformer, so if you wanted to do something like, for example, combine an Option and an Async into generalized syntax, you'd have to write your own wrapper monad thing, which wasn't that hard but was sort of ad hoc.

The general rule of thumb was that the first draft of software, we would try and keep as functional and pretty. If that was too slow, we allowed mutation but only within a function. If that was too slow, we'd allow global mutation but only with thread-safe stuff.

> where it didn't shine

Care to expand on this for the curious?

Answered in sibling thread.

Reply on news.ycombinator.com