I spun up two Haskell teams at work, and now it composes about half of our codebases. Happy to answer questions about the experience.
I consider it one of my larger mistakes. In hindsight the real problem was building the thing in isolation and then dumping it on someone else with minimal training time. I think different tools and languages are fine, but they need a bus factor of 2-3 to prevent this kind of stuff.
One of the motivations for adoption for us was strong interest on the team. Looking back on our team...
1. We started with me (enthusiast), our expert,and a Go developer with no Haskell experience.
2. Over time, at its peak, the team grew to us 3, Python developer who had dabbled, two Ruby developers who were interested but had no experience, a frontend expert who had dabbled, and another engineer who I didn't work with closely but I think had a background in Java?
3. Our expert left the team before or around the time that the last person (maybe-Java person) joined, I think.
4. On the second Haskell team I spun up, it's me, one person from the old team, a Ruby/JS developer, and a JS developer.
We were able to get all of these people productive in Haskell. It got easier once we had more experience teaching folks. There are a handful of _key differences_ between Haskell and ALGOL-family languages (mostly around evaluation model and effect tracking), and once you nail those down, the rest is pretty smooth sailing, and your SWE experience and intuition begins kicking in again.
Although we miss our expert very much every day (they were a very cool person! They drove a motorcycle!), their departure has not had an outsized impact on our velocity.
A sibling comment recommended a bus factor of 2-3. I think this is roughly correct, although I would think of this as not merely your "don't get hit by a bus" group, but also your core teaching group.
What's your team workflow around designing the code ?
Do you follow mainstream ideas or do you have very special tricks (say innovation on top of property based testing.. metaprogramming.. whatever)
Re: design workflow - this is not very different from any other code. You have module interfaces, logical subsystems, separately deployable service entrypoints, etc. Our Haskell code tends to take special care to avoid statefulness and globals, but this is not any more of a design burden than writing idiomatic code in any other language (e.g. planning out your classes in Java).
Re: special tricks - we used fused-effects to model our effects, which has been pretty useful (it lets us delay teaching transformers), but this is by no means a secret that we've discovered.
I think the biggest trick is to write simple code. I think when you give a junior engineer a sufficiently powerful hammer (like Haskell and its effect tracking), they will attempt to nail absolutely everything to the wall (e.g. build a taxonomy of every possible effect and every possible exception and write your whole program in this framework for Maximum Compile Time Safety).
Avoid this. It's just not a productive use of time, and the gains to safety often are not worth the effort and velocity and overhead you paid to build the underlying framework. Focus on the high-ROI piece of the program. Write the effects that you would use for testing, do the rest in IO for now, and come back later if you find that you need more granularity in your effects. The important thing is to ship the product.
(I go into much more detail about this in the Serokell interview linked in the other HN comment.)
Do you guys use parsec in production? What is the motivation for having Haskell to be half of the codebase? (Are there some open-source domain-specific libraries that are used intensively? Or do you guys prefer to implement things in-house (on top of libs that are more abstract)?)
And if parsec is used in production, any tips on how to best design/debug parsec parsers? (Thanks!)
Lastly, in your experience, what are some of the best monads / design patterns to improve team productivity? :{
Really curious about it
I think we waited too long to ship our initial prototype to customers, but a big part of that was that we were pretty sloppy in our product management at that stage of the company (I think we had closed the A about a year ago, and did not yet have our first dedicated PM), and our technical rewrite was also partially a feature and UX rewrite. It's been several years since, and we have clawed our way out of that hole, and I think our delivery process is actually in a really good place now.
If I were to give advice to people looking at adopting new languages, I would say to get it into production ASAP. I believe the keyword to search for is the "tracer bullet pattern". One of my favorite blog posts on this is https://blog.thepete.net/blog/2019/10/04/hello-production/
I think Haskell and OCaml are popular in finance because they're high-level and familiar to math/quant types (who are broadly--and this is a sweeping generalization--more familiar with expressing algorithms in recursion than in procedural sequences).
I don't have much experience with OCaml, but I can also say that Haskell in particular has incredible support for building very high-level libraries that are both expressive and have strong compile-time guarantees. I've found it's relatively easier for a programming expert to build such a library and a domain expert to consume the library in Haskell than in many other languages I've seen (perhaps Python comes the closest, but that feels to me more like an ecosystem thing than a language thing).
[1] https://serokell.io/blog/haskell-in-production-standard-char...
Click on "Area" column header to sort.
Or do you mean relative to Java?
Off the top of my head, 3 advantages stood out:
1. First, if you're not going that far off the beaten low-level path, Haskell has incredible productivity benefits. Effect tracking has enormous benefits for testability and understandability. If you've ever been down a debugging rabbit hole shaped like "there's no way this logging call is sending that API request", then you might be pleasantly surprised to discover that you can statically guarantee that this doesn't occur in Haskell programs! Pattern matching, algebraic data types (sum types!), and typeclass derivation make it much easier to make it impossible to construct invalid representations of data. Other languages are finally picking this up, but their versions of pattern matching often have caveats for backwards-idiom-compatibility. And monads are a very powerful abstraction. It's like being able to write your own semantics for async-await (I've talked more about this before at https://lobste.rs/s/7cllte/monads_part_six_really_what_is_mo...).
2. Haskell was a good domain fit for us. One thing we build is the FOSSA CLI (https://github.com/fossas/fossa-cli/), which runs in customer CI pipelines to analyze their builds. It's a very compiler-shaped problem: shell out to some tools, do a lot of parsing, think very hard, and then spit out a JSON blob to send back to the API. Our first version of this was written in Go. At the time of development, writing correct, testable parsers in Go was like pulling teeth. We have a relatively small headcount-to-product-surface-area ratio, and our team was running up against the overhead of rewriting traverse in Go over and over again (that's a Haskell-flavored joke, but if you've ever been annoyed at writing yet another for-loop in Go, you get it). We decided to hack out a prototype in Haskell, and it turned out to be a good fit.
3. Lastly, the kind of people who wind up working at FOSSA and are interested in the code analysis bits tend to be the same kind of nerds who love Haskell. We had lots of people on our team who were chomping at the bit to try it, so we decided to try it out. I really can't understate how big of a productivity difference it makes when people are working with tools that they actually enjoy rather than are merely forcing themselves to use. It is night and day.
If you want to learn more, we also did an interview with Serokell on this topic (https://serokell.io/blog/haskell-in-production-fossa), and discussed it on an episode of our engineering podcast (https://fossa.com/blog/fossa-podcast-adopting-haskell/).
Re: development process - it's very similar to development in other languages. Write, compile, yell at compiler, push, complain about how slow CI is, deploy. You know, the usual.
I think the most interesting difference is the _onboarding_ curve. Haskell's curve is pretty brutal, although I think most of this is because of bad pedagogy (many monad tutorials are bad, and beginners can't tell) rather than because of intrinsically difficult concepts. Some observations:
1. Empirically, zero to code review is roughly six weeks for a professional industry software engineer. It's not that much longer than other languages we've had to teach. But it _feels_ very brutal because zero to side project is roughly three or four weeks. Contrast this against Go, where zero to side project is about five minutes.
2. Having an experienced Haskell engineer on your team to start with makes a WORLD of a difference. You've gotten a type error - why? Is it because GHC is doing weird backwards type inference stuff again? Or is it because you've misunderstood this fundamental concept? Or is it because you've done a typo, and GHC has inferred a downstream site to be a type error? This sort of thing is very difficult to explain in words and in general, and much easier to pick up through experience and mentorship. If you do not have an experienced Haskeller at your disposal, I would strongly recommend starting with side projects first, and using the Functional Programming Slack (fpslack.com), who are some of the friendliest and most patient folks I've had the pleasure of talking to.
It appears monads truly are something you can either understand or explain, but not both.
I find it suspicious. I mean, plenty of Haskell devs out there, surely it's not that hard?
Most tutorials go off the rails because they confuse types supporting this pattern with "being a monad". For example, arrays in JavaScript support the pattern through the `flatMap()` method, but saying "a JavaScript array is a monad" is misleading because most of what people do with arrays are unrelated to this.
As a pattern it is very general. It strings a sequence of functions together, but doesn't care about the semantics of the functions or the types involved, as long as each function just return the same generic type.
But many explanations take the semantics of how some particular types use the pattern and generalize from that. E.g. list and option types are monads, so monads are explained as containers. Or IO and State uses the pattern to represent side effects, so monads are explains as a way to have side effects in Haskell.
This is what leads to the bizarre metaphors, like monads are boxes, monads are spacesuits, monads are train-tracks etc. Each metaphor matches some uses of monads but breaks down on others.
A Maybe monad just says the value may be Something(x) or Nothing. You won't know until you run the computation. If you use flatmap and give a function that takes an x and gives a Maybe[x], the monad will first map into Maybe[Maybe[x]] and then flatten into Maybe[x]. The computation has not happened until you execute it and internally all the functions have been composed together.
A List[A] just says, give me a function A->List[B], and I will flatmap (flatten `compose` map) it. So it maps each element into a possibly empty list of Bs, and then flattens it by concatenating them.
You can define your own monads, and as long as they obey the laws of monads, you get a bunch of stuff for free.
Each monad constructs a type that behaves differently and expects different things, but in general; the monad is defined by a unit/point/return function which brings a value into the monadic context, and a flatMap function aka bind, which further breaks down to “flatten after map”.
So List[_] is a type constructor, given a type T, it produces a List[T], which defines some transformations. Return creates a single element list, and flatmap takes the A->[B] applies it everywhere and then concatenates.
The nice thing about the wrapper analogy is that even though it is technically wrong, it is easier for people to get it because it follows naturally from OOP and it is a sufficiently useful mental model imho.
But then there are always people who start popping in and pointing out how those definitions aren't quite right. Which is true. But does it matter *for practical purposes* to give a dev a useful for now mental model that they can then use to figure the rest out later? I'd say no.
In Haskell, a Monad is a type class with a method `bind`. In Scala that would be a `flatten` or whatever.
If a person who's asking doesn't yet know what the "type class" is, or how to read signatures, your "monad tutorial" would not make much sense anywat. Guide them to learn the prerequisites first.
Knowing some prehistory about doesn't affect usage. And metaphors are very tricky and personal.
The personal part is where the "monad tutorials" fall flat. They assume shared context which may or not may not be actually shared by a random reader on the internet. Yes, some monads are really about "wrapping a value" or whatever.
Reading a post using this metaphor when you want such a wrapping and/or deal with chaining wrappers regularly can bootstrap your understanding in no time. But if you're reading "wrapper"-flavored tutorial while dealing with "pipes" then the spell breaks, you end up confused, and another one joins the "monads are uncomprehensible" group.
Teaching is hard enough. Writing good tutorials is even harder. Successfully giving an universally good drive-by explanation is next to zero probability.
I suspect this is a knowledge variant of "XY problem" and you have to establish more context before answering.
And another problem that "what is a monad" is already a meme. Everyone has the burning desire to ask it, but usually there's no practical need for the answer. Without that confusion ensues. Or, even worse, a false understanding gets locked in and starts to proliferate, sustaining the memetic chain reaction.
They're very general. Classes in OOP are general too — surely you can model lots of things with them — but a class always models a category of things with similar functionality, and an instance of that class is one of those things.
Monads are much more flexible than that. You can model nondeterminism with monads, as well as side-effects, exception-based error handling, state, (backtracking) search algorithms, and more. Could all of those things be an instance of the same OOP class? Surely not. Yet in Haskell, 'Monad' is just a "type class" (not dissimilar to an interface / abstract class in the OOP world).
Monad tutorials typically try to do one of the following things:
1. Try to explain the entirety of monads by giving a metaphor (burritos, anyone?) that only works for some of the instantiations of the pattern. It turns out to be hard to find a metaphor that covers all ground that Monad does, unsurprisingly. Personally I think this method is good, as long as you're honest about what it does not give you: an intuition for all Monad instances. It just gives you an intuition about some of them, but that could already be plenty to work with them usefully (and see 3. below).
2. Try to let the reader invent Monad by showing various instantiations and asking the reader to find the pattern. Personally I think this misses the point somewhat; yes, you can see the pattern, but that doesn't give any understanding per se. Why are those things similar, and why does it make sense to abstract over the idea?
3. Not actually explain, but instead say "work with them a bit and you'll understand soon enough". This is the one I'm most in favour of — after giving a special-purpose intuition for the monads you'll be working with as a beginner (IO, perhaps parser combinators, and not much else; perhaps lists (nondeterminism) to jump-start exploration into non-imperative monads).
Note that none of these give a nice and polished answer to what a monad is.
As alluded to above, the closest OOP equivalent to the kind of thing that Monad is, is a design pattern. It's a design pattern that can be encoded into a three-line definition in the language itself, and is super general.
Maybe (optionals) - chain steps and abort if any step is empty.
IO - perform IO side effects, and run the next step when this one completes. Just like a async/await.
You use monads all the time in other languages! Haskell just has many more kinds and allows the programmer to make their own.
I always twitch when I hear someone say you can’t both understand and teach them. Did I succeed?
Also note that you don’t have to totally understand monad machinery to use them productively, only to write your own.
No you don't. You are confusing monads with features which can be implemented using monads. In Haskell monads are used for modeling side effects, but this is not the case in other languages. In Haskell exceptions are implemented using monads, but this is not the case in other languages.
What these languages don't have and Haskell does is a way to talk about all monads, all at once. Instead of
replicateM :: forall m a . Monad m => Int -> m a -> m [a]
replicateArray: <T>(count: number, ts: T[]) => T[][]
replicateFunction: <S, T>(count: number, f: (s: S) => T) => (s: S) => T[]
replicatePromise: <T>(count: number, t: Promise<T>) => Promise<T[]>
After they’ve been working for a month or so, they’ll “get” monads on a gut level, and the details won’t be so confusing to them.
Explaining monads in terms of promises and futures will make a beginner associate them with asynchronous programming, which might make some sense for some monads. But now show such a beginner a simple list comprehension and explain it is also a monad, and I assure you they will be very confused! Explaining concatMap in terms of promises is a very convoluted way to explain something quite simple.
Haskell comes from a population that loves to understand things from the ground up, and most people learn by doing.
So, what if, instead of an abstract burrito metaphor, we taught people 4 monads: List, Maybe, State, and IO. We explain how do syntax does “something different” for each of them, and how IO is very similar to promises, but the others aren’t. Then we let them play with those for a few weeks, submit some PRs, let the panic of not understanding subside, and then teach them the theory?
Think about how JavaScript developers learn promises. They don’t understand how they work under the hood at all when they first start using them. Then, when they need to go deeper and look under the hood, that theory is connected to their practice, and makes much more sense to them.
Yes, I am on board with that.
> Maybe (optionals) - chain steps and abort if any step is empty.
Could this be (remotely) akin to the following shell pattern?
set -o pipefail
cmd1 | cmd2 | ...
Does it make any sense?
doStuff :: Maybe X
doStuff = do
r1 <- cmd1
r2 <- cmd2 r1
…
return rX
doStuff = cmd1 >>= cmd2 >>= …TL;DR: Monads are Promises and async/await, but generalized to different implementations of `.then`.
You are right to be suspicious; it is _not_ that hard! I think people just get confused and overwhelmed because (1) there are many separate different concepts that are all being introduced at once, and (2) you need to learn a new debugging methodology at the same time. I think the vast majority of this problem is the huge deluge of bad teaching materials (especially monad tutorials written by people who do not use Haskell in production) and the shortage of good teaching materials.
Re: 1 - some separate, orthogonal concepts that I have seen people mix up while trying to teach Haskell:
1. Effect tracking (which is a thing that we _do_ using monads, but is not intrinsically tied to monads - this would be better served by being taught as "dependency injection but stricter")
2. Non-eager evaluation (which is a language runtime evaluation choice that Haskell has made, and which motivates the usage of monads where most languages have function bodies, but is again not intrinsically tied to monads)
3. "Purity" (a more confusing way to explain 1)
4. "Mutable state" (another thing we represent with monads, but again is not intrinsic to monads)
5. Monads the math / category theory concept (don't bother with teaching this - it will not be helpful until you have intuition for writing and operating the programs themselves first, which takes at least a few months to build, and even then it is mostly useful as a source of advanced techniques for library writers rather than application writers)
At work, we've developed our own set of educational materials for teaching folks the language. I'm working on externalizing them in my spare time.
We started from nothing in January, shipped the app last month.
We have 2.5 developers.
Can you elaborate the process? The only reference I could only find this: https://wiki.haskell.org/Android.
How did you end up developing/deploying your UI?
1. mostly pair programming, so we all knew the current state of the codebase and the path forward
2. When we didn't understand something, we wrote a separate small prototype of that feature / library / reference to teach ourselves how it worked. ( async, monad transformers, lenses, many reflex features )
3. We paid Obsidian for a one hour a week meeting to help us when we couldn't figure it out by ourselves. ( https://obsidian.systems/ )
4. nix for cross compilation and build / dev automation
5. Several hours a week of explicit teaching each other what we knew. This might should go first in the list, as the culture of being open and teaching was a massive benefit. We had one hour a week where the nixpert taught nix, one hour a week where I taught beginning Haskell, another hour a week where I taught Advanced Haskell (things I was learning that might help). We also had two hours a week where we all got together and worked on the stickiest problem along the path to shipping.
Most software dev jobs I've had want me to "do the thing" and have zero time left over for teaching / training others. I wanted to take the opposite approach and this paid off far more than my already wild hopes.
Even more importantly, it tightens the loop between "write code, review code, change code, merge it finally" since you're basically doing all of that "at once" in the best case.
The process was learning Functional Reactive Programming, then learning reflex-frp, then getting a contract with obsidian (creators of reflex) for one hour a week where we could ask questions.
( https://github.com/reflex-frp/reflex-platform )
We had a grant requirement to create a phone client for Tahoe-LAFS, a Python application with a bunch of dependencies, including ZFEC, a forward error correction library.
( https://tahoe-lafs.readthedocs.io/ )
( https://github.com/tahoe-lafs/zfec/ )
We needed bug for bug compatibility with the Python codebase, so I ran Tahoe on localhost and tested the Haskell client against the Python server. We used servant to build the API, since it builds both client and server side from the same description.
I hope this helps
Should be named "private storage" when it does show up in the app store.
Even so, it won't be useful until you have a Tahoe-LAFS shared magic folder.
Hopefully we'll get funding to make a default shared folder for getting started!
Thanks!
https://whetstone.private.storage/privatestorage/privatestor...
Dev environment was our laptops, obsidian's "ob run" is friendly and helpful (but doesn't work well with haskell-language-server).
Dev environment was also our Android phones.
We used nix for cross compilation and build automation, lucky for me the other main dev is really good at nix.
Since the original codebase is in Python, I would have used that if possible, but I spent eight weeks failing to load a shared object into a Python interpreter on Android. That got us to this past January.
I have had a Haskell job before, though I was entirely unfamiliar with Android dev.
In two days I was able to load a shared object into a Haskell interpreter on Android, so we tried that route.
Have you tried fp-ts ecosystem? Code written in it looks pretty similar to Haskell, but it's still the same language, and you can gradually adopt it in the codebase instead of re-writing the whole thing.
(I've also sent a question about personal professional development to your email from profile, hope this isn't abuse of your kindness here)
Overall, we found that fighting against the language this way was the worst of both worlds - you paid a non-trivial cost and did not gain enough benefit. But our original codebase also predated TypeScript (remember TLDRLegal? That's us!), and much of our TS was bolted on after-the-fact on a pretty significant existing codebase that was written in very idiomatic JS. A team with more resources to do technical refactoring (we were quite small and very focused on building product) or starting from scratch may have a different experience.