The Meaning of Monad in MonadTrans | Hacker News Reader

The Meaning of Monad in MonadTrans

parsonsmatt.org

47 pointslemper60 comments

It looks like this article jumps right into the deep end. I'll try to speedrun some context:

* You have monads in your language:

    - Optional, Future, Promise, List, Observable, etc.

* Your language's type system can't refer to them generally. This is why you don't see the word Monad in your language. If it could, you'd see:

    - Optional<A> implements Monad<A>
    - Future<A> implements Monad<A>
    - etc.

* Why is this important? It means you get to reuse the same (familiar!) machinery even when faced with a new library:

    - Turning a List<Future<A>> into a Future<List<A>> uses the same few standard library functions as:
    - turning a Vector<Arbitrary<A>> into an Arbitrary<Vector<A>>.

* So what's the limitation of Monads? They only compose with their own concrete type:

  - Optional compose with Optionals.
  - Futures compose with Futures.
  - What if you want both, e.g. a FutureOptional<A>?

* Monad Transformers let you stack different types of Monads on top of each other, e.g:

  -  FutureT <Optional, A>
  -  OptionalT <Future, A>
  -  EitherT <L, ReaderT<E, State <S>>, R>

* This means you can write functions which use features from any of those monads, but importantly, you can reuse any code written for only one of those monads:

  - An Optional<A> cannot become a Future<A>, but
  - both Optional<A> and Future<A> can trivially become FutureT <Optional, A>>
  - This is called lifting.

Semi nit (but semantically quite important): not `Optional<A> implements Monad<A>`, but `Optional implements Monad`. The reason many languages can't encode the `Monad` type class correctly is due to this fact: type constructors like `Optional` (higher-kinded types, in the lingo) are second-class in a lot of languages, so you can't refer to them directly in your type class definitions/implementations. The practical upshot of this is that (similar to what's touched upon in the article) if you have a true `Monad` constraint you can use the `Monad` operations for any choice of `A` without requiring a different constraint for each possible `A` (which you would otherwise need, because there's no guarantee that `Optional<A> implements Monad<A>` implies that also `Optional implements Monad)`. If you have sufficient machinery available (universal constraints, basically) you can hack around the lack of HKTs by using a constraint like `forall A. Optional<A> implements Monad<A>` and having some kind of type-level map function that gets you from `Optional<A>` to `Optional` for any `A` and `B`. This is the trick people use in Rust, for example.

> there's no guarantee that `Optional<A> implements Monad<A>` implies that also `Optional implements Monad

In a language like Java (and presumably many others) a generic type argument is implicitly universally quantified. Optional<A> implementing Monad<A> would therefore imply that Optional implements Monad.

The problem for why you can't define "monad" as an interface in Java is because you can't refer to the type constructor (due to the lack of HKT, as you yourself explained). That means that you can't even define "map" generically. You can define it for Optional, or List:

  class Optional<A> {
    ...
    Optional<B> map(Function<A, B> f) { ... }
    ...
  }

  class List<A> {
    ...
    List<B> map(Function<A, B> f) { ... }
  }

but you can't extract map into a Monad interface because you'd have to write the return type as "T" or something and that's not possible (for a non-concrete T).

The universal quantification works for defining the _implementation_, but (by itself) not for defining the constraints. Yes, if you write an implementation for `Optional<A>` then it _is_ implemented for any `A` — but it's impossible for an implementation to _require_ it to be implemented for every `A` (and thereby be able to take advantage of that fact, e.g. to be able to know that `Optional<A> → Optional` is well-defined for any `A` and `B`). :)

Thanks for writing this explanation. Now I’ll explain the case against it:

I think I’d want write out the function for a transformation from a list of futures to a future containing a list, give it a name, and explain what it does. This is waiting for all the futures to settle, right? And you have to explain what happens when any of them fails.

And indeed, this is a standard library function [1].

Deriving it instead of naming and explaining it will make it harder to understand any code using the derivation because you can’t look up what Promise.all does. If the code and explanation is written in terms of Arbitrary instead of Future then the explanation will be difficult to write and difficult to read. Defining it for an arbitrary datatype instead of a List would complicate the explanation too.

The lesson of monad tutorials is that there are some levels of abstraction that are usually not worth it because they’re too hard to explain. Explaining how async works is difficult enough.

[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

And how are you going to write all the functions for the countless different combinations of types? This is what many users of the mtl library do and it's clear that it doesn't scale, it's error prone and the code is often not even much more readable. Clearly, some kind of mechanism that is easily composable without ceremony is needed. The real problem is that lifting/unlifting monad transformers which are not monad morphisms (mostly those that have some kind of state) is inherently unsafe. This is where MonadTransControl and MonadBaseControl classes come into play and the whole controversy surrounding them. Haskellers have debated for years about the correct semantics to use here.

I don't need to write all those functions that I'll never call. I write the few that I use in my own code, which is probably doing something fun or practical rather than abstract.

(But if you use these abstractions extensively, maybe Haskell is for you?)

> Deriving it instead of naming and explaining it will make it harder to understand any code using the derivation because you can’t look up what Promise.all does. If the code and explanation is written in terms of Arbitrary instead of Future then the explanation will be difficult to write and difficult to read.

To the contrary! Because I’ve already used that function (traverse) a million times and know exactly how to write it myself I understand exactly what it does without any knowledge about the details of your Future or Optional or whatever. True and meaningful abstraction. Which is truly an amazing super power!

That's okay if you're just writing for yourself. But the next person reading your code might never have used traverse and won't get it.

Then this becomes a question of what sort of common language we should expect of programmers. That's always going to depend on context.

Here are some people trying and mostly failing to explain what 'traverse' does:

https://stackoverflow.com/questions/7460809/can-someone-expl...

If you use a language which has monads, then these things quickly become second nature. traverse, sequenceA, etc are all sorely missed when I use rust, as they are quite fundamental in languages which allow the abstraction.

It honestly provides so much for free when using a new library.

I started learning haskell about 6 months ago and I'm quite enjoying it as it requires a different mental model compared to any C-like language (Java, C#, etc).

Learning Haskell has helped me to cut a good amount of BS from technologies (and engineers too). Haskell needs Monads, Transformers and more things that I don't understand yet because sometimes operations need to be sequenced. And on top of it because it's not possible to perform any non-deterministic (IO) action without telling the world about it, as any non-deterministic behaviour is wrapped in IO.

I also find it interesting that Haskell feels lot more polymorphic that every statically typed OO language I know, when polymorphism is one of the pillars of OO.

If you look at a 30 year old language and people are still trying to explain one of its major mechanics, at what point do you say that it might be taking away from people being able to focus on the programs they want to write?

I don’t think it will affect people’s ability to write good programs - in fact it probably helps.

That said, if a feature is really hard to figure out and it’s a constant pain point for newcomers … maybe it isn’t worth it …

No other language seem to bother with monads to the extent Haskell does.

People (high school and college students) are still struggling to understand calculus nearly 3 centuries after Newton’s death. Some topics are just difficult for many people. Does that mean we shouldn’t bother teaching them?

Calculus is necessary. It wasn't invented as a set of restrictions, it was just just discovered and formalized.

Haskell is not a discovery of something that already existed. It isn't the only programming language out there. There are dozens of examples of other languages that people use more easily. The problem was something invented, not something that has to do with the underlying principles of programming.

If there are easier ways of teaching calculus, maybe those should be used. Actually that is happening now as youtube videos and web pages help visualize concepts.

Monads are a discovery from category theory.

That's great for category theory.

If most people are confused why they're in a programming language, maybe it's time to admit that a language is influential but not made for general purpose productivity.

C++ doesn’t get this treatment when it comes to template metaprogramming, Python doesn’t get it about metaclasses, etc. Every language has tricky parts. The difference is that Haskell’s trickiest part, Monads, are actually very simple. They’re simple enough that they can be explained in a short article, so that causes people to write lots of articles about them.

This is all true except for the claim that monads are Haskell’s trickiest part. Monads are Haskell’s most infamous part but far from the trickiest. There are a lot of Haskell libraries that make the type system twist itself into a pretzel to achieve some really funky things. See Edward Kmett’s lens library for one example.

> C++ doesn’t get this treatment when it comes to template metaprogramming

Quite sure a lot of people hate C++'s template programming.

> They’re simple enough that they can be explained in a short article, so that causes people to write lots of articles about them.

Or ... there are a lot of articles because people keep finding new ones to read because they still don't get it after reading the previous 100 articles.

In their basic form templates solve an obvious problem in an obvious way. It doesn't make sense to make the same data structure for every type, you want to make the data structure generic and sub in the type every time you use it.

The extremes of template meta programming are self imposed by people being fancy while writing libraries and the value is questionable. On top of that no one thinks that C++ templates are the ideal way to do complex meta programming.

Monads are a bit like exiting vim. The complexity is exaggerated to the point of a meme, but in reality not very difficult, and they can be quite useful.

> and they can be quite useful

That's a bit of an understatement, seeing as it's... I mean is it even possible to write a non-trivial application without accidentally implementing a monad?

Most people write monads all the time, and then their head explodes when someone calls it by its name.

Can you admit that you just moved your own goalposts here? Your claim was literally that monads were invented rather than discovered.

I said that haskell was invented and not discovered. The person I replied to was comparing calculus to haskell.

Calculus is necessary

I agree. It’s very important.

It wasn't invented as a set of restrictions, it was just just discovered and formalized.

That’s a bit too casual of a dismissal of an active, major branch of philosophy of mathematics which holds that all mathematics is invented. For fun, and what I take as evidence supporting that argument, look up Michael Penn on YouTube to see how nonstandard analysis works and how you can define the operations of calculus in vastly different ways than you might have learned in school.

The main difference between calculus and monads is that the former traditionally belongs to the domain of analysis and the latter to algebra. That monads haven’t taken off to the same degree is a matter of luck. The vast majority of mathematics has found no use whatsoever outside of pure mathematics.

Finally, I want to say that I think it’s a mistake to think of monads as some set of restrictions that Haskell’s designers put in place to keep people out, like some twisted “you must be this tall to ride the rollercoaster” gatekeeping. They discovered an algebraic pattern in a bunch of commonly used data types and decided to make it into a type class. For them, it solved a major problem of the language at the time: how to deal with effects in a language that uses lazy evaluation.

That’s a bit too casual of a dismissal of an active, major branch of philosophy of mathematics which holds that all mathematics is invented.

This is the problem. You keep talking about "philosophy of mathematics" and I'm talking about programming.

I don't know if anyone thinks that. It just isn't useful to write programs with an arbitrary albatross around your neck because someone else is obsessed with philosophy and set theory.

You keep talking about "philosophy of mathematics" and I'm talking about programming.

You made a specific claim that calculus was discovered but that monads are invented. Are you walking back that claim? Fine.

It just isn't useful to write programs with an arbitrary albatross around your neck because someone else is obsessed with philosophy and set theory.

I've tried to engage with you in good faith but now it appears that you're just concern trolling. The article is an educational piece about monad transformers. Nowhere is it implied that you, specifically, need to use Haskell in your work. If you can't see why monads are useful, why not ask about that?

but that monads are invented

No I didn't, I said haskell was invented.

Are you walking back that claim? Fine.

Quote me where I said that.

I've tried to engage with you in good faith

You haven't engaged in the thing that matters: programming. You went off on tangents about philosophy and monads and set theory and pure mathematics, when my whole point is that they don't help the vast majority of programmers write programs.

Nowhere is it implied that you, specifically, need to use Haskell in your work.

I don't. I tried it and it was interesting but not useful.

If you can't see why monads are useful, why not ask about that?

My point is about the larger issue that after 30 years, people are still trying to explain step one of this language. In that time entire other languages have exploded and withered. At some point the expectations that this is healthy needs to be looked at. I like that haskell exists, but it isn't overall a good tool for making software.

Monads are similarly fundamental in the theory of programming/computation — when you start considering effectful computation at a deep level, monads come up pretty quickly, in a way that is independent of the programming language in question. That doesn't mean they'll stand the test of time as well as calculus has — computer science has had much less time than analysis (or physics, as calculus originally was) to get to the bottom of things — but it was at the time Haskell was designed the most fundamental way of talking about these things, and still a main contender. There are other abstractions with nicer properties, like algebraic effects (which compose nicely by default and therefore don't require an equivalent of monad transformers), but they're not quite equivalent in power — whether they're ‘good enough’ to become the de facto representation of effects depends on how we end up capturing the remaining part.

Haskell doesn't impose monads on a more fundamental notion of computation that other languages expose directly, which is how I interpreted your comment; rather, other languages force a particular monad on you at the language level, whereas Haskell lets you choose your monad and even mix and match different monads that suit your program. This does introduce some complexity in exchange for the additional flexibility, and people may reasonably differ on whether that flexibility is worth it, but the complexity results from exposing something more ‘fundamental’, according to our current theories of computer science, that other languages hide from you.

when you start considering effectful computation at a deep level, monads come up pretty quickly

You say that, but they only seem to come up when dealing with haskell.

Monads are there in computations that we care about, regardless of the implementing programming languages (e.g., Python, C, Assembly, Turing Machines). Just that Haskell is famous for considering Monads as a first class concept, while many programming languages do not name and reason about them (in Python, C, Assembly, etc.).

This is similar to structured loops and function calls (not GOTO), which are in computations that we care about, regardless of the implementing programming languages. Some languages do not name and reason about them (e.g., Assembly, Turing Machines).

While it is possible to program without raising the level of discourse, this is discouraged (considered harmful), as it lacks safety, hurts productivity (developer velocity/experience), and scales worse.

> > when you start considering effectful computation at a deep level, monads come up pretty quickly

> You say that, but they only seem to come up when dealing with haskell.

"when you start considering structured computation at a deep level, loops and function calls come up pretty quickly"

"You say that, but structured loops only seem to come up when dealing with high level languages."

This situation, where a higher level programming language delivers more power (in this case, scales with better developer velocity/experience) but a lower level programmer resists, has been observed many times. Paul Graham calls this the Blub Paradox [1].

    What's so great about Lisp? And if Lisp is so great, why doesn't everyone use it?...

    I'll begin with a shockingly controversial statement: programming languages vary in power.

    ... But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.

And using a powerful high level language is "The Secret Weapon" behind the success of Viaweb [1]. Haskell is also a secret weapon for those who know how to use it.

[1]: http://www.paulgraham.com/avg.html "Beating the Averages"

I can show you the assembly language for a loop and a function call in a dozen instructions. What is the assembly language for a monad?

our hypothetical Blub programmer

This is all patronizing rationalization to believe that people aren't picking someone else's favorite language because they "just aren't smart enough to get it".

Programming is hard enough without constantly trying to make languages themselves a silver bullet. Fancy macros, fance meta programming, 'pure mathematics' etc. never equals productivity over the long term. Library writers can get away with it, but simplicity wins overall because people can focus on making programs and making tools surrounding the language.

Lisp and Haskell were influential, but that doesn't make them good tools by modern standards. They had ideas that made it into other programming languages and that's enough. They did their jobs. Haskell has major programs with controlling order of execution because it pretends it can be abstracted away.

"The Secret Weapon" behind the success of Viaweb

Haskell users always talk about the same handful of programs that no one would have heard about if they weren't made in haskell.

Meanwhile 90% of software that people actually use is basically made in C++, python and javascript. The languages aren't perfect, but when people sit down to program they can move past the language and actually write software.

Languages that include Option, Promise, and Result other than Haskell beg to differ

Only because mainstream programming languages haven't caught up to Haskell. Not that they haven't tried. Java's interfaces are an application of research done on Haskell. It's just that some new ideas are harder to adapt to existing tools than others.

> It wasn't invented as a set of restrictions, it was just just discovered and formalized.

You share good company with this observation. Philip Wadler (who put the monads into Haskell) often talks about functional languages having been discovered, rather than invented.

Haskell is System F (The polymorphic lambda calculus) with the Hindley-Milner type system. (Both Hindley and Milner independently discovered it.)

Curry and Howard observed that this type system corresponds to 1st order logic - types are propositions and programs are proofs, e.g.

    modus ponens:
      P implies Q
      P is true
      Therefore Q must also be true.

    H-M 'App' rule:
      f has type P -> Q
      x has type P
      Therefore f(x) has type Q.

You don't have to dig far into Haskell to find the 'discovered' stuff.

Maybe you are just unfamiliar with Haskell, but the intro to the article mentions the "quantified superclass constraint" which is a relatively new feature.

The Haskell of today is not the Haskell of 1998. It evolves constantly. More than any other general purpose language dares to. Some people don't like that. Some do. It's fun to be on the bleeding edge, for me at least.

I simply wouldn't program if I couldn't use Haskell. I'd maybe clock in and roll my face on the keyboard enough to get paid, but I wouldn't allow my brain to be molded by other languages, lest I become dumb.

If you look at a 30 year old language and realise that most of the progress in mainstream language design over the same time period has come from copying it’s features, at what point do you start to think it might be worth learning?

This a good point. It's sadly common in languages (see also C++, APL, JavaScript etc). But of course lots of people program and use their concepts without having good theoretical understanding.

Python used to be good for this but it's had somewhat of a complexity explosion. What's the best now? Scheme?

Are there any field where haskell is heavily used, beyond PL research ?

I may start to look into it as some point for curiosity, but it would be nice if i'd knew that it would be at least a bit useful somewhere.

One production example I know: GitHub code navigation is written in Haskell https://github.com/github/semantic

There are exciting practical FP usage powered by the likes of Vavr and Algebird in the JVM land. I would like to suggest looking at Scala if you are interested in the conventionally practical. For example I found Scio incredibly exciting

On a more personal note, learning Haskell is more like learning FP and basic type theory, both of which are transferrable skills to most languages. How "useful" is debatable. Maps and filters you can use almost everywhere, ADTs are lacking in most languages, going into things like optics or type lambdas engineers will start feeling you are crazy but that's not even scratching the surface of PL research

[This is a half-joking note. Many real-world programs are written in Haskell and OCaml, but they really are the minority]

> ADTs are lacking in most languages

Native support for ADTs is lacking, but most languages can cobble them together out of OO constructs, and they're so useful that it's often worth the inconvenience.

Algebird is perhaps exciting in the same way as a plane crash. The only "functional" thing about this library is its utterly pointless focus on the simple algebraic structures/typeclasses like monoids and semigroups that are shoe-horned into the API. Yes, it so happens that both HyperLogLog and BloomFilter admit some kind of monoidal structure, but that's hardly the interesting thing about them. What you get is a crappy API centered around the wrong abstraction. The whole thing feels like it was written by an intern who desperately wanted to use some algebra he learned in a freshmen class... Certainly the code quality supports that theory.

For instance, pandoc is written in Haskell: https://github.com/jgm/pandoc

Haskel is being in used in quite a few fintech companies.

I am curious how that is useful at all? I imagine prototyping and the set of libraries that you have is quite limited compared to something like Python. You're also limiting yourself to a much smaller pool of developers which have to intersect both finance and Haskell! I suppose that last point is an advantage of some sort since that filters out a huge portion of "normie" applications.

My understanding is "safety". You need to cover all cases of enums, you can't implicitly coerce a number to a string, you can ensure that there's no side-effects in company code, etc.

I work at one of those fintech companies using Haskell in production.

Hiring was actually Haskell’s strong side. I managed to hire 6 engineers in only one month. There were more than 50 applicants to the position. Just the idea of using Haskell in production is attractive to many.

It’s been a great experience overall. I worked at a python company before this one and it’s been refreshing not having unexpected runtime errors. Moving money around requires more robustness than your typical app.

IOHK uses Haskell in production for the cardano blockchain - I worked there for around 18 months. Irregardless of your thoughts on blockchain tech, it was really fun to work with fairly well known Haskell developers!

Hasura, the hosted GraphQL company, is built on Haskell.

Maybe just a rumour:

https://discourse.haskell.org/t/hasura-migrating-to-rust/662...

Mercury (https://mercury.com/) uses Haskell extensively for pretty much all of its backend systems. It’s a great general purpose language.

Man, the FOMO. I had a friend who worked at Mercury and if I had met her just a few years later I'd probably be working as a Haskell dev there now. I had to chomp my way through _HPFFP_ first, though.

If anyone else is wondering, it looks like HPFFP is "Haskell Programming from First Principles" :)

https://haskellbook.com/

It's used professionally in "backend software development" at many companies. It doesn't dominate (it's a niche language), but it works great for that.

Are monads (I mean more complex than just Option) useful if my program is mostly without side effects? I get how the IO Monad is great, but what I don't really need it?

You can use monads to track where effects are used, and scope them. This allows you to write pure functions that uses effects in their implementation, but the users of the function need not worry about that.

Some monads are also handy for structuring certain kind of code---for example, the non-determinism monad can be handy for writing searches, etc.

Yes! I think side-effects are just one little use-case for monads. They can be useful any time you want to compose functions together but you want to do something else with the composition and data. As Bartosz Milewski so wonderfully explained here [1], Monads are just "a way of composing special types of functions" or "function composition with some kind of flair!"

I was recently writing a Unger-parser-like natural language parser and I wanted to compose functions that took the parse results from one parse function and then parse something else. I noticed that every time I wanted to pass the results to another function I had to do something with the leftover tokens and also coallate the error messages that might have been building up. What if I used monads to automatically handle the resulting tokens and error messages each time I reached for another parse function? I wrote a monadic binding function that handled the tokens and error messages for each parse function, and then I was able to happily compose my parsing functions in a much more clean and concise way.

Monads are great anytime you want to do something more along with your function composition other than just passing one piece of data directly from one function to another.

"And then you say... What if I used a different kind of composition? Also function composition, but with some kind of flair, something additional. If I redefine composition, I have this one additional degree of freedom in defining composition. If I use this additional degree of freedom, I am using a monad." - Bartosz Milewski [1]

[1]: https://youtu.be/i9CU4CuHADQ?t=2363

Yes! Consider jq. jq is almost a 100% pure functional programming language [0], and it's a lot like using the list monad in Haskell.

Various monads basically help you manage things like configuration parameters that affect your computations. Your code can all be deterministic, and yet using a monad can help you write cleaner code.

Just because the IO monad is used for impure actions doesn't mean that monads in general are used for impure actions.

[0] There are a few builtins that have side-effects: `input` and `inputs` (which consume inputs), and then there's `stderr` and `debug` (which emit output to `stderr`). Other than this, every jq program represents a pure transformation of its input.

Reply on news.ycombinator.com