Algebraic Effects in Practice with Flix

1. The effect data type admits an instance of monad, like arrays or option or result. Not sure why would you put those in competition, an effect is a supercharged ReaderResult. The only differences are in naming choices, traverse being called foreach, flat map being called "and then", etc.

2. There is a much more popular and robust implementation of effects for Typescript: https://effect.website/

I use effects to make guarantees about what the code will and won't do.

When Option was retrofitted onto Java, the NPEs didn't disappear. We got Options that could be null, but more importantly, devs (and the standard library) continued to use null liberally. We never ended up at the tautological goal of "if I have a Person, then I have a Person".

I am equally skeptical of bolting on effect systems after the fact. Even if one of these effect systems becomes 'the main one' and starts showing up in standard libraries, we'll still end up with declared-effectful code and undeclared-effectful code, but will again fall short of the goal, declared-effectful code.

I upstreamed all of my changes so really we should be linking to https://github.com/ocaml-multicore/ocaml-effects-tutorial instead of my repo.

in that same vein, “nothing runs before main” is actually an extremely based take. i suppose this is another take informed by my python (mis-)adventures, but it is so unfortunate that you don’t have just a liiittle more flexibility in terms of the order that code, imports, etc. are evaluated. it is hard for me to give a concise answer as to why i felt this pain, but it is an idea that immediately feels like respite, and gives programmers more leverage / lattitude in making libraries / other code simpler and ergonomic.

They are similar, but effect handlers are more powerful and more amenable to typing.

https://lobste.rs/s/q8lz7a/what_s_condition_system_why_do_yo...

I wasn’t aware of effects until I read the article, but I like the idea. The example you have is just an interface. Effects work a bit like boundary hooks. A pure function only works on things internal to its scope, and so it has no effects. An impure function might affect things outside of its scope, and may want to allow the caller to hook when those effects occur. Think a function that uses the gpu. That’s a side effect, and the function should be decorated to reflect that.

I’m guessing some languages would allow for checking to ensure that any induced effects are correctly noted and advertised so that the system as a whole can be more easily reasoned about.

I _think_ the distinction is that the interface tells you what MovieAPI _does_, but this system also tracks what are contextual requirements to do that thing. So when they write their version of the MovieAPI that calls out to an existing web api, they have to write `\ {Net, IO}` on the main method that uses them. When they write their test that uses a canned recommendation, they can just write `\ {IO}`, and they have a complicated pile of test logic and confirm just at a glance at the top-level signature that no network calls are made when the test runs.

So in java, you could write two distinct classes implementing the MovieApi interface as you defined it, one of which calls out to the web and one which doesn't, and nothing about the type indicates that difference. If you accidentally used the wrong class, your tests could make a network call and you might never notice, because you would not have tracked that effect.

For someone with a java background, it's really helpful to make the analogy to Checked Exceptions, which let us propagate information about what kind of handler must be required contextually.

It's similar on the surface. Another language, Effekt, does actually use interfaces for their effect declarations rather than having a separate `eff` declaration.

The difference comes in their use. There's two things of note. First, the implementation of an interface is static. It's known at compile time. For any given concrete type, there is at most one implementation of MovieApi. You're using the interface, then, to be generic over some number of concrete types, by way of only specifying what you need. Effect handlers aren't like this. Effect handlers can have many implementations, actually. This is useful in the case of ex. adding logging, or writing tests to simulate I/O without actually doing it, or just having different behavior at different places across the program / call stack...

    eff MovieApi {
      def getPopularMovies();
    }
    def main() {
      run {
        println("Alice's movies: ", getPopularMovies());
      } with handler MovieApi {
        def getPopularMovies() = [
          "Dr. Strangelove", 
          "Lawrence of Arabia",
          "The End of Evangelion",
          "I Saw the TV Glow"
        ];
      }
      run {
        println("Bob's movies: ", getPopularMovies());
      } with handler MovieApi {
        def getPopularMovies() = [
          "The Magic School Bus: Space Adventures",
          "Spy Kids 3-D: Game Over",
          "Twilight: Breaking Dawn: Part II"
        ];
      }
    }

Second, the effects of effect handlers are not functions. They're under no obligation to "return", and in fact, in many of the interesting cases they don't. The `resume` construct mentioned in the article is a very special construct: it is taking the "continuation" of the program at the place where an effect was performed and providing it to the handler for use. The invocation of resume(5) with a value looks much like a return(5), yes. But: a call to resume 1) doesn't have to happen and the program can instead continue after the handler i.e. in the case of an effectful exception, 2) doesn't have to be invoked and the call to resume can instead be packaged up and thunkified and saved to happen later, and 3) doesn't have to happen just once and can be invoked multiple times to implement fancy backtracking stuff. Though this last one is a gimmick and comes at the cost of performance (can't do the fast call stack memcpy you could do otherwise).

So to answer your question more briefly, effects differ from interfaces by providing 1) a decoupling of implementation from use and 2) the ability to encompass non-local control flow. This makes them not really compete with interfaces/classes even though the syntax may look similar. You'd want them both, and most effectful languages have them both.

Implementing an interface results in a class that implements it. From then on, the fact your class implements that interface is entirely obscured from your code. In the case of tracking effects, this is clearly undesirable. (Also, in a language that doesn't have classes, implementing interfaces is difficult for me to think about.)

In the case of algebraic effects, they're functions with type signatures explicating the effects the function has. Those don't go away until somewhere nearer the edge of your program, outside the core, you say "by the way, use this to handle those effects"

Effect handlers also give you control over continuations so you can sequence effects explicitly, which is—speaking as someone who writes a lot of code in a language that has algebraic effects as a core part of the language (Unison)—really powerful. Deciding in some instance how you prioritize IO vs Exceptions, making a thrown error go away and replacing it with an HTTP effect that, wayyyy far away from this code handles all HTTP calls by attaching a self-signed certificate when doing the SSL handshake, or whatever, is very nice.

The more I write in this style, the more interesting techniques I come across.

Flix does not have generic effects, yet it has generic traits.

This hints that effects are not meant to deal with data variability.

Effects are allowing variability of control flow.

Abstracting `try/catch/finally` or `async/await`.

For example, something like "durable workflows" is just a specific effect implementation, code stays the same.

To be fair, presumably debug printig could be "escaped" from the effect type checking if the designer of an effect system would want it. For instance, debug printig in Haskell completely sidesteps the need for the IO Monad and just prints in whatever context

I’m sure default effects could handle this, just as Rust has some auto traits and default trait bounds. It doesn’t even have to be the full console i/o effect, it can be a specific debug effect that outputs to console in a dev build and to a file or devnull or whatever in release builds, or you could disable it in release builds and the compiler ensures there aren’t stray debug calls left, without needing a specific lint for that.

(Rust does have a similar problem in that debug printing requires the Debug trait bound which can be really annoying when writing generic code. Would be nice if there were a sprinkle of RTTI involved.)

1. Nonsense [1]

2. It's implementation dependent, but of course you lose tracing, etc if you want to just log with language primitives, which is why you shouldn't when every effect system offers you tools to do so. If you want a system where each side effect is traced, monitored and encoded as a recipe, then you use the effectful version.

[1] https://effect.website/play#769a55e0ea0a

There is work coming from the "academic pedantism" sphere for exploiting single-resumability. For example: https://dl.acm.org/doi/pdf/10.1145/3632896

The Unison language supports algebraic effects and optimizes handlers that call their continuation at most once in tail position (we call these "affine"), so you can have the best of both worlds. Some links at the end if you're curious.

Here are a few places where "multi-resumable" stacks are still useful, even outside of nondeterminism:

* For instance, in a workflow engine a la Temporal, a `sleep` primitive might serialize and store the continuation in a distributed priority queue. The workarounds of not having access to the continuation are all not nearly as good.

* A pure interpreter of a structured concurrency ability is quite useful for testing, since it can test different interleavings of threads and produce tests that fail or pass deterministically. For Unison Cloud's distributed programming API, we have an (in-progress) chaos monkey interpreter that you can use for local testing of distributed systems.

* You can implement a simple debugger... as a library. It lets you set breakpoints and go forwards and backwards in time. Here's an example: https://share.unison-lang.org/@pchiusano/stepwise

Basically, any time you want to stash and do something interesting with the continuation, even if you only end up ultimately using it once, you still need the more general form of algebraic effects.

And then there are various nondeterminism effects that do call the continuation more than once. I'd say these are somewhat niche, but when you need them, you need them, and the code comes out much nicer. I especially like it for testing. You generally want tests to just be the logic, not a bunch of looping code or map/flatMap.

Some links:

* https://www.linkedin.com/posts/pchiusano_dan-doel-has-been-d... has some details on the optimization we do in Unison

* https://dolio.unison-services.cloud/s/blog/posts/optimizing-... is Dan's blog post on optimizing affine handlers

* https://www.linkedin.com/posts/pchiusano_kestrel-is-a-higher... is a typed query DSL that uses nondeterminism in an interesting way. For a declarative query DSL, it's nice to avoid explicit looping, similar to what SQL does.

Resumable code is extremely helpful and I'm pretty sure you're describing a chicken and egg issue: if resumable code was readily available in general, people would use it a lot more and it would look like a very normal tool that is often the best.

But when it's only a feature available in niche languages or many languages make it unsafe or unreliable, then very few people will get to the point where it naturally pops up in their mind as a solution to their problem.

First, you can write a function that's generic on the effects used, so you could write use it in a way that calls an LLM and use it elsewhere in a way that cannot, with the same logic.

But I see the need to change the code as a good thing when it is necessary: I don't want a dependency, inside or outside my codebase, to suddenly add network calls without me knowing it. Same goes for code that was pure and has become stateful. Not knowing this kind of fundamental change is a recipe for a nightmare.

It should be especially apparent with the recent supply chain attacks.

Regarding Effekt, here's an interactive introduction on how to use its effect system: https://effekt-lang.org/tour/effects

Other pages also contain some more advanced details and casestudies on effect handling

Koka, Effekt and 4 other research languages are mentioned earlier in the article though? This talks about practical languages, not research-only

This depends on the implementation. In effect typescript land, synchronous and asynchronous code is treated equally as soon as it's lifted in an effect.

One important thing that languages with algebraic types can usually do and most others can't is being generic in this "colorness" - precisely because this is the color problem they are solving.

So they can e.g. have a `map` function that can take a pure lambda, or an effectful one, and based on that itself will become pure or the given effect. Colors will stop being "infectious" without limits, you can now better barrier their reach.