Effing-mad, an effect library for Rust | Hacker News Reader

Effing-mad, an effect library for Rust

github.com

188 pointsagluszak72 comments

Open articleView on HN

This function signature is a work of art: https://docs.rs/effing-mad/0.1.0/effing_mad/fn.transform.htm...

Enough to make even a seasoned Haskell programmer blush.

Now the real question is blush in envy or embarrassment? ;)

I don’t see any GATs. Color me unimpressed. (/s)

What I worry the "Rust is hard" commentary and criticism makes people think of Rust.

99.9% of the time you're never going to touch voodoo like this. Unless you choose that path.

Or if you use a library that chose that path and you end up having to debug it.

I struggled mightily with the borrow checker and lifetimes when I first got started until I got some good advice to just use the heap. String, Vec, .clone(), etc. Just write the code you want and don't worry about allocations. You can fix that all up later when you're more comfortable. It was kind of a cognitive game-changer for me. And now I love Rust and I've gotten better at it.

This is the best Rust onboarding advice ever.

I don't think it is. I ran into the same pitfall. Instead of taking String and passing around &str, I did .clone() everywhere. Kind of gross? The better advice I think is "lean on passing around borrowed parameters as much as possible, otherwise you're going to run into Copy+Clone hell. I would be curious to learn from somebody "even better" at Rust if that is bad advice.

That and, I just wrap everything "hard" in some combination of lazy_static / Arc+Mutex.

Great way to learn your bearings. Then once you get the lay of the land you recognize that's not the way to do things.

It'd be neat if the language had a "beginner mode" flag where the compiler could recommend simple things like abuse of the heap. All you need is a day or two with it, then you can take off the training wheels.

Write one to throw away. Learning is messy and hands-on, not rote and academic.

I would put it as "when it isn't too tricky to do so write your program such that data flows one way".

When data flows downwards it's often simple to have the top layer own and then pass down references.

But if you start running into lots of issues doing that it's probably a fundamental issue with your architecture. So you need to either rethink it or give up and try a different kind of approach. Both are valid, but I think people who try to muddle through without either dramatic option are the ones who end up very frustrated with Rust.

I split my time between our hardware products (C and Rust) and our software product (JS everywhere, node in the back, React in the front).

Reading your comment I had two simultaneous thoughts....

1. <squinting appreciatively> Clever, cool way to think of it... ...and if you need information back at the top level, send it back via a Result....

2. <brows raised in horror, shuddering in React> State management hell!

:->

And then you decide to try nostd :)

I've saved most of this thread so that I can link it when someone asks me why I don't use Rust for systems development yet. In some ways Rust is worse than C++ with these completely out of orbit contrived math problems in the type system.

C is a simple language, but it's still difficult for beginners to understand systems software because it's complex. Adding something like this were people are inventing complexity in the language itself out of boredom is a recipe for disaster and is much worse than the terrible things than can happen in C.

Please just learn to write simple code and solve hard problems, rather than writing complex code to solve contrived problems.

> Adding something like this were people are inventing complexity in the language itself out of boredom is a recipe for disaster

Not really. This is a fun project to show what an effects system would look like. The consequences of this are pretty much strictly positive.

> and is much worse than the terrible things than can happen in C.

No. "Ugly code no one will use but that demonstrates a concept" is definitely not worse than "attacker has full control over the computer".

Complexity is where bugs hide. Do you really think Rust protects you from every class of bugs? It doesn't even protect you from all memory bugs.

Rust just exposes the complexity an equivalent program in C leaves you to find at runtime. It doesn't have to prevent every bug, just substantially more than the languages it might replace.

You can write simple or complex code in any language.

The Obfuscated C contests exist so I really don't see what points you think you're making.

Some people have fun pushing their tools to their limits. That doesn't mean they do the same thing in production code.

Rust is a very obvious security win.

you do realise this is essentially a joke right?

I'm sure every language has examples of code where it was written to be crazy on purpose, This is one of those.

Like when you write a raytracer in CMake, or Meson build, or C++ templates. Etc etc. Or when you compile your program to only mov instructions, because mov instructions are turing complete (should this be a reason we shouldn't use x86?). I'm sure there are plenty more examples.

Nobody is suggesting you actually use this stuff. Therefore it seems a bit silly to point at it and say 'don't use this language because people do silly things for fun in it!!!!11!!1'

Production Rust code doesn't look like this.

This readme is great!

> This means you have to use monad transformers. I don't really understand monad transformers, therefore they are bad.

LOL we all start here.

> Long story short, this library solves the function colouring problem

For a great example of this, see OCaml 5 support for effect handlers. It's basically a generalisation of exceptions, very lightweight and elegant. I can't help but think this approach would have been a much better fit for Rust than the current async effort. Monads like Async and Result are a slippery slope into the world of FP, something Rust will never excel at. Effect handlers are an alternative which arguably better align with systems programming.

I've heard this a lot about generic effect systems. Do you have any good resources on OCaml5 effect handlers I can follow? I dabble in type theory but prefer more pragmatic aspects. I sorta get the OCaml 5 examples I've read but don't know how it works into a generic effect system.

For example, Nim has an effect system (of sorts) and I've been curious what the leap to "generalisation of exceptions" would look like for a systems language. It does feel to me like error's and async's both should remain orthogonal to most functions but we haven't had a good theory for it. What are the missing pieces and what have the OCaml5 folks been up to?!

> Do you have any good resources on OCaml5 effect handlers I can follow?

I encourage you to have a look at the published paper, it's an easy read and covers the motivation and examples:

https://arxiv.org/abs/2104.00250

Here's an excellent talk that walks through modifying a non-trivial code base for concurrency using effect handlers:

https://youtu.be/k3oQwpyXmpo

> It does feel to me like error's and async's both should remain orthogonal

They aren't completely orthogonal though, as both are effects. Haskell and Rust model both with Monads. OCaml 5 can model both with effect handlers. It is desirable to track the difference in types, this is something I hope the OCaml folks will add in the future.

> For a great example of this, see OCaml 5 support for effect handlers. It's basically a generalisation of exceptions, very lightweight and elegant.

A similar approach is being explored for Scala 3:

- https://dotty.epfl.ch/docs/reference/experimental/canthrow.h...

- https://infoscience.epfl.ch/record/290885

- https://github.com/lampepfl/monadic-reflection (https://youtu.be/UmO-f0qTRSU)

See also https://www.unison-lang.org

For those of you following along with keyword generics in Rust, the current proposal is this [0]:

    trait ?const ?async Read {
        ?const ?async fn read(&mut self, buf: &mut [u8]) -> Result<usize>;
        ?const ?async fn read_to_string(&mut self, buf: &mut String) -> Result<usize> { .. }
    }
    
    /// Read from a reader into a string.
    ?const ?async fn read_to_string(reader: &mut impl ?const ?async Read) -> io::Result<String> {
        let mut string = String::new();
        reader.read_to_string(&mut string).await?;
        Ok(string)
    }

which was rightfully disliked [1]. There have been some other proposals however, such as this one I wrote about [2]

    fn foo<F, T>(closure: F) -> Option<T>
    where 
        F: FnMut(&T) -> bool,
    effect 
        const if F: const,
        ?async,
    { /* ... */ }

for which someone made a more detailed issue [3]. It is similar to a `where` clause in that the `effect` clause comes afterwards and defines the effects that a function can have.

[0] https://blog.rust-lang.org/inside-rust/2023/02/23/keyword-ge...

[1] https://old.reddit.com/r/rust/comments/119y8ex/keyword_gener...

[2] https://github.com/rust-lang/keyword-generics-initiative/iss...

[3] https://github.com/rust-lang/keyword-generics-initiative/iss...

If Rust had algebraic effects, your example would be:

  fn foo<T,E>(closure: impl FnMut(&T) -> bool | E) -> Option<T> | E

All that's needed syntax-wise would be a way to attach one or more type parameters to a function type. (I used "|" here but it could be anything)

And the way you'd do it with this library is

    // very complex and powerful API
    effing_mad::effects! {
        Read {
            fn read_to_string(bug: &mut String) -> Result<usize>;
        }
    }

    #[effectful(Read)]
    fn read_to_string() -> io::Result<String> {
        let mut string = String::new();
        (yield read_to_string(&mut string))?;
        Ok(string)
    }

and then to call it, you would do

    let handler = handler! {
        Read {
            read() => reader.read(),
        }
    };

    let action = handle_group(read_to_string(), handler);
    let result = run(action);

I really hope the ??? vomit isn't accepted.

You have to read the post related to this proposal (first link). It shows why it makes sense more than whatever is proposed in this thread.

I agree that it looks like trash, though.

No it doesn't, not in the form it is proposed.

Sometimes aesthetics matter though, and this is certainly a candidate

What you’re highlighting here is something we explicitly call out in the blog post as what we don’t want people to have to write. We agree the syntax is noisy and would be bad to work with. We instead would prefer it if people could write something like this instead, which would have the same semantics as `?const ?async fn`:

    effect fn read_to_string<R>(reader: &mut R) -> io::Result<String>
    where
        R: effect Read
    {
        let mut string = String::new();
        reader.read_to_string(&mut string).do;  // note the .do here
        string
    }

The exact syntax of the design is something we’re far less committed to than the semantics of it. But we didn’t do a good enough job at communicating that in our last post, so that’s on us.

Why would you want the function to run over all effects instead of explicitly outlining what effects you want the function to run on? It seems like the latter is more verbose but more explicit.

Not all effects, but a select set which are covered by the `effect` keyword. That set could change over edition bounds. The reason for that is because we don't just care about maximum expressivity, we also care about ensuring good readability. It would be really bad if every function, type, and argument had to repeat `?const ?try ?async ?panic ?send ?leak ?sized` and other effects over and over and over. Having a keyword to represent a set of effects which functions are expected to be generic over seems like the better option.

I do hope that in maybe 2027 or 2030 we can switch the stdlib over to const-by-default, so `const` would no longer has to be part of the effect set. But it's unlikely we'll ever want to implement "async-by-default" or "try-by-default". So there likely will always be a need to name a set of effects which functions can be generic over.

I think the issue about syntax is that the ?-mark soup doesn't look great, but many people likely won't mind something like the `where effect` proposal that I mentioned above. Still, I understand your point about having an effect keyword in general.

Yay, I’m glad that made sense!

It may help to know that "[computational] effect" is supposed to mean these things together: - a particular class of computation, in a programming context where we want to define the class and/or take advantage of computations being defined in this way - to this end computations are wrapped in a type constructor (typically with one argument e.g. State<T>, IO<T>, Maybe<T>).

The jargon is of course terrible and unhelpful (but widely used and used somewhat consistently). It is not necessarily about side-effects, it is not necessarily about interacting with environment. Calling it "algebraic effects" makes it a bit mysterious the various supported methods/callbacks are more or less like an "algebra", since they involve the type parameter T. To top it off, this use of "effect" is quite different from, say "type and effect system", and Moggi's 1991 never used "effect" in "Notions of computation and monads."

Aside from this stuff being quite cool & interesting, love the tone of the readme, especially the list of Cool Things :D

It does really stand out from how documentation on a low level library has traditionally been presented.

I was deep into effect languages (Koka) and did some effects on my own in Scala for some years.

After a year with trivial Go I just wonder why.

Could you elaborate? What did you find unnecessary or bad about effects?

I've dug deep into Haskell, and my day-to-day job is Go, so I can probably sort of answer this at least for myself.

A lot of time, heavy-duty functional programming advocates will use as their foil what you might call either the worst of imperative programming messes, or if you want to be more generous, an "average" imperative programming mess, where I include OO in the "imperative" category for the purposes of this conversation. Against this they will present their world of pure programming and recursion-scheme-based programming and super strong, complicated type systems, and claim it is better.

I agree, for the most part. I mean, I could elaborate and add half-a-book's worth of nuance to that, but for now, I agree.

However, when I, personally, sit down in front of Go and program with it, I am not some abstract "imperative programmer". I am someone who is writing Go having been informed by the lessons from Haskell and Erlang. My code is not perfectly functional, because that is not the optimum, but you don't see global variables flying everywhere in my code. You don't see my code being written where everything takes a dozen locks simultaneously to do anything. You see actors constraining things. IO may not be rigidly separated by a type system, but in a lot of my code you can see that I isolate IO behind an interface.

If you are feeling a bit generous and squint a bit, I write a lot of my Go code as a free monad with an interface being used as an interpreter, which allows me to use a "pure" implementation of that interface, if you also squint and allow the initial construction of the value to count as a "pure" call and the output of the test implementation to be considered as a pure output, as a test driver. This turns what superficially looks like imperative, highly stateful code that may even have extensive dependencies onto external state into pure code when used with its pure interpreter, which is great for testing. Then I can also write very impure, but highly focused, integration testing on to the "real" free monad interpreter to be sure it works as I expect, with its interface being minimized to just what the interpreter needs which makes the testing easier.

So when I sit down, in real life, with a real engineering problem, the choice I face is not "Write Haskell/Rust and be pure and wear the hair shirt" (as the saying goes) and "write imperative code that will bring the world down around my ears". It's between the first, and "write in an imperative language with guardrails inspired by functional and stronger languages that actually do a pretty decent job." As a result, the choice I face is not so much day versus night, but day versus "overcast but warm day". I don't deny there's still a difference and a value in the harder, stronger, more rigid languages, and there are absolutely still tasks I might reach for them for. But the value proposition the harder, stronger languages bring me are much more muted than they might be for someone else.

Effects are cool, I hope people continue to research them. I'd love to play with them sometime. Thumbs up to this Rust experiment too. But the simple truth is, they don't solve a problem I actually have right now. The "effects system" I implicitly get from already being careful with IO and my separation between IO-using code and the IO drivers solves my real problems, and nobody else on my team is reading my code and going WTF, because the value proposition is obvious after just spending a minute or two reading the test code.

Again, I want to emphasize, I'm not saying my solution is perfect and therefore anyone who uses any harder, stronger languages are wrong. I'm very explicitly saying the opposite. I'm just saying that when you include choices other than a binary "use the strongest, hardest language possible" and "be cast into the outermost darkness of pure imperative programming, where there is much wailing and gnashing of global variables", you may find that the ideal engineering balance isn't either extreme.

(Though I would say it's closer to the former than the latter. Undisciplined imperative programming is every bit the nightmare the functional advocates say it is, and what happens if you try to multithread without discipline hardly bears thinking about.)

All that said, I highly, highly recommend learning something like Rust or Haskell and becoming good at it. You can kinda sorta pick these principles up in other looser languages, but there is a ton of value in working in an environment where the compiler will rigidly enforce these practices. If those are still too strong, even Erlang/Elixir will give you a lot of practice, with a bit less rigidity. It is so much faster to learn in the stricter languages than in the looser languages. And it's a valuable skill that you may then someday deploy when you encounter a task where you need the full strength they offer.

"I am someone who is writing Go having been informed by the lessons from Haskell and Erlang."

This is what I saw with good Java developers 20y ago, E.g. controlling side effects and trying to distinguish IO and pure methods, in Java, 20y ago.

There really is nothing new under the sun. Below the furiously churning surface of the programming world are calm waters barely disturbed in the past 20+ years. And it's not all that far below, either. I've come to resist learning new techs not because it is hard for me at my age, but because it is too easy, and it is easy to get sucked into spending all my time learning the latest churny surface gloss on old ideas without ever exploiting a particular tech to do something useful because I'm moving on to the next churny surface gloss on the same ideas. Gotta actually build something at some point.

I didn't want to put down the effort of the OP, because even they admit it's an interesting experiment, not something they recommend that anyone use, and I do think it's an interesting experiment. But I have exactly this feeling in general.

I like the idea of Rust, but I feel it has, unfortunately, been taken over by the seductive idea that abstraction is the purpose of a programming language (e.g the C++ crowd, among others). I can say from long experience, that while it probably won't impede its popularity, this isn't real progress in programming language design. These abstractions just create their own problems to solve on top of solving the original problem you wanted to solve by writing a program in the first place.

Inevitably, that means that you end up having to limit yourself to some particular "idiom" or subset of the language in production, so that you can get anything constructive done without the code being inscrutable or unmaintainable or just overblown for the task that's being performed.

I knew it was over when they started debating adding more and more meta-language type system features, and then added async/await -- which is the very definition of creating a problem to solve a problem.

So, as much as I appreciate Rust, I am looking forward a newer systems language with more discipline in its design and direction.

Go isn't perfect, but it definitely trends to the right flavor of simplicity and design discipline.

I like Rust a lot, enjoyed working with it for a year, I'd just wish the borrow checker would be easier with structs that have data from different sources - so I also didn't want to put down the effort and quite frankly might take a look.

"been taken over by the seductive idea that abstraction is the purpose of a programming language"

This downed Scala.

I legitimately laughed out loud reading through the readme a few times. Also reminds me I should probably get around to really digging into Pin/Unpin.

Jon Gjengset's stream on pin/unpin is invaluable for understanding Pin semantics: https://youtu.be/DkMwYxfSYNQ

But the 10000 ft view is: Pin lets you tell Rust to never ever move a piece of memory. This is almost always done because the piece of memory has pointers into itself (it's self referencial) which would be come invalid if moved.

Algebraic effects is a fantastic type system feature that needs to break into mainstream ASAP.

I agree but Rust is extremely unlikely to go in this direction, even though it is a pretty good fit.

They're already working on keyword generics which is a limited version of this.

I don't think anyone would describe their proposal as a limited version of algebraic effects. It's missing the algebraic part for one thing.

That's how the authors describe it, in general terms at least: https://blog.rust-lang.org/inside-rust/2022/07/27/keyword-ge...

When I wrote that I didn’t really understand the difference in terminology between “effects”/“effect types” (e.g. keyword-based modifiers on functions and types) and “effect handlers” (e.g. typed co-routines).

Many effectful languages have both, so teasing them apart can be confusing. I think I can now more comfortably say that what we’re working on is an extension to Rust’s effect system. This has nothing to do with effect handlers.

?with ?lots ?of ?interesting ?syntax.

This project has the least cringey Rust vibes vibes I have seen so far. I love it.

Some over-the-top rust advocates (not naming names) almost made me ditch wanting to learn rust. I'm glad I stuck with it, as it has some lovely features, and after some weeks of frustration, it's fun to program.

Good effort but still this looks pretty awkward compared to Haskell and Scala cause of the type system limitations.

I mean, it's a library that implements a language feature. It's surprising it's possible at all, even if really ugly.

Less jarring than the keywords blog post to be honest. This doesn't skew away from existing Rust syntax at all, whereas the keywords proposal introduce more syntactic noise.

Well it introduces `yield` and multiple macros. Presumably these would turn into keywords/ syntax at some point.

Pedantically, `yield` is already a keyword [1]. As the project's README states, generators [2] are an unstable part of Rust, and the `yield` keyword goes with them.

[1]: https://doc.rust-lang.org/reference/keywords.html?highlight=...

[2]: https://doc.rust-lang.org/std/ops/trait.Generator.html

This sent me down a rabbit hole... I didn't know the "function coloring problem" was a thing, but this was a hilarious read https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

For context: https://blog.rust-lang.org/inside-rust/2022/07/27/keyword-ge...

Has anyone gone from not "getting" the value of algebraic effects to getting it? And if so, what was the tutorial/process by which you achieved enlightenment?

To each their own, ymmv. This was my gateway drug: https://arxiv.org/abs/1611.09259

Something I don't understand yet (maybe I should just read your linked paper, but maybe it doesn't answer this either): are algebraic effects necessarily founded on delimited continuations (of the reset-shift0 variety) or are there other kinds of algebraic effects?

I think that's largely an implementation detail, so not a requirement as far as I know. That said, delimited continuations certainly make them easier to implement, as I understand it.

I took a look at the paper and it too shows that the effects are just delimited continuations with named, algebraic-typed prompts. It even goes into the difference between shift0 and shift, but not by that name. (It claims that Koka and others are using shift, which isn't quite what I remember but okay.)

I guess it's a good marketing move because "multi-prompt delimited continuations" is scary. It also makes the types sane; I once worked out that expressions with delimited continuations of the reset-shift0 variety are typed by binary trees of ambient types, with a subtyping relation s.t. leaf nodes can be expanded.

Erdős thought God had a book of proofs. I think he was slightly off the mark: God has a book of abstractions, and the good proofs are the ones which use abstractions from the book most judiciously. Anyhow, algebraic effects are in The Book.

Reply on news.ycombinator.com