Rust to stabilize `async fn` and return-position `impl Trait` in traits | Hacker News Reader

Rust to stabilize `async fn` and return-position `impl Trait` in traits

blog.rust-lang.org

231 pointssaghm38 comments

Nice job! I'm glad the team was able to ship a useful subset of the desired functionality. Yes the current implementation shipping next week has limitations, but they're clearly marked, the compiler helpfully explains them and warns you away from some sharp edges.

I'm glad the Rust project is willing to ship useful but limited features quickly, see how people use them, and then iterate and slowly remove the restrictions in the future. I think it'll be more productive than taking another 3 years to solve all the remaining rough edges and problems.

In general, Rust should commit more to stabilizing important features like this. I think because of how easy it is to switch to nightly, there isn't as much pressure to stabilize features as there should be.

Congratulations to all involved. Huge milestone!

That said, I’ve been using it seriously on nightly for a while already… and must say its ergonomics are off putting.

For example I maintain https://github.com/plabayo/tower-async (a fork of tower), and on its own it looks to work (except for the boxing and dynamic dispatching others have already discussed).

But once you throw it into the tokio ecosystem, building on top of something like hyper, and you suddenly are back into nightly territory due to having to specify trait bound for your async fn trait methods (eg: call(): Send).

It works, and you can see in my early WIP proxy repo (https://github.com/plabayo/rama). It’s not pretty but does work… that said it does put you in a nightly position once again due to having to specify Send/Sync trait bounds of trait async fn methods opaque futures…

In contrast the RFC that would allow me to write ‘type Future = impl Future<…’ seems to fit a lot better in the existing tokio ecosystem. Just my 2 cents. Anyway, I get its hard work and WIP, so congratulations again and thx for all the work!

(Edit: I do understand that the main source of my issues is due to the combination of writing very generic code and using multithreaded async (via tokio). Still, it’s not pretty. But perhaps it’s also because I still have a lot to learn on how to use and write this. The latter is my hope)

Can you elaborate on why the `mixing` approach doesn't work? Specifically, if you want send bounds, doing this:

    trait HttpService: Send {
        fn fetch(&self, url: Url)
        -> impl Future<Output = HtmlBody> + Send;
    }

    impl HttpService for MyService {
        async fn fetch(&self, url: Url) -> HtmlBody {
            // This works, as long as `do_fetch(): Send`!
            self.client.do_fetch(url).await.into_body()
        }
    }

Of course that works. But only as long as everything is always hardcoded as Send. Which is a known limitation, so no complaints.

Edit: also the problem is about the method async returned future trait bounds, not the Self.

It's well known that what is being stabilized today is lacking the Send bounds stuff. In fact, there was a lot of discussion about whether they should completely block this feature until the Send bounds stuff was ready. Ultimately I think it is good that they shipped this part of the feature even though the other part isn't ready yet - Tower isn't able to use this yet, but other crates can.

Yes Alice fully agreed. I do understand this. I just want to share my experience as a warning to others. As what is shipped is nothing less than amazing. It would be ashamed if this results to disappointment due to wrong expectations, as happened to me. I’m however very grateful for what is already there.

My hope is that you can use Send variants generated with the macro to reduce the typing, in cases where you need that. In the future with return type bounds, middleware would just use the local variants and only the end user needs to specify the Send or Local variant.

Very important milestone reached for async programming, not to mention that this is actually a primitive that just happens to be useful for async as well. It’s not always needed but when you find yourself needing it, it becomes a major point of friction. Congrats to the team! Excited for dynamic dispatch and capture rules to be coming soon.

It's useful even without async too. Our codebase has some trait methods that return boxed iterators and I'm very much looking forward to switching them to impl Iterator instead.

Does Rust still require you to rely on external dependencies (or roll out your own async runtime) to run async code?

Yes, and that is unlikely to ever change.

But it would be nice to standardize the IO / time interfaces so that swapping out runtimes doesn't require writing a custom adapter layer :). I know you know, just saying that's really probably what OP meant.

That's unfortunate. I hope this gets fixed.

Cargo makes it rather seamless to include the async runtime that best suits your needs. I am curious why do you think this is a major issue?

I'm confused about the example for trait_variant:

``` #[trait_variant::make(HttpService: Send)] ... ```

In the given example, they create a trait with an async fn that returns HtmlBody, then the trait_variant macro makes a duplicate version of the trait but additionally with a Send bound on the future. Makes sense, because users commonly need the future to be Send.

But... what's the point of having two traits? Could there not just be one trait, with the Send bound? Doesn't the Send bound only increase the places it can be used? Are there scenarios where you need the future to not be Send?

It increases the places it can be used, but it decreases what type can be returned. Not all types can be sent between threads (in particular, types that feature unsynchronized mutability through shared references are not Send). If you executor is local to one thread, it can be a blocker if a trait you use suddenly requires your future to be Send.

ah, ok. so this is for scenarios where you, the library author, define the trait, and users will implement it. But since different users have different requirements, you provide two+ trait definitions, so they can pick if they need Send or not.

Yes. Say the HttpService trait comes from some third-party library. I implement that trait on my server, and then I run my server on a single-threaded executor. (This is analogous to how Iterator is in libstd but I can implement it on my own type and then use that Iterator with a for-loop / combinator in my own code.) My implementation returns Futures that are not Send and my executor does not need them to be Send, so it would be wasteful if the trait required the Futures to be Send.

What's the timeline for allowing async fn with dynamic dispatch?

You can do

    trait Trait {
        fn f(&self) -> futures::BoxFuture<...> {
            let fut = async move { /* ... */ };
            fut.boxed()
        }
    }

(which is essentially what `#[async_trait]` desugars to)

I'm not sure how you would even support dynamic dispatch in the general case without implicitly boxing the return value, which would be deeply unpopular within the Rust community and ecosystem. If we're saying we're ok with implicit allocations for futures then the whole "zero cost abstraction" of async/await seems pointless.

I guess the difference is that that isn't an async function. It's a normal function that returns a boxed future.

An async function in Rust is a function that returns a future.

Note that an "implicit allocation" doesn't need to be a heap allocation, it just needs to be dynamically sized. Stabilizing the allocator api (which seems tantalizingly close) and someway to pass a non-default allocator to the implicit box should cover zero cost sensitive folks.

Also note that "zero cost abstraction" means that "zero cost" over implementing it by hand there is no way to return an owned dyn sized type in current rust. Syntax that returns a box is inherently zero cost because there's no other way to spell it.

C++ `co_await` also forces an allocation for similar reasons and lets the optimizer get rid of them in the static case which predictably is a mixed bag in practice; at least in rust you can guarantee the static sized version doesn't allocate. That said I don't like forcing an implicit allocation it feels against Rust's ethos and the language will likely have to have better handling of dyn types writ large to get there.

I agree this is basically an ABI problem. Stripping away the async issue this snippet highlights the issue:

    trait T {
        fn f(&self) -> impl U;
    }

    trait U {
        fn g(&self);
    }

    fn h(t: &dyn T) {
        // What type is u? how big is it?
        let u = t.f();

        // How does this method resolve?
        u.g();
    }

If `U` is object safe, and `f()` returns something that is `Sized` then it's possible to define a non-heap allocated `dyn U` that is essentially a tuple `(<method table>, <anonymous>)` and this code would desugar to something like

    fn h(t: &dyn T) {
        let u: (<method table>, <anonymous>) = t.f();
        u.0.g(&u.1);
    }

But that kind of requires a special calling convention for `T::f` when invoked from a `dyn T` that also returns the size of its return type.

edit: Thinking about it some more, what could be done is that functions called from a `dyn Trait` object that are `impl Trait` have a special calling convention where they return everything on the stack. Then callsites take the top of the stack + offset of the method as the function arg and stack + sizeof the method table as their `self` argument and everything else is fine.

I'm not sure if LLVM supports this directly. I know some stack based language implementations and LISPs will do this to avoid writing register allocators so everything just gets dumped on the stack.

Also there's no way (afaik) in Rust to "spell" a non-heap allocated dyn Trait object. There's bare `dyn Trait` but that means something else and is preserved for backwards compatibility, iirc.

> Also there's no way (afaik) in Rust to "spell" a non-heap allocated dyn Trait object.

&dyn Trait is fine.

  let i: i32 = 5;
  let d = &i as &dyn Display;

Here's the current thinking on how to support dynamic dispatch without boxing: https://smallcultfollowing.com/babysteps/blog/2022/03/29/dyn...

this is so cool :) one other question for async Rust experts...

is it trivial to replace every use of the #[async_trait] with this, once it releases? async_trait was just a macro for adding a bunch of `Box<Pin<dyn Future<Output = BlahBlah> + OtherJunk>` and so on and so on.

Is async_trait entirely obsolete now, or are there still situations where async_trait must be used, where builtin async will not?

edit: if I read one more paragraph I would have seen the answer, it's right there in the post :)

For anyone looking for a TLDR, dynamic dispatch is the major missing piece. The only other reason is if you need to support a compiler version < 1.75

I've seen dynamic dispatch mentioned twice. What do you mean? Isn't that just `fn foo() -> Box<dyn Future<...>>`?

Dynamic dispatch is where you call a method on a Box<dyn T> type, where T is a trait.

Cfr: https://doc.rust-lang.org/1.8.0/book/trait-objects.html

This in contrast to generics which is static.

Yeah and I was trying to implement their workaround for missing dynamic dispatch in a generic-heavy multithreaded async (tokio, hyper) context..,

Cfr: https://github.com/plabayo/tower-async/pull/12

So far didn’t manage to produce a box that works without everything else in my tower async stack having to need send sync trait bounds as well… and of course needing nightly ‘call(): Send’ return bounds.

Despite that, haven’t found a reasonable working solution to support that in ‘tower-async’

Might be just me still having to b learn a lot. More then happy to hear feedbacks add help on these matters.

And of course this is a WIP. So fully understand. Respect to the entire async wg team. Onwards and forward.

`#[async_trait]` does dynamic dispatch, this approach does not.

If you need to use something like `Box<dyn MyTrait>`, then you'll want to continue using `#[async_trait]`.

nice! been waiting on this one for ages. Any idea if removing the implicit box-ing that async_trait uses has a measurable impact on performance?

Are there some cases, say with really big futures, where removing the boxing actually hurts performance?

I don't have numbers handy, but I can say with confidence that the answer to both questions is "yes". It depends on the use case. Boxing is a useful tool and in many cases the overhead is minimal, but you don't want to use it all the time. Likewise, static inlining can bite you in deeply nested cases and I've heard of this happening too.

The future we're working toward is for this to be a decision you can make locally without introducing compatibility hazards or having to change a bunch of code. Ideally, one day you can even ask the compiler to alert you to potential performance hazards or make reasonable default choices for you...

impl trait returns is so cool

Amen to that. Love it sooo much. Looking forward to the day I can use that for associated types. If I can get that for my birthday in May, it would be magical.

Congrats! Looks like it was a monumental effort

Reply on news.ycombinator.com