Overview of Rust error handling libraries | Hacker News Reader

Overview of Rust error handling libraries

blog.yoshuawuyts.com

222 pointsagluszak79 comments

This is a nice overview of Rust's available error-handling crates, and some improvements to the language that are error-related. But I'd like to quickly point out that you do not have to pick a library from this list! I don't want anybody to read this article and go away thinking "I need to learn nine libraries in order to use Rust effectively". In the majority of cases, I've found what the language and std offer is enough for me.

(In fact, it leaves my favourite solution out: using the derive_more crate to auto-generate From implementations for the ? operator to use. From is a general trait, useful for far more than error handling, which is probably why it's omitted here.)

Yeah, it does seem overwhelming from the outside. But all these libraries are just variations of boilerplate needed for the `std::error::Error` trait.

And the variety exists, because there are different needs and trade-offs:

• You may want an error type that takes 0 bytes, so that `Result` is cheap for trivial high-performance functions (e.g. parsing libraries that may fail on every byte).

• You may want your error type specify exact cause of failure in your library in a programmatic way, so that callers can turn it into a precise, localized error message for the end user.

• You may want to have a super rich detailed error object with all the context, e.g. an entire HTTP response for a 5xx status.

• And some application developers prefer having an "I don't care, just give me the backtrace" error wrapper for everything, to speed up the development and not spend time on micro-managing error types.

All of this works nicely and uniformly in `Result<Ok, Err>` with some `Err` type written (or generated) by you.

> And some application developers prefer having an "I don't care, just give me the backtrace" error wrapper for everything, to speed up the development and not spend time on micro-managing error types.

That's me tbh. I actually love Rust's error story atm, I always use an Error enum and I have made it do everything I want.. except one thing: Low (no?) cost backtraces.

My error enums give me all the API-level specifics I want, but often I make the variants grouped by general code area. It's guided by my program flow, not debug introspection. So if I don't need to branch a codepath from an error object, it typically just goes into a single variant and I shove some message in there.

What I'd love is to be able to easily and quickly just bake backtraces into this "non-program flow defining type". Bonus points if it could be done at compile time.

I've thought about using a macro at error creation time to make it a compile-time poor mans backtrace, but I'm unsure if just one level of stack trace is all that beneficial.

Either way I know I want something that's no more costly than:

    foreign_err.map(|foreign_err| MyError::Dump(format!("some description: {}", foreign_err)))

edit: I should add, internally my perspective is coming from Go, so wrapping repeatedly in strings is fairly welcome to me. Just adding some low-cost default context to this would be a huge help.

An unstable `std::backtrace::Backtrace` (https://doc.rust-lang.org/nightly/std/backtrace/struct.Backt...) exists in nightly Rust. If you cannot require nightly, you can use the backtrace crate (https://docs.rs/backtrace/0.3.40/backtrace/) on stable.

Crates like my SNAFU (https://docs.rs/snafu/0.6.0/snafu/) help abstract this away. Whenever the standard library implementation stabilizes, you will likely see many more errors including backtraces.

Are you familiar with how costly backtrace is?

I fear bubbling an error up and repeatedly adding a stack trace or some junk like that. Admittedly I have not looked too much into Backtrace, but I hope my concern makes sense.

I do believe it's non-zero, as the stack has to be walked and potentially symbols are looked up. The symbols can be delayed until display time, but the stalk walk is "now or never", as I understand it.

SNAFU addresses this by allowing `Option<Backtrace>` in the error struct. The backtrace is only captured when an environment variable is set by the end user.

It would be interesting for someone to port the Linux kernel’s ORC to user code. ORC backtraces are exact and more than an order of magnitude faster than DWARF.

> Are you familiar with how costly...

Checks the username of who you responded too: I'll blindly bet yes to that question for any amount you desire.

Please forgive the tangent, but I'm curious about how people are using Rust if it's not in their primary day job? My impression is that there aren't yet many Rust jobs, but many practitioners. What are people building with it? For Rust enthusiasts, how do you use it? Are you contributing to an open source project or do you have your own pet projects? In either case, what are the projects? I'm asking because I might have some time to play around with it over break.

Here's a list of applications written in rust: https://github.com/rust-unofficial/awesome-rust#applications

People are using rust as they do other general purpose programming languages that have somewhat of a focus on systems programming: to write code. Your question isn't really easy to answer in the same way that someone asking "what are people building with python?" isn't useful. The answer is everything from webpages to text editors to operating systems, and everything in between.

Programming languages lend themselves to different applications and the kinds of projects one contributes to in one's hobby time are different than those one contributes to in one's professional time. As such, I wouldn't expect hobbyist Rust users to work on the same distribution of projects as professional Python developers. In any case, I just want to know what people are inspired to work on with Rust; it's an open ended question, and it's okay that this question has no single easy answer. (Also, "what are people building with Python?" is a very interesting question to me; it's pretty much why https://github.com/trending?l=python and similar exist--and note, the kinds of projects that trend for Python are very different from those that trend for Rust).

We're using Rust for general purpose APIs. What would normally be written in Python or Go, for us. We just prefer it. We're also using it for a couple memory/generic intensive applications that were previously written in Go, but the lack of Generics made some API design awkward.

I use it for all my personal projects as well. Which are basically the same types of things I wrote in Go. Though, I also am experimenting with WASM DOM UIs, something I've not yet used in Go.

I've been using it as a way to learn some of the fundamentals of my field and write packages from other languages in Rust as a learning exercise. Then once those reach some level of usability / goodness, I've started building small CLI's for $work.

I have a feeling that inside of 5 years Rust will be blowing up on the job market. It's just so good and so practical that once you pick it up, anything less well designed will feel like a kludge and hack that might blow up on you at any point.

I use it for essentially every programming task, with the exception of using matplotlib to generate figures. I've personally used Rust in all of these areas to some extent: APIs, data processing, blackjack engine, operating systems, bioinformatics, etc.

My current pet project in Rust is writing a functional language compiler that compiles down to System F/F omega and eventually I'm going to build some kind of virtual machine

Even if you're not working in a rust company if you have some leeway you can use rust anyway for non-critical parts.

E.g. I have written a cache-warmer that preopens and does readaheads (posix_fadvise) for assets from NFS and offers the already open file descriptors to the calling process. The NFS share offers decent throughput but terrible latency (compared to NVMe). Opening files is slow, walking directories sequentially is slow. And then hitting cold pages is slow. Doing all this in rust and handing the file descriptors over to node lets me reduce minutes to seconds. But if someone doesn't like it, there's a flag that turns it all off, back to the slow path.

Another use is essentially replacing shell-scripting when some REST APIs, JSON parsing and so on are involved. E.g. to glue together some status reporting in CI. There are so many different unreliable APIs without client libs involved that I appreciate the error handling.

We're using it for a lot of non-critical stuff, and a few small semi-critical jobs. It's useful training to re-implement system test utilities in Rust, for example, then extend them once they work. Rust makes some things much easier (getting a backtrace on error, compared to C++ :)

I use rust for commercial projects because I can specify the language and sometimes rust is the right language (most of the time python is the right language). I was hired into this role many years ago as a PHP developer. And we haven't needed rust devs. So I have no experience with the rust job marketplace.

I also have a lot of personal pet projects written in rust: an MMORPG-style game, accounting software, various WASM widgets, some OS fiddling, some work extending servo as a browser that implements umatrix-like control but lets you set options per domain (e.g. 3rd party cookie, user agent setting, etc, all per-domain).

I've also contributed a number of small libraries that I wrote for my projects to open source (textnonce, float-cmp, mailstrom, formdata, mime-multipart, solvent, ddsfile, pemmican, resolv-rs, email-format, and others).

I tried out a few of the error handling libraries, but these days I reverted to just defining a simple Enum with the types of errors that occur in my library, for most use cases that seems to be the simplest thing to do. I implement `std::error::Error` and `std::fmt::Display` for that enum and add `From` implementations as I need them to tidy up my code.

It's usually not much more code than I would have written with an error handling library and it's fully transparent, understandable for anyone with a rudimentary understanding of Rust. And I get to have one less dependency.

I used to do this as well, but I came across thiserror (which is featured in the article), which is just macros that do exactly this and nothing more.

It's certainly less transparent, but I prefer this method of error handling, and writing out the Impls manually can get out of hand (as well as having to adding new fields to the enum).

In the early days of Rust 1.0 the `Error` trait was rather cubersome (error messages needed a static lifetime). That is what caused people to reach for crates like quick-error and error-chain. But then the `Error` trait got extended and the old methods (description and cause) got deprecated and replaced with default implementations. It is much nicer now :-)

I've never quite understood all the interest in these error-handling libraries. I've had great success even in large projects by impl'ing From when I need implicit conversion between error types. Most of the time, though, I want to actually handle the error at the calling site. I don't "bubble up" errors because they mean different things at different levels of the stack.

I think the biggest benefit of these is automatic backtraces. That's what I tend to miss from e.g. Python. But this is usually more of a concern for applications than for libraries. If my application spits out "IO error: nonexistent file", I need to know which line it came from to know which file it's talking about. But in a library it's usually more obvious.

I mean, don't just bubble out errors until they lack context then. Rewrap them to add more context like ConfFileReadError(io::Error).

I wholeheartedly agree with this sentiment. Another alternative, at the cost of some log noise, is to emit a meaningful log message at every call site that has to handle the error condition.

That would be a major point of my library, SNAFU, discussed in the article.

https://docs.rs/snafu/0.6.0/snafu/

The problem I identify with this is that the error creator needs to capture the backtrace for it to be useful. By the time the error has reached the application, the innermost parts of the stack have been popped and a backtrace is of limited usefulness.

This is part of the reason that SNAFU tries to make it lightweight for library authors to add backtraces and allows application developers to turn them on.

> If my application spits out "IO error: nonexistent file", I need to know which line it came from to know which file it's talking about.

Last time I needed this, I created a simple ErrorWithPath<E> wrapper around Error which also stores the path as a String, and a helper function which calls a closure and wraps any errors on its result with that wrapper. (If you want to take a look, it's at https://github.com/cesarb/filestatrec/blob/master/src/error....)

I think the interest in dynamic errors (which is what most of the error libraries deal with) comes from (1) people learning Rust and trying to get them to do something besides `unwrap`, (2) people prototyping and not wanting to use `unwrap`, and (3) applications where a lot of errors you just want to bubble up with some context.

Coming from other languages I always thought of Errors as something 'other' that just kind of magically work within their own little exceptions code path. So in Rust I was expecting errors to be similarly complex and 'other'.

I had a major aha moment recently when reading something that described errors as just types. When put that way it suddenly made so much more sense to me.

I like file and line numbers with my errors:

  // No need to use this directly. See the e!() macro.
  #[derive(Debug)]
  pub struct ErrInfo {
      pub file: &'static str,
      pub line: u32,
      pub msg: &'static str  
  }

  impl std::fmt::Display for ErrInfo {
      fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
          if cfg!(debug_assertions) {
              write!(f, "[{}:{}] {}", self.file, self.line, self.msg)
          } else {
              write!(f, "{}", self.msg)
          }
      }
  }

  /// Use this with `failure` crate's `ResultExt` like this
  ///
  /// ```
  ///    $r = function().context(e!("Function failed"))?;
  /// ```
  ///
  /// Error context will then include that message, and the file and line number.
  #[macro_export]
  macro_rules! e {
      ($msg:expr) => {
          $crate::ErrInfo {
              file: file!(),
              line: line!(),
              msg: $msg
          }
      };
  }

Some of the highlighted error crates will include a full backtrace if you are using nightly rust. Not sure when it will hit stable.

Some also produce backtraces on today's stable Rust and previous versions as well. SNAFU can produce backtraces back to stable Rust 1.31.

I've been doing that sort of thing in firmware for a long time. Record the file, line number and the caller and restart. 90% of the time it's obvious what the problem is by inspection.

Rust brings some great features to the table but the combination of them opens up a new level of challenges to deal with which is why there is so much talk in the community on error handling libraries.

Some of the competing needs that need to be dealt with to understand why include

- Quickly create errors and extend them for prototyping while scaling to programmatically reacting to them

- Easily convert errors with `?` has drawbacks. `?` wants you to implement the `From` explicit conversion trait but that becomes part of your public API, exposing implementation details (switch dependencies or one bumps major is technically a breaking change for you). Another problem is you can't both implement the `Error` trait and support `From` from anything that implements the `Error` trait because it will conflict with Rust's blanket implementation of `From` converting yourself into yourself. We need specialization to solve this.

- You can return `Result<T, E>` from main but it prints `Debug` information rather than user-friendly `Display`. So we need to deal with `Debug` printing the right thing most of the time while printing something user-friendly in `main`.

- Rust users have their errors wrap underlying errors in a giant chain. Rendering of this chain needs to be decoupled from a single `Error`'s `Display`.

- We aren't clean which errors in a chain are context and which are meant to be actual root causes to programmatically act on

- Supporting `no_std` for embedded cases.

On top of all of that are unsolved problems like process exit codes specialized for the error being returned, debug-only information (e.g. opt-in to rendering an original `errno` or `HRESULT`), error cloning (important for caching failable-operations), error serialization (important for network proxying APIs).

I think some in the Rust community are seeing Yoshua's comaprison and feeling like we are converging on a solution. While I think that is true for `derive(Error)` (code-genned trait implementations), I don't think this is true for our patterns for how to structure and communicate content in errors or how to do Dynamic errors. This post made me feel like Rust is in a rut. People have taken a pattern or two from cargo's source and refined it without branching out more into other ways of solving the problems. This ends up leaving a lot of the problems i listed above unsolved or less than ideal.

I'm toying with ideas for an alternative approach for solving most of these problems; just need to get some more time to get it in a shareable state.

> Another problem is you can't both implement the `Error` trait and support `From` from anything that implements the `Error` trait because it will conflict with Rust's blanket implementation of `From` converting yourself into yourself.

I believe this issue would only come up if you're trying:

    impl<T: std::error::Error> From<T> for MyError

which seems like a really clumsy way to handle it. In most real cases, wouldn't you write the From impls for the specific types of errors you're expecting?

> which seems like a really clumsy way to handle it. In most real cases, wouldn't you write the From impls for the specific types of errors you're expecting?

In an application, yes. The problem is in libraries because the `From`s` are public so you have put the error type for your dependency in your public API and if you change dependencies or upgrade past a major, you've broken your API. Granted, it is unlikely someone is going to use it ... but I wouldn't put it past people to take shortcuts like that.

A benefit of SNAFU is that the `From` trait is implemented against the internal-by-default context selectors. Since enum variants are public if the enum is, SNAFU also offers a way to wrap an opaque newtype around the internal error, allowing the implementor to choose exactly what public API their error exposes.

One error handling pattern I’d like to figure out how to implement in Rust is where you trace the error out from the point it was created rather than capturing a stack trace at creation time. Tracing the error this way is impactful when your application spans multiple processes or machines. Client/server systems, for sure (you can know what the server did to error, without extra noise in the trace from below where it received the command, and you also get a trace of the client’s side. It’s substantially more useful when the workload is distributed more.

The best I’ve come up with is using a custom ResultExt class which implements std::op::Try, marks its trait methods as inline(always), and then captures the program counter whenever you use `?` on it in order to build a trace. It’s not the worst, but it still feels a bit odd — particularly the forced inlining since that will surely break if you’re working with std::ops::Try from behind a trait object.

Could you provide a deeper example of the difference you mean?

Additionally, have you seen tracing (https://docs.rs/tracing/0.1.10/tracing/)?

You can also make it never inline, and use the return address instead of program counter. That's what zig does to implement this feature ("error return traces").

This would mean an extra function call for _every _ use of `?` (rather than a conditional jump that the branch predictor has been well-primed for), even in the non-error path. My Rust code for some projects use `?` pretty frequently, to the point that I’d be concerned about perf. Of course, the only way to be sure is to profile. Do you have any perf measurements with this technique in zig?

By default this feature is enabled in the Debug build mode only, so its perf impact isn't a concern in release builds. But that being said, the function call only happens in the error path, when returning an error from a function, right? Are you concerned about the perf of the error path? Maybe I'm not understanding something about your implementation. But yeah the way to be sure is to profile. When I get around to it I'll add a flag to allow enabling this feature in release builds and do some profiling.

> But that being said, the function call only happens in the error path, when returning an error from a function, right?

std::ops::Try looks like: - fn into_result(self) -> Result<Self::Ok, Self::Error>; // called on `x` whenever you use the `x?` syntax. - fn from_error(v: Self::Error) -> Self; - fn from_ok(v: Self::Ok) -> Self;

It's been a while since I explored this. I was thinking you'd `#[inline(always)]` the `into_result` method, but now that I look at the trait again, I think you would only force inline the `from_error` method and it should still work but without impacting the non-error path much.

I really don't like this whole "screw the type system, just throw string errors that you don't know the type of" behind all of this. It feels like backporting golang's error handling to rust, which feels like a step backwards.

That's a really unfair characterization of Go's error handling, and something only really novice Go developers do.

https://dave.cheney.net/2016/04/27/dont-just-check-errors-ha...

And pretty much every production golang codebase I've seen.

I've been using Go for 7 years and I don't think this is a mischaracterization at all. I definitely don't think it's something "only really novice Go developers do"--I see it all the time, including in popular open source libraries and even the standard library. I don't think it's contentious that Go's error handling needs some improvement (by which I mean "more standard, structured error handling" and not "syntax support for eliminating error handling boilerplate"--the latter is more contentious as far as I'm aware).

Every time you use `fmt.Errorf` and `if err != nil { return nil, err }`, you're returning a stringly typed error up the stack.

The entire stdlib does this. An error stack is never present from stdlib functions' errors, and third party libraries I see it just as infrequently.

If only novice go devs do that, than the entire go core team are novices, as well as the authors of popular go projects like kubernetes and docker.

"Stringly typed" means literally a string. Go doesn't always distinguish between errors, but the error type does distinguish errors from non-errors and that's important.

It seems better to have coarse-grained types than to standardize the wrong type hierarchy?

Even some std packages do it.

The std lib is not the paragon of go virtue as some people make it out to be. The maintainers have been frank with some of the mistakes and things they'd do differently now but because they place the Go 1 promise above everything else (a good thing), it has some patterns not many would recommend these days, error handling among them.

And kubernetes, docker, golang.com/x/*, etc.

And it's not like they couldn't add better types to errors without breaking backwards compat. Returning a type that implements error instead of fmt.Errorf doesn't break existing semantics.

Almost every large Go codebase does this with very few exceptions. Until the advent of github.com/pkg/errors there wasn't an easy way to uniformly implement error chaining so people would just prepend strings (if they have any context at all) and used fmt.Errorf religiously. These days, fmt.Errorf is still used but now you can in principle root-cause errors with errors.Cause -- unfortunately many Go libraries do not use github.com/pkg/errors (because of the nested vendor problem that existed for many years) and so you are stuck if you want to root-cause an error in an older library.

All of that being said, modern Go (written in the past few years) could avoid these pitfalls. But very few production codebases were both written in the past few years and only use libraries written in the past few years.

I've worked on Go codebases for the past 5-6 years.

Some factors for this

- When prototyping, deferring worrying about exact error types is helpful

- In applications, you aggregate so many types of errors, sometimes it is easier just to dynamically accept any.

For sure.

1 - Just have a Todo(String) in your error enum while prototyping. Remove it as you clean up a prototype, and you get a nice set of compiler errors telling you everywhere that needs to change.

2 - If you're aggregating so many error types that it's unwieldy, you're probably doing to much in one place. That has all sorts of down stream issues with testability and what not. Hacking your error types to make it easier is probably not the best option.

> 2 - If you're aggregating so many error types that it's unwieldy, you're probably doing to much in one place. That has all sorts of down stream issues with testability and what not. Hacking your error types to make it easier is probably not the best option.

I'd disagree. In a static-site-generator, I need to deal with

- file format errors

- syntax highlighting errors

- io errors

- Template language errors

- Adhoc errors I construct throughout the application

- Regex errors

- RSS errors

- jsonfeed errors

- sitemap errors

- git errors

Without even getting into any errors from the webserver that is being run. Yes, I could carefully construct an error enum for each subsection of the application and bubble those up ... but why? I'm not programmatically acting on these but sending them up to the user.

By dumping all those together, you've lost context as to why or where these errors have come from. Just yolo displaying io errors is a terrible experience for your users.

While dealing with errors in any langauge is irritating, I think Rust has the best ideas.

* Pattern-matching for success/error check

* Error passing with the `?` operator

* The correct error seems to appear when panicing due to an `unwrap` call

The most irksome thing remaining is to have to create your own error objects..

> The most irksome thing remaining is to have to create your own error objects

That's generally the purpose of the libraries outlined in the article — reducing the pain of creating the custom error types.

It bugs me to no end that Rust _almost_ got conditions and restarts. Apart from proper macros, conditions and restarts are one of the few things that CL has that no other language has mimicked.

Sure, I have seen it faked in scheme with call/cc, but that will never be a proper, fast solution.

I think guile at least got prompt and abort functionality which makes it mostly doable?

Also, bizarrely, http://p3rl.org/Worlogog::Incident exists.

Not only doable, but rather efficient. Delimited continuations are superior to undelimited ones in every way.

Where can I read about conditions and restarts?

Peter Seibel talks about it in Practical Common Lisp and in the talk he had at Google about the book: https://youtu.be/VeAdryYZ7ak

The book is available online I think.

Thanks!

That is definitely an interesting concept.

  struct Error {  
    XAtLine47,  
    XAtLine88,  
    YAtLine13,  
  }

This way you have error AND error place as contextualization.

To be honest, I can't tell if you're being serious.

Like the sibling, I'm not sure if you are serious, but my library SNAFU (https://docs.rs/snafu/0.6.0/snafu/) is similar to this at a step or two higher level. The idea is that you wrap lower-level errors (a file system error) with a higher-level context (opening the config file).

Taken to the extreme, you have a unique context for every possible error source inside your library, which means you could directly search for the name to find the line number.

This does make me wonder how I could add line / file information to an error automatically though.

I'm wondering if there's any equivalent in Rust to how Zig does error traces?

https://ziglang.org/documentation/master/#Error-Return-Trace...

I like the idea of TypeScript-like error handling.

Or more generally, functions that can return different types without having to define an `enums`. If only because naming these single-use `enums` is hard :)

Is it employing anonymous enums? They come up for helping errors in the Rust community from time to time. The main dropback I see on relying on them is that they make it easy to accidentally expose your implementation details as part of your public API, making it a breaking change to upgrade or change a dependency.

In typescript you can return union types. Rust doesn't allow that, at least not ergonomically.

As a Rust newbie, this is why I end up just using .unwrap() everywhere. result::Result, io::Result, is something a Some/None or an Ok/Error, etc, etc. What (non-string) errors does this function return, etc? I find it easier to check for and propagate integer error numbers correctly in C.

One pattern is to use your own crate-level Error type and then Result::map_err to bubble up other error types into your own thing. Example:

https://play.rust-lang.org/?version=stable&mode=debug&editio...

This makes error handling a little more verbose but you can use some quick and dirty macros for that.

> but you can use some quick and dirty macros for that.

That is the point of most of the libraries described in the article — to prevent having to recreate this particular wheel each time you need it. Do all Rust programs/libraries need one of the libraries? Not at all, but I find that using one that has some nice bells and whistles will encourage the programmer to improve the errors and their messages.

Use `Box<dyn Error>` as your error type, and you won't need to know about any other error type. They all convert automatically and propagate through the `?` operator.

failure and anyhow crates are `Box<dyn Error>` on steroids.

Box<dyn Error> for life

when your error handler can also fail

The top level error handler is

    inner_main().expect("it should work")

Reply on news.ycombinator.com