Rust has practically one error, it's the Error trait. The things you've listed are some common ways to use it, but you're entirely fine with just Box<dyn Error> (which is basically what anyhow::Error is) and similar.
An easy rule before you make a knowledge based choice is Thiserror for libraries, helping you create the standard library error types and Anyhow for applications, easy strings you bubble up.
Or just go with anyhow until you find a need for something else.
* Someone tells me to use enums for errors, in a comment like yours
* I try writing the enums by hand, implementing the error trait
* I realize that in order to use the ? operator I need to implement From on my errors (I’ve read so many comments about how awfully verbose Go errors are, so I assume I’m supposed to use ? in Rust). There are also some other traits IIRC but I’ve forgotten them.
* I realize that this is pretty tedious, manual work, so someone points me to thiserr or similar
* Now I’m debugging macro expansion errors and spending approximately the same amount of time
* I ask around and someone tells me not to bother with thiserr and to just write the boilerplate myself or else to use anyhow or boxed errors everywhere
* I try using boxed errors everywhere, which works, but now I have all of these allocations which feels like I’m doing something that will bite me later. Oh well, but now I need to annotate my errors so I can figure out what is actually happening. I guess I should use anyhow for this?
* Anyhow mostly works but this is approximately as verbose as the Go error handling that I’m told is Very Bad, and when I ask for code review most Rust people are telling me not to use anyhow because errors should be enums, at least in the API surface
I’m sure I’m doing it wrong, but as with many things in Rust, the Right Way is so rarely clear and every other Rust person gives different advice about how to solve my problem and the only thing they seem to agree on is that Rust has an easy solution and that I’m following the wrong advice. (Similarly when I had lifetime problems and half the community told me to just use clone and Rc everywhere until I had performance problems, so instead I just had different static analysis problems).
I don’t love Go’s error handling. It feels like there has to be something better than its runtime-typing. But it largely gets out of the way—creating an error is just implementing the Error method, and if you need a concrete type you use Is/As/AsType. Wrapping is fmt.Errorf. All of this is built into the stdlib and used pretty ubiquitously across the ecosystem—I don’t run into “this dependency uses a different error framework”. Error handling is marginally more verbose than with Rust if you are actually attaching context in both, and neither solves the problem of which call frame attaches the context about specific function parameters (e.g., which level of error context specifies that the function was called with path “/foo/bar.baz”). It’s terrible, but it works—feels like the least bad thing until the Rust community can arrive at some consensus and document it in The Book. Or maybe I just need to try again in the LLM era?
The example on the docs page is quite clear:
https://docs.rs/thiserror/latest/thiserror/#example
Including all kinds of errors: Strings, tagged unions and automatically converting from std::io::Error with added context.
That one page document is the entire documentation for the thiserror crate.
Common in HTTP land. The HTTP system returns a different error type than the network I/O system, but they can be sorted out.[1]
[1] https://github.com/John-Nagle/maptools/blob/main/rust/src/co...
(there's a lot of this in Scala too, because of the various monads/containers, eg. the built-in Future, and then Scalaz.IO, FS2, Cats, ZIO, etc...)
regarding lifetime and performance problems, the best practice seems to design the rough scaffolding of the program first, with the structs, so the who owns whom can be figured out. but this is far from trivial. Rust is very good at forcing developers to stare at these problems, but solving them requires practice and patience.
for me the tech toolbox that makes sense is TS by default (because of the super convenient type system and tooling), and Rust when the circumstances really justify it (latency, throughput, scalability, cost effectiveness, or a need for a single native executable [though nowadays this is also pretty simple with Deno], or more safety/control [no GC])
> Now I’m debugging macro expansion errors and spending approximately the same amount of time
This never happens once you've learned the language a bit more. Anyhow and thiserror are a cinch.
> I realize that this is pretty tedious, manual work, so someone points me to thiserr or similar
Claude writes Rust so effectively. It can do all of this for you now. It's effortless. In fact, I don't see any reason to use any other language unless I'm targeting web or some specific platform, or dealing with legacy code. Rust is now the best tool for most problems.
> Similarly when I had lifetime problems and half the community told me to just use clone and Rc everywhere until I had performance problems, so instead I just had different static analysis problems
Do this for a month, then it'll click and be second nature. Also Claude will make quick work of it now.
> feels like the least bad thing until the Rust community can arrive at some consensus and document it in The Book
It's difficult because it's so different. But once you get used to it, you'll realize it's the best approach we have right now.
> Or maybe I just need to try again in the LLM era?
Seriously this. You'll be writing Rust code as quickly as you would Python code. It'll be high quality. And the type system will mean that Claude emits better code on average. You'll pick it up quickly.
> And the type system will mean that Claude emits better code on average.
I’m curious if this is true. I believe that it emits better code than with a dynamically typed language, but as with people I don’t know that the sweet spot is at the extreme. Or maybe it is at the extreme when the context is small but as the context grows perhaps code quality suffers as it has more constraints to balance?
I'm so sorry for this btw.
The problems are trivial once you've used Rust for n hours, for some value n. It's just that these folks forgot the learning and headache they went through.
You're going to build that same recognition and familiarity using Claude over time. It'll seep in pretty quick, I'd imagine.
> I’m curious if this is true.
Being forced to emit an Option<T> or Result<T,E> and then having to actually use syntax to get at the goods forces the code to deal with errors the appropriate way, clearly, idiomatically, and typically in a good flow that is amenable to readability and easy refactoring. Other languages without Option, Result, and sum types baked into the language so fundamentally do not have this advantage.
I feel it every time I have to work in a TypeScript codebase, for instance. It's a strongly typed language, and can emulate sum types via discriminated unions. But that doesn't convey the same advantages because it doesn't enforce anything. It's far too lose to have the same advantages Rust has.
I think you'll feel the same way as you use the language more and more.
All good. It happens in every community, but it seems like Rust has more of these problems where there are N ways to do something and none of them are obvious. Reminds me of Python where everyone swears they have a package manager that will fix all previous problems and then you invest a bunch of time into their suggestion only to find that it introduces half a dozen new glaring problems (e.g., pipenv taking 30 minutes to resolve a lock file for a relatively small project).
> The problems are trivial once you've used Rust for n hours, for some value n.
Heh, I've been taking a real stab at Rust every year since 2014. `n` can be a pretty large value! I'm optimistic that Claude and friends will help me get there though.
> Being forced to emit an Option<T> or Result<T,E> and then having to actually use syntax to get at the goods forces the code to deal with errors the appropriate way, clearly, idiomatically, and typically in a good flow that is amenable to readability and easy refactoring. Other languages without Option, Result, and sum types baked into the language so fundamentally do not have this advantage.
Honestly I use Go as my daily driver these days and while I wish it had sum types for error handling, it's really not a problem. It's more aesthetics than anything, and LLMs do just fine with managing errors in Go. There may be other advantages for LLMs with respect to Rust's rigorous type system, but I could easily see it going the other way as well (additional constraints for the LLM to focus on, taking more of its context budget that could go to the fundamental product constraints). I really don't know what the right answer is here--I suppose time will tell.
1. the compiler gives very high quality error messages. It helps humans, and also helps LLMs
2. Rust reduces memory management to local reasoning (via the borrow checker). This means that it performs well even as context grows, because checks in one function/module are well-encapsulated to that function/module.
3. Rust can more easily obtain this encapsulation for more general properties than many other statically typed languages. In particular, rust's type system is very strong, so it's easy to take a function `func(x: T)` that relies on some implicit assumption on `x` (say that it is non-zero), and turn it into an explicit requirement. By this, I mean you define `pub struct NonZero(T)`, and provide constructors `pub try_new(t: T) -> Result<NonZero<T>, _>` that error if the condition doesn't hold. If you additionally only provide public methods on `NonZero<T>` that uphold the invariant, you can lift runtime runtime assertions to the type level. This is both good practice, and helps out LLMs quite a bit.
This is to say that rust makes it quite easy to encapsulate implementation details (both regarding memory management, as well as other details) essentially completely. Sometimes you still have invariants that need care/can't be encapsulated in the type system, but such invariants should be marked `unsafe`, so it can be easier to audit the LLM's output.
Anyway, the "more constraints to balance" is only problematic if all the constraints are inter-dependent. It's definitely possible to get LLMs to generate spaghetti code like this, but the way you fix it is the way you fix similar issues in other languages.
Don’t get me wrong, I like this aspect of Rust, but I can’t make heads or tails as to whether it helps or if they just have to iterate more to figure out how to make something work. LLMs already do pretty well with a comment “this value can’t be zero” in my experience, so I’m unsure how much value the static typing provides. Maybe it lets you get by with a lower quality model, but that model will likely just spend more tokens on iteration so I can’t discern an obvious win. (shrug) I hope I’m wrong though—if I can have super fast code with the ease of LLM generation then I’m happy.
> Rust reduces memory management to local reasoning (via the borrow checker). This means that it performs well even as context grows, because checks in one function/module are well-encapsulated to that function/module.
I don’t think this is true, right? Changing a single lifetime in a function signature can easily propagate across your entire program. Maybe I’m just a Rust noob, but any time I change a field from owned to borrowed or vice versa I have to propagate that change pretty broadly, which to my mind implies consuming a lot of the context window. Garbage collection (I know, ewww, shame on me, etc) allows for local reasoning in a much more meaningful way however morally impure it may be. :)
Anyway, making such a (breaking) change in rust immediately tells you all of the callsites that break. You have to chase it through, but that's mechanical/low context work. More formally, you can parallelize across files with sub-agents to not pollute your main agents context window. So it really should be a "zero context window" cost.
Whether or not strict typing is strictly better is really a correctness/velocity tradeoff, the same as it always has been. For most projects something in the middle is right.
As for owned vs borrowed and things propagating quite a bit, sometimes it happens. It's often avoidable with a couple of tricks
1. always default to borrowed unless you have a good reason to otherwise
2. make your function signatures more permissive so they can support either way. This can be done by modifying f<T>(x: T) (or f<T>(x: &T)) to f<U: AsRef<T>>(x: U). The later can be equivalently written as f(x: impl AsRef<T>).
When you say "change a field from owned to borrowed", I'd generally suggest not doing that. It's generally easier to start with some owned type MyType. You can then have function signatures take &MyType as input. This borrows all the fields, and is often good enough for most functions.
If you have a more esoteric function (that needs a combination of borrowed and owned inputs), it's typically easier to define a struct for that function. The steps are
1. Define a FunctionInputsRef<'a>
2. write `impl<'a> From<&'a MyStruct> for FunctionInputsRef<'a>`, then
3. update your function to take as input FunctionInputsRef<'a> rather than `&MyStruct`, and
4. update callers with `input -> input.into()`.
It has the benefit of less churn, as you're maintaining the old def (which might be useful elsewhere), and only updating the callers in a fairly trivial way. `FunctionInputsRef<'a>` can also be defined local to the function, so it is modularized better. If you later have other functions with other requirements, it's a relatively easy pattern to duplicate as well.
1. thiserror just does codegen of the "standard" enum things people do. if you find debugging thiserror difficult, just write out the enums manually. sure it's uglier, but (roughly) equivalent. so its preferable as synctatic sugar for enums, but doesn't have any technical benefits (in the same way that syntatic sugar never really does).
2. for boxed errors, you only get allocations on your error path. Hopefully this is a cold path so it shouldn't matter.
There is a general theme behind rust error handling though which it can be good to internalize. In particular, the more details of your errors you encode in the type system, the more powerful things are. Any error type could just be
pub struct MyError(String)
the issue is that this gives very little information to a caller on what to do with your error. If you have no callers (e.g. are making a binary) it's fine, and (roughly) what `anyhow` does.
That all being said, when designing errors a natural question to ask is "can my caller do anything meaningful with this error"? For example, in Rust stdlib, `Vec::push` can allocate. This allocation can fail, which panics. "Proper" error handling would use the fallible allocation API, and propagating an OOM error or whatever through results. For most applications, this is not an error that is worth investing that much time into guarding against, so using the (potentially panicing) `Vector::push` makes things easier.
You can take this same perspective in other settings as well, in particular separating out errors into
1. structured data, that a caller should be able to extract/process to handle the error, and 2. unstructured data, that is more used for logging, and you expect the caller to pass up the call stack without inspecting themself.
Handling both types of these errors with `thiserror` can be tedious for little benefit. I've found it useful to instead solely use `thiserror` for category 1, and category 2 does other things. This could be using `.expect(...)`. There are some crates that make this nicer (e.g. `error_stack`). But the point is that it can significantly clean things up if you only encode in your error enums failures that you expect someone to handle, rather than just e.g. log.
This does somewhat validate your point that the "right way" I've been experimenting with (and mentioned above) is not just "use this-error".
Also: a big issue with `thiserror` is the tedium of handling the large error enums (or giving up on using it "properly" and shoving together multiple error variants in some unstructured error type). that is somewhat better in the LLM era, as you can have the LLM handle the tedium.
enum AllocError {
SizeTooLarge { size: usize },
// etc.
}
You'd describe it as a tagged union in some languages. So when you say you'd return an error with extra information, what that information is is associated with the specific variant of the enum.
Using yuriks AllocError as an example, if the error is SizeTooLarge, it has the size field. Other errors may have no additional data, others may have different data.
When you return an error from your allocating function, it's a known size, the size of the largest enum variant + the discriminant (tag).
right. If allocerror only has size field, where do you stash the unwind information (which could be of arbitrary size)
What value would a stack trace that includes internal allocator functions be to you? What do you lose by having to collect the stack trace at the point where your function receives an AllocError?
enum MyApiBindingCrateError {
// You didn't provide an
// API key. Maybe we should
// design our interface to
// make this impossible
ApiKeyMissing,
// Client was unauthorized
// to make this request
AuthorizationError,
// The entity you requested
// did not exist (404'd)
NotFoundError,
// You're sending too many
// requests to the server
TooManyRequests,
// That specific error with
// the API
// Maybe users can't delete
// folders until they're empty
// Whatever
SpecificApiIssue1,
// Some other specific error
// with the API
SpecificApiError2,
// Server didn't respond the
// way we expected.
// Here's what it told us
UnexpectedHttpResponse {
// HTTP status code
status_code: StatusCode,
// If it had a
// string-encoded body
body: Option<String>,
},
// Unhandled Issue with IO
IoError(io::Error),
// Unhandled Issue with
// request library
ReqwestError(reqwest::Error),
}
Your application can be a little less structured if you want. Though with LLMs, I'm using anyhow and thiserror a lot less.
but for most libraries I on allocation failure I don't expect any fancy logging system. maybe even panic is fine.
Stack traces are only useful for errors that indicate a bug in the program, i.e. something a programmers has to respond to. It's not useful for the vast class of bugs that are a result of wrong input, wrong external state, or infrastructure issues.
Rust projects tend to favor panicking over error handling for programmer bugs (which does indeed give you a stack trace depending on environment variables), or even better encoding the invariants in the type system, but there are cases where an error coming from a library are truly, actually unexpected, so both `anyhow` and `thiserror` do provide support for attaching a stack trace in those situations.
>Stack traces are only useful for errors that indicate a bug in the program, i.e. something a programmers has to respond to. It's not useful for the vast class of bugs that are a result of wrong input, wrong external state, or infrastructure issues.
This is a personal opinion, not something you can declare as the objective truth. There is a lot of value in seeing what path the program took before it encountered a eg. validation error.
>but there are cases where an error coming from a library are truly, actually unexpected, so both `anyhow` and `thiserror` do provide support for attaching a stack trace in those situations.
This is wrong because it's up to the library to attach the stacktrace, not the userland code using the library, so saying "you can get them if you want them" is not true. If the author of the library did not decide to attach the stacktrace, your only option is wrapping it yourself, which you can only do if you already know up front all the paths that can fail. Also, you are not supposed to expose errors from a library with anyhow, they are only for application/top level code.
The only thing a library gathering a stack trace instead of you gives you is that it includes traces through code you didn't write & ostensibly aren't responsible for. If you're going to go to the effort of tracing through a dependencies code, you might as well add the stack trace yourself; it's a single line of code from the standard library to collect it, std::backtrace::Backtrace::capture().
EDIT: capture will only actually grab a trace when env vars say it should, you can use force_capture to ignore those. To get to why this isn't the default for errors you're asking for, here's a line from their documentation:
> Capturing a backtrace can be both memory intensive and slow
Since Backtrace::capture() is already tied to an env var, we could have the backtrace on Result without affecting performance, since you would only enable it for debugging. This would allow you to eg. easily track down a situation where you see in your prod logs that you are encountering a lot of "validation error: string is too long" but you can't tell where it is coming from. Flip the env var, redeploy the application, read the backtrace, turn off the env var, fix the problem.
Capturing a stack trace is a hefty operation: making it happen on _every_ error creation, which would include creating an error in response to another error (like <failure to allocate> causing <failure to create object>) could easily grind a production server to a halt. Especially if there's correctly handled errors happening: every one of them will pay this cost, every time.
It sounds like a really specific problem here; the log line that's happening is generic enough that it doesn't identify which line of code is emitting the log, so you can't just add `capture` to that line (what logging system even does this? printf logging?).
It is a specific problem, so what? You see that you are sending 500 from an axum handler, and you are logging "serde deserialization error: line 4 invalid", wouldn't it be nice to see where that came from, without instrumenting all the places you are deserializing something?
It sounds like what you want is the errors you log to always log stack traces. Which is a fine position, I do something like that. It's just not something that can be the default, because it can't be done everywhere.
Some languages have exceptions as the only error handling mechanism (C#, Java, scripting languages), and it sounds like that's what you're used to. But this is also broadly agreed to be a severely limiting factor of those languages, resulting from being designed at a time when we didn't know better.
If you want to go fast (and Rust does), you cannot be catching exceptions in the hot path, and you certainly can't be throwing exceptions that carry stack traces, because walking the stack to build up the stack trace is many orders of magnitude slower than returning an error value.
Rust's error handling modes are designed with the benefit of hindsight from all those other languages from the last few decades, and reflects the fact that errors broadly fall in two categories: validation failures and programmer errors. The former should be a cheap error code that can be handled, the latter should terminate the program/thread/task and give you enough information to diagnose the problem.
Obviously you need to explicitly include the line numbers since the error reporting Rust is closer to C (errno) than Java or JavaScript.
https://users.rust-lang.org/t/getting-line-numbers-with-as-i...
Just do what feels right. I personally am a big fan of Java stack traces.
You can't have a stack trace on an error in the error path that failed to allocate. If you have a "jumbo sized" error and the error fails to allocate, it won't get reported. The only reporting you will get is that the error failed to allocate and this new allocation error overrides the error that failed to allocate.
If you need to handle an allocation error in the error path, then the error reporting path must abort, which means that the allocation error must be bubbled up.
There is no real solution to an allocation error inside the error path. Even if you preallocate an arena for errors, the error might be large enough that it won't fit inside the arena.
Hence the best thing you can do from that point onwards is to have an error enum with an AllocError variant that doesn't allocate. Said error won't contain any information beyond line numbers of the allocation error since you just don't have the space for it.
In the end you will basically end up with panic free code, but the error still bubbles up like regular unwinding.
So yeah you can do it, and I will do it in the future, but I personally think that the people who think this is some huge deal breaker don't understand the problem in the first place.
In Rust you can do that (mostly on nightly although not only), and yes there are alternatives for this case, but it's rare anyway.