Agreed.
I left out any commentary on `.unwrap()` from my original comment, but it’s an obvious example of something that should never have appeared in critical code.
I left out any commentary on `.unwrap()` from my original comment, but it’s an obvious example of something that should never have appeared in critical code.
Using unwrap() in Rust is Okay
https://news.ycombinator.com/item?id=32385102 https://burntsushi.net/unwrap/
They aren't presenting a coherent philosophy. And when asked for examples, or to engage directly with examples in my blog, they can't or won't do it.
But yes, of course it's okay to use unwrap(). It's just an assertion. Assertions are fine.
Ignoring that is bypassing the type system. It means your types are either wrong, or your type system is incapable of modeling your true invariants.
In the case of the cloudflare error, their types were wrong. That was an avoidable failure. They needed to fix their type-level invariants, not yolo the issue with `.unwrap()`.
Your willful persistent lack of understanding doesn’t mean my philosophy is incoherent. Using `.unwrap()` is always an example of a failure to accurately model your invariants in the type system.
You can't engage with my examples and you provide none of your own. So continuing discussion with you is a waste of time.
This is literally what “invariant” means, and what a type system is built to model.
Declaring an invariant in the type system that you then violate is not correct code. I truly can’t even begin to guess at why you’re so voracious in your defense of this particularly poor practice.
[edit]
HN rate limits kicking in, so here’s my reply. I work for a FAANG but I’m not going to say which one. You or a relative are, with almost 100% certainty, relying on code written to that philosophy, by me, daily and widely.
Insisting on examples and evidence to support an argument isn't combative. It's appropriate when extraordinary claims are being made.
If you've published code using this philosophy that is used by real people in real systems, then I would be happy to take a look at that as well. If it exists, I would bet it's in a niche of a niche.
I've had these arguments before about this very topic. Some people have taken me up on this request and actually provided examples. And in 100% of those cases, it turned out there was a mismatch between what they were saying and what the code was doing.
It should very clearly state my position. And it provides the examples that I previously referenced.
The GP got a link to this blog in the previous HN thread. They dismissed it out-of-hand without engaging with it at all. And tossed in an ad hominem for good measure. So your issue with me specifically here seems completely inappropriate.
Cool story bro.
Like, even interpreted maximally charitably, your statement still doesn’t provide GP’s requested published code. Not “take my word for it” ostensibly deployed software—code; the discussion here is about code constructs for modeling invariants, not solely about runtime behavior.
I’d be interested to see that code discussed in context of the blog post GP linked, which seems to make a compelling argument.
Not being able to envision how it is in fact possible to write code with these invariants encoded in the type system is a fundamental fault in one’s ability to reason about this topic, and software correctness in general, in the first place.
Code proving that it’s possible to avoid branching into an abort (the concept, not necessarily the syscall) was not what the original GP requested. Nor was a copy of your employer’s IP. Published examples which demonstrate how real-world code which intentionally calls panic() could be better written otherwise was my interpretation of the request.
And I’m requesting that, too, because I am interested in learning more about it! Please don’t assume I’m asking out of inexperience with safety critical systems, dick-measuring, faulty reasoning ability, or unfamiliarity with using type systems to avoid runtime errors (where—and this is the source of this discussion—practical and appropriate). If you work on your tone, that would make it much easier to have educating discussions in contexts like this.
“Can do X” is not an invariant. “Will never do X” (or “Will always do Y”) is an invariant. “Can do X” is the absence of the invariant “Will never do X”.
> Using `.unwrap()` is always an example of a failure to accurately model your invariants in the type system.
No, using .unwrap() provides a narrower invariant to subsequent code by choosing to crash the process via a panic if the Result contains an Error.
It may be a poor choice in some circumstances, and it may be a result of mistakenly believing that code returning the Result itself had failed to represent its invariants fully such that the .unwrap() would be a noop—but even there it respects and narrows the invariant declared, it doesn't ignore it—and, in any case, as it has well-defined behavior in either of the possible input cases, it is silly to describe using it as a failure accurately model invariants in the type system.
What’s silly is the desire to pretend otherwise because it’s easier.
The invariant is that either condition X applies or condition Y applies. "Panic and stop execution if X, continue execution with the invariant Y if Y" is not unsound and does respect the original invariant in every possible sense.
It may be the wrong choice of behavior given the frequency of X occurring and the costs incurred by the decision to panic, but that’s not a type-level problem.
You guys really will go to any possible rhetorical length to justify lazy programming practices in error handling.
If a proof trivially demonstrated that a given program’s behavior was indeed “proceed if a condition is satisfied, crash otherwise”, then what? Or do we not trust the verifier with branching code all of a sudden?
There's no reason to use them as the language provides lots of safer alternatives. If you do want to trigger a panic, you can, but I'd also ask - why?
Alternatively, and perhaps even better, Rust needs a way to mark functions that can panic for any reason other than malloc failures. Any function that then calls a panicky function needs to be similarly marked. In doing this, we can statically be certain no such methods are called if we want to be rid of the behavior.
Perhaps something like:
panic fn my_panicky_function() {
None.unwrap(); // NB: `unwrap()` is also marked `panic` in stdlib
}
fn my_safe_function() {
// with a certain compiler or Crates flag, this would fail to compile
// as my_safe_function isn't annotated as `panic`
my_panicky_function()
}
It is not like an Erlang service would be able to make progress with an invalid config either.
What happened here is systemic: the config file contained an issue severe enough that it precluded the system from running in the first place and unfortunately that caused a runtime error when in fact that validation should have been separate from the actual use. This is where I see the problem with this particular outage. And that makes it an engineering issue much more than a language issue.
Bad configuration files can and do happen, so you take that eventuality into account during systems design.
Rust is built on forcing the developer to acknowledge the complexity of reality. Unwrap acknowledges said complexity with a perfectly valid decision.
There are a few warts from early days like indexing and the ”as” operator where the easy path is doing the wrong thing.
But unwraps or expects are where Rust shines. Throwing up your hands is a perfectly reasonable response.
With your approach, what should Cloudflare have done?
Return an error, log it and return a 500 result due to invalid config? They could fail open, but then that opens another enormous can of worms.
There simply are no good options.
The issue rests upstream where deployments and effects between disparate services needs to be mapped and managed.
Which is a truly hard problem, rather than blaming the final piece throwing up its hand when given an invalid config.
Why is it inappropriate to be able to statically label the behavior?
Maybe I don't want my failure behavior dictated by a downstream dependency or distracted engineer.
The subject of how to fail is a big topic and is completely orthogonal to the topic of how can we know about this and shape our outcomes.
I would rather the policy be encoded with first class tools rather than engineering guidelines and runbooks. Let me have some additional control at what looks like to me not a great expense.
It doesn't feel "safe" to me to assume the engineer meant to do exactly this and all of the upstream systems accounted for it. I would rather the code explicitly declare this in a policy we can enforce, in an AST we can shallowly reason about.
Right now you all the instances where the code can panic are labeled. Grep for unwrap, panic, expect etc.
In all my years of professional Rust development I’ve never seen a potential panic pass code review without a discussion. Unless it was trivial like trying to build an invalid Regex from a static string.
Unwrap, slice access, etc. are not.
Thanks for making abundantly clear how such a feature wouldn’t solve a thing.
Copying this so you see it too -
The Cloudflare outage was a multi-billion dollar outage. I have personally been involved in multiple hundred million dollar outages at fintechs, so forgive me for being passionate about this.
Several of the outages I've been involved in were the result of NPEs or incorrectly processing runtime data. Rust has tools to enforce safety here, but it doesn't have tools to enforce your use of them. If also doesn't have a way to safeguard you from others deciding the behavior for you.
There is potentially a very easy set of non-onerous features we could build that allow us to prevent this.
As the poster here said, the place to build in features that would have prevented this from happening is the DB schema and queries. 5NF would be onerous overkill here, but it seems reasonable to have some degree of forced normalization for something that could affect this much.
(Requiring formal verification of everything involved here would be overkilling the overkill, otoh.)
This doesn’t seem to be a principled stance on making the language safer. It feels a bit whack-a-mole. “Unwrap is pretty easy to give up. I could live without slicing. Malloc seems hard though. I don’t want to give that up.”
https://news.ycombinator.com/item?id=46060907
Malloc is fine. We can and do monitor that. It's these undetectable runtime logic problems that are land mines.
In distributed systems, these can cause contagion and broad outages. Recovering can be very difficult and involve hours of complex steps across dozens of teams. Meanwhile you're losing millions, or even hundreds of billions, of dollars for you and your customers.
Someone unwrapping() a Serde wire message or incorrectly indexing a payload should not cause an entire fleet to crash. The tools should require the engineer handle these problems with language features such as Result<>.
Presently, who knows if your downstream library dependency unwrap()s under the hood?
This is a big deal and there could be a very simple and effective fix.
The Cloudflare outage was a multi-billion dollar outage. I have personally been involved in multiple hundred million dollar outages at fintechs, so forgive me for being passionate about this.
I don’t necessarily agree that malloc should be okay (buggy code could try to allocate a TB of memory and OOMKiller won’t fix it) but I can understand that it’s probably workable in most cases.
Unfortunately I think the fix here would require a compatibility break.
- https://rust-lang.github.io/rust-clippy/master/#unwrap_used
- https://rust-lang.github.io/rust-clippy/master/#indexing_slicing
- https://rust-lang.github.io/rust-clippy/master/#string_sliceDunno ... I think runtime assertions and the ability to crash a misbehaving program are a pretty important part of the toolset. If rust required `Result`s to be wired up up and down the entire call tree for the privilege of using a runtime assertion, I think it would be a lot less popular, and probably less safe in practice.
> Alternatively, and perhaps even better, Rust needs a way to mark functions that can panic for any reason other than malloc failures.
I 100% agree that a mechanism to prove that code can or cannot panic would be great, but why would malloc be special here? Folks who are serious about preventing panics will generally use `no-std` in order to prevent malloc in the first place.
In one of the domains I work in, a malloc failure and OOMkill are equivalent. We just restart the container. I've done all the memory pressure measurement ahead of time and reasonably understand how the system will behave under load. Ideally it should never happen because we pay attention to this and provision with lots of overhead capacity, failover, etc. We have slow spillover rather than instantaneous catastrophe. Then there's instrumentation, metrics, and alerting.
A surprise bug in my code or a dependency that causes an unexpected panic might cause my application or cluster to restart in ways we cannot predict or monitor. And it can happen across hundreds of application instances all at once. There won't be advanced notice, and we won't have a smoking gun. We might waste hours looking for it. It could be as simple as ingesting a pubsub message and calling unwrap(). Imagine an critical service layer doing this all at once, which in turn kills downstream services, thundering herds of flailing services, etc. - now your entire company is on fire, everyone is being paged, and folks are just trying to make sense of it.
The fact is that the type of bugs that might trigger a user-induced panic might be hidden for a long time and then strike immediately with millions of dollars of consequences.
Maybe the team you implemented an RPC for six months ago changes their message protocol by flipping a flag. Or maybe you start publishing keys with encoded data center affinity bytes, but the schema changed, and the library that is supposed to handle routing did an unwrap() against a topology it doesn't understand - oops! Maybe the new version handles it, but you have older versions deployed that won't handle it gracefully.
These failures tend to sneak up on you, then happen all at once, across the entire service, leaving you with no redundancy. If you ingest a message that causes every instance to death spiral, you're screwed. Then you've got to hope your logging can help you find it quickly. And maybe it's not a simple roll back to resolve. And we know how long Rust takes to build...
The best tool for this surely can't be just a lint? In a supposedly "safe" language? And with no way to screen dependencies?
Just because somebody's use case for Rust is okay with this behavior doesn't mean everyone's tolerates this. Distributed systems folks would greatly appreciate some control over this.
All I'm asking for is tools to help us minimize the surface area for panics. We need as much control over this as we can get.
The core issue is that resilience to errors is hard, and you can't avoid that via choice of panic versus non-panic equivalents.
As appealing as the idea of a #[cfg(nopanic)] enforcement mechanism is, I think linting for panic() is the optimum, actually.
With a more rigidly enforced nopanic guarantee, I worry that some code and coders would start to rely on it (informally, accidentally, or out of ignorance) as a guarantee of completion, not return behavior. And that’s bad; adding language features which can easily be misconstrued to obscure the fact that all programs can terminate at any time is dangerous.
Lints, on the other hand, can be loud and enforced (and tools to recursively lint source-available dependencies exist), but few people mistake them for runtime behavior enforcement.
You can keep unwrap() and panics. I just want a static first class method to ensure it never winds up in our code or in the dependencies we consume.
I have personally been involved in nearly a billion dollars of outages myself and am telling you there are simple things the language can do to help users purge their code of this.
This is a Rust foot gun.
A simple annotation and compiler flag to disallow would suffice. It needs to handle both my code and my dependencies. We can build it ourselves as a hack, but it will never be 100% correct.
This is why I want it:
> Rust needs to get rid of .unwrap() and its kin.
Now you say:
> You can keep unwrap() and panics.
So which is it?
> I just want a static first class method to ensure it never winds up in our code or in the dependencies we consume.
Now this is absolutely a reasonable request. But it's not an easy one to provide depending on how you go about it. For example, I'd expect your suggestion in your other comment to be a non-starter because of the impact it will have on language complexity. But that doesn't mean there isn't a better way. (I just don't know what it is.)
This is a classic motte and bailey. You come out with a bombastic claim like "remove unwrap and its ilk," but when confronted, you retreat to the far more reasonable, "I just want tools to detect and prevent panicking branches." If you had said the latter, I wouldn't have even responded to you. I wouldn't have even batted an eye.
> This is the Hundred Billion Dollar unwrap() Bug.
The Cloudflare bug wasn't even caused by unwrap(). unwrap() is just its manifestation. From a Cloudflare employee:
> In this case the unwrap() was only a symptom of an already bad state causing an error that the service couldn't recover from. This would have been as much of an unrecoverable error if it was reported in any other way. The mechanisms needed to either prevent it or recover are much more nuanced than just whether it's an unwrap or Result.
This sounds like the kind of failure Bobby Tables warned about a long time ago. An entire new, safe language was developed to prevent these kinds of failures. “If it compiles it’s probably correct” seems to be the mantra of rust. Nuts.
Coordinated systems that test and rollback are way beyond the scope of what a compiler can check.
Apparently that isn’t true.
Edit: isn’t the whole idea of C/C++ being flawed pivoted around memory management and how flawed the languages are? Wasn’t the whole point of rust to eliminate that whole class of errors? XSS and buffer overflows are almost always caused by “malformed” outside input. Rust apparently doesn’t protect against that.
If you exit, a known bad thing happens.
No language can protect you from a program's instructions being broken. What protective measures do you have in mind? Do they still result in the service ceasing to process data and reporting a problem to the central controller? The difference between "stops working and waits" and "stops working and calls abort()" is not much, and usually the latter is preferred because it sets off the alarms faster.
Tell me what specifically you want as correct behavior in this situation.
I would expect such a critical piece of code to be able to hot-load and validate a new configuration before it is put into action. I would expect such a change to be rolled out gradually, or at least as gradually as required to ensure that it functions properly before it is able to crash the system wholesale.
I can't say without a lot more knowledge about the implementation and the context what the best tools would be to achieve this but I can say that crashing a presently working system because of a config fuckup should not be in the range of possible expected outcomes.
Because config fuckups are a fact of life so config validation before release is normal.
But the operative point in this sub thread is whether unwrap() specifically is load bearing.
If instead they bubbled up the error, printed it and then exited the program---without ever using unwrap---then presumably they still would have had a denial of service problem as a result of OOM.
And even if unwrap were load bearing here, then we would be in agreement that it was an inappropriate use of unwrap. But we are still nowhere near saying "unwrap should literally never be used in production."
And if that config doesn't validate, what should the process do? Maybe it had a previous config, maybe it didn't. And if it keeps running the old config, that adds extra complication to gradual rollout and makes it harder to understand what state the system is in.
> I would expect such a change to be rolled out gradually, or at least as gradually as required to ensure that it functions properly before it is able to crash the system wholesale.
Me too. Note that doing a gradual rollout doesn't care whether the process uses unwrap or uses something gentler to reject a bad config.
> I can say that crashing a presently working system because of a config fuckup should not be in the range of possible expected outcomes.
By "working system" do you mean the whole thing shouldn't go down, or the single process shouldn't go down? I agree with the former but not the latter.
“If it compiles it’s probably correct” has always been a tongue-in-cheek pithy exaggeration. I heard it among Haskell programmers long before I heard it in the context of Rust. And guess what? Haskell programs have bugs too.
If you say so, I believe you. That isn’t how it comes across in daily, granted pithy, discourse around here.
I have a lot of respect for you Andrew, not meaning to attack you per se. You surely can see the irony in the internet falling over because of an app written in rust, and all that comes with this whole story, no?
Nobody credible has ever said that Rust will fix all your problems 100% of the time. If that's what you inferred was being sold based on random HN commentary, then you probably want to revisit how you absorb information.
Rust has always been about reducing bugs, with a specific focus on bugs as a result of undefined behavior. It has never, isn't and will never be able to eliminate all bugs. At minimum, you need formal methods for that.
Rust programs can and will have bugs as a result of undefined behavior. The value proposition is that their incidence should be markedly lower than programs written in C or C++ (i.e., implementations of languages that are memory unsafe by default).
Heard, chef.
You can't force a config file loaded at run time to be correct at compile time. You can only decide what you're going to do about the failure.
We also don't get surprised at runtime. It's in the AST and we know at compile time.
The right API signature helps the engineer think about things and puts them in the correct headspace for systems thinking. If something is panicking under the hood, the thought probably doesn't even occur to them.
While lexically the unwrap actually puts the unhandledness of the error case as close to the source of the issue's source as possible. In order to get that lexical goodness you'd need something much more fine grained than Result.
#![deny(clippy::unwrap_used)]
cargo clippy -- -D clippy::unwrap_used