Hacker News Reader

Comment by munificent | Hacker News Reader

I don't mean this to sound superior or critical.

I found this article a little surprising because the authors seem to have an intuition about the compiler that I have never had. I've never expected potential stack overflows from unbounded recursion to be caught by the compiler. I've been delighted every now and then when a compiler gives me a warning when it can detect it. But I have always had stack overflows in my mental bucket of "stuff it's my responsibility to think about".

Maybe this says something about how compilers have gotten smarter and more helpful since I first cut my teeth on them.

View on HN

What I also think is weird is that people think seg faults and crashes are "unsafe".

In the terms of Cardelli, a stack overflow is a trapped runtime error -- and thus a SAFE behavior of a language runtime.

What's not safe is when your program keeps running in an unknown state:

- buffer overflows - who knows what it does after memory corruption?

- integer overflows, especially in C where the behavior isn't defined. But in Rust's release mode, these runtime checks are omitted, so that's unsafe too.

---

On the other hand, the behavior of null pointer exceptions and a stack overflows is SAFE. It protects you from undefined behavior at runtime.

Crashing is a DEFINED behavior -- it's also what happens on divide-by-zero by default, and apparently when Rust's allocators run out of heap memory.

This isn't a theoretical distinction, because diagnosing and fixing NPEs and stack overflows is easy compared to the other two.

And crashes can lead to denial of service, but not remote code execution, which is WAY worse.

I think that you're the weird one here.

The most common reason to GET a segmentation fault is memory corruption. Frequently from a buffer overflow. Which makes them the canary in the coal mine for bugs that often turn out to be exploitable security vulnerabilities.

So if a program segfaults on me and I don't know exactly why, I should dial my paranoia up and expect that it is insecure. That an attacker who looked at it could use it to take over my computer instead of simply leaving me with a core dump.

> So if a program segfaults on me and I don't know exactly why, I should dial my paranoia up and expect that it is insecure.

This is true in that the program contains at least one bug, but some memory errors are safer than others. A SIGBUS is safer than SIGSEGV (usually means you mmapped a file and got an I/O error reading it) and a read is safer than a write.

This comes up if you're prioritizing crashes from fuzzing.

Instead of stack overflow causing a crash, it could silently overflow into heap memory. Do you want that?

What I'm saying is that crashes are the thing that PROTECT you from undefined behavior at runtime.

---

If a C program crashes, then you might suspect memory corruption.

But here's the thing: if it DOESN'T crash, you actually have NO less reason to suspect memory corruption! (This isn't theoretical, I'm certain that there are thousands or millions of C programs that have memory unsafety 100% of the time at runtime, especially in embedded systems.)

If a Rust program crashes, would you suspect memory corruption? It could be, but it's still BETTER if it crashes 100% of the time than if it nondeterministically crashes.

---

Do you think there are any other options besides crashing or silent undefined behavior?

I agree that segmentation faults are the right thing to do in a bad situation. But the fact that we are in that bad situation says that there is a problem that we should worry about.

As for memory corruption, you're right that nothing can ever prove that code has no memory corruption. But that doesn't stop crashing from being a pretty good signal that THIS program is experiencing corruption NOW. Which should raise our alert level.

I don't have enough experience with Rust directly as opposed to C to have a great sense of how valuable this signal is for those programs. But then again the programs I see segfaulting all of the time tend to be C programs. So the problem is kind of moot.

> And crashes can lead to denial of service, but not remote code execution, which is WAY worse.

Some crashes in the past have lead to remote code execution, because a program that was supposed to be guarding/filtering something was caused to die, and the system failed open.

Yeah that's an interesting consideration, but it's more of a systems design issue, outside the scope of what the C or Rust compilers and runtime can do anything about. (Or Python, JavaScript, etc.)

Rust has a particular definition of unsafe that they've documented here: https://doc.rust-lang.org/nomicon/what-unsafe-does.html

They have also documented some frequently discussed behavior that is not considered unsafe here: https://doc.rust-lang.org/reference/behavior-not-considered-...

For each of the items on the second list, I believe there are languages that would consider those behaviors "unsafe", but rust doesn't - ultimately it's an arbitrary choice. So, it's important to be clear about what model of "unsafety" we're talking about.

Yes definitely, which is why I mentioned Cardelli. Previous comment on the same subject: https://news.ycombinator.com/item?id=21832009

A program fragment is safe if it does not cause untrapped errors to occur. Languages where all program fragments are safe are called safe languages. Therefore, safe languages rule out the most insidious form of execution errors: the ones that may go unnoticed.

-- Luca Cardelli

It's certainly fine for Rust to have a keyword called "unsafe". But as you note, users should remember that this isn't equivalent to semantic safety.

---

As a trivial example, integer overflow IS unsafe in Rust, but it's allowed WITHOUT using the unsafe keyword.

Any language that traps integer overflow, regardless of debug/release mode is safer than Rust in that respect.

Python is also safer than Rust in that respect, because it allows arbitrary sized integers.

Rust made a tradeoff for efficiency, which is appropriate for its use cases, just like C did at the time.

---

Programmers often use/abuse terms from computer science, which is perhaps OK when the difference doesn't have engineering consequences.

Here it absolutely has consequences -- a crash is SAFE because it protects you from undefined behavior at runtime. Crashes are also easy to debug.

I think the fallacy is that there's any other option. Many errors can't be caught at compile time, because they're simply incomputable problems. The mathematically BEST behavior is to crash.

e.g. "Regular expression" was abused by programmers, and that one also has significant engineering consequences. O(n) time and O(1) space are desirable engineering properties.

So if people let one definition of "unsafe" take over, then they've lost the ability to think clearly about the problem.

There are languages that are safer than Rust now (e.g. Python), and there will likely be languages in the future that are both safer and more efficient. (Maybe Mojo if they do something different WRT bounds checking).

> As a trivial example, integer overflow IS unsafe in Rust, but it's allowed WITHOUT using the unsafe keyword.

To reach this conclusion you need to decide that even though the behaviour (in compiler modes which don't panic on overflow you get wrapping) is well-defined it's somehow an "untrapped error" which sounds a great deal like nonsense to me. Maybe you've been talking too much to the gentlemen Babbage complained about, "if you put into the machine wrong figures, will the right answers come out?"

If you can afford to do overflow checking in production and would rather panic, today your two options are: checked arithmetic (e.g. 127i8.checked_add(127i8).unwrap() will panic) or to switch on the overflow checks even in optimised builds.

Some day, perhaps a Checked<T> will join Wrapping<T> and Saturating<T> so that you can explicitly express this in the type itself. I actually find Wrapping<u8> is rather pleasant to work with, having the same properties as unsigned char in C but with the overflow behaviour more explicit as part of the type.

As to what can't be done. You're think about General Purpose languages, but there's no good reason that so much of our software should be written in a General Purpose language. When the language can't express "Email the new model schematics and 2024 financials to our competitors in China" even if that was specifically what you wanted to do, good luck to hackers trying to make your program do that when it wasn't supposed to. This is the rationale for languages like WUFFS.

WUFFS doesn't have integer overflows, they don't compile. That is what safer than Rust looks like, not Python.

Dereferencing a null pointer is safe only on systems with an MMU to mark the 0 page as inaccessible so the processor can trap when it's accessed. On embedded systems with no MMU then 0 is a valid address just like any other.

By making null pointer dereference unsafe the compiler doesn't have to insert null checks for every single pointer access to maintain the same behavior as a CPU with an MMU.

how is integer overflow unsafe? integer overflow in rust is not ub, unless you opt into the *_unchecked methods

> On the other hand, the behavior of null pointer exceptions and a stack overflows is SAFE.

For many languages only if the OS (sometimes with a bit of help from the runtime/ compiler) protects you against them. Not all language specs require programs to abort when they occur, if only because checking for them on systems without a MMU is expensive.

The branding (or at least the zeitgeist) around Rust is that it is a "safe language", which implies that bugs are all logic bugs or somehow the fault of the programmer for doing something "unsafe" (hence the keyword).

Anyone who thinks about it for a bit knows there are limits to guarantees of safety. It's notoriously impossible to prove that a program terminates appropriately, for instance.

So I can understand some shock to run into the arcane all of a sudden. It must be especially shocking because it feels especially out of place.

Rust is a safe language for a specific definition of safe regarding a class of memory errors.

Regarding multiple classes of memory errors I think. Many languages are memory safe from control flow takeover, rust extends this to data races corrupting data.

Rust (without `unsafe`) protects you from Undefined Behavior. Behavior that is defined as crashing your program doesn't fall under that.

Rust is pretty clear about “safe” meaning no memory unsafety in unsafe-free code. It certainly doesn't market that code without unsafe will be bug free (this would force the safe subset to no longer be Turing complete).

It can still be surprising because in other languages it may be the case that a stack overflow can lead to memory unsafety. However, in Rust, a stack overflow leads to a guaranteed abort on major platforms, and a best-effort abort on niche platforms (the reason being that LLVM only provides stack probes for major platforms, so niche platforms have to make do with only guard pages).

My understanding is that there is a decent amount of undefined behavior on stack overflow in Rust, due to its use of TLS.

https://github.com/rust-lang/rust/issues/43146

I'm unclear how that would lead to undefined behavior on stack overflow. Stack overflow on Rust is specified to immediately abort the entire process (with some aforementioned wrinkles as to how achievable that is on certain targets), which necessarily nukes all TLS.

The fact that the linked issue isn't tagged with "I-unsound" suggests that nobody has ever indicated that that might somehow be a soundness issue. If you think otherwise, I encourage you to bring it up so that any soundness issue can be properly tracked.

To push this point a little further: in Rust when there's not a clean way to prevent memory unsafety (or when you haven't requested to), it aborts the process. Out of array indexing, stack overflows, etc.

Minor clarification: there's a difference between a "panic" (which may either unwind or abort, based on compilation flags) and a "guaranteed abort" (which is not negotiable). Array-index-out-of-bounds is merely a panic, as are almost all "unrecoverable errors" in Rust. Things that are guaranteed to abort are rare in Rust; I can only think of stack overflows, and OOM for certain stdlib types.

Various refcount overflows also abort: mpmc, Arc, Rc: https://doc.rust-lang.org/src/alloc/rc.rs.html#2628

I think we're lucky the usual "aha rust isn't so safe after all, checkmate rustaceans!" crowd hasn't run into these aborts yet, because they won't have any kind of debugging spew so are not very friendly. You have to intentionally leak refcounts to hit these aborts or else you first run out of memory just storing the pointers; but then again, intentionally leaking things is just the sort of thing the anti-rust crowd is likely to try.

Mostly once they discover they can leak things, Rust is pronounced "not really safe" by such people anyway.

My understanding is that the code which aborts the process contains the potential for UB.

As far as I know, in the case of stack overflow, the code that aborts the process should literally just be a single line of assembly that contains an invalid opcode, resulting in a (safe) segfault. What has given you the impression that there is potential for UB in these code paths?

EDIT: ah, I see your other comment.

Stack overflow in Rust does abort the process, but not immediately: the signal handler first distinguishes stack overflow from other sources of SIGSEGV/SIGBUS, and then prints a message containing the name of the current thread. Both of these require accessing TLS, which in turn may panic, or invoke other non-async-signal safe functions. That is UB.

I don't know what the definition of a "soundness issue" is, but I think there are some nasal demons to be found here; it will be fun to try at least.

edit the linked issue missed a case! Rust aborts by using libc::abort(), which is not async signal-safe in glibc prior to 2.26.

I have put a bug report where my mouth is. https://github.com/rust-lang/rust/issues/111272

Thanks for the report! We can continue there. :)

My point is more that debugging a core dump (and similar activities) would be all the more foreign. The engineer might even be shocked that the need to do that sort of thing even exists for a piece of Rust code.

I have chosen this as a hill to die upon:

Philosophically, the Halting Problem and the Pigeonhole Principle are very similar. They both talk about what is not possible with arbitrary input, but they have very little to say about practical input. You can't compress a random number generator. You can compress human communication, in any form, some of them famously. The existence of PP or HP are only boundary conditions, as entropy goes to infinity.

Humans also call entropy 'chaos', which has a connotation that's doubly meaningful in information theory. Chaos obfuscates. It's only useful when obfuscation is the point (encryption). We run away from it as fast as we can, and the farther we get from in the more interesting opportunities open up. Languages that disallow certain patterns can solve the halting problem for pieces of the application, maybe even entire 'useful' applications.

For some categories of applications, or even components of applications, the inability to meet a deadline in handling an event is a safety violation in the category of "or else people could die". I mentioned the halting problem instead since it's close enough to those requirements for an HN comment.

Rust has nothing special to say about that definition of safety, except that it thinks it has a better model for avoiding certain kinds of memory bugs that could contribute to those issues. Given that, I can see that "safe language" as a concept can be confusing and even leave engineers a little unprepared for unexpected categories of bugs.

Anyway, I brought up the halting problem to illustrate. I'm sure there are other illustrations you can substitute regardless of your opinion on the halting problem.

It's actually very easy for rustc to raise a compiler error if you have recursion inside an async fn. It naturally falls out from the way async is implemented, because the resulting Future type would be infinitely sized.

An `async fn` essentially compiles to a state machine where each `.await` is a transition to another state, and local variables and the future being `.await`'d are data of that state. If a future were to `.await` itself, that state data would contain the future itself, which means the future would need to be larger than itself, which is impossible. Similarly corecursion would simultaneously require that `FooFuture` contain `BarFuture and `BarFuture` contain `FooFuture`, which would require both to be bigger than the other, which is again impossible.

The solution to the infinite size problem is to wrap the future in `Box<dyn Future>` before `.await`ing it, so that the state data is a fixed size (`Box<dyn Trait>` is always two pointers long, a data pointer and a vtable pointer). Since TFA was using the `#[async_trait]` macro which changes the function to return `Box<dyn Future>` (as the stable alternative to async-fn-in-traits / TAIT), it ended up suppressing the recursion error too.

I have the same thought! But then again when I tried a simple example in clang the -Winfinite-recursion warning triggered. In the compiled code, clang compiled the function as just `ret`, which is alright because C++ standard says if a thread does not eventually do an atomic op or call certain functions then it's undefined behavior.

Still wouldn't recommend anyone to rely on the compiler to catch this.

> But I have always had stack overflows in my mental bucket of "stuff it's my responsibility to think about".

I remember working on kernel modules, and coming from userland being surprised to find kernel stacks of 16k or 24k are common. Anything that could help with that without having to do kernel debugging on weird errors would be great.

Static stack analysis is a thing, though it's a thing which annoyingly lacks any real open-source implementation, and any good tooling whatsoever in my experience. It does limit what you can do though, since dynamic dispatch and recursion can quickly make it fail.

It's certainly a Rust thing, and definitely a part of the appeal.

It doesn't remove the "stuff it's my responsibility to think about" but it certainly alleviates it. Your comment actually sounds like the first step to becoming a 'Rustacean'.

Reply on news.ycombinator.com