Comment by Nacraile | Hacker News Reader

Comment by Nacraile | Hacker News Reader

Parent Full thread

> I ask this in bad faith: I encourage you to share a single nontrivial codebase which actually creates the abstraction you've described and religiously adheres to using it throughout. As to why this is in bad faith: I'm definining "nontrivial" here to mean using 3rd party APIs - which will operate on C style arrays, not your project specific safe wrappers - and thus by definition won't be "religiously" sticking to said abstractions when using said APIs. By these definitions, the codebase I'm asking for doesn't exist - by definition. Even relaxing the "third party" rule, I haven't actually worked on a nontrivial C or C++ codebase without buffer overflow problems.

I work on a C codebase that does this, although in the slightly weaker sense that it does drop the abstractions at a few isolated interaction points with external APIs (think openssl, linux system calls, and not a whole lot else). Yes, there is quite a lot of NIH. With essentially-uniform use of checked data structures, and an extremely comprehensive suite of automated tests getting run under ASAN (originally Valgrind), memory safety errors almost never get so far as being committed to the main branch. This is a complex, >1M SLOC distributed system that has seen several years of production use at this point, and as far as I can recall we have not seen a single memory safety related issue in production (a few have managed to to get as far as certification testing). General resouce-leak class issues have struck a few times, but are also pretty rare.

Proprietary, naturally, so I can't actually show you (sorry), but it absolutely can be done in practice. It isn't even really all that difficult, it just needs to be done from the start, and then you just need a bit of discipline to keep it up.

>I work on a C codebase that does this

And more power to you. Note the beginning of the parent comment, however:

> Just because you 'can' write such an array implementation doesn't mean you will

So yes, even if the codebase you work on does have these 'mythical', hard-to-achieve properties, that doesn't mean that most or even many C codebases will.

Good engineering entails observing what problems actually occur and working to fix those. Memory safety issues do commonly occur in C codebases. Regardless of whether the fix in C is simple or even trivial, programmers aren't doing it. So, Rust has some value because it forces the programmer to produce code that is largely free from this type of issue.

Enforcing norms like 'be more disciplined when writing C' or 'stop using external libraries' is much harder than simply using a different language.

> I work on a C codebase that does this […]. Yes, there is quite a lot of NIH. With essentially-uniform use of checked data structures, and an extremely comprehensive suite of automated tests getting run under ASAN (originally Valgrind) […]. This is a complex, >1M SLOC distributed system that has seen several years of production use at this point […].

>

> […] it just needs to be done from the start, and then you just need a bit of discipline to keep it up.

Here's a neat idea: wouldn't it be cool and save a lot of time if the compiler did this for you automatically, from the start?

Of course, you say it's easy to do it manually, but something tells you your company might have needed to pay less for development if the compiler did it automatically with no human intervention required.

But Rust still won't warn about an out of bounds access (when accessing using a variable) at compile time, and your code will panic at runtime. This isn't the "safety" anyone ought to be expecting from a language billed incessantly as safe.

Rust, at least in this regard and probably others too, is no better than C, and for me it isn't enough to justify the horrible and complex syntax.

Safety has to do with memory corruption that lead to security exploits and unrecoverable data, not panics.

Panics can be a vector for denial-of-service attacks. OTOH that's still better than remote-code-execution, and unwinding panics can be caught. OTOH I see failing to do so across a FFI boundary is UB, which could get back into RCE territory through sufficiently convoluted shenanigans...

What do you expect a language to do if you index an array using a runtime-calculated value?

Rust checks at runtime and panics if your program exceeds the bounds. You can opt-in to asking if the bounds are exceeded and fail gracefully if you like, or if you want to promise the compiler you know for sure your bounds are tight, you can use unsafe blocks and act like C. Opt-in to danger.

C lets you do it with no checks. You have to opt in to the safe path of checking and failing. You don't automatically segfault if you exceed the bounds; instead you read arbitrary memory. Welcome to the land of undefined behavior. You may crash, but more likely, you will read some value from an unexpected place, and carry on executing incorrectly for who knows how long. Opt-in to safety.

That's what people mean by Rust is safe by default. And that's just bounds checks. Carry that notion over to pointers, references, threads, lifetimes, ...

What ever made you think "Safe Rust" meant "compile time checks of runtime values are possible" or "C is just as safe because it lets you index outside an array"?

> What do you expect a language to do if you index an array using a runtime-calculated value?

Optimizers already try to prove index bounds to eliminate unnecessary checks, and static analysis tools to demand necessary checks. Turning the latter into compile time errors is a reasonable approach if your language can provide sufficient information to deal with false positives - likely by forcing you to add your own bounds checking to explicitly handle out-of-bounds cases.

A language John Carmack was using or researching at one point comes to mind, which had this kind of thing going on IIRC. I'm afraid I can't find it off hand, so I might recall in error.

Sure, but this can be the same in Rust or C, which was the context of this discussion. New languages may provide more tools to eliminate more checks, but in general, if you have a runtime value and you index an array with it, someone has to check somewhere.

And the point of this thread is that Rust's default (check index bounds and fail at runtime unless the check can be proven unnecessary by the compiler or optimizer) is safer than C's default (don't check anything by default and hit UB if the index is out of bounds).

>You don't automatically segfault if you exceed the bounds; instead you read arbitrary memory.

GCC and clang both have sanitizers either built in or available for them. Sure, it's not default, but let's not act like there is no choice in C but to account for every OOB access while programming or to read memory you don't want to.

Furthermore, I never said that compile time checks of variables are possible, but rather we could move to using dependent typing, or at least a way to judge whether a variable would work as a subscript based on the type of the array and variable.

The Rust designers didn't do that. Instead they put in a feature common in the two most popular C compilers to "panic" at runtime instead of accessing memory. That's nothing. It's rubbish. And if you know to "catch" the panic, why don't you check the value of what you're subscripting with? Saying you can catch the panic is missing the point of unintentional OOB accesses, which is that they're unintentional.

C is just as safe with regard to OOB accessing, and to be honest that's pretty poor in 2017.

> let's not act like there is no choice in C

If you are using gcc or clang, you have more options. True. But not all C compilers give you those options. However, the point is moot, since I never said you can't catch these things in C; I said it wasn't the default. Which you agree with.

> I never said that compile time checks of variables are possible, but rather we could move to using dependent typing

You didn't say anything about dependent typing. You said "Rust is no better than C". And I'm pointing out that it is. Dependent typing may be even better in some cases; I'm not arguing otherwise.

> Saying you can catch the panic is missing the point of unintentional OOB accesses, which is that they're unintentional.

No one said you should catch the panic. You can use Vec::get() for example if you are using runtime-derived indices and want bounds checking in an ergonomic fashion.

And saying a panic for unintentional OOB is the same as in C is not true, since you get a panic by default in Rust, and to get one in C, not only must you be using a specific compiler or two, you must have the sanitizers enabled for every source file in your program. Not "by default" by any stretch.

> C is just as safe with regard to OOB accessing, and to be honest that's pretty poor in 2017.

It is nowhere close, and saying it is is pretty poor in 2017 as well.

And you are still ignoring pointers, references, lifetimes, threads, ... you know, the other things that also help Rust make "Safe by default" and C "dangerous by default".

If you are not sure whether the index is in bounds or not, you could use `slice::get`, which returns an `Option<&T>`. Sure, it's a little more typing compared to `[i]`, but how would one solve this in any language?

https://doc.rust-lang.org/std/primitive.slice.html#method.ge...

Do you know what a panic is?

C will actually read arbitrary memory, Rust won't, that's the difference.

We're talking about a situation where the length of the array is not statically known and you access it out of bounds at runtime. Rust checks first if it's out of bounds, and if it is, DOES NOT blindly read the memory anyway (as C would) but exits.

I don't understand how you could think the two situations are at all equivalent.

> Here's a neat idea: wouldn't it be cool and save a lot of time if the compiler did this for you automatically, from the start?

Yes, but when the project started the only existing compiler that met all requirements was C (also C++, although that was not chosen, by reasoning I disagree with). We are in a domain where we derive material benefits from the low-level control C gives us (we have a bunch of highly specialized memory management and I/O), and are not willing to accept GC pauses. There's a common sentiment that we would have used Rust if it had existed when we started, but it didn't so we didn't and so it goes.

Would you care to point to or write a blog about the principles of this strategy? It sounds immensely useful to the many C developers out there as a potential set of best practices to adopt.

Reply on news.ycombinator.com