Hacker News Reader

Comment by pizlonator | Hacker News Reader

I think that Rust's clearly stated guarantee holds if you never say "unsafe".

That's still a clear statement, because it's trivial to tell if you used "unsafe" or not.

Maybe I just misinterpret what you mean. When you say "no way" and "all cases", I take your meaning literally. The existence of pointers to bypass the borrow checker, disabling runtime bounds checks and unsafe blocks are exactly that: escape hatches to break Rust's safety, in the same way type-casting is an escape hatch to break C's (anemic) type safety, and unsafePerformIO in Haskell is an escape hatch to break every bone in your body.

That’s fair.

FWIW, Fil-C’s guarantees are literally what you want. There’s no escape.

Trivial is not a word I would use here! Rust's `unsafe` gets fuzzy as you transverse an operation's dependencies! There are many applications where marking a function as `unsafe` is subjective.

True.

I do think the ideal kind of memory safe language either has no "unsafe", or has an "unsafe" feature that only needs to be used in super rare an obscure cases (Java is like that, sort of).

Fil-C has no "unsafe", so in that sense Fil-C is safer than Rust. You don't need an escape hatch if the memory safety guarantees are dialed in just right.

In Virgil, native targets have a couple of unsafe operations available, one of which is to be able to forge a closure from a pair of a code pointer and an object reference. This is used, e.g. to implement Wizard's JIT and fast interpreter, which generate new machine code at runtime. I can't imagine the level of proof necessary to make that safe--and not just the safety of the generated machine code, but it often lacks bounds checks because it relies on running verified Wasm bytecode, which cannot go out of bounds. So the proof would have to include a complete proof of correctness for the code validation algorithm (i.e. part of Wizard itself).

I dream of a JIT API that lets you propose machine code that is checked using an abstract interpreter to ensure that you have adequate checks to stay within the host language's type system.

Someday, man

This will kill baseline JIT performance.

I am trying out a couple of new directions, e.g. generating more of the tiers from a more abstract description, constantly shrinking the amount of hand-written compiler/interpreter code. My hard requirement is the end result has to be pretty darn close to what I'd write by hand.

One thing I am thinking about now is how to make more use of the implementation language's (e.g. Virgil) compiler to be able to paste together machine code templates gotten from writing in the implementation language. Think copy-and-patch compilation, but as language primitive. E.g. "please emit an inlined copy of the machine code for this function (first-class ref to said function) into memory here, under this ABI".

> This will kill baseline JIT performance.

You say that and then you describe exactly what I would have used as a solution: copy and patch.

Just have the checker check the templates that the baseline JIT is stitching together and then have a safe way to ask for the prechecked templates to be stitched together.

Bunch of details in getting that right obviously, but it doesn’t seem impossible.

We're not that far off from each other.

> Just have the checker check the templates that the baseline JIT is stitching together

Sure, from my second paragraph, the templates it's using could be opaque things it got from requesting the static compiler generate a template from a first class function ref (at compile time), in which case the verification has already been done.

I think we're saying the same thing

I don't get this, can you explain?

Sure.

If I told you that I have a snippet of machine code that:

- obeys the ABI of your safe language (ie it has exactly the calling convention that safe language uses)

- corresponds exactly to a function body whose signature is T->U (or whatever, different safe languages have different function type syntax)

- obeys the language’s type system.

Then you could run an abstract interpreter to check that the machine code follows that type system. Simple example: given the above claims, if we further assume that the host language impl puts argument one into register 5, and the first argument’s type is “pointer to an array of bytes”, and we know that arrays have a 64-bit length prefixed to the start, then the abstract interpreter would just need to check that any deref of register 5 is preceded by a bounds check on whatever was loaded at offset -8 from register 5. And so on, for every possible thing you can do in the language.

Then the JIT would just have to make sure it puts checks in all of the places that the absint expects them. If the absint fails, then the machine code is rejected.

I think there’s a sense in which Rust is safer than Fil-C, though: Rust allows abstractions with little to do with memory safety but that still can’t be broken without 'unsafe'. So a struct called EvenNumber can fairly strongly guarantee that it contains an even number.

But Fil-C objects (at least for now?) only seem to allow one single capability type, and that capability grants unrestricted read/write access to the object’s bytes.

I wonder if one could build a handle system in Fil-C that would allow this to be extended. Or if a different variant of a Fil-C-like system could distinguish between pointers with different access levels to an object and could allow only the correct piece of trusted code to increase the permission of a pointer.

I’ve thought about adding such things to Fil-C but have held off on going there because it feels like a bridge too far.

What I mean by that is: the memory safety issues of C are a total dumpster fire, while whether a number is even or not (and whether you can prove that) is maybe like icing on the dumpster fire. It just doesn’t matter by comparison.

So I want to decisively fix the memory safety issues and not lose focus.

The problem of C code migration to memory safe languages is that legacy C projects aim for extremely high performance. Garbage-collecting languages would also be safe in any situation, but I want to note that the recent tendency toward Rust derives from its type-system based approach that imposes very few runtimes checks such as bound checking. I myself hope something like F* gets more attraction in the industry.

That is the usual myth, back when I initially learned C, hobby programmers in Assembly could easily outperform machine code generated by C compilers, that is why books like those from Mike Abrash exist.

C got its performance fame thanks to optimizing compilers that abuse UB semantics.

Microsoft team on .NET, especially the great Stephen Toub blog posts, has been showing off how much performance can be squizzed out of a managed language compiler toolchain when people actually care.

Also lets not forget Apple only moved away from Object Pascal due to an internal team doing MPW initially as kind of submarine project, due to their UNIX roots, and still their focus was C++, not C.

I hear the perf claim a lot and yet most of the time when I write C/C++ code it's not because it's the most performant. It's either because I'm editing a codebase that's already in C/C++, or I'm using libraries whose best (or only) bindings are C/C++, or because I want to make syscalls (and safe languages don't expose those as nicely as C).

For desktop, server, web, mobile, etc. This holds true. Not so much for embedded systems, anything with unnatractive memory capacity or processor performance. Rust is starting to make it's way in, but C and even assembly is still king AFAIK.

And yet there are embedded systems running JavaScript and Python.

Are there any real-world, production systems based on JavaScript and Python?

Toy systems, yes. Hey, I too, think CircuitPython is really neat. But I'm skeptical someone would base a PLC (or similar) on it.

Above was talking about embedded systems, and I really don’t know if the James Webb Space Telescope counts, but…

The James Webb Space Telescope runs JavaScript, apparently [1].

[1]: https://www.theverge.com/2022/8/18/23206110/james-webb-space...

Have you thought about building a C verification framework around Fil-C?

Fil-C doesn't necessarily have to run in production. It just needs to catch the bugs, e.g. by making it easy to fuzz C code compiled via Fil-C.

I think you should run Fil-C in production.

Why not both?

But the Rust ecosystem is littered with unsafe, so good luck getting the actual benefits of Rust. :(

The actual benefits seem manifest: there aren't nearly as many public reports of memory corruption (much less exploitable corruption) in Rust components, even when those components make extensive use of unsafety directly or transitively.

(This seems like one of those "throw the baby out with the bathwater" cases that people relitigate around Rust -- there's ample empirical evidence that building safe abstractions around unsafe primitives works well.)

> The actual benefits seem manifest: there aren't nearly as many public reports of memory corruption (much less exploitable corruption) in Rust components, even when those components make extensive use of unsafety directly or transitively.

Not sure the data is clean enough to draw meaningful conclusions because of confounding factors.

The biggest confounding factor is that Rust is relatively new, code written in it is even newer, and folks who research vulns may not have applied the same level of anger to Rust as to C.

That said, your point about "throwing the baby out with the bathwater" is well taken. I would expect that Rust has much fewer vulns than C/C++. My point is only that it's an unproven expectation.

> Not sure the data is clean enough to draw meaningful conclusions because of confounding factors.

I'm thinking of things like the Windows user- and kernel-mode font parsers; these have a pretty long and steady public history of exploitation that seems to have mostly stopped with the Rust rewrite 1-2 years ago. I don't think that's because vuln researches have stopped looking at them!

But yeah, I would like it if Google and Microsoft (among others) would put more hard data out there. I don't think of the Windows kernel teams as typically suffering from hype-driven development, so my abductive conclusion is that they have strong supporting data internally.

Edit: here's a hard data source from Google, showing that Rust has contributed to a marked decline in memory unsafety in Android[1].

[1]: https://security.googleblog.com/2022/12/memory-safe-language...

That's good data, but: is the reduction in vulns because Rust is safer, or is it because vuln researchers assume it's safer and so choose to look for vulns in C/C++ code because it's what they're familiar with?

It's hard to say.

Wait, what? Unsafe in Rust _is_ the actual benefit. That is to say, what `unsafe` does is that it allows you to implement a fundamentally tricky thing and express its safety invariants in the type system and lifetime system, resulting in a safe API with no tricky parts.

That's the whole point.

There's tons of trivial unsafe in the Rust ecosystem, and a little bit of nontrivial unsafe, because crates.io is full of libraries doing interesting things (high-performance data structures, synchronization primitives, FFI bindings, etc.) while providing a safe API, so you can do all of that without writing any unsafe yourself.

The point of Rust isn't that you can implement low-level data structures in safe code, but that you can use them without fear.

Saying that unsafe is a benefit is backwards, I think. Unsafe is the compromise Rust made to achieve its other goals, but of course Rust would be better if there was some way of doing things without unsafe.

Consider the split_at_mut function. It takes one mutable slice, and returns two mutable slices by chopping at a caller-specified point. It's in the standard library.

The operation is completely safe, as after the call the original mutable slice is no longer live, but the borrow checker won't let you write such a function yourself unless you tag it as unsafe, so that's what the implementation must do.

The same thing happens in the implementation of Vec: there is low-level code that is unsafe, used to provide safe abstractions.

That’s not a benefit of Rust. In other safe languages you could implement those things without writing a single bit of unsafe code. (This is true in Fil-C for example.)

Is that really fair? Neither Fil-C nor C have anything that particularly resembles & or &mut. If Rust had an &shared_mut style of reference, you could presumably split it without unsafe. For that matter, Rust does have various interior-mutable types, and you could have a shared reference to a slice of AtomicBool or whatever, and you can split that without any particular magic.

Fil-C doesn’t have those things because it doesn’t need them to achieve safety.

How does it make sure data races are safe?

Rust can split up an array without unsafe if you use Rc. That has overhead, but is it more than Fil-C in this situation?

Data races are memory safe in Fil-C.

Yes but how do you accomplish that? If you do nothing at all, pointers can tear. And allocation sizes too if they're stored inline.

The capability is internally a pointer to a monotonic GC-allocated object.

That capability pointer is always stored and loaded using monotonic 64-bit accesses.

Therefore, in the worst case you'll get a pointer that is torn from its capability, and then you'll trap accessing that pointer. But you'll never get a corrupt capability.

If you don't want the pointer to tear from its capability, just use `_Atomic`, `volatile`, or `std::atomic`. Then Fil-C uses lock-free shenanigans to make sure that the capability and pointer travel together and don't tear from one another.

Okay, in that case I do think one of the data-holding types in rust is a fair comparison, adding some runtime checks to let you be looser about ownership. Then you can write a function to split/share an array without a single bit of unsafe code.

I think the disconnect here is that some people want their data races (a) not to result in memory safety violations; some people want them to also (b) not result in nonsensical control flow or code producing results that look impossible, and some people want (c) data races to simply not occur.

You seem to like (a), Linus Torvalds seems to like (b), and Rust targets (c).

Rust doesn’t target (c). That’s a hugely disingenuous claim.

The truth is: Rust’s approach to memory safety only works if you also have no data races. This makes it a strictly inferior approach to memory safety, since programs sometimes do have to race (sometimes it’s just the best solution). So, Rust has data race prevention not because it’s a good idea but because the whole language falls apart without it.

> Rust has data race prevention not because it’s a good idea but because the whole language falls apart without it.

In terms of the design goals and evolution of the Rust language, this is exactly backwards. Rust was originally conceived as a garbage-collected, green-threaded language designed for concurrent programming -- think something similar to Go but with a stronger type system and no data races. The ownership model was created, first as foremost, not for memory safety but to prevent data races; and, more broadly, to help programmers reason about the correctness of their code.

Midway through development, as people started using the language & getting a feel for it, the language designers realized that the ownership model was powerful enough to express not just data race freedom but full memory safety without needing a GC (some writing from this stage in Rust's development: https://smallcultfollowing.com/babysteps/blog/2013/06/11/on-..., https://pcwalton.github.io/_posts/2013-06-02-removing-garbag...). So they removed the GC, and that decision positioned Rust where it is today: a nicer, memory-safe "C/C++ competitor", popular for systems code where the performance overhead or runtime complexity of a garbage collector is considered unacceptable.

I don't see the interest in split_at_mut. I can get the same thing by reslicing in Go. And also the GC will do the job the borrow checker foists off onto the unlucky Rust programmer.

Pfft, whatever. Rusters gonna rust, I guess.

I'm not a Ruster, I've spent my career writing a ton of C++. But I'm interested in Rust as an alternative because it doesn't need GC.

But in this case, it's not just the memory safety I'm interested in, it's the data races. If we have multiple threads but can guarantee that any object either has only read-only references, or one mutable references and no readers, we don't have data race issues.

Great reason to use Fil-C

Reply on news.ycombinator.com