Hacker News Reader

Comment by pjmlp | Hacker News Reader

Indeed, Zig still hasn't a story for use after free issues.

The question is if this should be done at the language-level or library-level. With sufficient metaprogramming capabilities, an enterprising programmer could write safe resource management abstractions for Zig. However, this wouldn't make the language "safe by default," which seems to be what Rust programmers are bringing with this criticism.

Even if Zig doesn't bring safety by default to the language, however, I think you could get it in practice by making the standard library enforce safety. Then, unless someone goes out of their way to write their own libraries from scratch and eschew all of the safety mechanisms, they would likely live in this safety bubble.

Can you write a borrow checker that enforces alias-xor-mut using metaprogramming? I'm skeptical, because of the flow sensitivity you really want to make it practical.

You could with some changes to the type capabilities, but I don't think that's the right direction. The main question is do we want soundness or not? This is a general question for various correctness properties, and, at least in the formal methods space, the answer seems to be "not always." Soundness has a cost, and stopping 100% of UAF bugs for the cost of making the language more complex and even adding a few bugs of other kinds, might not be worth it if you can stop 99% of them for a fraction of the cost. I think the goal should be to not have soundness -- IMO it has more downsides than upsides in this case -- and make good runtime detection joined with fast compile/test cycle and even automatic test generation.

I mean, you're essentially saying here that memory safety is not worth having, which is a position increasingly at odds with the evidence.

Not at all. I'm saying that sound guarantees are not the only way to achieve memory safety. The goal isn't to use a language that makes sound guarantees, but to write correct programs (even Rust programs don't give you sound guarantees for memory safety as many of them depend on unsafe code that isn't soundly proven). That the cheapest way to write such programs is to have sound guarantees for everything is an interesting hypothesis, which would get you a language like, say, Idris, but it is not the consensus. There are many paths to safety and correctness, and not all of them go through soundness. I'd venture to say that most properties your life depends on in safety-critical systems are not soundly guaranteed.

When I say "memory safety" I mean "language-enforced memory safety". You're saying that having the language enforce memory safety isn't worthwhile.

> even Rust programs don't give you sound guarantees for memory safety as many of them depend on unsafe code that isn't soundly proven

There's no such thing as 100% memory safety; at the extreme end there are bit flips caused by cosmic rays. But the evidence suggests that practical language-enforced memory safety is actually worthwhile, despite the fact that no theoretical absolute memory safety is possible. All memory-safe languages have runtimes and FFIs, which in Rust is the unsafe blocks. This doesn't change the fact that, empirically, memory safety, even the imperfect memory safety we have to live with in the real world, is a meaningful improvement in stability and security.

> That the cheapest way to write such programs is to have sound guarantees for everything is an interesting hypothesis, which would get you a language like, say, Idris, but it is not the consensus.

Straw man. Nobody is talking about proving all code correct. What is reasonably describable as consensus nowadays is that having the language enforce memory safety is worthwhile. The idea that enforced memory safety has, in your words, "more downsides than upsides", is increasingly at odds with the consensus.

To get concrete, I see no reason to believe that quarantine (Zig's current solution to UAF) is a meaningful solution to use-after-free problems, given that quarantine has been deployed for a long time in production allocators in other languages and has failed to eliminate this bug class in the wild.

> You're saying that having the language enforce memory safety isn't worthwhile.

I am saying that it may come at a cost, and overall it may not be worthwhile; but, of course, there are different kinds of safety bugs, different kinds of soundness, different costs, and different alternatives.

> But the evidence suggests that practical language-enforced memory safety is actually worthwhile, despite the fact that no theoretical absolute memory safety is possible

What is it that the evidence suggests exactly? There are so many variables. For example, Zig's runtime checking could be much more effective than similar systems for C or C++, because it is much easier to track all pointers in Zig, just as its overflow protection is far more effective than sanitisers in C, because there's no pointer arithmetic (unless using unsafe operations) and all buffer sizes are always known. So you can't compare what Zig can do to what C can do. Then there's the question of what the safety mechanism is. Then, for each point in this coordinate space, what is the fitness you're looking at? Reducing a particular bug or improved overall correctness?

> What is reasonably describable as consensus nowadays is that having the language enforce memory safety is worthwhile.

No, this is not true, or at least, that depends on what is the fitness function and what exactly it is compared against. To put it bluntly, there is absolutely no consensus that Rust would more cheaply produce programs that are more correct overall than Zig. It's possible this is the case, as is the opposite or that the two are about the same, but we just don't know yet.

> To get concrete, I see no reason to believe that quarantine (Zig's current solution to UAF) is a meaningful solution to use-after-free problems, given that quarantine has been deployed for a long time in production allocators in other languages and has failed to eliminate this bug class in the wild.

But that's not how we look at correctness. The goal isn't to eliminate all UAF bugs. Of course completely (or close to it) eliminating a specific bug is more effective at reducing it than not completely eliminating it. If my only goal was to eliminate UAF, then I'd choose Rust's approach over Zig's. But usually our goal is to write a program that meets our correctness level for all bugs combined in the cheapest way possible. Investing a huge language complexity budget in completely eliminating UAF, as opposed to, say, reducing it by 99%, has not been shown to be the best way to achieve what we actually want.

I am not saying that it's not reasonable to believe that soundly eliminating this particular bug with something like Rust's ownership and lifetime system ends up being better overall, but it is equally reasonable, based on what we know, to believe that Zig's way is more effective overall, or that the two end up the same. There's just so much we don't know about correctness.

On the other hand, then one can just go the Apple way and create a safe C dialect instead, while keeping the same tooling if 99% is good enough.

See "Memory safe iBoot implementation" section,

https://manuals.info.apple.com/MANUALS/1000/MA1902/en_US/app...

Sure, but the goal of new languages like Rust or Zig isn't to only add safety.

True in regards to Rust.

For Zig I really don't get what it offers over Modula-2 other than a more C like syntax and compile time metaprogramming.

Maybe I will be proven wrong.

Not sure, but I became optimistic of there being some solution after I saw someone implement affine typing using stateful template metaprogramming: https://godbolt.org/z/PnPPrnPjY.

That's neat, but also flow-insensitive (and wow, generating a definition for every single use of a variable can't be good for compile-time memory usage).

Yeah, I'm unaware of any push to bring flow-sensitive typing, or a mechanism that affords flow-sensitive analysis, to Zig.

Not sure what 'story' is intended to mean here. Is use-after-free undefined behaviour in Zig?

There isn't really anything in the language that prevents use-after-free, partially because allocation isn't even part of the language and exists entirely in userland code. However, IIRC the standard library does include an allocator that tries to detect use-after-free.

Isn't that an intentional design decision in Zig? As far as I understood, Zig isn't trying to "solve" memory management, it's trying to provide manual memory management with better tools than C to help you do it right, but it's explicitly not trying to save you from yourself in this respect.

Except that C and C++ already provide similar tools, one just needs to actually enable them.

I don't follow. You recently pointed out to me that most of the serious security bugs in the Chromium codebase (C++) are rooted in memory-management. [0] It's no simple thing to fix these bugs. Not even Google can do so in their most treasured code.

[0] https://news.ycombinator.com/item?id=26862330

The point being that Zig improves very little regarding that, because it is based on the same kind of tooling.

> allocation isn't even part of the language

It's not supposed to be. In most cases, memory allocation is an operating system concept. As a systems programming language, zig must not abstract this away from the programmer.

You don't need allocations to be part of the language, you just need destructors (or linear types.) But the Zig authors intentionally omitted destructors because they want all control flow to be explicit. There's more at [0].

Side note: the proposal at [0] seems very close to [1].

[0]: https://github.com/ziglang/zig/issues/782

[1]: https://www.haskellforall.com/2013/06/the-resource-applicati...

Too late to edit, but: I'm an idiot, I was thinking of memory leaks, not use-after-free.

Allocation isn’t part of the core language in Rust either, yet it prevents use-after-free generically via RAII

Currently I'm using Rust in a project without RAII. RAII is neither necessary nor sufficient to prevent use-after-free. In Rust use-after-free is prevented by the borrow-checker. In C++ with RAII use-after-free is still possible, because normal references aren't tracked with regard to the lifetime of pointees.

Microsoft and Google are trying to improve it via lifetime analysis, but after 3 years it is still pretty much WIP due to C++ semantics.

Is use-after-free ever really defined behavior?

Yes. Undefined behavior exists only in standards. It's perfectly possible for implementations to define everything.

https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost...

> I first heard about this from one of the developers of the hit game SimCity, who told me that there was a critical bug in his application:

> it used memory right after freeing it, a major no-no that happened to work OK on DOS but would not work under Windows where memory that is freed is likely to be snatched up by another running application right away.

> The testers on the Windows team were going through various popular applications, testing them to make sure they worked OK, but SimCity kept crashing.

> They reported this to the Windows developers, who disassembled SimCity, stepped through it in a debugger, found the bug, and added special code that checked if SimCity was running, and if it did, ran the memory allocator in a special mode in which you could still use memory after freeing it.

That would depend on the language and the libraries.

You could roll your own 'safe' memory pool in C such that use-after-free doesn't produce undefined behaviour. You'd still have a bug, but it wouldn't have to invoke UB at the level of the C programming language.

Reply on news.ycombinator.com