TIL: Rust does not prevent memory leaks | Hacker News Reader

TIL: Rust does not prevent memory leaks

26 pointsakagusu22 comments

I'm not a Rust developer so I have no clues about what Rust does, how it does or why it is the way it is.

I had a short interaction on another thread and some user told that Rust does not prevent memory leaks.

People talk so much about Rust memory safety and how everything must be rewritten in Rust, but if Rust does not prevent memory leaks why are people so crazy about it?

View on HN

> People talk so much about Rust memory safety and how everything must be rewritten in Rust, but if Rust does not prevent memory leaks why are people so crazy about it?

The primary goal of rust is to provide a low-overhead, low-level language which is memory safe without the need for a runtime. Memory leaks are not a memory safety concerns[0], although they can cause programs to crash they are not UBs, they don't cause grave vulnerabilities (they might lead to DOS but not something like a compromised device)..

Furthermore while the Rust team and community discovered they could not reasonably prevent memory leaks[2], at least not without a very different languages and years if not decades of development, a normally constituted program is unlikely to have unintended leaks[1], the main vehicle to get that being a refcount cycle.

[0] although in Rust the realisation that leaks could not be prevented did force the reconsideration and binning of entire APIs because they could not be made safe anymore

[1] since it was realised that leaks were a fact of life, the standard library added a few APIs whose entire purpose is to leak objects

[2] much to everyone's dismay and displeasure, I assume you can still look for "leakpocalypse" for recountings of the events

Memory safety is not about memory leaks. It's concerned with the corruption of data structures, particularly the stack and the heap.

To elaborate a little, consider security. If your software's only vulnerability is potential memory leaks, an attacker can only get away with denial-of-service attacks. If your software is vulnerable to buffer overflows, an attacker can use return-oriented programming to get arbitrary code execution, exfiltrate all your data, and add your machine to a botnet to perform subsequent attacks on yet other systems.

To add to what was said, one of the main ways to leak memory is `Box::leak` [0], meaning most memory leaks are explicit and intentional, rather than implicit and accidental. Likely the main implicit/accidental way to leak memory are Rc cycles.

Looks like https://cglab.ca/%7Eabeinges/blah/everyone-poops/ has more background on why this is allowed.

[0]: https://doc.rust-lang.org/std/boxed/struct.Box.html#method.l...

The "Memory Safety" of Rust is not that memory leaks are not possible, rather its how memory is accessed. Specifically, using "safe" Rust, it should be impossible to:

1. Access memory that hasn't been explicitly allocated to your program 2. Access memory that was was previously allocated and freed by your program 3. Access memory that could be modified by a separate thread / function of your program, outside of some locking mechanism.

I'm sure there's a more academic definition, and stronger guarantees, but those are the basics.

Now, these guarantees impose restrictions on how code is written, and runtime overhead to enforce. There are many cases where a human can write optimizations and ensure those guarantees in a way the compiler cannot verify. This is why Rust has an "escape" of writing "unsafe Rust", where the compiler does validate those guarantees. Unsafe Rust includes expanded primitives, such as "Box::leak()", that enable writing these optimizations, and may explicitly leak memory without maintaining a reference

If a program is written using purely safe Rust, it should not be possible for a malicious input / request to corrupt internal data structures, read memory that shouldn't be read, or all the other "interesting" things that get highlighted by Heartbleed and other big security vulnerabilities.

Note that I say "should not be possible" - the Rust compiler can and has had bugs that break those memory safety guarantees under certain conditions. They aren't common, but it's unfair to say "Rust is perfect and 100% safe"

I strongly recommend everyone learn Rust, at least to the point where you "grok" the borrow checker and what it's trying to enforce, even if you have no intention with regularly using Rust. After learning Rust, I found the patterns the borrow checker enforces incredibly useful when writing C++, and ensuring I don't make silly memory mistakes that may result in a crash or vulnerability.

You're conflating memory safety and memory usage/management. Safe Rust prevents you from addressing or accessing arbitrary memory, null pointer dereferencing, and buffer overflows, while remaining performant relative to other memory safe languages (e.g. Java, Python, Go, Ada, etc). In particular, it doesn't have a Garbage Collector.

There is no actual difference between a memory "leak" and intentionally allocated memory except the INTENT of the developer. It is important for good language design to avoid accidental memory leaks via well-designed constructs. Ultimately there will always need to be a mechanism for a developer to create a block of allocated memory and free it only on their schedule.

Rust's language design does make it harder to leak memory, and well written Rust would rarely do so. However, if Rust made arbitrary allocations impossible, certain use cases would also be nearly impossible and performance for others would suffer greatly (e.g. allocating a buffer ahead of time, for the lifetime of a worker). It is a balance, at some point you have to trust the developer to make decisions.

Memory leaks are different from memory safety. What people are talking about is that Rust helps prevent the use of memory that wasn't intended. That's the big win there.

Disclaimer: I am not a fan of Rust, this is just my understanding of this aspect of the benefits of it.

(Not a regular rust developer, so take this with a large grain of salt)

Rust can absolutely prevent memory leaks. It depends on how much unsafe rust code you use. Many rust developers find the convenience outweighs the risk of leaks, and take the tradeoff offered by the `Rc` type.

>It depends on how much unsafe rust code you use.

What does the term "unsafe" mean. If a language can have "unsafe code", then it is not safe. I heard the term "unsafe code" used everywhere in relation to c as a excuse for bad code.

So, if you are correct, you can still have leaks in rust, thus removing the number 1 reason to use rust.

Up and down this tread there are people saying "it cannot have leaks" and people saying "yes it can". That tells me moving to rust is not worth the expense and risk because no one knows how safe it is.

Leaks aren’t a memory safety issue, so that’s not one of the reasons for using Rust although the modern language design does make it less prone to accidental leaks.

Unsafe has a strong name to encourage care about using it but that still doesn’t make it as unsafe as C is by default. You’re telling the compiler that there are certain areas where you have to do something which would normally not be allowed because of some information available to the developer but not the compiler. For example, Rust targets systems programming and it’s likely that you’ll need to do things such as dereferencing a pointer if you have to interact with a C API. Unsafe lets you do that without giving up other protections like the borrow checker and it confines that to specific blocks which you can audit more carefully.

https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html

If I might use a car analogy, going from a mid-20th century language model like C to Rust is like buying a car which has automatic braking and lane assist. Yes, you can still crash if you turn those features off but the ratio of preventable problems to special circumstances where you know you’re doing something tricky is incredibly high.

This is a hyperbolic comment. You are either simplifying to the core, are misinformed or ignorant. There is a nuance.

You can have leaks in every language including garbage collected ones. Allocate a shared OS resource like a semaphore and don't give it back, there you go you have a leak that survives until the next reboot.

Safety as Rust means requires below:

- The program cannot call an operation that accesses invalid memory i.e. uninitialized memory and invalid memory areas.

- The program cannot substitute a memory location that defined as a particular type as another type.

- The program cannot make calls to an operation that causes the runtime environment (OS, CPU interrupts, running modes, system registers etc) that causes the two issues above.

If you have already allocated memory that just stays as it is, this is safe. The action of allocating it can be unsafe if the program makes a call to libc allocator since libc calls are outside of the Rust compilers' control. Similarly directly calling the page allocator of the OS will also be unsafe since Rust compiler doesn't have complete control of the OS and internal structure of the CPU. Similarly freeing memory may also change the page tables.

Moreover Rust makes it easier to create data structures that clean after themselves. This is not a direct safety issue. It is a language feature. The standard library makes extensive use of these. So the common programming patterns usually mitigate the risks of leaking resources but not eliminate.

Box::leak() and std::mem::forget() are safe code. Memory leaks are unrelated to memory safety.

"Safe" is unfortunately overloaded here. I wasn't trying to say anything about memory safety. `leak` and `forget` may be memory-safe, but they can only be implemented with `unsafe` code to circumvent the liveness/"relevant" guarantees of the type system.

While this is not the direction the standard library took, I think it would be possible to write an alternative stdlib that doesn't choose to allow leaks.

My original statement was probably too strong though. Leak-free code might be possible with rust-the-language, but is probably not possible with rust-the-ecosystem without significant development.

This is the kind of post I see on Reddit. I don't really understand what this is doing on the front page, it is a low effort question, OP says it doesn't have any clues about how Rust works and has apparently spent 0 time searching about it.

And yet it, and the replies, cleared up a misconception I too had about memory management, in Rust and in general. There's no value to elitism in this circumstance.

I daresay any Turing-complete language will be unable to prevent all memory leaks, since the simplest memory leak is just using more memory intentionally and never letting any of it go. Unless you want to eliminate resizable data structures like linked lists, growable arrays, trees, and hash tables, you can always leak memory.

What Rust and most other modern languages do is make it very difficult to unintentionally and/or unknowingly leak memory. There are times when even this can't be guaranteed without breaking other constraints (e.g. one of the solutions to the setenv unsafety issue involves leaking memory instead), but they are rare and documented.

Rust does help to prevent memory leaks, but it is not considered a soundness bug to leak memory, whereas it is a soundness bug to (e.g.) use memory after it is freed.

Historically, what happened in Rust is that shortly before 1.0 was released, someone found a situation where you could get a reference-count cycle with the standard container types, and there was a lot of consternation as to whether or not this was actually a bug. This is termed the "leakpocalypse." The ultimate resolution was to accept that destructors are not guaranteed to be run, and therefore the kind of leak that was created is not actually a bug.

Yes, you are not alone in being surprised by this. Another thing that sometimes surprises people is deadlocks, and maybe the Fearless Concurrency slogan is partially to blame.

To Rust's defense, both are extraordinary hard problems, and Rust's tooling is better than that of other languages in the space.

For memory leaks we have (above and beyond the usual suspects from the C/C++ world like valgrind) the leak detection in MIRI. For deadlocks we have the parking_lot crate.

Memory leaks aren't unsafe generally, unless they are triggered in a way that could lead to DoS attack.

That's not memory safety that rust advertises.

As already said, Rust puts guards when using memory. Forget what everyone is doing and start writing some programs - from simple to advanced - to understand what Rust is all about. There are some great resources (TRPL, books, blogs, YT videos, playgrounds etc) out there that should get you going.

Think in terms of sharing memory across threads and what you need to do to avoid contention, use after free, races etc - and you will realize Rust helps you avoid all of that - and more.

Granted Rust has its idiosyncracies - which language doesn’t?

Try Rust yourself and don’t worry about the hype and what others are saying. Using LLMs would help you understand, write, troubleshoot some advanced concepts. We live in an amazing time where this is possible. Take full advantage of this.

std::mem::forget() explicitly leaks a resource, including memory[1]. The whole point is to allow circumventing running a destructor. There's also Box::leak() to quite explicitly leak memory[2]. Handy for dynamically allocating memory that needs to live for the remainder of the program (making `'static` lifetimes at runtime).

[1] https://doc.rust-lang.org/std/mem/fn.forget.html

[2] https://doc.rust-lang.org/std/boxed/struct.Box.html#method.l...

Rust does prevent memory leaks, however memory leaks are still possible, as they are in any language. Is it even possible for a language to make memory leaks impossible? A memory leak is different from safety, btw.

Why is this flagged?

Reply on news.ycombinator.com