Hacker News Reader

Rust sounds like what you want, though be prepared to deal with near C++ levels of complexity at times (lots of Rust code is too macro happy for my tastes).

That said, the way to deal with memory management in C is to... not do much of it. I know that sounds like a cop-out but the patterns you're supposed to use in languages like Zig and Odin are the same ones you'd use in C to keep your mind sane.

Do not do Reference Counting or Single Owner + Borrowing (RAII), it'll drive you insane without the automation languages like Swift and Rust or C++ give you.

Instead, the way you're supposed to do things, are big "manager objects". Only these manager objects are ever explicitly allocated and freed, everything stored within them is managed by them and they expose only safe handles to the outside world (e.g. generational handles).

Most of the time these manager objects will store some dynamic arrays, hash maps or pools within them that get freed when they are also freed. Note that unlike RAII there is no real "nesting". The managers are responsible for the lifetimes of their whole "object tree".

Any temporary data should be allocated using a temporary allocator (like an Arena allocator) which gets freed at the appropriate place for the application (end of a frame in a game, or end of a request in a web server). Never store pointers to something within the temporary allocator in "long lived" data structures inside the manager object.

Follow these rules and things get manageable, Zig and Odin have lots of facilities in their standard libraries to make this easier, in C you're mostly on your own but there are some libraries you could use like the Apache Portable Runtime.

I’ll admit, the fact that Hello World requires a macro, dissuaded me from learning Rust for years.

You’re using the past tense. Did your mind change?

Technically, you don't need a macro at all:

    use std::io::Write;

    fn main() {
        std::io::stdout().write(b"Hello, world!\n").unwrap();
    }

The reason people often use println! is that println! is variadic and can support any number of arguments to format in the string. For example:

    println!("x is: {}", x);

This also has the benefit of allowing type checking at compile time, as opposed to using a function like printf in C, which does not have such power. Additionally, this allows Rust to take references to the variables entered without the programmer's specification, in order to prevent unnecessary copying.

These reasons tie directly into Rust's philosophy of making it easy to write reliable, performant code.

Yeah this kind of framing always rubs me the wrong way. I don't "dread" manual memory management. Indeed, from a personal standpoint I really like thinking about stuff like that while I'm programming. It's part of the fun puzzle aspect that got me into all this. But as a professional, I know that there is heaps of evidence over many decades that it is unwise for me to indulge this fancy, when working on real systems that real people use and which may be exposed to the internet. I consider it a bummer, but I'm very persuaded that this is unwise for pretty much all of the software I work on.

You can have safe manual memory management. The main cases of bugs are:

1. Null pointer deref. Can be fixed by having optional types and requiring that possibly null pointers have to be wrapped in them.

2. Out of bounds references. Can be fixed by making the type system track how big all objects are, and having the compiler insert bounds checking.

3. Use after free. Can be fixed by the free function zero'ing heap objects smaller than a page (eg 4KB), and unmapping larger ones so that future accesses are a seg fault. The heap also needs to not create new objects at the same address as deleted ones, but we have 64-bit address spaces, so maybe that's fine.

These all have costs, but so do all solutions to these problems.

I can't think of any reasons that a language with manual memory management has to be less safe than one with a Garbage Collector / ARC.

By "manual memory management", I think of explicitly allocating and freeing memory. Assuming you're thinking of the same thing, are you suggesting that the compiler (or other static analysis) could catch all of the issues you listed? If so, the natural question is, why is it necessary to manually insert the allocations and frees, if the compiler knows where they are supposed to go?

This path leads you to something like Rust, which is safe without garbage collection or ARC, but I also wouldn't call it "manual memory management". The trade off they took for this is complexity in the language.

An additional tradeoff is that many safe data structures or algorithms are impossible to implement in safe rust. Doubly linked lists being the canonical example. However I feel like the juice is very much worth the squeeze and most rust developers won’t run into these limitations regularly.

Yeah, but I think this is also captured under the complexity trade off. It is possible create a correct safe interface over an unsafe data structure, but it is complex to do so.

GP's solutions to problems 1 and 2 are the same as Rust's; the difference is problem 3 ("temporal memory safety"). Rust solves this problem with statically analyzed lifetimes, which, in addition to the safety and correctness advantages of compile-time checking, also permit RAII (which answers your "natural question" with "no, it's not necessary to do that"), which makes programming more pleasant. The disadvantage is, as you said, language-level complexity, since it's not enough for the programmer to be personally satisfied that the lifetimes are correct; they have to be represented in the code in a way that satisfies the borrow checker.

GP's proposal is to drop lifetimes and RAII (thereby simplifying the language semantics), make the programmer responsible for allocations and frees (as in C), and solve the temporal memory safety problem by doing additional work at runtime to ensure that use-after-frees reliably crash the process instead of overwriting return addresses or doing other arbitrarily bad things. Whether this is more fun to program in than RAII depends on whether you think it's better to suffer from too much abstraction or too little; people have sharply diverging intuitions on this and it's been a holy war since forever and it probably always will be.

The clearer-cut problem is that such a language would be slower than C or Rust, both because freeing memory involves extra work that C and Rust programs don't have to do, and because the requirement that use-after-frees must reliably behave a specific way inhibits optimization, since the compiler can't assume that use-after-frees don't occur. Also, using memory from a small allocation that's been zeroed out doesn't reliably crash the process unless a pointer in that allocation is dereferenced, and even then, this (contra point 1) would require the compiler to assume that null pointer dereferences can happen and must segfault, which, again, inhibits optimization.

I agree with all of that. How theoretical are the optimization gains from the extra constraints guaranteed by Rust's borrow checker?

Well put!

> Assuming you're thinking of the same thing

Yes.

> are you suggesting that the compiler (or other static analysis) could catch all of the issues you listed?

Yes, the compiler for the first two and the standard library's heap implementation for the third.

> This path leads you to something like Rust

There are stops along this path before you get to Rust. If you just add the 3 things I mention above to a C like language, it would still be perfectly possible to leak memory. But that isn't a safety problem.

The 3 things don't include a borrow checker. You could still make doubly linked list and graph data-structures.

That makes sense! This isn't the set of tradeoffs that most appeals to me, but I can see your point that it's a different set of trade offs that may be good ones.

the last thing is safe concurrency. this is the greatest achievement of the rust borrower checker imo. the ownership model means data races are detected at compile time, eliminating a huge class of concurrent programming bugs without forcing a message passing architecture. deadlocks are still possible if you have two or more mutexes but that’s a much harder problem

You are correct and Odin supports all of this.

`Maybe(^T)` exists in Odin.

Bounds checking is on by default for all array-like access. Odin has fixed-length arrays, slices, dynamic arrays, maps, and #soa arrays, all of which support bounds checking. Odin does not have pointer arithmetic nor implicit array-to-pointer demotion which is pretty much removes most of the unsafety that languages like C have.

Odin also has built-in support for custom allocators which allows you do a lot more with extra safety features too beyond the default allocator. Use-after-free is usually also a symptom of an underlying value responsibility and lifetime problem rather than a problem in itself, of which is fundamentally an architectural issue. Ownership semantics in a language in Rust does deal with this issue BUT it does come at a huge cost in terms of architecting the code itself to accommodate this specific way of programming.

There is a common assumption amongst many of the comments that if you have manual memory management, you are defaulting to memory unsafety. This is untrue and memory management and memory safety are kind of unrelated in the grand scheme of things. You could have C with GC/ARC and still have all of its memory unsafety semantics.

Odin and Zig solve almost all the memory safety aspects, except for use after free. Having bound checks, union types, etc. is good and a real improvement over C, but I think that use after free is a memory unsafety issue. It's the one that's harder to tackle because it's more dynamic ("temporal memory safety"?).

"Use after free" is a symptom of other problems. You can "solve" it by making it very different to do in the first place with something ownership semantics, but there are usually better ways of dealing with it in the first place.

One really good approach is to not use pointers in the first place and use handles. I highly recommend this post for more information: https://floooh.github.io/2018/06/17/handles-vs-pointers.html

Because use-after-free is a responsibility problem, handles are a way to make sure that a subsystem has responsibility over that memory directly rather than have it spread out across the program.

This is why Odin nor Zig "solve" this problem: solving it at the language level is not necessarily the best option.

That’s essentially what Rust does: it makes using handles to get temporary access to memory that are owned in a single place very attractive because otherwise the borrow checker will yell at you.

If you had this belief say, 10 years ago, and because of it you decided to write your serious application in say, Java, you would have been in for a huge unexpected surprise last year when the now infamous log4j vulnerability was made public.

See, having memory safety did not prevent the language from causing arbitrary code execution vulnerabilities. Having the log4j project be open source and popular did not prevent that either (so much for the "enough eye balls" theory).

Going back to Odin, when I think memory safety is not as big a concern as people make it out to be:

Your only source of concern is C.

This would be like judging SQL statements as fundamentally unsafe because websites written in PHP tended to (specially in the early 2000) be written in a very unsafe manner where user input was put directly into SQL strings.

The lesson that people took is not to throw SQL out the window, but to properly sanitize user input before passing to the queries, and to never use plain string concatenation when doing that.

So for manual memory management, the lesson to take from the vulnerabilities that C has caused is not that manual memory mangement is bad. It's that you need some facilities in the language to minimize the chance of them occurring by several orders of magnitude.

Odin does this by providing the slice type (and string type) that have their length known and providing several custom allocators out of the box.

The cool thing about the slice type is not just that the length is known: the language provides facilities for iterating over the slice that automatically never goes out of bound:

    for item, index in slice {
        // do something
    }

This, and providing a "string builder" type into the core library that lets you dynamically construct a string in a safe way (you don't have to write the code to grow the string dynamically because it has already been done).

These features make the "fear" of unsafe memory access largely unwarranted anymore.

What remains is a matter of what attracts you to programming: are you interested in having explicit control over a system to make it do what you want, or are you more interested in expressing some abstract ideas in an abstract mathematical virtual machine? If the latter, you might find Haskell or Lisp more appealing.

Highlighting a different systemic problem is not at all an argument against avoiding this problem. Indeed, if memory safety were the only problem to worry about, then I'd be less worried about it, because we could spend more time on it. But it isn't, there are all sorts of other things to worry about, simultaneously.

> This would be like judging SQL statements as fundamentally unsafe because websites written in PHP tended to (specially in the early 2000) be written in a very unsafe manner where user input was put directly into SQL strings.

This is indeed a good example, but supports my argument rather than yours. If we had devised ways to make it impossible (or incredibly hard and weird) to introduce sql injection vulnerabilities at the language level, then that would be excellent. One less thing to worry about!

I don't have a unique grudge against memory safety issues. It's just one type of issue that we've spent a lot of time devising solutions to that don't just boil down to "be very careful" and I'm generally supportive of any solution to any problem like that, if the tradeoffs are acceptable.

But I agree with you that it's a great thing to have language and library support that make memory safety issues significantly less common and problematic (to be clear, I only just heard of Odin from this article, but I think it looks pretty awesome on initial glance), and that that gets to the level of solution that we have for sql injection in practice. I think there are somewhat better alternative solutions available in the case of memory safety, but they have different tradeoffs.

> What remains is a matter of what attracts you to programming: are you interested in having explicit control over a system to make it do what you want, or are you more interested in expressing some abstract ideas in an abstract mathematical virtual machine?

I'm mostly interested in the first thing, but I think these are both false choices. There is a language that provides that explicit control with more memory safety (rust) with a different trade off (language complexity), and most other languages are memory safe without being focused on expressing abstract ideas in an abstract mathematical virtual machine (go, java, python, etc. etc.).

This is the usual speach against memory safe languages adoption, while ignoring the nice security equation,

    Σ exploits = Σ memory_corruption + Σ logic_errors

Having Σ memory_corruption ==> 0 is of course much welcomed outcome, even if Σ logic_errors > 0.

Seems like you need Go then? You mention you want memory management, type safety and proper abstractions, hence the suggestion.

Of course, you also mentioned C's syntax as awkward for you, so you might feel the same with Go's syntax.

Go is not a fast language compared to C/C++/Rust. It’s more in line with Java as it’s also garbage collected.

That's fair. Go is not as fast and cannot manage resources as well as the languages you mentioned.

Go, despite using GC, is still a compiled language. From what I've seen (including various test results), Go is usually faster than Java.

There is also the issue of, what is fast enough? A lot of times, people don't really have the speed requirements they say or think they do. To include the code they have written, could be better optimized. It's often better to be more specific about what the requirements are, before making blanket statements about a language not being fast enough. Clearly, many people are fine with how fast Go is.

You state that as a blank and white fact, but there's nuance.

https://github.com/lotabout/skim/issues/317#issuecomment-652...

The wikipedia definition of systems programming seems to exclude it

https://en.wikipedia.org/wiki/Systems_programming

I'm not sure what you mean by "it" but nothing in that page precludes languages with runtimes from writing system software. I'd say Go can definitely claim to be a systems language.

I think Rust is what you need. It basically won’t compile if your code would have runtime errors or memory leaks, except for cases you need to explicitly handle. The linters in VSCode also make it pretty practical to use. It’s been a joy to program in since I started about a month ago.

> It basically won’t compile if your code would have runtime errors or memory leaks

While Rust does move a lot of errors from runtime to compile time, there are still a lot of ways to create runtime errors (which must obviously the case if your program handles any input at all).

Rust also does not stop you from creating memory leaks; there's even the `Box::leak()` method[1] that allows you to simply leak a heap allocation.

[1]: https://doc.rust-lang.org/std/boxed/struct.Box.html#method.l...

The difference is you need to explicitly handle cases that would cause runtime errors where you have to use .unwrap() or match selectors, which makes it safer in general. Since the values are wrapped in Result or Option enums. Code that doesn’t create these types isn’t possible to have runtime errors which reduces cognitive burden when coding.

And yes you can write non-idiomatic Rust that lets you leak memory but it doesn’t happen by accident like in C/C++.

> Code that doesn’t create these types isn’t possible to have runtime errors

Rust code can still panic and unwind at runtime. There's some ongoing work on supporting guaranteed-not-to-panic code for very specific uses (similar for guaranteed-not-to-leak, which is a related problem), but it's a long way off and will not be applicable to anything that must interact with the system in any way.

And the compoper error messages are fantastic for beginners.

What is "compoper"?

A typo of "compiler", I'm fairly certain.

Surely a compoper makes a new pope.

Exit code of white smoke signals success.

Be sure to run decompope first though.

Why call it the Great Schism when we can call it the decompoposition?