Hacker News Reader

Comment by quelsolaar | Hacker News Reader

This is a tool issue and not a language issue. I think that if you do some thing in code that results in unintended behavior, then that's a bug that should be communicated to the developer during development.

A buffer over-run isn't a bug, its the symptom of a a bug in the arithmetic that computes how to access the buffer. The way to fix that is to communicate and address the bug during development, not to try to mitigate the symptom at run time.

I am a C programmer, and to me buffer over-runs are a rare class of bugs, and a class of bugs that are (at least on the heap, and often on the stack) trivial to find and fix with the tools i have.

View on HN

Why are they so trivial to detect for you while they bedevil the rest of the software industry? Is it possible that there are many such bugs hiding in your code that have not been found, because your code has not been subjected to the amount of scrutiny that projects like cURL, Windows and Chrome have?

I think i have fewer, bugs but this is a wild guess as I have no data supporting this even though i have significant number of users.

I dont know why others struggle with it, but I suspect:

-Most programmers are not used to thinking about memory the way you should when using C.

-Most developers are not aware of the tools available for debugging C.

You should write an operating system, clearly Torvalds and the Microsoft and Apple kernel developers have not reached your level yet.

I don't find either of the Kernels you mention to be particularly unsafe, despite being written in "unsafe" languages. Can you point to better kernels written in Rust?

Redox?

That being said, a kernel is a huge undertaking. There is nothing else with the scope of Linux. And substantial parts of writing a kernel involve dealing with architecture nastiness, and, in the absence of formal architecture models, no language will help much here. The Linux x86 low level code is a terrifying mess, and it’s also the best and most capable implementation I’m aware of. (I’m obviously biased.) A language like Rust would help only a tiny bit.

(Something like NMI handling on x86 is fundamentally memory unsafe. If you get nested NMIs, your stack gets clobbered. Thanks AMD. Linux has code, mostly in assembly, that detects and recovers. Good luck writing it in any memory safe language.)

I once skimmed seL4, and I would not use its x86 code for serious work in its current form.

As someone who helps write an operating system, some areas of kernel code can be less subject to the usual C memory safety issues than most user code. This isn’t because kernel programmers are special, but because the kind of code that gets written is different. I rarely find myself allocating memory at all in kernel code, and if I don’t allocate memory, I’m less likely to overrun a buffer.

In other areas of kernel code, buffers common, but the data is completely opaque. You won’t find kernel code parsing a buffer written to a filesystem because there is nothing to parse. The buffer is literally just a pointer and a length.

That being said, I would love something like Rust for lifetime management and thread safety and for the occasional data structure.

(Large sections of Linux are indeed quite scary from a memory safety perspective. Anything related to /proc or /sys comes to mind.)

> I am a C programmer, and to me buffer over-runs are a rare class of bugs, and a class of bugs that are (at least on the heap, and often on the stack) trivial to find and fix with the tools i have.

What does your testing setup look like? Does it include regular fuzzing and security audits? Buffer overruns are so common because normally your mental model of how the code should work assumes they don't happen, so you forget to test for their presence (or prove their absence, like Rust nudges you to).

I tend to have tests on a lot of the time (see below about my test lib). As someone who has worked with C for a long time, I think my mental model is all about direct memory access, so it probably comes more natural to me.

That sounds like the opposite of the mindset I have when writing C. Keeping buffer limits in mind all the time, and having them communicated around the code/function calls and respected, is one of the most basic things.

I don't do fuzzing, because I leave string parsing to tools like re2c. I tried it in the past with my mail message parser written in C/re2c and it was not very helpful.

There are also other low-level languages besides C and Rust

I use Pascal, and I never have any buffer overruns in my code. If there were any, Pascal's automatic bound checking on array indices would raise an exception

The only issue is, should I keep the bound checking in the release builds for maximum safety, or disable them for maximum performance?

> The way to fix that is to communicate and address the bug during development, not to try to mitigate the symptom at run time.

You'll write bugs and they'll make it past the development phase because bugs are inevitable. So now the choice is whether they're a crash or an rce.

How are they trivial? The issue with buffer overruns is that they're usually silent. If you have an array of pointers, and you overrun that array, you could end up assigning garbage to another pointer variable. Then when you deref that, if you're lucky it'll be outside the mapped memory addresses for that process and you crash. If you're unlucky, you read or write to a random location.

Basically the point is, it's very easy to end up in a situation where you know you have a bug but you're 2+ levels of indirection from the actual logic error. Those sorts of issues are always the biggest time sink.

I have a debug mode in my code base, where I wrap malloc, free, and realloc with debug versions that over allocate all allocations that are made, and put magic numbers before and after the memory blocks. These are periodically checked to make sure nothing has been overwritten that shouldn't be. You can find this code on my website, www.gamepipeline.org as part of the "forge" library.

The system tells me very precisely what has gone wrong with buffer over-runs, and it also make memory leaks and double frees trivial to find.

Visual studios debug mode does something similar with buffer overruns on the stack, but I tend not to put very many buffers on the stack so that much more rare as an issue.

If you want to go even more hardcore, windows has "gflags" that you can turn on to find overruns using memory protection.

LLVM also have the "Checked C" compiler that also finds these issues.

My own system is so good that, I haven't used gflags in a few years and I also haven't tested Checked C so I cant vouch for it, but there are loads of possibilities in this space that i hope to explore further in the future.

These sound good, and I've used things like asan and ubsan which do a lot of the same thing. The issue is not everyone's running in a traditional environment where you have the conveniences of things like... an OS.

I sometimes write code for micro controllers, and I tend to wrap all system calls so that I can run the same code on my PC just to get access to better tools. On small hardware platforms its common that you are forced to use C because of the limited resources, so the options are limited.

This testing-based approach is great at finding buffer overflows that occur within your test suite. But especially for code like curl that runs on untrusted data, you need a more adversarial mindset. Unless your tests cover every possible input, a clever attacker might always find input data that you didn't think to test.

Here's an example from one of your own projects, where code that works fine on normal input has a buffer overrun on malicious or corrupted data:

https://github.com/quelsolaar/HxA/issues/4

Reply on news.ycombinator.com