Comment by lbrandy | Hacker News Reader

Comment by lbrandy | Hacker News Reader

> "In the C programming language, you're regularly forced to deal with the painful, dangerous concepts of pointers and explicit memory allocation. "

> "I'd wager the majority of programmers alive today have never once worried about malloc(). I call this progress..."

Anger. Must... resist... anger.

I've come to believe that Jeff Atwood is on a personal mission sent by God himself to piss me off.

Just remember, Jeff: every time you fire up that pretty C# code of yours, and make all your little variables through your ORM, letting the garbage collector clean up the mess you've made and then proceed to pump out those cute little web pages through your cute little templating language, to be sent off to some other computer through some magic wire to some magic browser to be rendered... that thousands and thousands of programmers (who, it should be noted, are almost all still alive, today) worrying about "dangerous" and "painful" concepts like pointers have slaved to make your silly little ORM, your database, your templating language, your network stack, your web browser, your web server, and your operating system.

As I've said before, some of us are in the trenches worrying about all the "dangerous" stuff, like pointers. The primary difference between you and I, though, is that I actually know you exist. I don't believe that at some point in the mid 80's, a network stack and a file system (each created in the "before time") really really loved each other and out popped a web server.

I know its somewhat trendy to pick on Jeff Atwood, but I don't really know what you're getting angry about.

He never said that malloc and free are irrelevant. And manual memory management is dangerous and painful. Anyone who's written a modicum of C would agree with that.

That a majority of programmers today don't have to worry about it is, in fact, progress.

> He never said that malloc and free are irrelevant.

It would be a good point if it was true.

Jeff's basically said as much going as far as calling C an "obsolete" language and saying it is not worth learning. He said he doesn't know C and sees no reason to learn it. Is that not the definition of calling malloc and free irrelevant?

um, I'm talking about this particular post of his that you specifically quoted. I'm not talking about his general position on learning 'C' (which I also disagree with)

Anyone who's written a modicum of C would agree with that.

I've written tens of thousands of lines of C and maintain a codebase of over 50KLOC. I also am a somewhat less active maintainer on another project with almost 400KLOC. And I would strongly disagree with that, so you shouldn't make wide generalizations like "anyone."

I have devoted probably less than an hour of my time out of over a year of development on this codebase to memory management. I have never understood why people think it is so hard to call free() a couple times when your program shuts down.

At least in my experience, if you're finding memory management to be "hard", your program is structured in a bad way to begin with. Your program shouldn't be covered in mallocs and frees.

>I have devoted probably less than an hour of my time out of over a year of development on this codebase to memory management. I have never understood why people think it is so hard to call free() a couple times whenhttp://news.ycombinator.com/item?id=435267 your program shuts down.

I'm completely and utterly unable to comprehend how a code-base of the size you mentioned could have its memory managed by a couple of free() calls when your program shuts down. (And if you're program is shutting down, why would you even need to call free() at all? )

Are you using some kind of garbage collector tacked on top of C?

Edit: Can you elaborate on the nature of these programs? Because if you're building any sort of non-trivial data structures in your code at all, then you almost certainly need allocations and deallocations

We have to call free when the program shuts down because the program is available through a library interface, and so it can be called as a linked library in addition to being used as a standalone program. If it wasn't for that, we wouldn't even need free.

The program is structured so that based on the parameters it is given, we know exactly what needs to be allocated: how many video frames need to be malloced and so forth. This is all done in the initialization functions and after the program has started, it never needs to grab any memory ever again. This is not a case of "we're allocating the max we need and distributing it later"--it's a case of since we know how the program works, we know what exactly what it will need to do and how many frames it will do it to simultaneously.

There are only four things in the entire program that are malloced:

1. Video frame objects (big structs with tons of pointers to data that goes along with a video frame). A new_frame function exists to malloc and return a video frame. This makes it easy to check mallocs. We know exactly how many frames will be needed on startup. These frames make up the vast majority of memory usage of the program.

2. One data struct for each thread. Each thread has its own local struct that it passes around containing data currently being worked on.

3. A scratch buffer, per thread. This is a small malloced buffer used for a few calculations that require variable-size buffers, usually depending on video frame width (i.e. something known at initialization, but not known at compile-time).

4. Bitstream output buffers for each thread. In the extremely rare case that a bitstream exceeds the initial allocated value, this is realloced on demand.

Every single one of these can be free'd trivially at the end of runtime.

Well, of course you find malloc easy if you know your exact memory requirements when your code starts. If you are writing something like a long-running server process, you will be more able to appreciate garbage collection.

Interesting. Thanks for the details.

I'm guessing this is a non-interactive library\application which takes input at start-up, does work and then exits?

(because otherwise, I assume it would be impossible to reason about how much memory is required)

The program is x264 and it can be interactive to some extent--it has a "reconfig" function that allows you to reconfigure the parameters on the fly, though it has some limitations as to what you can adjust and what you cannot.

One potential purpose of this is a "speed control" feature where you adjust settings on the fly to ensure realtime operation, such as for a broadcast encoding server (my company does this).

If you are writing good, and highly reliable C, that is meant to run for long period of time, there will be no mallocs() and frees() inside the main loop because memory fragmentation can eventually crash your program. You will dynamically allocate and re-use memory outside of the main loop. You will use the stack for temporary creations. You can actually calculate the maximal stack usage, and ensure you have enough space, instead of 10 days into running that malloc() decides to return a null pointer because of fragmentation.

And one (very good) reason you call free() before your program terminates is so that you can use tools to figure out if there are any "real" memory leaks in your code. This way you don't have to wade through piles and piles of "oh that's not a 'real' memory leak because it'll get cleaned up when the program terminates".

I estimate that I've written about half a million lines of C code and I will have to strongly disagree with that.

Manual memory management just isn't that much of a big deal once you are experienced enough with C to organize your software into modules (usually individual C files with one corresponding header file). You just encapsulate the creation and destruction of elaborate data structures so that it's easy to verify that everything is cleaned up correctly.

These days I program mostly in Java and I spend a lot more time tracking down memory leaks than I ever did while I was programming in C.

I don't think he is saying pointers et al. aren't important. Consider the Whitehead quote: "Civilization advances by extending the number of important operations which we can perform without thinking about them."

Even so, I imagine that most garbage collection proponents still think people should learn about pointers and GC, just not use them (unless there is a good reason to).

Reply on news.ycombinator.com