Hacker News Reader

Comment by giomasce | Hacker News Reader

The problem is not even just safety. You can write safe C, if you are careful (and you need to be careful also for C++, smart pointers do not just magically make everything right). The point is that OP makes the point that C requires to write less code, and this doesn't even seem true: you have to remember, each time some non-trivial object goes out of scope, to call its destructor/deallocator, which results in a lot of code (which can at times hard to read, especially if you have not completely linear control flow). Looking at just the declaration is a small part of the issue.

View on HN

It depends. One approach to avoiding tricky memory management issues is to avoid memory management altogether. In the kind of applications where C is a good choice, you might consider strategies like allocating all the memory you need to work with besides small structs which fit on the stack when the program starts and then never deallocate. If you're worried about performance this is often a very good strategy.

We are talking about a renderer. Renderers definitely need to deallocate or otherwise manage memory :)

Why would a renderer need to deallocate memory? And when would a renderer need to deallocate memory?

Renderers need to deallocate memory all the time. For example, they often need to store per-frame data that gets dropped when the frame is over, debug strings, track temporary access to texture/buffer data, manage command buffers, etc. They also have to track resources for lots of data types external to the program (such as GPU data) which often forces them to work with external allocators, as well as having to synchronize stuff like resource destruction using GPU-CPU fences. Moreover, most resources are either immutable or can be accessed optimally when they are immutable, so you often end up benefiting significantly from being able to destroy and reclaim the memory of stuff you're no longer using rather than trying to update in place.

Even if you try to only allocate one buffer (for example) to split up and use for all your logical vertex buffers, you still have to manage memory within that buffer, which ends up leading to pretty much the same kinds of complex reasoning you need for ordinary dynamic allocation. Failure to do this properly results in resource leaks on both the GPU and CPU side which can be really nasty.

Of course, this all depends on the complexity of your renderer. For simple usecases you might be able to get away with no deallocation until exit. But they'd have to be really simple.

> For example, they often need to store per-frame data that gets dropped when the frame is over, debug strings, track temporary access to texture/buffer data, command buffers, etc.

Exactly. You can just bulk allocate all the memory for the frame at the beginning and drop the entire thing after the frame is finished. This is a very easy case for manual memory management where you have one allocation and one deallocation per frame.

Or you can do one better, and keep an arena for each framebuffer, and then just recycle them and never deallocate at all. If you really need some level of dynamism in terms of memory usage, you can just double the size of the arena every time you fill it, and you still don't need to deallocate, since chances are you will need that space again for some frame in the future.

I love arena allocation (which, incidentally, does entail dynamic resource destruction if you use it per-frame, and which is pretty easy to get wrong if you're just storing raw pointers everywhere--so good bye nice use after free properties), but you absolutely cannot just use arena allocation for everything. GPU resources in particular need to wait on fences and therefore can't have a neatly defined lifetime like what you're proposing unless you block your whole program waiting for these fences, while other GPU resources are (like I said) immutable and need to live for multiple frames. Where exactly do you store this data? How do you access it? How do you free it when it's done? How do you perform slow tasks like recomputing pipelines and shaders in the background without being able to dynamically track the lifetimes of the old/new backing GPU allocations? More generally, how do you track resources other than the framebuffer/surface, which are usually managed very dynamically and often appear and disappear between frames?

I think maybe it would be helpful if you would go look at the actual implementation of a nontrivial renderer and try to find one that doesn't allocate. Because the strategies you're describing don't match my experience working on renderers, at all.

I am speaking from experience, I have written a lot of renderers.

> GPU resources in particular need to wait on fences and therefore can't have a neatly defined lifetime like what you're proposing unless you block your whole program waiting for these fences, while other GPU resources are (like I said) immutable and need to live for multiple frames.

I don't quite understand why fences are a problem here. As long as you know what frame a resource corresponds to, you just to have to guarantee that the entire frame is finished before reusing. You don't need to know what's happening at a granular detail within the frame.

As far as static resources, that's easy: another arena for static resources which grows as needed and is shared between frames. Most of that data is living on the GPU anyway, so you just need to keep a list of the resource handles somewhere, and you're going to want a budget for it anyway, so there's no reason not to make it of finite size.

> More generally, how do you track resources other than the framebuffer, which are usually managed very dynamically and often appear and disappear between frames?

See the whole issue is trying to solve the issue "more generally". In many cases, you can know exactly the resources you will need (or at least the upper limit) when you start rendering a given scene. If you need to do any memory management at all, you can do it at neatly defined boundaries when you load/unload a scene.

The only time when you need some kind of super dynamic renderer is when you are talking about an open world or something where you are streaming assets.

Most of the serious renderers are written in C++ using custom allocators, which are just some kind of arena allocator under the hood anyway.

Sorry, but if you think you can just get away with never deallocating any static GPU resources (as you're implying by "just use an arena"!), one of two things is going on:

(1) You are manually managing an allocator on top of the GPU resources in the arena (reintroducing the usual use after free problems). (2) You have written a highly specialized renderer that cannot be reused across programs, as it requires all GPU resources used in any scene to fit into memory at once (and for their exact quantity to be known).

Once you acknowledge that asking for dynamic GPU resource allocation, the rest of what I asked about follows; for instance, you can't reuse whatever portion of the already-allocated resources were freed at the end of the frame unless you can track used resources across frames; you can then either trace the open command buffers each frame or whatever (tracing all the live resources) or use something like reference counting and free once the fence is done (tracing all the dead resources). Hopefully you will acknowledge neither of these is the same as arena allocation.

"dynamic GPU resource application isn't needed for a useful renderer" certainly sounds good, but again does not jive with my experience with renderers (admittedly, most of this involves rendering an open world with streaming assets, but most examples I've seen that aren't super tightly constrained examples require resource reuse of some sort).

I think you are not thinking through all the possibilities here, but you would be surprised how far you can get with this approach

I would indeed be surprised. Like I said, point me to a production renderer used by more than one application that actually gets away with pure arena allocation except at blocking scene/frame boundaries. Until then, you're just asserting you can work within these restrictions without providing any examples. Considering that even stuff like hashmaps generally require allocations, it strikes me as fairly unlikely, but I could be convinced with code.

> Renderers need to deallocate memory all the time. For example, they often need to store per-frame data that gets dropped when the frame is over

Zero the buff and reuse this same frame as the new frame.

I don't believe that it is possible to write safe C (or C++), even if you are both very careful and also among the most skilled C developers out there. Every sizable project in C has had critical vulnerabilities. It is not possible to train engineers on a team of any meaningful size to consistently write bug free code and catch bugs in code review. Sanitizers, fuzzing, and static analysis all help but are insufficient in the face of the utterly impossible task of writing safe C programs, let alone evolving safe C programs.

Look at all the very smart people that tried and completely failed to write libraries that do such basic things as copying strings.

> You can write safe C, if you are careful

In practice this pretty much cannot be done unless you use a formal verification framework, which almost no one does. Even the most well-resourced projects written in C tend to have trouble with low-level bugs. Same goes for C++. Both the Linux kernel and Chromium have plenty of these issues.

My comment seems to have sparked a lot of reaction about safety, but that wasn't the main point. The main point (and the one OP talks about) was about conciseness, and I can't really see how C can be considered a concise language.

Agreed. C is more concise than assembly, but it's highly verbose compared to most modern languages. C++ can sometimes be a good deal more concise.

> You can write safe C, if you are careful

You can also live forever, if you have right combination of genetics and environment.

Like when someone says "Lisp is homoiconic," and that automagically creates a thread of people tripping over themselves to make the same points they do every time "Lisp" and "homoiconic" appear in the same sentence.

There should be a list of these language snowball avalanches somewhere. They work every time.

I assume Lisp homoiconicity is closer to fact than "You can write safe code in C".

I mean it is possible. In the same way all molecules of air could bunch up in one corner of the room suffocating you. It's a possibility.

It is possible to write safe-C, but C is far more error prone than C++. C has more implicit type conversions than C++, which may results in bugs and undefined behaviours. C lacks RAII (Resource-Acquisition Is Initialization) that is useful for memory and resource management. C will actually require more code than C++ since, the C standard library lacks generic data structures such as vectors, hash maps, linked lists and so on. The implementation of those data structures requires lots of preprocessor macro hacks.

It is possible to write safe C, only if one uses static analysis tools and undefined behaviour sanitizers. In the case presented by the article, as it is related to game, safety does not matter much, unlike device drivers, operating systems or embedded systems where C bugs can introduce security vulnerabilities.

Regarding the compile-time complaint, it is possible to reduce the compile-time by using forward declarations; forward template declarations; template forced instantiation and isolating large parts of a project in a static library using CMake.

> C is far more error prone than C++

I don't know about that... C++ (both the language and the library) is orders of magnitude more complex, and the opportunities to make mistakes have grown almost proportionally. (Two characteristic examples recently discussed here on HN: auto references and iterator invalidation.)

But C lacks even strings, lots of C bugs and vulnerabilities are related to memory management, memory ownership and string handling. Even the C subset of C++ is better than C since it at least has more explicit type conversions that forces the developer to state his or her intent. One example of the C string problem is the strcpy(buffer, char* string) that copies a string to a buffer. If an external actor discovers how to manipulate the string size, he or she can take advantage of this buffer overflow vulnerability and even execute arbitrary code remotely if it is used in a server. If the application with this problem is a file, one create a specially crafted file to take advantage of this design flaw.

However using C in the case of the original poster does not matter much as the application is game-related not subject to untrusted input.

strcpy - is a standard library issue, not language one. Most C projects creates their own "string handling" routines.

It is pity there is no alternative "standard library" with safer data struct and operations.

There are some attempts, for example, relatively wildly known klib: https://github.com/attractivechaos/klib

There are two problems here, one being complexity, the other being abstraction power. The two correlates. C is an easy language without much complexity, but with laughably little abstraction power. Text-based macros are the worst thing ever, and other than that, the language can’t even express reusable data structures, only with convention.

C++ on the other hand has good expressivity that can better deal with complexity of programs (eg. just simply having string, vector, etc) at the expense of some added language complexity. But unfortunately most of that is due to backward compatibility.

My opinion is that program complexity is inherent for anything interesting, so the C++ tradeoff is worthwhile. Also, by sticking to a good subset of C++, one can minimize the “bad” complexity of the language, imho.

Reply on news.ycombinator.com