How to write better game libraries
handmade.network
handmade.network
Sure some of those libs may have some define you need to set in one of your .c files so the implementation will go there but then whenever you change that file it will result in a recompile of the third-party lib. So then you go and create a separate .c file just for the lib and we're back to why the lib author didn't do that in the first place?
No, I think what sqlite does is pretty neat. If you're doing development in the sqlite code-base you have lots of files (so you can manage it) but if you just want to use sqlite, you have the amalgamation which is one .c and one .h.
In languages that do have "real" macros, you don't see the same problem. (On the other hand, in most of them, you see a need to call out to a C compiler to generate FFI code, which is sometimes just as bad when you're using a PL with a managed runtime and your build-env didn't otherwise have any need for a C toolchain.)
Google, and therefore Chromium
https://google.github.io/styleguide/cppguide.html
Mozilla, and therefore Firefox
https://firefox-source-docs.mozilla.org/tools/lint/coding-st...
I'm pretty sure WebKit also doesn't use exceptions. It's not listed in their style guide but searching the repo I don't find any instances and the repo history shows turning them off.
From my perspective, banning exceptions is the exception (a ha ha); not the rule. I've worked on embedded software that (I expect) didn't support exceptions in the provided compiler, but I don't consider that an explicit ban on exceptions - they simply weren't possible.
> pleasant
Nice joke. Hour-long compilations, type-unsafe templates with unreadable errors, and circular references are considered pleasant now?
- templates are type safe. it is just the error messages are not helpful a lot of times. C++20's concepts should help
- circular references. that's a thing in C as well. as soon as you need interact with your libraries user to allocate/deallocate resources.
C++ has a large surface to interact with. Yes you can shot yourself in your foot. However, the more you know you can use the features to your advantage instead of shotting yourself in the foot. At the point you gain more than you lose.
'Modules via copy-and-paste' (= #include) is what I would hurl at C++ these days. Though they are working on that as well, I heard. Though https://vector-of-bool.github.io/2019/01/27/modules-doa.html doesn't sound hopeful.
That description only applies of you venture into the dark realm of template metaprogramming, and anyone who ventures into those lands is wise enough to understand that the tool is not the one to blame if one decides to abuse a feature to apply it in a way it was not (initially) designed to be used.
Concepts would presumably enforce some kind of thematic relationship between different overloads for the same group of functions and operators. (At least that's how we are doing it in Haskell with typeclasses. But we have a more forgiving syntax that allows us to make new operators.)
That's an ignorant statement both wrt generic programming (you know, the whole purpose of C++ templates) as well as the history of C++, in particularly how C++ template metaprogramming was discovered by accident after C++ templates were already implemented, widely used (see the STL), and standardized.
Templates are of course type unsafe, I'm not sure why you're lying while also admitting that they will only become type-safe in C++20.
Circular references deserves more comment: there is a difference between unmanaged and managed memory. C is unapologetically unmanaged, which cements its position on the embedded scene. But C++ has tried to be the in-between language: you have your raw pointers, and you have your refcount GC (“smart pointers"). And the thing that strikes me is the Cppistas have overwhelmingly chosen GC, but their language's GC is the worst kind, with circular references, performance costs (yes, smart pointers have a lot) and ultimately without memory safety (as raw pointers are still there). They've truly chosen tge worst of both worlds yet don't have the integrity to admit that the future is with managed, traced GC. C++ is like driving a car from the fifties and bragging about how comfortable and modern it is with that new internal combustion engine!
If your project takes hours to compile them you only have yourself to blame.
> type-unsafe templates
There is no such thing.
> with unreadable errors,
That's an implementation issue, not a language issue.
> and circular references
If you write buggy code then you only have yourself to blame.
When it comes to exceptions and rtti the sad reality is that about 50% of the C++ community doesn't use (for one reason or another), so if you do use them in a library it can have an impact.
Point in case for lack of portability, UE4 doesn't support exceptions unless you do a custom build.
Anyone providing a library written in pure C, better be serious about security and prove that they have taken all the required steps to handle memory corruption and UB exploits.
We really need more liability on software development.
Functions and opaque handles are often way simpler for the user.
That's important here. Some of my projects have something around 10 cross-compilation toolchains for various platforms. If a library is written in standard C99, adding it to the project is a breeze. Otherwise...
C is easier in this regard just because the language evolves so slowly that even though the build and library systems are bad, theyre a known bad that people can work around. No such luck with C++, where its bad and ever changing.
Why do you believe picking a specific build system is relevant wrt libraries, particularly a build system whose main selling point is build speed.
I usually recommend to write library-level code in the "subset of C99 which compiles both in C and C++ mode on GCC, clang and MSVC". This is basically a version of C that's somewhere C89 and C99 (basically a "C95").
Another quite valid option is to define a pure C-API first, but implement the "inner library code" in a simple C++ (just be careful with "modern C++", since this usually results in increased compilation time and binary size).
enum {
FPGAPARAM_FOO,
FPGAPARAM_BAR,
FPGAPARAM_BAZ,
NUM_FPGAPARAM_KINDS
};
struct FpgaparamInfo {
const char *name;
int address;
int args;
};
static struct FpgaparamInfo fpgaParamInfo[NUM_FPGAPARAM_KINDS] = {
[FPGAPARAM_FOO] = { "FOO", 0x1337, -1 },
[FPGAPARAM_BAR] = { "BAR", 0x666, -1 },
[FPGAPARAM_BAZ] = { "BAZ", 0x42, -1 },
};
The FpgaParamInfo is basically a mapping from a FPGAPARAM_??? value to additional information. We can make as many of these mappings as we want, and can define them where we want, which means it's all nicely modular. That's not really possible when modelling in a OOP fashion.I really like this style of programming since it's data first and it minimizes the amount of actual code. Designated initializers are important because the order in which the items in "fpgaParamInfo" are given doesn't matter. Without designated initializers, programming in this style would probably lead to many hard to find bugs when the enum is changed and not all associated data items are updated.
Nevertheless I received enough requests to make my C libraries 'C++ compliant' that I gave in :)
Might be worth touching on error callbacks/logging as an error handling strategy.
Sometimes an error is not recoverable in the sense that the calling code can't really do anything about it, but the library should attempt to make progress anyway instead of halting the entire program.
By allowing users to specify an error callback, this means they can log errors, capture stack traces, assert, or whatever.
This isn't that helpful for smaller libraries with smaller-scoped processes, but if it's something like a renderer or interactive audio lib, those often just need to be given a bunch of frame time to do work with the complex input you've prepped and fed to it, and trying to propagate error codes up out of that simulation step would both contort the inner code and not be as helpful as an error callback.
With an error callback you can assert, set breakpoints, or do whatever. But more importantly, by default you can have it just log so when you inevitably in a bug it doesn't prevent everyone else from getting work done while it keeps asserting until you fix your shit.
This will be a less common need than the other standard error reporting mechanisms, but is important to get right if your library has these complex internal preconditions that you want to make visible to the client when violated.
I've used some libraries that do some of these things, but I can't think of any that does all of them.
This could be very well flipped over. If I have the source available, I'd much rather deal with C++ bindings.
C++ ABIs are rather complex and much harder to interface with. The existence of concepts like virtual functions and exceptions significantly complicates the implementation of foreign language interfaces.
https://wiki.osdev.org/System_V_ABI https://itanium-cxx-abi.github.io/cxx-abi/abi.html
It does require some discipline, but C++ has many portable constructs that make life significantly easier when writing business logic.
> Always prefix your names to avoid name collisions
Solved by C++ namespaces.
> Use header guards instead of #pragma once.
#pragma once is supported by all mayor compilers, header guards can introduce bugs, also #pragma once builds measurably faster: https://github.com/electronicarts/EASTL/blob/3.15.00/include...
> Expose constants to the user using constexpr variables.
Modern C++ has strongly-typed scoped enums for such constants.
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-1193...
Other than that, I'm not surprised the file in question is C. Even ignoring the fact that it was you to get the link. And even ignoring that it's about a security problem...
Other than that, from a user perspective, software written in C is among the most reliable software I'm using. I'm looking at Linux, vim, xterm and so much of the infrastructure that I don't even know by name.
Some people value "security" more than they value the beauty of a nice, a maintainable, or a practical program. That's ok as far as I'm concerned...
https://msrc-blog.microsoft.com/2019/06/14/prevent-the-impac...
https://security.googleblog.com/2019/08/adopting-arm-memory-...
https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Pr...
Seatbelts and helmets are also a nuisance, maybe we should get rid of them, that is my perspective.
They are not random, only to those that don't care about improving the quality of our eco-system.
Software liability couldn't come soon enough.
I did, in my first comment.
C++ namespaces lead to much more readable code, compared to these prefixed names.
C++/11 scoped enums eliminate a class of bugs: when you have many different constants on the API surface, multiple functions accepting them, and erroneously use the constant of a wrong function. Example:
enum Lod { Low, High };
enum SomethingElse { Other };
void setLoD( enum Lod v );
void bug()
{
setLoD( Other );
}— Every language out there has a way to call into C
— If your code is slower than C, someone will rewrite it in C.
— If your library is written in C it means it can be used on any OS, console or mobile device and even on the web.
— Not everyone wants to use C++ (some prefer C).
— It is easier in general for a C++ user to use a C library than it is for a C user to use a C++ library.
— C++ is not as easy to write wrappers for in other languages.
— Unless you limit which C++ features you use (to the point where you are pretty much left with C) a lot of people won’t be able to use your library.
PS. If your original comment had been phrased the way you put it here I might have made the same comment, but I would not have downvoted it. Here, you’re at least providing some supporting evidence for your assertions which makes it a much more valuable contribution to the conversation.
When used correctly, C++ is not slower than C. Sometimes faster, a classic example is C qsort versus std::sort.
> If your library is written in C it means it can be used on any OS, console or mobile device and even on the web.
C++ is good in that regard. I know only 1 mainstream platform where C++ adds significant friction compared to C, that’s iOS, because their objective C is a superset of C. The rest of them (Windows, Linux including embedded, game consoles, android) support C++ just fine.
> Not everyone wants to use C++ (some prefer C).
Most people are OK with C++, especially in the context of game development.
> to the point where you are pretty much left with C
No, not with C. Namespaces and scoped enums are awesome.
Another thing, inside the implementation of the library, you can use whatever C++ language features you please, even the features that would be inappropriate when exposed at the API surface of the library. For example, MS implemented parts of their C runtime library with C++ classes, RAII, lambdas and templates, eliminating duplicated code for char/wchar_t routines. Obviously, you don’t need C++ to consume that library, just C is enough, but it’s implemented in modern C++. On my system, that source is in "C:\Program Files (x86)\Windows Kits\10\Source\10.0.18362.0\ucrt\stdio\output.cpp".
Apple’s compiler supports overlaying Objective-C features on top of C++ instead of C; it’s called Objective-C++.
Then I’ve got an e-mail from a developer who asked a few things how to back port my changes to C. I answered their questions, but I was curious why. They replied it’s because Objective C and iOS. Personally, I haven’t been developing for iOS for several years, but I don’t think people would do such things for lulz.
C ABI happens to be the mixed up with OS ABI, on OS written in C like UNIX clones, on mainframes, and other competing OSes that isn't the case, because they use other systems languages on their stack, or even some kind of bytecode based interoperability format.
However since the context here is game libraries, C ABI == OS ABI pretty much applies everywhere (except WebAssembly or Android JNI) and lets leave at there.
Since C++98, writing a library in C++, even if exposing it as extern "C", provides the following benefits for the quality of code implementation:
- less implicit conversions
- use of reference types instead of pointers for memory accesses we can be sure are never allowed to be null
- use of namespaces instead of Assembly style programming of having to come up with prefixes for code organization
- bounds checking for strings, vectors and other related data structures provided one uses the library types. They can even be left turned on for release mode, if the profiler shows there is no visible impact on hot paths
- use of RAII to manage library internal state and reduce leak occurrences
- ISO C++ working group is actually striving for reducing the amount of UB from its 200+ use cases, unlike ISO C group
- strong typed enums introduced in C++11 don't have implicit conversions and must map to their underlying types
- type safe compile time code execution, specially helpful in games, used for stuff like generating trigonometry tables
- templates as replacement for pre-processor magic that eventually goes subtly wrong when the #include order gets misplaced or too few parenthesis are used
If this still sounds absurd, well all major C compilers are now implemented in C++, Microsoft rewrote their C standard library in C++ with extern "C" entry points, Android NDK is actually implemented in a mix of C++ and Java (via JNI) also using extern "C" calls.
Even in 2019, the C ABI is the one universally agreed-upon cross-language ABI for languages compiled to native code.
However given that at least C++ does support ways to tame C, after all the whole purpose was for Bjarne never to repeat his Simula into BCPL rewrite experience ever again, it is up for security conscious to decide what legacy they want to leave, when the option is between both those languages.
On the other hand maybe C should be used to write Skynet, so that we stand a chance.
C is a perfectly fine language for tasks of all kinds. The fact that it expects programmers to take greater care when using it is not a reason enough to dismiss it as a general-purpose language. Not everyone needs (or wants) to ride a tricycle wearing knee pads and a helmet to get from A to B.
The reason is very simple, the billions of wasted money fixing security exploits caused by industry's adoption of C.
Morris worm is more than 30 years old, and the old ways can still be used to attack modern systems that people insist in writing using C.
https://msrc-blog.microsoft.com/2019/06/14/prevent-the-impac...
https://support.apple.com/en-us/HT210348
https://msrc-blog.microsoft.com/2019/07/16/a-proactive-appro...
https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Pr...
https://security.googleblog.com/2019/08/adopting-arm-memory-...
C++ defines don’t respect namespaces so you are pretty much screwed in anycase.
BTW, you can disable these windows.h macros by defining NOMINMAX. I usually do, because I prefer min/max from <algorithm>; in some edge cases std::min / std::max can be twice as fast because compiler guarantees to compute arguments exactly once.
I've never understood what's the practical difference between NS::foo() and NS_foo() with regards to preventing name collisions. Can someone enlighten me?
I know some disadvantages of the namespacing variant, though. There are now multiple names for the items defined in the namespace: the qualified one and the unqualified one. The latter is often not unique in practice, since the programmer relies on the qualified name for uniqueness. In effect, making simple text searches for identifiers is very unreliable. Note I do use IDEs, but I also code in vim, and I need to do simple text searches even when working in Visual Studio.
Additionally, there's no guarantee that there are no spaces around the scoper. I believe "NS :: foo" is just as valid, which makes me uneasy with regards to text search, as well.
Another issue I have:
$ cat test.cpp
namespace NS { int foo() { return 0; } };
$ g++ -c -o test.o test.cpp
$ nm test.o
0000000000000000 T _ZN2NS3fooEv # I hate my life
> Modern C++ has strongly-typed scoped enums for such constants.I haven't found those working for me. Apart from the namespacing issue described above, I have issues with explicit enum types. One issue is that I often need to put sentinel / "missing" values (typically the value is -1) where an enum value is expected. Even more often, I want to iterate over the values of an enum. C++'s enum "type safety" makes working like this really unergonomic.
The way I go about this is I don't even use names for my enum types, and I fully qualify the enumeration values.
enum {
FPGAPARAM_BLA,
FPGAPARAM_BLUB,
FPGAPARAM_FOO,
NUM_FPGAPARAM_KINDS
};
struct ASDF {
int fpgaparamKind; // obvious what kind of values are expected here...
};
for (int i = 0; i < NUM_FPGAPARAM_KINDS; i++) {
struct ASDF asdf;
asdf.fpgaParamKind = i;
do_asdf(&asdf);
}
In programming, the slightest mistakes, like putting a "-" instead of a "+", result in program bugs. These mistakes are much more likely to be made (and much harder to spot) than mistakes involving enum values from the wrong set. I won't let programming ergonomics be ruined in the name of "type safety".They both do the job. There’re 2 practical differences.
1. You can write `using namespace` inside functions or the whole .cpp files. This often makes the consuming code more readable.
2. Sometimes you want to replace implementations. With prefixes it gonna be massive changes likely to introduce new bugs. With namespaces, replace `using std::vector` with `using eastl::vector` and you’re done.
> Even more often, I want to iterate over the values of an enum.
I only need to do that rarely. When I do, I cast types like this:
enum struct eParamKind : uint8_t
{
Bla, Blub, Foo, valuesCount
};
for( uint8_t i = 0; i < (uint8_t)eParamKind::valuesCount; i++ )
{
const eParamKind pk = (eParamKind)i;
// Whatever
}
> I won't let programming ergonomics be ruined in the name of "type safety".I disagree on ergonomics. VS makes much easier to consume API with strongly typed enums: ePar<Ctrl+Space>::f<Enter>, to type eParamKind::Foo It’s similar with namespaces versus prefixes BTW, IDE will first auto-complete the namespace, then only list members of that namespace once you type the `::`
Update: another C++ feature relevant for game development is overloaded operators. Games often do non-trivial amount of math on small vectors, matrices and quaternions. Overloaded operators make sense for them.
> replace `using std::vector` with `using eastl::vector` and you’re done.
The pipe dream of reusability.. If I ever happen to be in a situation where that will work, I'll happily use a text replace to change my identifiers. Or just link a different library if it has the same names.
It doesn’t on my PC.
I’ve copy-pasted C enum from your example, when I type FPG<Ctrl+Space> there’s no way to auto-complete just the FPGAPARAM_ part, to be able to then press F to get FPGAPARAM_FOO. Using VC2017 here, with latest Visual Assist.
Which C++ IDE are you using?
> The pipe dream of reusability.
Did it more than once.
Here’s one open source project where I’ve replaced most parts of the C++ standard library with EASTL: https://github.com/Const-me/vis_avs_dx
Here’s my header-only C++ library which allows users to switch between 16-bytes/32-bytes wide SIMD by using different C++ namespace, either Intrinsics::Sse or Intrinsics::Avx: https://github.com/Const-me/IntelIntrinsics/
Regarding prefixes, I advice that you start by writing the library in C and then wrap it in C++ for a variety of reasons that you might want to consider. In C++ you should indeed always use namespace.
Header guards have the advantage over pragma once that they are standard and you can also use them to check if a library is included. I might remove that since maybe it's not that important and people might different views.
Regarding constants, I was referring to things such as numeric constants for which you would constexpr in C++. Maybe I can be more explicit there. Thanks for the feedback.
It’s very hard to write correct C code which does IO and supports multithreading. Take a look at Microsoft’s implementation of fprintf, copy-pasted from Windows 10 SDK: https://gist.github.com/Const-me/f1bb320969adde6c79694265ea6... They use RAII to set & revert the locale, and to lock RAM buffer to avoid corruption by another threads. They use C++ lambda for exception handling. They even use C++ template to avoid code duplication between printf and wprintf.
But these C++ shenanigans are not exposed to user, user calls their `printf` (possibly in a code built by C compiler) and it just works.
This is just a random unsubstantiated statement. There's nothing particularly "hard" about writing IO libraries that is language-specific. Multithreaded or not.
The hard part is making sure you release stuff every time you acquire stuff, exactly once. C++ RAII makes it almost trivially simple, but standard C has nothing comparable. When you only targeting gcc and clang can use __attribute__(cleanup) in C, it helps but still it’s more limited and more error prone compared to destructors.
Oy vey... you can't be serious. That's rudimentary basics of using any API.
There’re also other programs in wide use, which need to reliably work for hours, sometimes weeks. Some of them have huge amount of code they built from, written by many people over many years. Combine that with large enough APIs (some peripheral devices have hundreds of writeable registers of state; or D3D11 exposes huge amount of very complicated state, only limited by VRAM amount which is measured in gigabytes) and it’s very easy to make bugs in such programs.
Leaks of memory, handles, sockets, and many other resource types e.g. GPU ones. Deadlocks caused by locked mutexes, or threads which exit but forgot to release something they needed to release. Unwanted changes to global or thread state, both internal to the process and external (locales, formatting options, console colors, process and thread priorities, current directory, environment variables, CPU registers like FPU flags and interrupt masks, GPU render states) caused by some code changing stuff but not reverting the changes back. Unwanted state changes of custom peripheral devices, due to the same reason.
C++ RAII is not a silver bullet, but it does help a lot for all these things.
If you need to rely on RAII in order not to screw things up, then it's an issue with the coding style or the application design. That's what needs fixing. Not the language choice. You got it backwards.
The problem is only tangentially related to API semantics. The problem is mutating state. The state is not necessarily managed by an API, for instance CPU registers aren’t, you modify them directly. Same with other global state like I/O formatting options and locales, these things are just global variables.
The lowest common denominator for build instructions that works everywhere is literally "set your include paths like so and compile these sources".
> The lowest common denominator for build instructions that works everywhere is literally "set your include paths like so and compile these sources".
Yes, but that's quite inconvenient and error-prone ; a typo in a -D flag is very likely to not compile what you wanted.
Make is not a multi-megabyte program with a frack tonne of dependencies. Having your users to install it in order to build your project is far from being completely outrageous (unless for coughmicrosoftcough cults that consider the command-line is evil).
For usage from C code, there's cimgui, which is an automatically generated C-API wrapper:
If, on the other hand, you’re in the habit of shotgun debugging, where you make repeated changes to the problem code until it appears to work, you’re quite likely to leave behind various problems that will be hard to figure out.
Often, experience is the best teacher. If you’re not exposing your program to malicious users (aka. the public at large), the most serious issue you’re likely to run into is either a program crash or data corruption— nothing that will really harm your computer, but may cause you grief as you try to figure it out. In that process, though, you’ll learn an awful lot about how everything works. So, go write some programs for your own use and see how they crash and burn so that the next thing you make is more stable. Eventually you’ll start to intuitively spot trouble before it actually happens.
Writing Secure Code
https://www.amazon.com/Writing-Secure-Second-Developer-Pract...
Secure Programming Cookbook for C and C++
http://shop.oreilly.com/product/9780596003944.do
SEI CERT C Coding Standard
While UB quirks exist, they are WAY off the beaten path and it takes an effort to run into them. Doubly so if you are just starting with the language.
Just treat C as a thin convenient layer over the hardware that expects you to think and act responsibly in exchange for this nearly raw access.
Not to mention compilers make fun-times out of this by sometimes zeroing memory in debug and then not doing so for release builds (Hi MSVC!)..
The "thin layer over hardware" idea is a thing of the past as soon as optimizations come into the picture, and even then.
If you are interested in learning more about writing complex software in C consider checking out HandmadeHero.
Sadly there need to be more good resources on learning how to write good C and low level software. I am hoping my article can be a starting point for people who wish to learn about library design for example.
I don't have the exact numbers, but this seems debatable.
Six of the eight most popular programming languages (from https://www.tiobe.com/tiobe-index/) have C-like syntax. I think most people would agree.
Edit: I think that in this context, Reelin's comment is more appropriate: This is not about understanding, but about interoperability.
Why not write it in Rust, and provide C99 bindings? Using stock C for the actual library doesn't sound like a good idea to me, when today there are better alternatives.
And for sure avoid using Metal lock-in directly. Use Vulkan, which Apple should have supported from the beginning. For lock-in targets, there are translation options from Vulkan.
Also if people don't have a Rust toolchain setup, compiling the library and using it from source would be hard. In some cases integrating Rust in their toolchain could be hard.
Regarding metal and vulkan I will edit the article to mention Vulkan there too alongside metal. Thanks for your feedback.
As for toolchain, Rust can be set up basically anywhere llvm can, which is quite a lot. There are rare cases where llvm wasn't ported yet to, but I don't think they are enough to make C a compelling option in general, and I don't think any of them are gaming related.
If you are in such case - then sure, but otherwise, I'd still prefer Rust.
So if you do choose to make the implementation in another language, there are extra considerations that you have to take into account.