Brian's Rules for Writing Cross Platform 'C' Code (2008) | Hacker News Reader

Brian's Rules for Writing Cross Platform 'C' Code (2008)

ski-epic.com

87 pointsnkurz52 comments

I developed 9 ports and maintained 13 ports of RAD's Telemetry middleware for the game industry over a period of 3 years. My biggest takeaway from that experience is that build management, testing and release management are the hard parts once you get beyond the first few platforms. The only platforms that posed even minor problems in the traditional coding sense were ones in their infant stages (e.g. WiiU, Xbox One and PS4 years before their release) where hardware, APIs and even compilers are full of holes, and you have to deal with SDK updates (including deprecated APIs) and devkit refreshes. Automate everything you can and grit your teeth for the rest of it.

Purely on the coding side of things, I would caution against extremism when fighting code duplication between platforms. There is much to be said for having a single C file per platform that is self contained, even if it entails a certain level of duplication. The alternative very easily leads to an increasingly bifurcated decision tree that tries to account for similarities and differences between platforms (autoconf syndrome).

It is somewhat outdated, but yes, most of the points apply today.

Some things changed. Qt is pretty solid today. We use it as native Linux platform. We use native Windows and Mac front ends too.

Using Qt solves lots of issues very easy, like OpenGL drawing capabilities.

So in Windows and Mac we have two versions of our own software, one with native widgets, one with Qt ones. The Windows version using Qt is much faster and efficient, albeit a little ugly, because of OpenGL issues(instead of official DirectX) and font sizes or colors a little different from Windows and so on.

Reminded me of Subsurface maintainer's talk on Qt: https://www.youtube.com/watch?v=ON0A1dsQOV0

He found it was very good for cross platform work.

The obvious benefit for me is that Qt provides cross platform GUI creation which is, after all, what it's designed for. Building your own frameworks is great if you have the man power to do it once, but I think there is a strong case for using something like Qt even if it locks you into using its data types.

If you're just writing a utility of some sort then sure, you probably don't need the kind of overhead that Qt dumps on you. That said, I've written very performant code using Qt's threading and concurrency libraries and it t was surprisingly easy. Think industrial image processing that needs to be done at 150Hz on two camera streams simultaneously.

I think there are some good, sound principles here; I've dealt with some to one extent or another, and I couldn't find anything to disagree with. I worked on a port of a large Java application to C++ on a different platform, and we went back and forth for a while trying to choose between utf-8 and utf-16 as a standard character encoding. We ultimately settled on utf-8, even though the Java code had used utf-16 (because it's Java's native encoding); I'm not surprised to read here that utf-8 is the way to go. Utf-16 is at odds with the expectations of every C and C++ standard library function and the language itself, which would have been a source of annoyances that would never end.

It's very, very hard to make any generalization about programming, and so I suspect that in his confident pronouncement that cross platform should never add more than a small amount of time to a project, there is something he's ignoring. The application they sell is for backup, and so it would probably be close to ideal for making cross-platform. I think he underestimates how complex GUI code can get when dealing with requirements less trivial than a backup application.

He also basically ignored Macintosh and ObjectiveC. I don't recall what the state of Macintosh was at the time this piece was written, but I'm pretty sure ObjectiveC was the standard language on Mac then as it is now.

I think there are sage principles here, though.

ObjC and C go extremely well together, more so than C++ and C. You just have to make sure platform specific stuff that is in ObjC and GUI stuff is separate, which he advocates for any way, if I understood correctly.

I think you are right about separating the ObjC. C++ and ObjC don't play so well together, either, so if your primary language was C++, I think you'd need a plan around how to deal with ObjC.

Ever heard of Objective-C++? It is also supported by Objective-C compilers.

The problem is that C++ and ObjectiveC have overlapping purposes. The issue isn't that it's impossible to use them together. The issue is that you will have to have a strategy for dealing with conflicts before they occur.

That is true. Actually such considerations should be part of any mixed language project, specially in the presence of portable code.

gcc and clang can handle mixed Objective-C and C++ in a single source file (some call it Objective-C++). The only tricky part is linking since C++ and ObjC++ use a different symbol name convention than C/ObjC; but you can still link between them by using extern "C" declarations in the right places.

UTF-16 is a sad encoding and there's essentially no reason to ever use it except when interfacing with something that insists on it.

UCS-2 made a lot of sense back in the day when Unicode was supposed to be 16-bit and you'd just have one 16-bit "char" for each Unicode code point. Bliss! Then Unicode breached the BMP and this didn't work anymore. UTF-16 is just a way to let all that old code that assumes 16-bit Unicode to keep working. You get all the disadvantages of UTF-8 (no 1:1 correspondence between code units and code points) and none of the advantages (widespread compatibility with non-Unicode-aware tools, mostly-reliable discrimination between actual text and random binary data that might look like text).

One can kind of make a case for UTF-32 in some cases, but UTF-16 is almost never the right choice unless there is literally no other.

btw, the wchar_t typedef is 16 bits for MSVC, but 32 bits for gcc (even for MinG on Windows!). UTF-16 is the worst of all worlds.

Original author here....

This is why we do not use wchar_t as a cross platform type. We like "char *" with NULL terminated Utf8 strings, which works really well on all platforms. Unfortunately on Windows, some of the core API still take Utf16 (2 bytes per char). Since it is a Windows call it must be in an #ifdef anyway, and so IMMEDIATELY BEFORE calling into the Windows call at the last possible moment we convert the string from Utf8 to Utf16. It sounds like a pain, but after you have wrapped all your OS specific calls (like File I/O) you're calling the higher level anyway, and it was completely worth it.

That helps put some things in perspective. Unicode is not a well-understood topic among programmers, I've noticed (at least in the US), and for something so important it's surprisingly difficult to find information. The distinction between UCS-2 and UTF-16 must have made sense at one time, but it makes no sense anymore, and it's not necessarily easy to track down the explanation for why there are two 16-bit formats. Thanks.

The whole thing is pretty much incomprehensible without understanding the history. It's a bit crazy how much historical baggage Unicode has managed to accumulate in just a couple of decades (and most of it from just the first few years).

It sounds like you have it figured out, but just in case: UCS-2 is the original pure 16-bit encoding from the days when nobody could conceive of needing more than 65536 characters for anything. UTF-16 is the hacked up "oops, it turns out there are a lot more Chinese characters than we realized" version that tries to maintain some compatibility with software built for UCS-2.

Given that there was only a five-year period between the first release of Unicode and the introduction of UTF-16, it's kind of amazing how pervasive the 16-bit encoding got. It's thoroughly baked into Java, where "char" was supposed to be a unicode character, but ended up being a unicode character or a piece of a UTF-16 surrogate pair. Windows APIs and Cocoa's NSString went through the same process. JavaScript too, and the spec still allows implementations to only support UCS-2.

The c and c++ apis seem to make things worse. The right solution for charset conversion seems to be iconv.

Author here (late to the discussion):

> He also basically ignored Macintosh and ObjectiveC

At Backblaze (my company) we write all lower level common libraries in 'C' and C++ so like a cross platform HashTable, and file I/O functions. It turns out ObjectiveC on the Macintosh is a PERFECT SUPERSET of 'C' and C++ so luckily all the Objective C code can call into our OpenSSL for encryption and can call into our base libraries for File I/O.

#3 is bad (or at least incomplete) advice when you consider portability to 64-bit. What it should say is to use the C type that correctly represents the data.

Translating DWORD to `unsigned long` is not correct! unsigned long is 32 bits on x86_64 Linux, but 64 bits on x86_64 Windows. Using long is essentially never correct for cross-platform code.

The right thing to do depends on what the value represents. If you want a 32 bit unsigned integer, use uint32_t. If you want something representing a pointer or size, use the corresponding type (uintptr_t, size_t). This doesn't just make porting easier, it adds useful hints for anyone reading your code.

It's actually vice versa. long is 64bit on amd64 linux and 32bit on amd64 windows.

really, the issue is, if you want a fixed bitwidth data type, use the types in unistd.h - uint64_t or uint32_t.

but fixed width data types are super freaking slow if your doing intensive processing.

um.. what do you mean, exactly? all types in C are fixed bitwidth. but the uintx_t types make it obvious what what programmer wants.

I'm not buying it. I want to see your timings and proof. First of all, this only applies to a very few internal loops that don't touch disk, so you can use the "cross platform rule" for 99.999 percent of code. But if you are down to that last inner loop that runs your 3D video game, I'll admit you will need to have platform dependent code paths. But this describes an infinitesimally tiny subset of the code in the world. The rest of the time, write easy to understand cross platform less buggy code!

The last part of that entry makes me wonder if the guy even understands what's going on:

"The most important place to apply this rule is in cross platform library code. You can't even call into a function that takes a DWORD argument on a Macintosh, so the argument should be passed in as an unsigned long."

If DWORD is declared, then your Mac code can use it fine. If it's not, then it won't even compile. At no point can you ever reach a point where "you can't even call into a function that takes a DWORD argument".

Author here (late to the discussion, nobody told me my 2008 article was posted on hackernews...)

> If DWORD is declared, then your Mac code can use it fine. > If it's not, then it won't even compile.

Correct -> as a WINDOWS programmer you think it's fine because it compiles just fine. The point is it then won't compile on the Mac, or on Linux, or any other platform. The whole point is if you want a 32 bit integer, use a cross platform 32 bit integer from the beginning. When a Macintosh programmer goes to compile, it will compile. ONLY IF you need to call a Windows specific call, then type cast or convert into the Windows specific type, and then you do it inside an #ifdef. But most all of the time Windows programmers seem to love their DWORDS and they use them all over the place, even for public APIs that some other poor Macintosh programmer has to replace later. Why not start with the cross platform version? What does a DWORD get you that "int32_t" does not? (NOTE: we make sure "int32_t" is defined and a 32 bit integer on all of our platforms, then everybody uses the same thing.)

Hey, brianwski the author here, unfortunately I didn't realize my (2008) web page was on Hacker News until somebody told me. :-) Anyway, in my old age I feel that the "ambiguous size types" in 'C' are outright evil. Size is REALLY important, it's not like you can just declare an "int" and know in your heart it is "big enough", you need to absolutely be sure that the values you put in it will fit. So I like using explicitly sized integers in all situations, why not? The ONLY argument I can fathom for "let the compiler decide on this integer's size" is performance, and frankly if that is your performance problem (and not disk I/O) then you need to be focusing on that problem WAAAAAAY more than just letting the compiler decide how big your integer is.

'But on BOTH PLATFORMS they call the same line of code -> BzUiUtil::PauseBackup();'

That's some funky C code you got there Brian.

Pretend he wrote bzuiutil_pause_backup(); and the rest of the advice holds up.

Original author here. Exactly, thank you! I thought everybody knew 'C' and C++ both, and specifically that:

1) 'C' is the most basic, it only has functions. A function call looks like "bzuiutil_pause_backup();"

2) 'C++' is a super set of 'C' and if you want to group several static functions together, you can put several static functions in one class. This defines scope, and also allows programmers to find where the function is written, in this case "BzUiUtil::PauseBackup()" means that there is a 'C++' class called BzUiUtil and it has a static function with EXACTLY the same contents as the 'C' version has inside "bzuiutil_pause_backup();"

At Backblaze, we tend to void writing 'C' code in favor of 'C' code in static C++ wrappers as described in point "2" above. There isn't any downside as long as all of your compilers on all platforms are C++ compilers anyway. I honestly didn't know a pure 'C' compiler still exists. Specifically, Microsoft Visual Studio does both seamlessly. On the Macintosh Xcode does both seamlessly. On Debian Linux (the flavor we use most and is also the slowest to adopt anything) the g++ compiler ships for free on every linux box that does both seamlessly.

The code examples are all C++, but from what I understand the rules apply to C as well.

I thought that, too, when I saw that example, clearly in C++, after about 100 mentions of 'C'.

I've encountered programmers that habitually refer to C++ and ObjectiveC as C. I suspect he has that habit.

Original author here:

No, we use 'C' and 'C++' on Windows PCs, and we use 'C' and 'C++' and 'Objective C' on the Macintosh. Objective C can call into 'C' libraries just fine on the Macintosh. Objective C can also call into C++ just fine on the Macintosh.

The distinctions are very important to us. :-) Objective C cannot compile on Linux or Windows, but 'C' and 'C++' can compile on all three platforms (Macintosh, Linux, and Windows) using the default development environments on all three platforms (GCC on Linux, Xcode on Macintosh, Visual Studio on Windows).

the stuff he talks about applies to C and C++ - i think we can forgive that faux pas :)

#4 is not completely true, as the "standard" ifdefs are defined by the compiler, not the platform. Thus, they may differ when building on Windows using MSVC and building on Windows usign MinGW.

Kind of falls out of his using one compiler per platform. There tend to be few differences though unless you use something less standard (like ARM's C compiler).

Original author here. Yeah, my goal was cross platform APPLICATIONS where the first step is choosing your tools. I suppose if you are writing a library like OpenSSL that you want to compile with every last flavor of every last compiler the problem gets worse.

I stand by my guideline for applications.

Another way to think about it is if you want, use all your own custom compiler flags, then if you can manage to get your compiler flags set by a creative combination of BUILT IN compiler flags, that way your code will compile right out of the box correctly on all platforms. What I dislike is that out of the box the code compiles on zero platforms. Instead, you run a "configurator" program that sets all the cross platform flags correctly. If at all possible, write your code so that it self configures correctly with zero changes on all compilers and all platforms. Please notice the "if at all possible". I think if you are relatively careful and just use basic types this can be done easily. Is it really that amazingly hard to find a native "int32" and "int64" type using ifdefs on every platform/compiler?

Worse come to worse, you include a header file that errors out if anything but one and only one of the platform flags is set, with an error message of something along the lines of "you are using a compiler that does not automatically set the correct flag for this operating system. Please set exactly one of the following flags: ..."

_WIN32 used to be defined for Xbox - it probably still is for Xbox360 and XboxOne too... - as well as Win32/Win64. All very different systems. Cue hilarity!

You really, truly are best off making your own defines. (If you absolutely don't want to specify them in the build setup, you can usually autodetect many of your own defines by checking for _WIN32 and _M_IX86 and _MSC_VER and __GNUG__ and so on. Then there's at least only one place to get the logic wrong.)

This is a remarkably old article, but its truths mostly hold today. I also think that cross-platform compatibility is a day-0 affair, you need to get to that mentality from your first line of code. Adapting later will always be harder and more error-prone.

This post is also on BackBlaze's blog - http://blog.backblaze.com/2008/12/15/10-rules-for-how-to-wri...

When I read "Platform" I thought the article would be about CPU architectures. Still a good read though.

I'm not sure if this is standard terminology but I like to refer to multiple operating systems as cross-platform and multiple CPU architectures sets as cross-architecture. It looks like the author is doing something similar.

After all the problems you encounter porting from Unix to Windows are different from what you hit porting across architectures. With cross-platform you're worrying over file IO functions or the lack of fork equivalent. With cross-architecture you must ensure the code is not abusing ints for storing pointers or assuming sizes of primitives.

> With cross-architecture you must ensure the code is not abusing ints for storing pointers or assuming sizes of primitives.

You should ensure that anyway, even when you're working on a single architecture. Those are very bad habits to form.

I'm just starting C, what should I be using instead?

Don't make any assumptions about the architecture that your program will run on. If you have to go into variables with a different type than the one with which they were declared make sure you do so in a portable way.

So, for instance, if you want the lower 8 bits of an int don't declare a pointer to a character and then lift out the byte, but do (value & 0xff) instead.

That way if you move your program to a big endian machine it will still work. Storing pointers in ints assumes that pointers and ints are the same size, which is definitely not always the case.

So store pointers in a space sizeof(whatevertype*) rather than sizeof(int).

If you need to store a pointer in a integer variable, first stop and look very hard at what you're doing and think about whether you really need this. It's usually a really, really, really bad idea.

If you have decided that you do really need it, then use the standard C type for "integer type large enough to hold a pointer". Yes, there is one. Actually two. They're called intptr_t and uintptr_t depending on whether you want signed or unsigned.

Similarly, if you subtract two pointers and need to store the answer (this is less of a cause for reflection, but still not too common) then don't toss it into an int and hope that it works. Use the language-provided type for "difference of two pointers", ptrdiff_t.

If you have a value that needs 32 bits of storage, don't throw it in an int and say, hey, int is probably at least 32 bits. Use the language-provided types int32_t and uint32_t.

Understand what types are provided (Wikipedia has a decent summary at http://en.wikipedia.org/wiki/C_data_types), what they mean, and what the guarantees are. Don't fall into the trap of "it works for me, here, now, so it's fine". If you need a particular guarantee (such as the ability to hold a certain range of numbers), ensure that the type you're using provides it.

I recently published an open source win/mac wrapper library in C. It is not well tested, and only covers the specific subset of functionality I needed for my current project. It may still be useful for others learning which pieces of each os handle which common tasks - things like CoreGraphics vs GDI, file system quirks, how to play audio, etc.

https://github.com/tylerneylon/oswrap

The praise for OpenSSL („I can't recommend OpenSSL highly enough, […] we just could not have done better at Backblaze.“) is kinda funny now after Heartbleed and the massive code cleanup of the LibreSSL people …

Original author here -> technically Heartbleed was not a bug in the OpenSSL low level library, it was specifically an HTTPS bug (it was a bug at a higher level than the library I'm saying I liked then and still like). Specifically it was a bug in the "Heartbeat Extension for the Transport Layer Security" protocol (the HTTPS level is the TLS layer, the underlying encryption functions did not have this particular bug).

All that is a technicality, software has bugs. Heartbleed was a particularly bad security bug. This doesn't mean you throw out the baby with the bath water. As a society we all use HTTPS and it had a bug -> so we fix this bug and move on as best as we can.

As far as I know, something like 99 percent of routers and OS systems such as Linux, Macintosh, and Windows use the OpenSSL library and it has very little to do with Heartbleed. On the other hand, it is fairly important your system vendor stays on top of security patches and applies them.

OpenSSL is software, and therefore has bugs. But the OpenSSL encryption layer did not contain the Heartbleed bug. Or put differently, the OpenSSL encryption libraries that encrypt AES 128 and AES 256 had nothing to do with Heartbleed, and those are the parts we continue to use at our company completely for free, without paying any royalties to anybody, and they are a lot more secure than anything we could have hand-rolled ourselves even if we took years to do it.

Simply compiling and running on many different platforms (Windows, Linux, 32/64 bit, ARM, ...) has helped me find a number of truly weird issues that would have surfaced later. Compiler warnings, diverging stdlibs, alignment...

I'm a strong believer in this approach.

Original author here. I COMPLETELY agree. Anybody who defends their little one-platform-world as "special" should run their code through several compilers. It's like running a spell checker on your essay for the first time - the spelling errors jump out at you. Different compilers seem to be amazingly different in catching really obvious bugs. When I get something fully working on Windows and Linux, then the Macintosh XCode points out a problem I think to myself "WHAT THE HECK, HOW DID WINDOWS ALLOW THIS?!" It is worth doing cross platform development simply to raise the quality of the software you are working on, even if you never ship on the other platforms.

Reply on news.ycombinator.com