What you should do instead is write all your code so it is little-endian only, as the only relevant big-endian architecture is s390x, and if someone wants to run your code on s390x, they can afford a support contract.
The vast majority of modern network protocols use little endian byte ordering. Most Linux filesystems use little endian for their on-disk binary representations.
There is absolutely no good reason for networking protocols to be defined to use big endian. It's an antiquated arbitrary idea: just do what makes sense.
Use these functions to avoid ifdef noise: https://man7.org/linux/man-pages/man3/endian.3.html
You should actually use format-swapping loads/stores (i.e deserialization/serialization).
This is because your computer can not compute on values of non-native endianness. As such, the value is logically converted back and forth on every operation. Of course, a competent optimizer can elide these conversions, but such actions fundamentally lack machine sympathy.
The better model is viewing the endianness as a serialization format and converting at the boundaries of your compute engine. This ensures you only need to care about endianness when serializing and deserializing wire formats and that you have no accidental mixing of formats in your internals; everything has been parsed to native before any computation occurs.
Essentially, non-native endianness should only exist in memory and preferably only memory filled in by the outside world before being parsed.
These days it's bi, actually :) Although I don't see any CPU designer actually implementing that feature, except maybe MIPS (who have stopped working on their own ISA, and now want all their locked-in customers to switch to RISC-V without worrying about endianness bugs)
ARM works the same way. And SPARC is the opposite, instructions are always big-endian, but data can be switched to little-endian.
This argument is pretty silly: visualizations can always be changed. For some time I have been thinking that hexdumps on little-endian systems ought to be written right-to-left: in fact, when I once decided to include such a right-to-left dumper in my own software, it took me very little time for me to get used to, and I immediately started regretting I don't have it available everywhere.
> it’s easier to draw pictures as a matrix of bytes that are transmitted from left to right and top to bottom.
There are many reasons for big endian... but that is not one of them :)
> But processors are all over the map
That's not true anymore, big endian is dead. Upstream Linux is refusing to support big endian riscv at all, and is making serious noises about ripping out the existing big endian aarch64 support because the companies that ship the hardware that needs it don't work upstream.
If the data stream encodes values with byte order B, then the algorithm to decode the value on computer with byte order C should be about B, not about the relationship between B and C.
The point (for me) is not that your code runs on a s390, it is that you abstract your personal local implementation details from the data interchange formats. And unfortunately almost all of the processors are little, and many of the popular and unavoidable externalization are big...
[0] https://commandcenter.blogspot.com/2012/04/byte-order-fallac... [1] https://github.com/apple/darwin-xnu/blob/main/EXTERNAL_HEADE...
Their x86 changeover moved the CPU's to little-endian and Aarch64 continues solidifies that tradition.
Same with Java, there's probably a strong influence from SPARC's and with PPC, 68k and SPARC being relevant back in the 90s it wasn't a bold choice.
But all of this is more or less legacy at this point, I have little reason to believe that the types of code I write will ever end up on a s390 or any other big-endian platform unless something truly revolutionizes the computing landscape since x86, aarch64, risc-v and so on run little now.
Most CPUs (including x86-64) have variants of the load and store instructions that reverse the byte order (e.g. MOVBE in x86-64). The remaining CPUs have byte reversal instructions for registers, so a reversed byte order load or store can be simulated by a sequence of 2 instructions.
So the little-endian types and the big-endian data types must be handled identically by a compiler, except that the load and store instructions use different encodings.
The structures used in a data-exchange format must be declared with the correct types and that should take care of everything.
Any decent programming language must provide means for the user to define such data types, when they are not provided by the base language.
The traditional UNIX conversion functions are the wrong way to handle endianness differences. An optimizing compiler must be able to recognize them as special cases in order to be able to optimize them away from the machine code.
A program that is written using only data types with known endianness can be compiled for either little-endian targets or big-endian targets and it will work identically.
All the problems that have ever existed in handling endianness have been caused by programming languages where the endianness of the base data types was left undefined, for fear that recompiling a program for a target of different endianness could result in a slower program.
This fear is obsolete today.
Just treat the data on disk / on the wire as if it were in some encoded format. Parse on load. Encode back out to the expected format when you save it. Within your program, just use your language's native int formats.
For example, in C I use something like this:
uint32_t read_be_u32(uint8_t data[4]) {
return ((uint32_t)data[0] << 24) |
((uint32_t)data[1] << 16) |
((uint32_t)data[2] << 8) |
((uint32_t)data[3]);
}
You can do the same thing in Rust, Go, or any other language. No special type definitions or macros necessary.
In fact, I'd be surprised if you made a big endian arch and then ran a browser on it if some large number of websites would fail because they used typedarrays and aren't endian aware.
The solution is not to ask every programmer in the universe to write endian aware code. The solution is to standardize on little endian
(1) for JPG for embedded TIFF metadata which can have both.
[2] https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.ht...
The only question that matters: Do your customers / users want to run it on big-endian hardware? And for 99% of programmers, the answer is no, because their customers have never knowingly been in the same room as a big-endian CPU.
The second sentence, weather your customers know if they have been in the same room with a big-endian system (CPU alone doesn't matter) is irrelevant when the point is to write correct code. Many of then aren't interested in this or other details and that is ok as they are not responsible for the implementation.
Changing the endianness either direction did have show bugs to me several times, that could be fixed, and it was worth it for that alone.
However if designing a new network protocol, choosing big endian is insanity. Use little endian, skip the macros, and just add
#ifndef LITTLE_ENDIAN
#error
Adding other architectures to your build system also tends to reveal nasty bugs in general, e.g. you were unknowingly triggering UB on all architectures but on the one you commonly use it causes silent data corruption whereas one with a different memory layout results in a much more conspicuous segfault.
FWIW I doing hobby-stuff for Amiga's (68k big-endian) but that's just that, hobby stuff.
Or you can just be a nice person and make your code endian-agnostic. ;-)
The adjacent POWER architecture is also still relevant - but as you say, they too can afford a support contract.
Of course that’s not what people will do. They’ll write code and not have any idea which parts have a dependency on endianness. It won’t be given a thought during their design or testing and when they need to make it work on a different architecture, it will needlessly be a giant pain in the ass.
It costs nothing other than having separate instructions for the different endian types.
The reason for this is that on the transistor level it takes exactly zero transistors to implement a byte swap since all you are changing is in which order the wires are connected.
Forcing software to deal with the pain of big endian support in exchange for saving a nonexistent cost in hardware is such a bad trade that it's on the same level of stupidity as not applying a clear coat on a car and then seeing them rust and expecting the owner of the car to thoroughly wax the car frequently to prevent the inevitable formation of rust.