Hacker News Reader

Comment by CJefferson | Hacker News Reader

It's really hard to test big-endian software -- because almost no-one (and in particular no devs) really run big-endian CPUs anymore.

There are a few programs I work on where we "officially" disabled big-endian support, because it's just impossible to find devs to work on it. It's also hard to test, you can't test it on github actions for example.

I'd be happy to see some people step up for testing big-endian, but at the moment it would be a huge time-sink for many projects, when it's not even clear if there are any users.

View on HN

ARM has selectable endianess, but it's difficult to find an OS that will run big endian ARM in any sort of out-of-the-box way.

The simplest one I could find is netbsd, which can run big-endian on an Rpi3 and some other boards.

https://wiki.netbsd.org/ports/evbarm/ (see the earmv7hfeb note)

One approach I've seen to writing big endian safe software is to not load integers directly, and apply the same logic regardless of endianness.

Eg. Instead of:

       uint16_t x;
    #ifdef BIG_ENDIAN
       x = swap(*p);
    #else 
       x = *p;
    #endif

You can write:

       uint16_t x;
       unsigned char *q = (unsigned char*)p;
       x = q[0] | (((uint16_t)q[1]) << 8);

(Pardon the rushed job of writing C in an HN form on mobile.)

This exercises the same deserialization from little endian regardless of the hardware representation of integers. This means you are testing how it will act on big endian even when running on little endian.

The problem is that it's hard to be certain that you actually did it correctly everywhere if you aren't testing it. Writing endian-neutral code isn't terribly difficult, but you can easily accidentally write endian-dependent code here and there.

This works when your serialization format is designed to be either big or little endian. But for some programs, the serialization format is memcpy the structure to disk. This isn't a good strategy, but unfortunately it can be difficult to undo historical choices.

So the padding, alignment, endianness, all leak. And it sounds like X does something like this: clients expect the endianness of serialized data to match the local machine and the server can optionally account for this.

I've done things like this.

First, the structs need the nonstandard pragmas and attributes to not pad.

Then, all accesses of 16, 32, or 64 bit quantities need to go through a helper function.

Even with that baggage, the syntax of having struct member access is can still feel more convenient than more verbose serialization code.

Code like that second one (using shifts) is generally what I do always (dealing with small-endian, big-endian, or PDP-endian data), anyways. Hopefully, the compiler will be able to optimize the code so that it works like the first one. (I don't know if the compiler actually does optimize it or not, though. However, it seems to me that a code like that should be common enough that it would make sense to implement such an optimization.)

So.. ntohl?

If you put that in a helper function, then kind of.. except ntohl and ntohs deal with big endian. And is a no-op on big endian. And there is no ntohl for 64 bit quantities.

Neither of those are best practice; you should use some kind of uint16_from_le() macro.

Yep. Rust's APIs do this right. Eg u32::to_be_bytes() / u32::to_le_bytes() and u32::from_be_bytes() / u32::from_le_bytes().

There's also methods for native byte order, but thats almost never the right thing to use.

I think Go does the same thing. I heard about this approach first from Rob Pike, who would rant about this to anyone who would listen. He said to think of "bytes on disk/network" and "an integer in RAM" as different types that you need to convert between. When you do, the API should specify the endian-ness of the bytes on disk / in the network. He argued you shouldn't do "byte swapping" at all based on endianness. Just parse the int, byte at a time and use the same code on every computer.

The real problem is that C makes it easy to convert between a pointer to an integer and a pointer to some bytes in RAM. Just never do that using pointer casting, and everything works out great.

https://doc.rust-lang.org/std/primitive.u32.html#method.to_b...

We agree. I'm writing it out as an example. In real life you would put it in some kind of helper function. (Or macro.) I'm not suggesting writing it out inline on every access.

In that case I disagree. You can't guarantee that the shift-and-or will be optimized out by the compiler in cases where no swapping is needed so you're making the code potentially slower.

why not just use POSIX hton{,s,l} ?

https://linux.die.net/man/3/htons

It's really hard to test big-endian software -- because almost no-one (and in particular no devs) really run big-endian CPUs anymore.

BE is still common in networking hardware (due to being the traditional "network byte order") and mainframes. The most widespread BE hardware would probably be the cheap un/rebranded routers with MIPS SoCs.

I know MIPS was still common for low-end 802.11ac routers/APs (eg. MT7621-based devices). Is it being used by anyone for systems designed around newer standards (802.11ax/WiFi 6/6E, multi-gig Ethernet) or has the market finally shifted over completely to ARM?

The Cudy X6 supports 802.11ax yet uses MIPS.

I guess it's not too surprising somebody paired newer radios with the MT7621 SoC. I see now that MediaTek even lists that application on the product page, so they're probably not going to retire that processor anytime soon: https://www.mediatek.com/products/home-networking/mt7621

MIPS, yes, but not big endian. Last time I started building binaries for my (utter junk) EdgeRouter-X I ran into issues with mipsel stuff not being well supported. If Marvell and MediaTek aren't putting out big endian MIPS stuff I'd be surprised if anyone was.

AFAIK mostly ARM these days. And headless network devices aren't a concern of GUI-related programs like X anyway.

Most networking gear control plane CPUs at this point run little endian (and x86 is surprisingly common). Even the ARM SOCs in most APs are running LE.

MIPS can use either big or little endian modes, though. SGI used MIPS in big endian mode, but some of the stuff out of China using MIPS today is little endian. Some devices include a hardware switch that lets you choose.

ARM also supported both, but you don't really see ARM used in big endian mode today.

> because almost no-one (and in particular no devs) really run big-endian CPUs anymore.

This is a big problem, because we may be introducing subtle bugs that are actual bugs that just happen to work because the hardware is arranged in such manner.

A lot of our stacks these days is built from source and very portable. We can certainly do better than everyone using the same CPUs.

because we may be introducing subtle bugs that are actual bugs that just happen to work because the hardware is arranged in such manner.

It's arguable whether that's a bug if it doesn't have any practical effect. I'd say that it's not.

IMHO, it's a bug if it's not specified. If your code looks portable and compiles on a big endian machine and fails, that's a bug. You should really at least fail compilation.

This is different than say a video game that works fine on hardware, but breaks in emulation, as sometimes happens. The game isn't buggy, the emulation is (or at least, it isn't complete), even if the game is writing to 'weird' addresses; it works on the specified hardware.

Before we even get to endianness, there's a lot of C and C++ code out there in the wild that is technically broken because it assumes much more basic stuff like int being "large enough" - but the language spec only guarantees you 16 bits.

I think at some point we should just bite the bullet and say that this kind of stuff is not really meaningfully standard anymore, and the few niches that need it can be accommodated with extensions. It doesn't really make sense to place a tax on everything else.

IMHO, it's a bug if it's not specified. If your code looks portable and compiles on a big endian machine and fails, that's a bug. You should really at least fail compilation.

I don't care about non-8-bit bytes either, and a bunch of other things, but having to deliberate add checks to "say no" just because someone might decide to use your code on some odd machine is a pure waste of time and even encourages the sort of lock-in and "environment nitpicking" that proprietary software vendors love.

As the saying goes, common sense is unfortunately not so common these days.

> As the saying goes, common sense is unfortunately not so common these days.

Ignoring the existence of big-endian computers is not really common sense. They DO exist. And it'd be very useful to test in both little and big-endian architectures. I cover the endianness tests by running my TravisCI pipelines on amd64 and s390x, but both run on Linux, which is sub-optimal. I should probably add AIX so that I'm not implying the OS is Linux and importing linuxisms into the code when it should be more portable than that.

> It's arguable whether that's a bug if it doesn't have any practical effect.

If doesn't have any effect until it does and, out of nowhere, you start getting crashes on some new hardware category that you thought was supported. That's not a great place to be.

Even before the move to x86, Apple was making sure OSX (which was based on NeXTSTEP, which was very portable) would not bake in PPC-isms that would hurt its portability. Apple learned the lesson with the painful port of MacOS classic to PPC and that care with OSX paid handsomely with the moves to x86 and then to ARM.

I've compiled and tested software for big-endian, using a s390x/debian Docker container, running it with qemu-s390x-static. It is quite slow though. To make it somewhat bearable, it is important to do the actual compile unemulated (using a cross-compiler), and then only actually test the binary under emulation. Although I was doing it all on a little Intel NUC, it would probably be somewhat more bearable on beefier hardware.

Not because I had any actual need to do this, just because it was fun.

You can get a small AIX system from IBM Cloud for US$50/month. I've thought about doing that – maybe some day I will. If I really wanted to I could afford it.

What big-endian hardware worth mentioning exists now, if we exclude historical hardware which we maybe want to keep up, but which is not produced any more?

Asking unironically.

AFAIK many Mikrotik MIPS routers are big-endian. Also Ubiquiti Edgerouters based on Cavium Octeon MIPS CPU.

Honest question, is there any network router that runs an X client or server? I have never seen something like that...

People run lots of junk on openwrt routers, no reason not to run X clients from them.

Except that web interfaces will probably always be a better way to remotely access those devices.

IBM Power chips can run in both big and little endian modes

In fact, Power ISA is probably the last architecture where both modes still have notable importance: AIX, IBM i and OpenBSD still run Power big, FreeBSD runs it big but work exists to run it little, and most Linux distros run it little (not counting 32-bit PowerPC, which pretty much always ran big).

Most of the other notionally bi-endian architectures nowadays have settled on one. For example, SPARC is just about exclusively big; conversely, ARM64 got rid of their big-endian mode.

My Tadpole Ultrasparc laptop that I love...

Oracle's SPARC servers are still manufactured and sold.

IBM mainframes are big-endian, including both IBM mainframe Linux (z/Linux) and the traditional mainframe operating systems (z/OS, z/VM, VSEn, z/TPF)

It's annoying that some file formats (and of course protocols) are still using big endian. Otherwise yes, the world has effectively standardized on little endian, it's fine to let big endian die outside of some weird niches.

> because almost no-one (and in particular no devs) really run big-endian CPUs anymore

It is a network protocol, just put a proxy between client and server that byte swaps the affected fields and changes the endian flag. No need for hardware.

.. so can byteswapped clients run through that?

They should. Such proxy would allow any mismatched client and server combinations. And, considering how X makes the detection, the matching could be automatic.

Better yet: Throw out the endian flag and define the network protocol as always being big/little endian. Network protocols which can swap endian order is a terrible idea. Let it die.

Convert from bytes in (whatever) endian order to bytes in memory when data is read and written using a uint32_t parse_le_uint32(uint8_t[4]) method or something similar.

Debian and Gentoo still provide current installation images for various big-endian architectures such as hppa, m68k, ppc32, ppc64 and sparc64.

> hppa, m68k, ppc32, ppc64 and sparc64

None of which are supported officially by Debian, with notable exception of ppc64el (POWER8/9 little-endian). The only official port with big-endian is s390x. [0]

[0] https://wiki.debian.org/SupportedArchitectures

QEMU?

Reply on news.ycombinator.com