A binary coder for Swift | Hacker News Reader

A binary coder for Swift

mikeash.com

108 pointschmaynard23 comments

the numbers are byte-swapped to be endian agnostic

Regardless of how fast it is, it seems wasteful to do all this byte-swapping when the vast majority of systems today, outside of more specialised applications, are little-endian.

You can do byte-swapping in one instruction on x86, so it really isn't something worth worrying about. There aren't many big-endian systems out there, but it doesn't really matter that much. If you wanted to, you could change this so it always stores the numbers in little-endian (and does a swap on big-endian systems) but a lot of the uses of this will likely be for networking, and people generally expect big-endian in that situation.

> You can do byte-swapping in one instruction on x86

Well considering the vast majority of swift code runs on Apple's ARM chips (iDevices) rather than x86 processors (Mac), I think that's kind of a moot point

Moot twice, since ARM also has a single instruction for byte swapping.

Little-endian ARM, to be precise. AFAIK Apple's only big-endian products were those based on the 68K and PowerPC.

"1 instruction on x86" doesn't really tell us much about the cost of executing that instruction. x86 has a wide variety of instructions that take varying processor resources to execute.

Yes, you're technically correct, but I can assure you that byte-swapping is fast.

It seems I should have just left out the fact that byte-swap is a single instruction on x86, I was trying to make the point that it is fast but I suppose the point missed its mark and the fact that it's a single instruction really doesn't matter. You can do the swap with a few shifts and it still takes no time at all. If speed really becomes such a factor that this is too slow doing the encoding and decoding, you'll likely be looking at replacing this framework with a more hardcoded solution, not removing byte-swaps.

1 or 2 uops, 1 or 2 cycles (depending on data size). So quite fast. There's also the newish MOVBE (load/store big endian), which currently performs fairly similarly to MOV + BSWAP per Agner, but could be made into a single uop in the future.

Big endian is traditional and it doesn't really matter much either way. A more sophisticated version of this coder would expose that choice as a configuration option when creating the encoder/decoder objects.

A choice is the often _worst_ possible solution, assuming it is encoded into the format (which it may not be in this case).

With a choice, when reading every platform has a conditional swap rather than some platforms having an unconditional swap (and the others having no swap at all).

A choice will only be beneficial if writes happen more often than reads (as providing a choice means that platforms can do no swapping on write). This can only be the case if some written data is never read.

Also, conditionals in general are not good from a performance perspective, so even more writes than reads doesn't make it clearly better to have a choice.

This coder doesn't really have a format, it's more a way of defining a format. It should be flexible enough to work with many existing binary formats, and endianness configurability would need to be a part of that.

Performance is not really a concern for this implementation. I would guess that the indirection from calling through all the Swift protocol methods will completely swap any cost of having extra branches (which will be predicted correctly about 99.99% of the time if there's a lot of them).

Most network protocols, including TCP and UDP, are big endian, so little endian really isn't that ubiquitous. And I imagine a big use case for the binary coder will be for networking.

TCP/UDP/IP are BE, but that's largely an artifact of the time they were invented (when the LE/BE split was more even), hence networking hardware is probably where BE is still commonplace.

Most application protocols I've worked with are LE; serialisations like Avro, BSON, capnproto, protobufs, etc. come to mind, as well as numerous other proprietary ones. ASN.1 is a notable exception, but it's also much older.

But even though the protocols are big endian, is it generally the case that you always make the data itself big endian? (Asking out of ignorance, not rhetorically or sarcastically.)

It's up to the programmer. In the systems I've worked on, we always sent the data itself in little endian.

How stable are swift releases now? What are some good resources to start learning swift?

I think, as a starting place, Apple's Swift Programming Guide is hard to beat

In my experience, it's difficult to go through because it's so comprehensive and thorough, and I guess it has to be because it has to assume the lowest common denominator audience. Personally I'd like a shorter guide that assumes you know a few languages (C#, ObjC, JS, Java) and skips a bunch of the tedium of how programming languages work in general and gets right to what's different about Swift. It may seem like it does that, but look at their page about control flow[1] and tell me it's not an unreasonably long-winded way of explaining switch, for, for-in, while, break, continue, and if.

[1]: https://developer.apple.com/library/content/documentation/Sw...

I agree but Swift is a new enough language that I'm not sure such a resource exists yet. When I learned Swift I just skimmed through the parts that weren't interesting and that worked for me. It was a good overview of the language.

As an aside, the book, Advanced Swift, by the objc folks is fantastic but assumes you already know Swift. It's not what you're looking for, but something to move onto after you grasp the basics.

Maybe there is room for a publisher that makes books for people that already know how to program?

1.x was rough every point release broke major stuff

2.x was stable all through, but a lot of work to migrate from 1.x

3.x minor changes but a big renaming effort leading to a lot of work in accepting proposed changes in Xcode. Laborious more than difficult