Comment by userbinator | Hacker News Reader

Comment by userbinator | Hacker News Reader

IMHO much of the issues around validation are largely self-inflicted by the use of needlessly complex/verbose/flexible formats. Having worked (fortunately, not much) with SOAP, I can safely say it's one of the worst offenders. Text-based formats add so much more flexibility that they dramatically increase the space of both valid and invalid inputs over a simple binary protocol. The article shows how this can happen with numbers, but my favourite example is a boolean - if you use a textual format, you may think "true" and "false" are valid, but what about "True", "TRUE", "FALSE", " FALSE" (with whitespace), "tRuE", "falso", "yes", "no", etc.? Maybe you will think it makes sense to allow only the first two, but in a nearly-optimal binary format like ASN.1 PER, there's no need to even think about such things: a boolean is a single bit, representing exactly the two states which need to be communicated. There's no ambiguity[1], and (even better) no room for invalidity!

[1]I guess if you really stretch things, you can say 1 is false and 0 is true, and I have actually seen that before, but I can't think of any other ambiguities a single bit could have.

That is a very interesting take!

Adding to that line of thinking: obviously we need the textual representation for reading - so we need something to translate it. That situation is kind of dual to programming languages and compilers - we use compilers to make the textual representation machine readable, here we need a decompiler to make the machine readable data textual.

But if you start from a programming language, it surely do have something to describe data structures. That would already be unambiguous. That is why JSON is useful.

So just use protobuf/thrift as the payload, problem solved.

protobuf still requires a schema to distinguish between 1 and true.

Their example of asn.1 also needs a schema.

No, ASN.1 has separate boolean, integer, and enumeration types, as well as separate "printable string" and "octet sequence" types.

Yes, which makes it a schema

OK, then what term would you like me to use to mean "the thing protobuf uses to define messages, commonly called a schema, that lives outside the encoded data itself"? The point is that ASN.1 can do certain things without it which protobuf cannot.

Two functions applied on input (Trim and ToLower in C#) solve all the examples you listed and any similar cases. Yes and no are obviously very different from true and false.

> Yes and no are obviously very different from true and false.

The text-based language YAML allows “yes” and “no” for Boolean in addition to “true” and “false”. So if multiple bright people think about this topic they’ll evidently come up with different approaches. Not so with a single bit.

Non-native English speakers also exist. Around here, it’s not uncommon to see “S” for true and “Ó” for false.

Reply on news.ycombinator.com