Hacker News Reader

Comment by ajross | Hacker News Reader

A "parser" is the piece of software that translates unstructured input (usually text) into structured output that better reflects the runtime of a programming language. A "security bug" in a parser is normally construed to be the existence of an input that causes the software to do something incorrect/undocumented/unexpected.

Nothing in the article discusses a parser or anything like a parser bug.

The article doesn't like that the semantics of the user-facing API wrapped around the parser is, I guess, "easy to make mistakes with". That's an article about API design, at most. But that's boring and specious and doesn't grab clicks, so they want you to think that Go's parsers are insecure instead.

View on HN

I would consider the case-insensitive key handling in Go’s JSON parser and the malleability in the XML parser to both be “classic” examples of differential-prone behavior, which is the ultimate point made by the article.

(This is why the more formal definition of a parser is useful to ground on: a parser is a type of recognizer, and disagreements between recognizers that claim to recognize the same thing can be exploited. This doesn’t require a bug per se, only difference, which is why it’s a footgun.)

> case-insensitive key handling in Go’s JSON parser

This was an explicit decision for convenience, because the Go struct fields will be Capitalized to export them but JSON tends to follow a lower case convention.

Just because it’s convenient doesn’t make it a good idea! It’s also not what Go’s other parsers do per the rest of the post.

The "other parsers" in stdlib are gob and encoding/xml.

Gob is a format made for Go, so it does not have a field naming convention mismatch; what's on wire is the Go convention.

encoding/xml isn't really structured as a "convert between equivalent-shaped structs and a message" utility, its struct tags are more like a query language, like a simpler cousin of XPath. Most real-world code using it does things like

    type Person struct {
        FirstName string   `xml:"name>first"`
        LastName  string   `xml:"name>last"`
    }

Where as with JSON there was a clear desire to have

    {"email": "jdoe@example.com", "name": "John Doe"}

parse with as little noise as possible:

    type Person struct {
        Email string
        Name  string
    }

Again, I understand the rationale. I just don’t think it was a good idea.

> Nothing in the article discusses a parser or anything like a parser bug.

And it doesn't claim to. The article is titled "footguns" not "bugs". A footgun is just something that is easy to misuse due to unintuitive or unexpected behavior.

> And it doesn't claim to.

Yes it does. The title is literally "Unexpected security footguns in Go's parsers". The article didn't identify a single footgun. This is just bad design.

The article links to CVEs directly caused by some of the less intuitive behavior here. I feel like that’s sufficient to qualify as a footgun

That title is clearly constructed to imply that Go's parsers are insecure. The text of the article isn't about parsers at all, and only tangentially about Go[1].

It's deliberately misleading clickbait. You know it. I know it. We all know it.

If you want to have a considered discussion about pitfalls with the use of automatic serialization paradigms across trust boundaries, I'm here for it. If you just want to flame about Go, get better source material. This one isn't the hill to stand on.

[1] Which, again, has a really first rate serialization story; but not one fundamentally different from any of a zillion others. Cooking data from untrusted sources is just plain hard, and not something that anyone (much less the author of this awful blog post) is going to solve with a serialization API.

I think you and I read a different article. For example, about treating JSON keys without case sensitivity:

> In our opinion, this is the most critical pitfall of Go’s JSON parser because it differs from the default parsers for JavaScript, Python, Rust, Ruby, Java, and all other parsers we tested.

It would be kind of difficult to argue that this is not about Go.

Don't get me wrong, I love Go just as much as the next person, but this article definitely lays bare some issues with serialization specific to Go.

I disagree.

It was a bad design choice to allow JSON keys with different capitalisation and to serialise all public struct members by default.

These decisions can easily result in the creation of an insecure system.

I don’t think the title is clickbait. You’re fixating on just one example in the post, but the others demonstrate a clear pattern: there are footguns in Go’s various parsers, including forms of malleability or flexibility that enable parser differentials.

This isn’t intended to be a ding against Go; it’s a universal problem across programming languages (otherwise there’d be no differentials at all). But they’re worth noting, and I think the post amply demonstrates their security relevance.

Of course it's about Go. They decided to build not just opt-out unmarshaling of objects, but fragile opt-out.

See eg rails' strong params for the opposite approach: opt-in.

Exactly right. Better have a domain layer with data types representing the domain object 1:1 and add one or more API layers on top for interacting with those for some modality. Creation, deletion, verification, auth etc.

The security failure is not the parsing library, but failing to model your application architecture properly.

Reply on news.ycombinator.com