Hacker News Reader

Comment by _-___________-_ | Hacker News Reader

> Because in golang everything is UTF-8. After all, Rob Pike authored both golang and utf-8.

That's a terrible reason. HTTP header values are not necessarily UTF-8, and an implementation that blindly assumes they are is broken.

> Now, what are the security implications of this? (beside the chunk thingy)

Many variations of the "chunk thingy" where values considered safe in one context may be unsafe in another context.

Like so many parts of Golang, this is a good example of how "good enough" engineering often really isn't good enough. HTTP is pretty thoroughly specified, and the HTTP stack included in a language should be thoroughly tested against that spec. If you can't achieve that, just leave it out of the stdlib!

View on HN

> That's a terrible reason. HTTP header values are not necessarily UTF-8, and an implementation that blindly assumes they are is broken.

Actually, they are explicitly specified as ASCII ("follow the same generic format as that given in Section 3.1 of RFC 822"): https://tools.ietf.org/html/rfc822#section-3.1

> Actually, they are explicitly specified as ASCII

They are not, only new headers added after RFC 7320 are ASCII, pre-existing ones may contain high-bit characters which implementations compliant with 7230 version of HTTP/1.1 SHOULD not parse. They stopped short from making it a MUST so that applications compliant with RFC 2616 (where they were to be treated as ISO-8859-1!) was allowed. Or, more simply, they stopped themselves from making RFC 7230 describe new, incompatible HTTP/1.2.

https://tools.ietf.org/html/rfc7230#section-3.2.4

nope: https://tools.ietf.org/html/rfc7230#page-25

> That's a terrible reason.

Consider this: all strings are utf-8, so working with pure ASCII is much more work in golang.

It leads to all kind of issues, not only security problems.

But that's a Golang issue then, not http

I know, I am just trying to explain how this could happen :)

Yes, but this simply means that header values are not (Go) strings, they're byte arrays. Many languages represent them this way, with a (fallible) function to get the string value iff the byte array contains valid UTF-8.

The whole "it would be more work" as a justification for not doing things properly is one of the roots of the more general issues with Go I alluded to in my post above.

1) Go string type is specified as a sequence of bytes. Full stop. Any encoding, binary blob, string does not care. Particular stdlib string functions _do_ care, expect UTF-8 encoding and document the fact. Other stdlib string function work with the sequence as a sequence.

3) Go specification requires UTF-8 in only one place: the encoding of Go source files string literals.

2) Pure ASCII is a subset of UTF-8, ie. pure ASCII _is_ valid UTF-8. There is _no_ more work and there are _no_ security problems when working with pure ASCII as UTF-8, because that's what it is.

Reply on news.ycombinator.com