The 7-bit Internet | Hacker News Reader

The 7-bit Internet

blog.tabini.ca

222 pointsfogus160 comments

My personal "wow, it's just this?" moment came when, in high school, I figured out how to IRC from a Unix box that didn't have an IRC client installed, just by telnetting to the IRC server. The only particularly annoying part was that you'd get disconnected if you were idling and didn't respond to a PING with a PONG quickly enough.

It was also a fun way to learn programming. mIRC at some point added a sockets interface to mIRC script, and after that it was pretty easy to write toy clients for various text-based protocols, doing simple things like checking whether a URL was a 404. I can't say I did anything particularly useful with that, but it was a nice way to learn something about programming.

I remember just about the same, some 15 years back. Discovering networks and being able to mess with them, even though my programming skills were still very basic. This is a good part of what made me enjoy programming. Exploring this world, feeling it within grasp.

I think TAOUP sums up a few important points about text protocols [1] and why going binary is not always wise. Obviously the main drive here is money and I'm sure we all have stories where short term money translates later to large unforseen and/or hidden costs.

Bandwidth capacity is growing exponentially so do we really need this, even for the savings ?

I'd rather see people invent clever new ways to do wonders with text streams than go back to opaque and obscure data structures that bring back to me rememberances of old days proprietary protocols.

After all, if our fathers made the choice of text streams at a time when each byte cost much more than now, there may be good reasons.

[1] http://www.catb.org/esr/writings/taoup/html/ch05s01.html

For some reason, I feel as though the people in this thread against text-based protocols have never used telnet to inspect or debug an HTTP request/response. Or used telnet to help develop a client for something like memcached.

The ease of discoverability of a protocol should not be underrated.

Yeah. For some things seeing truly is believing.

I guess if you've never had that moment where it all clicked for you, just by looking at traffic, the significance of human-readable protocols is easy to underestimate.

But hey, Google says that with binary-compressed headers of sorts, they can fit all their tracking cookies in one TCP-packet and that means you wont feel inconvenienced by the tracking payload they are throwing at you everywhere you go on the internet.

You know what? I want that huge bag of tracking cookies to hurt. If you violate my privacy, that should come at a cost.

Google: You can take your SPDY and keep nice, old HTTP alone.

I feel as though the people in this thread against (well-designed) binary protocols have never tried implementing a binary protocol.

Length-prefixed fields and direct encoding of binary data go a long way to simplifying implementing a protocol.

The ease of implementation of a simple binary protocol should not be underrated.

It's not just the binary. It's that HTTP 2 is hugely more complex. Excessive complexity is a general engineering smell. It can't be all that difficult to implement a protocol for applying a few verbs to URIs.

Complex + non-discoverable = bad engineering

HTTP is already painfully complex to implement correctly. It's also a terribly wasteful and poor engineered protocol for anything other than fetching documents.

> It's also a terribly wasteful and poor engineered protocol for anything other than fetching documents.

I wonder if that was the original purpose of the protocol;)

Those who do not know History are doomed to repeat it. Back in the day of the OSI stack, the telecom guys were huge fans of binary protocols. The arguments were exactly those described here on HN: they're easier to implement; with the right tooling, they're just as readable; they're more efficient.

The fact is, on the application layer, text based protocols wiped the floor with binary protocols. The answer is in the little details, which everyone knows is where the devil spends its time. They are not easier to implement, because they're more difficult to debug (and because text parsers are, let's be honest, a solved problem); they are not as readable, because debug tooling is never perfectly implemented and never ever present in every system; they are more efficient, but at this layer the efficiency does not pay off (i.e. the size of the HTTP header leading that 5MB image does not really matter).

One of the comments here pulled a comparison I did not remember any more: SS7 vs SIP. Go and have a look at both protocols. They are both mature, so you don't really need to wait for the fantastic debug tooling to appear. Study both ecosystems and then form your opinion on binary vs text application protocols.

Those who don't understand history are doomed to misinterpret its lessons.

Do you remember the OSI binary protocols as designed back in the day?

ASN.1 as far as the eye could see. OIDs. MIBs. Loosely defined extensibility. Terrible, complex protocol designs.

This had nothing to do with the protocols being binary or text; the protocols themselves were painfully complex, making it difficult and frustrating to write a working implementation, much less a complete and interoperating implementations.

> ... at this layer the efficiency does not pay off (i.e. the size of the HTTP header leading that 5MB image does not really matter).

"I don't care about RTT time, the lack of bidirectional communication, and the inability to inline binary data in the protocol stream" ... said no mobile/wireless/desktop developer, ever.

> One of the comments here pulled a comparison I did not remember any more: SS7 vs SIP.

I've implemented SIP. It was a massive hassle because it was text-based and annoying and difficult to parse. The available SIP libraries tend to be buggy and incomplete, and people actually pay for workable SIP stacks. I'd rather not look at SIP ever again.

That isn't to say SS7 is better, it doesn't appear to be. As far as I can tell, that's because it was badly designed, not because of a flaw inherent in using binary encodings.

> "I don't care about RTT time, the lack of bidirectional communication, and the inability to inline binary data in the protocol stream" ... said no mobile/wireless/desktop developer, ever.

http://www.igvita.com/slides/2013/breaking-1s-mobile-barrier... if you want to read about this from a mobile perspective.

Easy to implement in C, perhaps, but not Python, PERL, and the ilk. Most importantly, it's not as easy to do it with a shell and nc to build a simple server.

Implementing binary protocols in Python is quite easy. I'll take length-prefixed and encoding-tagged binaries over "text" any day.

So you are saying you are using a tool (telnet) to view/interact with the current protocol and this is good because it is easy to read and use.

Now we build a different tool, let's call it http2net. With HTTP being so prevalent, perhaps it isn't unreasonable to think it becomes part of most OS distributions.

So now you are using a tool (http2net) to view/interact with the new protocol and this is good because it is easy to read and use.

In summary, is binary vs text really the big problem here?

I think the reason I like telnet is because I control what is being sent down the line. It's not like telnet has any contextual knowledge of what I'm sending, so it can't do any translations on it; I get to choose exactly how the bytes get sent to the server. This means that telnet can be used for every text-based protocol out there (so long as it accepts \r\n line breaks), whereas your proposed http2net tool would be specifically for turning commands into bytes to be sent to HTTP 2.0 servers, and that response.

> so it can't do any translations on it;

You'd think so, but while it's true we can mostly pretend telnet gives you a raw TCP connection, telnet intercepts some character sequences. It just doesn't interfere much with plain text.

My problem with the hypothetical "http2net" tool is that to this day I still have to deal with systems that doesn't have basic tools like tcpdump, curl or wget. But telnet, nc or some other way of getting a semi-raw tcp connection is pretty much always available.

So I have every expectation that it'd take 30+ years before this "http2net" tool would be available everywhere I'd want it.

No, Telnet does not, in fact, intercept any character sequences unless when connecting to the telnet port (port 23).

This is not true on the client I have on my OS X box at work, nor on my Debian and Ubuntu machines at home, nor, I believe, with pretty much any other telnet client I've used in the last twenty or so years. I just tested several to verify whether I'd somehow missed something so basic.

Regardless of the issue of negotiated options, most telnet clients goes into a command mode if you enter an escape character, defaulting to ctrl+], unless you explicitly change it or tell it not to with a command line switch. This happens for me regardless of port.

tcpdump and strace also confirms what I thought I knew, namely that it also does LF => CR+LF conversion also when connecting to other ports.

EDIT: Here's a simple demonstration of how it is decidedly not 8-bit clean:

    echo -e "test\035help" | telnet www.google.com 80

Not only will this get you the help text for telnet rather than send the unmodified byte stream to Google's unsuspecting web server, but with default options not a single byte will generally get sent over this connection on Linux at least, as the client starts out line buffered and gets put into command mode without getting a line feed.

Confirm with tcpdump, or this way if you have strace installed:

    echo -e "test\035help" | strace telnet www.google.com 80 2>&1 | grep sendto

(the sendto calls you will get are DNS lookups)

Right, the interactive escape character. That's easily fixed with "telnet -E". That is,

    echo -e "test\035help" | telnet -E www.google.com 80

will not give any help text output.

(I was, in fact referring to telnet option negotiation which will not happen if not using the telnet port. Since the escape character for interactive use is so easily remedied, it did not occur to me to consider it a problem.)

I agree. Much of that attitude is also pompous.

Engineering should be about taking complex things and making them simple and easy (the best engineers do this), on the other hand you have engineers that try to add complexity because they understand something and over engineer it to obscure it on some premature optimization or cool factor. Taking something simple and making it more complex is the epitome of bad engineering.

Cool is useful and useful is simple or at least simple parts. Keep entry to technology simple, just like good games are simple to start, deeper in it gets more difficult. The door should be easy to enter even though the labyrinth might be immense.

Engineering is about making complex things simple and easy for your users. Sometimes that means increasing complexity at one level of the stack to increase performance and functionality at a higher level of the stack.

Many times in engineering the users are other engineers. Complexity only where necessary. The OSI layers beneath this layer are much more complex for those reasons. But in the end they are still simple parts to an overall complex computing and delivery system. Each engineer, if they are good, simplify things even for other engineers, this leads to less hurdles on the next platform innovations.

HTTP improvements are necessary. If this raises the bar past what some developers can manage, that's unfortunate, but necessary -- the current system actually prevents what better developers want and need to do.

I've written a couple parsers and stacks for SIP, a HTTP-derived protocol.

The text-based aspect isn't bad in and of itself. It's the crazy moronic rules and pointlessly flexible syntax that makes it bad. It's the fact that "text-based" is often taken to mean "should allow humans to be flexible in writing it", instead of "uses ASCII". Oh, and UTF8 if you're lucky. If you dare want a non-ASCII value that needs to go into a header, you're toast.

HTTP: Comments inside headers, line folding, context-sensitive header value parsing, and on and on. There is zero legitimate need for these things, yet RFC writers cannot seem to help themselves and design the most complicated syntax as possible.

Text-based formats frequently trick people into thinking they're right because it looks easy on the surface to get something to work. Shit, most HTTP clients/servers aren't actually fully compliant.

It opens more possibility for security holes due to potentially ambiguous parsing (where one implementation misses an edge case, but another doesn't). Protocols are not programming languages. They don't need flexible syntax.

There's also crap left over from a time when people read/write these protocols by hand. Idiocy like the IETF preferred date formats "Sun, 06 Nov 1994". Really? Including "Sun", the English 3-letter abbreviation, in a protocol? How is that useful?

HTTP is being used more and more for high-performance work. Take a look at a high-perf text-based parser, and you see all sorts of ugly hacks. Like doing bitwise comparisons on word-sized integers to determine the request method. (Both my personal code and nginx ended up with similar solutions, so it's safe to say it's a common approach for perf.)

Having a proper binary format that can be quickly, unambiguously, and safely parsed is a huge boon for interoperability and performance. The slight detriment for analyzing raw bytes (which, when encrypted makes it all moot) is not worth it. For a popular protocol like HTTP, you're gonna have plenty of tools to properly parse and analyze.

IP, UDP and TCP aren't text based, and I have no problems regularly analyzing them, nor do they seem to have adoption issues. But you can bet your ass if UDP specified port numbers as a flexible text field, you'd find all sorts of fun bugs and implementation issues.

Edit: After skimming the HTTP 2.0 spec, implementing this as a text-based protocol sounds like a nightmare with no benefit. It's not like you're going to write multiple streams out by hand or something (and use some terrible multipart-mime approach, or another "fun" ASCII-delimiting-binary thing).

IMAP is the protocol I love to hate with a burning passion... I get that it is enormously flexible. But it is also a total, utter nightmare to work with, for not very good reasons.

Though I don't really agree with you that you need a binary format for interop and performance. You'd get the same benefits if people just constrained the text protocols. E.g. Your performance "hack" for HTTP request methods is only a hack because method names can be variable length. We could constrain method names to 4 bytes and make the whole thing cleaner while still getting the text protocol benefits of being able to manually inspect request/responses. Similarly, picking a sane date format like iso-8601 and restricting it further to a specific set of options would do wonders while still being readable/writable for humans.

I do agree with you that once a protocol gets as widespread as HTTP most of the benefit is lost as we get better tools to use anyway, though. Especially with debug tools built into most browsers these days.

The issue is not that non-text protocols are impossible to analyse, but that each new non-text protocol requires new tools to do so. For IP, UDP and TCP we've had decades to create good tools. If a common binary serialisation format was agreed and most protocols stuck to that, it would be much less of a big deal to ditch text-only protocols and rely on client side tools to allow reading or manipulating them as text.

> Protocols are not programming languages. They don't need flexible syntax.

Or, as the langsec crowd claims, protocols don't need flexible syntax, lest they actually become programming languages.

Relevant talk: Meredith Patterson on Turing-complete protocols and weird machines - https://www.youtube.com/watch?v=3kEfedtQVOY

I disagree with your last line, if you know what http chunked encoding is, it is a 'simple ascii delimited binary thing' - it just doesn't have a 'stream id' in it, but there could be. Of course, people always forget to check the chunk length is <0 which makes for a constant stream of security holes over the years, but you would have the exact same problem in binary.

I think it's sad that HTTP/2.0 is primarily just trying to multiplex TCP and the rest seems like pure micro-optimizations (the one exception being server push; I haven't read a lot on that yet). It seems like the wrong layer in which to attack the problem, and is basically a huge 'meh'.

What, exactly, prevents a binary protocol definition from being overly flexible and complex? I've seen quite a few like that, and I've seen quite a few simple and easy to parse text protocols.

Attitude. One of the big advantages that seems to be touted for text protocols is easy to use manually by humans, and with that can come a desire to make it forgiving of human error.

You can have simple text protocols and complex binary protocols, but it's more tempting to make text protocols more complex.

(In this comment, simple/complex is in terms of syntax only)

Writing new protocol modules for tcpdump isn't difficult.

After that you can see what? You also have to use something like sed, xxd, and nc at least to make a request unless you are really good with bitfields and a hex editor. You want to keep a connection open or use what comes back for he next request? Maybe you can quickly cobble together a crude client in expect and tcl, perl, or python.

A lot more difficult than using netcat though.

It's incredibly rude to just dismiss everyone who disagrees with you like that.

What do you expect from a group that thinks trans-compiling C++ to JavaScript is a good idea...

I don't understand why SPDY / HTTP 2.0 is being framed as "fitting the needs of a few major players".

Google is not the only one who benefits from reduced latency, fast page loads, efficient use of SSL sessions, and server pushed resources.

If anything, the people who benefit the most are those who _can't_ afford massive forward deployed CDN networks, large servers, and fat network pipes.

The mind-bending complexity of the proposed HTTP 2.0 does very little to gain the little guy, or a young generation exploring the web: in fact it shuts them out.

If the internet should have taught us one thing so far, it is that it's the open technologies which are built to be easy to understand and explore for humans which have driven the net forward.

HTTP 2.0 is massive step backwards in that respect, and the only thing we're getting in return is slightly shorted response times.

You know what? We can deal. We're getting better and better bandwith. We're getting more and more computing power. A few milliseconds doesn't hurt anyone.

Let's not create a shit protocol violating everything the internet was built on, just because we're all of a sudden feeling resource-constrained. Now, of all times.

rolls eyes

> We're getting better and better bandwith. We're getting more and more computing power. A few milliseconds doesn't hurt anyone.

Yeah, but the speed of light isn't getting any faster, and little guys can't afford to have their servers everywhere. If HTTP 2.0 reduces the round trip count, then it will make a big difference in download times for small pages.

HTTP 2.0 is going to happen whether we complain about it here or not.

While we can certainly wait and see whether people will balk and make changes, the only true way to stop it is to replace it.

If you can write a better set of protocols that meet the requirements, do it. If you can't then work with or support those that will.

Like IPv6? Its taken more than 10 years to approach 1% traffic share. Perhaps HTTP 2.0 will happen but I wouldn't bet money on it being everywhere anytime soon.

Consider that until people start being told "no, you can't get more IP v4 addresses" or some services are unavailable via IPv4, the incentive to switch to IPv6 is quite low.

We have IPv6 connectivity from our colo provider in one location, and a tunnel to our office, but two years in we're still only testing it for this reason - there's never enough time, and it's not yet urgent enough.

I can't switch to IPv6 by downloading a browser. I could switch today to SPDY. In fact, it's got 55% market share including IE 11: http://caniuse.com/spdy

I see the proposals of binary protocols, push content, etc. as symptoms of a deeper issue.

Different than the spirit of HTTP, those protocols have little to do with publishing content anymore. They are just more kludges, in the history of kludges, to patch browsers into application platforms.

That mainly benefits the big web monopolists, who require the browser to be the ultimate application platform, where they can track to their hearts content, display unsolicited advertising, and basically extort business to advertise on their channels to remain relevant on the web. Not quite the idea of "information repository" that spawned the web in the first place.

Right. Because the alternative, bloated pages out the wazoo makes so much more sense than having smarter Apache and Nginx servers who will do all this for you. Remember, most of the web will be connected by cell phones, we need better protocols for that.

Did you read the comment or just skimmed? I don't question the technical merits, but the underlying motivations, which I believe are fatally flawed.

Perhaps I can reply more clearly after some sleep :) What I meant is that it's to support an evolution of content from static print and "data" to dynamic, interactive and responsive content the likes of which we've never seen before on such a wide scale of platforms and devices. Write once, run anywhere truly exists for content that conforms to web standards, and HTML5 Components will push that boundary even farther. I'll admit, authoring tools and publishing techniques need to play catch up, but that's not a reason to stop progress on standards. That's only a reason to slow or change adoption patterns and usage. Build a better Wordpress, I say, one that's two way. It's not always for the big advertising players, you know...

Read. That's why I mentioned Apache and Nginx, since the Google SPDY team has focused on making sure that there are plugins for each, plus CloudFront support, so that it can be applied "to the little guys". And in turn, I would ask you: Have you not used Web Sockets? How about HTML5 Web Components? New JS functionality? As "hackers", I would expect more excitement than this -- since you're going to enable a whole new generation of low latency cell phone apps, OSes based on web browsers and apps for them (e.g. Win8.1, ChromeOS and Webkit) and do so in a very standards-based way, rather than ad-hoc distributing SPDY through your proprietary browser. Short of security flaws like SSL's header compression, what could go wrong? :)

> That mainly benefits the big web monopolists, who require the browser to be the ultimate application platform, where they can track to their hearts content, display unsolicited advertising

Please. All you need for advertising is plain text, all you need for tracking is cookies.

Read again. I'm not saying the technologies enable advertising. I'm saying it's in their best interest to have people do all useful work inside a browser, with a technology stack they control, increasing ad impressions and being able to provide ubiquitous user behavior tracking. This is the holy grail of advertising and would strengthen their monopoly even more.

For Google, a .0001% reduction in user overhead may translate to millions of dollars of resource savings. Google is willing to do crazy things like run their own modified Linux kernel that most companies wouldn't bother with.

What is the end-user benefit in performance using HTTP 2.0? The only analysis of SPDY I saw indicated low single digit percentage benefit vs HTTPS. Whoop-dee-do. http://www.guypo.com/technical/not-as-spdy-as-you-thought/

It's funny, isn't it? We gave out more domains to speed things up, but now it slows us down (referring to your link).

I would highlight that HTTP 2 > SPDY as far as standards go, and the "beta" protocol of SPDY is already at 55% market share including IE 11: http://caniuse.com/spdy

For those who are interested in speed optimization for 2013 and beyond, plus why SPDY's faster-than-SSL descendants are crucial to the web's success, have a look at http://www.igvita.com/slides/2013/breaking-1s-mobile-barrier... -- I believe there's a video out there too somewhere. Edit: Video link is on first page of the slides.

What's interesting too is that this talk focuses as much on what's achievable without SPDY as what might be with, down the road. We really need faster SSL negotiation for that first time connection cost.

Google can afford to dedicate a bunch of engineers just to deal with all these complexities and inconveniences of binary protocols, but others can't. And this is far more important than some hypothetical tiny improvements in performance.

Yeah. It's too bad there aren't free web servers out there that everyone can use that implement these protocols. Oh wait, there are.

By making things more efficient, HTTP 2.0 is really going to make startups spend more money. Oh wait, no, the opposite thing.

These web servers aren't that useful for everyone, some of us have to write our own.

EDIT: It's not the matter of implementation anyways, it's that something important only for Google is proposed as a standard for everyone, which is insane.

What startups do you know that are writing their own web servers?

More efficient web servers will, if anything, be even better for startups than they are for Google, since they lower the barrier to entry.

> If anything, the people who benefit the most are those who _can't_ afford massive forward deployed CDN networks, large servers, and fat network pipes.

You win some, you lose some. The benefit of cutting down on some infrastructure should be weighed against the mind-boggling complexity of HTTP 2.0 and the added difficulty in debugging.

If you're tempted to argue against this by pointing out that very simple tools can make irrelevant the change to a non-textual protocol -- then you are, I suspect, missing the point. There's a real psychological difference between debugging something that is, at bottom, human-readable versus one that isn't. I'm willing to concede that it isn't really that big of a difference. For the sake of argument I'm even willing to concede that it's largely illusory -- because that doesn't matter. Who cares how irrational it may be? If it's that prevalent in the human psyche, you're just better off not fighting it.

Like scripting languages, a text-based protocol doesn't just make it easier for you to get your hands dirty: it practically begs you to. The value of that sort of encouragement to the adoption of a global standard should not be underestimated.

HTTP 2.0 is not a deprecation of HTTP 1.1/1. You can have your cake and eat it too.

Never thought that it was. This is just another reason why HTTP 2.0 adoption is going to be an uphill battle.

>At first, this struck me as a very odd thing; text is inefficient, and machines, not humans, are meant to interpret protocols. A binary setup would save bytes—bytes!— and be all-around more manageable by software.

>It wasn’t long, however, before I realized the true genius behind this decision.

This reminded me very much of a talk by Jonathan Blow where he's talking about how he "nerd raged" while reading the Doom source code. He found a section that wasn't optimized, but in fact not optimizing it had so many other benefits (besides, the performance gain would have been negligible). Starts at 14 minutes: http://the-witness.net/news/2011/06/how-to-program-independe...

I don't get all the for-telnet arguments. Who uses telnet for this anyway? I'd bet that 90% of telnet-folk do:

  cake:~ mali$ telnet google.com 80
  Trying 74.125.235.8...
  Connected to google.com.
  Escape character is '^]'.
  GET / HTTP/1.1
  .......
  HTTP/1.1 200 OK
  Date: Wed, 10 Jul 2013 22:25:02 GMT
  Expires: -1
  Cache-Control: private, max-age=0
  Content-Type: text/html; charset=ISO-8859-1

At which point they go "oh cool" and go do something else. For the rest of us who use curl, wget, requests, Chrome / Firefox developer tools every day (who are we kidding, it's everyone, you liars! :P), the binary transformation would be transparent.

Hell, if you're going for pure cool-factor, how is pulling out your hex editor less cool? But in reality, you'd never do this.

For a non-standardized and obscure protocol where tooling would likely be lacking, I can see why human readability is a good idea. But we're talking about the very protocol that makes up the fabric of the internet. Seriously, why?

Give me one good reason.

You've never needed a basic TCP test without installing wget/nmap/etc? Most operating systems and hardware devices have telnet available for this kind of low-level troubleshooting.

You've never needed to test an SMTP connection to see what the rejection message was on the remote server (when a user can't get you the bounced message you require)?

You never wanted to see if an SSH port was open and what version was running?

Telnet is available in every router and firewall I have, I can't install curl onto a router to generate a request from a remote network, and I'll never see wget there either.

I do this stuff every day as part of my job, telnet is the go-to, the other utilities are fine, but they usually mask what I'm really looking for anyway, if they are even available on the platform I am using in the first place.

When its easier to write tools, tools are more plentiful. Same binary protocols make it an afternoon's work to implement most protocols, and even less time if all you want to do is open up a socket and send some EHLOs.

Text protocols, on the other hand, require writing a parser, dealing with encoding back and forth between string representations and binary data, handling line delimiters, etc.

I'll take binary protocols any day of the week. Any cost they incur in not being human readable is offset by the value of them being so easy to implement.

It sounds like your argument is a preference for a pre-written parser library over having to write the parser yourself. Yeah, sure, no one would disagree with that for day to day use. It's when you don't have the pre-written library available that a text-only protocol will save your day.

Since binary protocols are easier to write tools for -- and difficult to us without them -- the tools get written.

Slowing down everyone forever just to ease telnet debugging is a misdirected optimization.

The tools might get written. But they won't be installed on those routers, remote servers you're not getting to install stuff on and all kinds of other places where people who work with networks frequently want to be able to talk protocols from.

It's largely irrelevant to me if there are tools out the wazoo to work with some binary protocol if I'm unable to run that tool everywhere.

And there's a huge range between advocating arbitrarily complex and flexible text protocols vs. binary protocols. You can "easily" do text protocols that are picky about field lengths and that use formats that can be parsed much faster than the more complex protocols.

If your protocol is using a small enough, regular enough grammar, it'd also be fairly trivial to allow a binary serialisation of requests or responses as an option without much extra overhead. E.g. start client connections with a word indicating it wants to "switch on" binary and length prefix any variable length fields instead of relying on an end of field marker, for example. (Or make human clients type out a word to switch to the text serialisation).

But very few protocols are so affected by latency in request/response exchanges that binary vs. text is a huge deal. For HTTP moving to a pure binary protocol might make sense because of how heavily we depend on it. But most protocols are not HTTP.

This is just scaremongering. Installing the command line LDAP tools (which is a horrendous binary protocol) is no harder than installing netcat or tcpdunp.

Do you have netcat compiled for Cisco IOS and JunOS? If so, please send them to me, and let me know how to get them into the devices firmware.

It might sound like scaremongering to you. To me, the reality is every network I've worked on have had devices that I can not install and run arbitrary code on. Most non roll-your-own routers for example does not give you a shell where you can install arbitrary applications. But many of them do have tools like ability to telnet to help diagnose problems.

So how do you debug the myriad of binary protocols that already exist today? How do you debug SSL services?

For that matter, how do you perform more than cursory debugging of HTTP services? Do you seriously sit there and carefully type out HTTP 1.1 compliant requests, along with requisite headers and maybe even cookies? Does that actually work for debugging complex issues, and does it really differ that substantially from the debugging one performs to see if a binary protocol service is up and accepting requests?

I was just going to reply to some of these comments when I realised that everyone saying "HTTP 1.1 will still be accepted" is right. After all, the mere fact that you can telnet to so many web servers that will serve gzipped content over SSL at a drop of a hat shows how backwards-compatible the web is. Don't fool yourself into thinking that just because your connection over Telnet works, everyone's using that. That Google example makes me laugh -- since Google already does SPDY! ;-)

And what if I asked for application/vnd.google.protobuf ? I don't always need 7-bit text over HTTP, after all...

I'll give you two, both occurred to me in the past month:

1. Debugging which headers were causing Amazon CloudFront to MISS requests originating from Android. tcpdump, observe request, repeat using netcat until the problem was pinpointed.

2. Debugging failed authentication on a dovecot IMAP, on one specific scenario. Again, tcpdump, reproduce, isolate and fix.

Have you never had to send a specific request/response to a server? I've often found it easier to store the request in a text file and pipe it instead of using curl or another tool to craft it.

When I was a webdev (over the past few years) I did this at least once a month.

I'd daily look at the HTTP headers, though. Also, I'd often add custom headers for debugging purposes.

Edit: It's also not just telnet, it's being able to use simple scripts to automate requests or responses for many reasons.

Most likely all HTTP/2.0 servers will continue to support HTTP/1.1 forever, so you will still be able to do this.

And clients will need to support it. I feel like the HTTP/1.0 to 1.1 switch allowed for a good deal of code reüse, but this won't.

Minimal clients can continue to use HTTP/1.1; full-featured HTTP libraries shouldn't have much trouble adding HTTP/2.0 support.

A lot of MUDs still use telnet. Even with MCCP (which has to be negotiated over telnet anyway), it's not uncommon for legacy clients to connect over raw telnet.

the cool kids use nc(1)

Actually, we use socat.

Hrmmm, looks like socat expanded the concept to any socket. Very nice.

Oh sure, let's keep buffer bugs, waste tons of bandwidth and storage, burn millions of CPU hours in parsing computer-generated text, completely ban encryption and screw it for all non-en_US-speakers so that it could be debugged via misuse of a protocol from 1983 that no one sane uses.

I watched simple text-based internet protocols eat the efficient, ASN.1 OSI protocols for breakfast. From the OSI side of the fence.

You can talk about efficiency until you're blue in the face, but human-readability is very useful.

I used to be able to pretty much parse a hexdump of an X.400 P1 PDU by eye (certainly if I could reformat in a text editor), but even today I found it useful to eyeball a recalcitrant programs's HTTP request/response cycle by simply catching it under strace and grepping for "HTTP".

It's a massive reduction of friction to have human-readable protocols. I used to make the argument that it didn't matter and I was wrong.

> * but human-readability is very useful.*

For who, how often and when exactly? As far as I have understood, we are doing something wrong if we have to deal with debugging protocols which implement an abstraction, rather than just let the abstraction do it's job.

This is like saying that "Well for programmers x86 assembly instruction mnemonics are useful compared to machine code bytes!", to which one could say that for an average programmer that makes no sense.

I didn't quite care about stuff like this until I got introduced to information theory and really started thinking about what it means to send and receive information. The amount of totally unnecessary waste is astounding.

I'm starting to feel like I start all my HN posts with "Hi, I'm a sysadmin."

Hi, I'm a sysadmin. Sometimes we have to debug things that are on the other side of the world. When we do that, we need to have a mental model of what we're doing. You're familiar with that as a software developer: you have a mental model of the capabilities of the language that you are using, and you are fitting that in with the mental model of the problem you are solving and the context of existing program code.

To sysadmins, protocols are like programming languages. That's why we like text-based protocols: we can fit them in our heads and type them out, slowly, like ancient creaky teletypes that make lots of mistakes and pause in between commands. Meanwhile, we're running down checklists: can we connect? OK, there's no IP-based packet filtering. What does the banner say? Can it support STARTTLS? OK, disconnect and try again with telnet-tls. Hey, it doesn't negotiate...

Totally unnecessary waste? Sure, assuming someone has already written a tool which does all the testing for you, and you know that tool exists, and you have access to it right now over your tiny smartphone's 3G connection at 0400 while you thought you were on vacation.

Hi, I'm a programmer.

As a sysadmin, do you really have to deal with HTTP protocol by hand on such basis that a tool could not do the task for you if the protocol wasn't human readable?

No.

I need to deal with a dozen protocols in an emergency situation where I can't guarantee the appropriate tool is going to be available. (When it's not an emergency, yay tools!)

For the same reason, I can use vi with no vim extensions. For the same reason, I like configuration files written in text formats, not binary blobs. Binary would be faster, sure. When it breaks, you need the precise tools to know what you're doing.

I commend to you RFC 3117 - http://www.rfc-editor.org/in-notes/rfc3117.txt - as a discussion on how to figure out what a network protocol has to do, and how.

Under what circumstances are you logging into machines with only telnet available? Presumably you also lack SSH, otherwise you could use a tunnel, so I assume you're physically logging into a bare-bones OS that lacks even basic functionality. Is that really something you do often? And if so, why?

Don't want to debug it? Don't turn it on. The only reason you would is if it had more benefits than drawbacks. Standards bodies make standards, they don't force people to pick one.

If you are using telnet on your smartphone at 0400 on vacation you should probably have access to an SSH client that you can use to get on a real system to run curl on.

> For who, how often and when exactly? As far as I have understood, we are doing something wrong if we have to deal with debugging protocols which implement an abstraction, rather than just let the abstraction do it's job.

Any time you're debugging "one layer above" and you're in Sherlock Holmes territory (i.e. the problem seems impossible, so one of your basic assumptions must be incorrect) you have to check stuff.

And if you check your protocol by using the protocol-parsing tool which comes with your protocol implementation, you're not doing an independent test that it's actually working OK.

As a more concrete example, I'm using a library doing AWS request/responses over HTTP. There are various places I could add instrumentation to dump information but:

- it takes work to add or enable. I can grab the on-the-wire protocol with strace or tcpdump (this a reason why it's always good to provide a non-SSL option for your protocol)

- whatever is causing the problem could be below the layer I'm logging at

- they are all error prone. Maybe I miss a part of what goes on the wire. The data-on-the-wire is the only thing which matters for the protocol. The other end has no additional state.

When you're debugging, you have to validate stuff. And look for patterns. Human-readable protocols facilitate both of those difficult activities, reducing friction.

Human readability too could mean base64 encoding.

Instead of ACKNOWLEDGE you say ACK or mere A. Instead of REQUEST PAGE FROM <path>, you say RPF <path> and so on. This is my main point of hatred towards "human readable formats", because they waste bytes for no reason.

I can run the water for as long as I want without absolutely no consequences for me, but why would I do it if I can avoid it? Why would I not save resources whenever I can, even though I don't need to do it? It's more a philosophical question, to which I would answer with "save anything you can, whenever you can and make no waste.". Very simple.

I don't disagree that human-readable doesn't mean "verbose", but in answer to:

> Why would I not save resources whenever I can, even though I don't need to do it?

It's a cost-benefit. You don't spend your evenings clipping coupons all the coupons you can (which would save you some pennies). Or if you do, you don't stay up late to do it. The benefit to you of some free time is greater than the benefit of saving the pennies.

Similarly, the cost of mild verbosity is balanced against the benefit.

Basically, I don't buy an absolutist position of "save wherever you can". There are costs to saving, make a judgement whether it is worth it.

Well actually, your assembly example is a good one because C#, Go and a bunch of other similar languages all have refractor information compiled into the output binary because debugging machine code is a pig (even with assembly mnemonics). So even the average developer enjoys the luxury of wasted information in modern specifications.

My point was that a average programmer does not understand a thing about assembly languages, because they are abstracted away by well working implementations of compilers and debuggers. I live under an assumption that such is also the case with HTTP, or at least should be, because for me it seems that a protocol as simple as HTTP would be abstracted away ages ago by libraries and implementations.

Plain text HTTP (and all the other ASCII-based networking protocols) is already a layer of abstraction on top of TCP/IP data packets. So a better comparison would be HTTP as the higher level programming language and the raw TCP/IP packets as the assembly. In which case we're back to the point that developers would care if they had to write websites in binary.

The problem with many web developers these days is that they don't understand or don't care about the networking side of things. Web development is such a high level view of programming that many developers who've only grown up with targeting the web, those kinds of developers don't also appreciate just how many layers of abstraction there are between them and the users navigating their site. As far as they're concerned, they just bang out some PHP, copy the files onto some shared hosting provider and let the sys admins worry about the rest. Which is fine if that's all they want to do, but there's a whole plethora of technology at work - even beneath the HTTP protocol.

As for tools to query HTTP, I swear by curl:

    curl -i --silent example.com | head    # http headers (written by web app)
    curl -I example.com                    # http headers (written by web daemon)
    curl -v --silent example.com | more    # verbose output; great for tracking down faults
    curl -H "host:example.com" ip.address  # set the host header; useful when using named based virtual hosts
    curl -A "opera mobile" example.com     # sets the user agent; useful for working around mobile / desktop redirects

...etc. Rarely does a day go by and I'm not using curl.

The problem with ASN.1 was ASN.1, BER, and DER themselves not the fact that a binary encoding was used.

I agree. HTTP with chunking is already way beyond the human-emulatable-via-telnet threshold.

I'd like the re-iterate that "ban encryption" line -- do we really need to be listening in to packets on the wire? Better to capture before the wire at the web server, or after the wire in the browser.

This text vs. binary discussion is pretty high level. HTTP/2.0 has a mux layer (let's temporarily put aside whether that's a good idea) and some of the payloads have to be binary (e.g. JPEGs). There are basically three ways you can do this:

A. base64 everything. Obviously this has high overhead.

B. Escaping (aka byte stuffing). This is somewhat slow to escape and unescape, the overhead is variable (in rare cases 100%), and it's fragile to read or write by hand.

C. Byte counting. This is the most efficient, but extremely inconvenient to write by hand.

You could create an efficient "text" mux protocol (basically BEEP), but the result is so non-human-readable/writable IMO that I don't think people would be any happier. HTTP/2.0 is not complex because it's binary; it's binary because it's necessarily already complex enough that text doesn't save you anything.

You have many of the same issues with HTTP/1.1 pipelining.

I don't think a simple binary protocol for HTTP would be bad at all. HTTP servers are still going to support text-based connections forever, so it only matters if you're snooping on existing traffic; but the ability to use Wireshark or tcpdump to grab the text out of packets directly is already disappearing because of encryption (SSL), which is such a good thing in general that it far outweighs that drawback. Therefore more specialized tools are required, and those may as well parse such a protocol - it wouldn't be that hard. The compression is a significant benefit, and can't be done well below the level of HTTP.

What I don't understand is why we need both SPDY and QUIC.

SPDY provides fast multiplexing, compression, and guaranteed SSL. QUIC provides fast multiplexing and guaranteed SSL at the transport(-ish) layer. QUIC does at least one thing that can't be done at the application layer - faster opens - and seems nice to have as a generic base for multiple protocols, so I call it a good thing (and I'd like to see it in the kernel). But the plan seems to be to run SPDY over QUIC - I can't actually find enough information on the Internet (and am too lazy to look through the source) to find out whether packets are going to be double encrypted for the time being, but even if/when that is avoided, the multiplexing and optional encryption seem to be wasteful complexity. I would prefer if HTTP/2.0 were a simple and easy-to-parse protocol providing compression only, expected to be used over QUIC.

As I see it, SPDY/HTTP 2.0 is for the next ten years. QUIC is for the ten years after that.

I agree with most of the article, but would strongly argue against any use of the 'robustness principle'. There have been huge security vulnerabilities because systems disagree with each other about what is a valid request and response.

And billions of money wasted because browser vendors, at some point long ago, decided to accept invalid HTML.

and lots of experiments, new websites and happy users appeared because HTML has been accepted leniently

would those experimental users have given up right when the browser reported "Missing </p> tag"?

That end tags were an option and not mandatory was a feature of the SGML language that HTML was based on. I'm not sure specifically if this was allowed in the description for the P tag, however I suspect it is.

So this was entirely by design and probably not all that difficult to implement. That it might not be appealing to some I can understand. I suspect the creators of SGML had very different concerns.

I like that HTML has a very humane interface.

They would probably be more encouraged and happier to develop even more because they would understand HTML, which is a simple but powerful concept of block markup, and not some random pile of characters in angle brackets.

And we've wasted vast mountains of money dealing with the resulting brokenness of the web.

Those people would have figured out how to ship a web page; they'd just would have had so much trouble figuring out why one they shipped was't working, because the browser would have told them.

> but would strongly argue against any use of the 'robustness principle'.

It's useful for humans talking to each other. Apart from that I agree with you.

Am I the only one who thinks its ironic that the title references 7-bits? This shows that you're already binary encoded enough that your terminal has to parse it out. Why not just have a dumber terminal?

Plus, it's SSL based. SSL means you're already used to not looking at things through telnet. Are you really saying we shouldn't use SSL because "it's not just 7-bit plaintext"? That's the point, after all...

> SSL means you're already used to not looking at things through telnet.

Someone has never used `openssl s_client` before.

Very few people actually delve into and edit the TLS protocol. People, including devs, often delve into the HTTP protocol and craft requests by hand, or with a simple text-processing script.

To most people, TLS is transparent. This HTTP 2.0 protocol is not transparent.

If you're using openssl s_client, why not use another such client? That's my point. The fact is, you type ascii because it's inconvenient to type binary, but you have software interpret bytes, so why not send what output you want, monitor what you want, using new tools. Haven't you ever used Fiddler, Charles Web Proxy, WireShark or anything else that makes binary protocols easier? Oh wait, yes, you have: openssl. My point exactly. You'll be able to unwrap the binary to ASCII representations as much as you like, no question, no harm, no foul. Or it won't get adopted.

Because I'm not working at the TLS layer, I'm working at the HTTP layer. I'm sure if I had to often write TLS handshakes by hand, s_client wouldn't be my tool of choice, just as curl isn't always my tool of choice.

Making the protocol I'm working in binary doesn't do me any favors.

If you're not working at the TLS layer, then why make the request over HTTP 2.0? You can just ask for 1.1 instead. If it works for Google.com, it should work for you too, right?

I don't understand, HTTP is not TLS. So you're saying that we should continue to maintain 2 protocols?

As some others have said, I feel that making a "webapps" protocol, a streamlined websocks or something, maybe this HTTP2.0, and keep HTTP for stateless resource representations, like it was designed to do.

"maintain 2 protocols" - actually, yes, I do expect browser vendors to maintain compatibility with HTTP 1.0, 1.1, 1.2 and 2.0+. By the way, you might be interested in 1.2: http://www.infoq.com/news/2011/04/http-1.2-released

The way I see it, we can have our cake, both flavours. Why make someone remember different URLs? "http://" is too entrenched for when people type it in, and I'd rather not worry about browsers trying to sniff out faster protocols as they currently do with SPDY. Better to make it a version number, even if you personally never use it.

HTTP 1.2 is an April fools joke.

Sigh. And I knew that. Just not at 2am. (Though the way I phrased it, maybe it could slip by as an intentional link...) Thanks for pointing this out to my less tired self and anyone else who stumbles upon this thread. It's perhaps worth pointing out that HTTP 2.0 then is not an April Fools joke :D

(1.0 and 1.1 was all I knew, but Google distracted me in to thinking it was real again with those dang uncorrected news articles...)

Stateless? What about that hack known as cookies? Aren't they useful? How about HTML5?

And what happened to the calls for SSL everywhere? Defending your cookies from the NSA and MITM? Of course we need two protocols, perhaps more than two. I'd love to see something more interesting happen with multicast given talk of IPTV in 4K.

I'm shocked that your only objection ends up being the protocol name. Worse, it's the version number. But hey, it is semantic versioning... Don't use it if you don't want to. It's not like anyone else will notice if your site is already fast because it's one request with no state.

> Stateless? What about that hack known as cookies? Aren't they useful? How about HTML5?

I dislike both.

> And what happened to the calls for SSL everywhere?

It should be, it's just not HTTPs job to provide that

> I'm shocked that your only objection ends up being the protocol name.

I get upset whenever the next version of something is entirely different in philosophy and design than its predecessors. Use spdy:// or something. You can use the HTTP upgrade mechanism, or something like STS to tell a browser that it accepts the new protocol.

I work in telecom, and this makes me think of two protocols I work a lot with, SS7 and SIP.

SIP is like HTTP, in fact I believe it was modelled after HTTP and has a lot in common with it. You can troubleshoot SIP issues very easily using a packet inspector like ngrep (using Telnet might be a bit difficult as there are some timers that expire if you don't respond fast enough, but that's besides the point).

Then there's a protocol like SS7 which is binary based. It's all structure binary bit fields. Even though it's binary based, and not very human readable, we have tools that decode the bits into a human readable format, which in turn makes it just as easy as SIP to troubleshoot.

The question I ask myself is, which one would be easier to implement, text based or binary format? I guess I'd lean towards text based--in fact I have implemented a SIP client myself with success. But have yet to implement any kind of SS7 or binary based protocols. Perhaps experience has something to do with it, would having experience implementing binary protocols make it any bit easier? And to be fair, SIP is also much well more documented then SS7. As well it's easier to test SIP.

I'm just starting to work with binary protocols, so it will be interesting how much progress I make. I realize there are also some libraries that help with working with binary/bit field data.

binary protocols can be easier to develop, because generated and parsing the protocol stream is usually straight forward: if there's a variable length field, you will typically get the length first and then the field; no need to scan byte by byte for a terminator(s) or handle escaping of terminator(s). There's often less need to be liberal in what you accept too (but that may depend on how the protocol is versioned)

But if you're developing for a protocol with garbage documentation (or no documentation), a text based protocol at least offers some amount of intrinsic documentation.

This sounds similar to an argument you could make for writing all of your code in assembly instead of C (or other higher-level languages.) Yes, there's a layer of obscurity between a program written in C and the resulting assembly: a tool (yes, a tool, spouting abstraction all over the place!) called a compiler. As a core tenet of engineering, we create tools that often leverage abstraction in order to let us tackle more complex problems without wrapping our heads around (or writing code to handle) the details.

It's a matter of balance, and we can all choose our own camps along the abstraction <=> ease-of-understanding-every-detail continuum (I'd tend to plop myself toward the former.) Just as we can create a reliable C => assembly compiler, we can create reliable tools to let us leverage abstraction for the better and focus on the bigger picture rather than the smaller.

I find your analogy curious, because I think it's backwards. If we're going to use C and assembly, the author is arguing for C: easier for humans to read, but we'll have to translate (compile) it so that the machine can read it. Assembly is easier for machines to read (well, not really), but harder for humans to read.

My "not really" bit is because the better analogy would be C to machine code.

The author's point is that HTTP/1.1 is a text-based protocol. It's inefficient that we have to translate that text into meaningful bits on either end, but it makes it easier for humans to inspect. HTTP/2.0 allows for binary communication, removing the need to translate text, but making it harder for humans to read. It's removing layers of abstraction, not adding them. The introduction is quite clear on this point:

The Hypertext Transfer Protocol (HTTP) is a wildly successful protocol. However, the HTTP/1.1 message format is optimized for implementation simplicity and accessibility, not application performance. As such it has several characteristics that have a negative overall effect on application performance.

I'm not necessarily saying I agree with the author; I don't know enough about the tradeoffs in this domain to say if the loss of abstraction is worth it. But I do think it's worth understanding his arguments on its merits.

I think we're addressing different points in the article, especially in that 'readability' and 'complexity' are distinct. Also, I think the author does make valid points, which is why I qualified that we can all choose our preferred places on that continuum of abstraction.

My point is that it is sometimes (or from my perspective, often) worth it to deal with more complex (as in higher-level) technologies via tools that allow us to de-abstract those technologies than to focus on using only technologies for which we can understand every detail of their implementation. Trying to understand assembly written by a human vs assembly spat out by a compiler (with its optimizations, etc.) is a very different beast, but thanks to the tools of higher-level programming languages and compilers we rarely need to fiddle around with machine-generated assembly.

Similarly, with reliable tools, we would not need to fiddle around with raw HTTP 2.0 workings, and thus rarely would the concern of immediate human-readability be an issue.

There's a broadly-held idea that passing around compiled images ready to be copied into memory as software is too fragile, arcane and low-level and that we should just put the human-readable source on the target machine to be consumed by the runtime.

Isn't that directly equivalent to having a preference for a textual wire protocol?

I'm not sure the two relate. Textual is still made of bits, just more of them. As a stream, they can be checked via TCP for completeness and order, iirc. The danger, I suppose, is that they might be interpreted incorrectly, but that's why you've a versioned spec. Like SSL and HTTP itself, negotiate based on what the browser says it supports and the server prefers, and that's that. Binaries for software or "golden master" images for the OS are fragile not because they're binary, but because they aren't versioned as easily and they can deviate from the golden master. While on the web, a new golden master of your app/doc is a refresh away and there is such version negotiation.

The original idea behind the web was a publishing platform that, like a classic car, would be easy to explore, maintain and develop.

The new vision is the application. Whether it's on a phone, or running as a web site, these are not publishing so much as services.

Perhaps it's time to fork protocols: one for publishing, one for interactive applications.

This has been on my mind, too, for a long time. Many applications don't fit the semantic framework of the WWW anyway; why not call it a day and fork it? This way the web app crowd will be able to do their somethingJS magic without worrying about what the WWW was meant to be, and the people who use WWW for publishing can get a sane platform back on track.

This is all very interesting...

I feel that the document-centric request-oriented nature of the web has been a powerful influence on the design of "web applications." But we still haven't really figured out how to do it.

The web has been full of applications since the beginning, no? There was no golden age of pure document sharing. So much of the web throughout its history has been CGI, ASP, JSP, ad hoc setups to provide dynamic sites. Session cookies, inscrutable URLs, forms with two dozen hidden fields encoding cryptic parameters...

(Here's a www-talk thread from 1993 discussing the new script support in NCSA httpd http://1997.webhistory.org/www.lists/www-talk.1993q4/0485.ht...)

But since the formulation of the REST architecture, it seems that more and more people are interested in what the WWW was meant to be. Maybe we're only beginning to understand the possible implications of the WWW design, and to purify that into a powerful conceptual framework.

Does that essentially involve a text-based HTTP protocol? I don't think that's the most important part.

The REST philosophy seems to say that the sane platform of the web is more crucially about clearly defined paths to typed resources providing a uniform set of actions and discoverable associations. This is not just for publishing -- what makes it so interesting and fruitful is the way it can be used to structure many kinds of applications in the form of published resources...

"Calling it a day and forking it" seems in some way to mean giving up on this fruitful encounter between two paradigms. Then you get a binary socket protocol with no resource structure but with a lot of potential for shiny stuff, versus a document-centric protocol that's slower and more restricted... And then all the big players do the shiny thing, and the document protocol is left for enthusiastic hobbyists and legacy applications...

> The web has been full of applications since the beginning, no? There was no golden age of pure document sharing. So much of the web throughout its history has been CGI, ASP, JSP, ad hoc setups to provide dynamic sites. Session cookies, inscrutable URLs, forms with two dozen hidden fields encoding cryptic parameters...

True, but many of those dynamic pages actually had semantic meaning in their structure. Think of a discussion forum, for instance. That's a case for which the WWW is perfectly fit.

> Calling it a day and forking it" seems in some way to mean giving up on this fruitful encounter between two paradigms.

On the other hand, this encounter doesn't look too fruitful in applications like Google Docs, Meebo or Prezi.

In the meantime, one side tries to push for things that are primarily relevant for non-semantic applications (think HTTP 2.0's binary protocol, which is all around HN nowadays), pissing off the people who primarily want to transfer semantic content, while the primarily semantic-oriented additions keep dragging the other side back. People have been building web apps with fancy UIs for almost a decade now, and there's still no decent UI builder to speak of.

That would be a big win if you can pull it off.

I wonder what is the best way to do it. convincing all owners of web apps to migrate to a new infrastructure strikes me as extremely hard. convincing the millions of 'publishers' of web writings (including all bloggers) to migrate sounds really hard, too.

What do you think 'native' platforms are? They're the platforms built to support app development.

While I agree that it sucks to lose text based protocols, we already have an underlying protocol that is completely binary in nature and requires tools to look at it (TCP). Same can be said for IP. We've survived a long time with those technologies and we have some awesome tools for working with them. While it's not as easy as ascii based protocols, it's more space efficient, which I'm fine with.

But one of the beautiful features of TCP is that it has the properties necessary to treat it as a text stream -- you can build text-based protocols on top of it. Since troubleshooting an application-level protocol seldom requires you to delve into the TCP-level, it works very well in practice. The HTTP 2.0 draft not only makes the application-level protocol binary, but it eschews the simpler stream metaphor in favor of replicating a bunch of that TCP-level complexity in the higher layer.

There are reasons for all of this, of course -- it's not being proposed on a lark. But it does have a very real complexity cost.

So don't support HTTP 2.0 if it doesn't benefit you and your use case. It's that simple. In a world where HTML5 says browsers need to pretend to be Netscape Navigator when asked, you'll never lose HTTP 1.x no matter what happens. Never.

> You should follow me on Twitter.

This meme needs to die. It's like ending every post with "Just kidding, I don't know what I'm talking about, I'm just cargo-culting blog attention-desperate person trying to build my Klout score to impress myself."

I don't get the fuss. HTTP 2.0 is not a deprecation of HTTP 1.1/1.

"I’ve been in this business for two decades, and the only way I know to hand-debug a SOAP transaction is with a hammer and a straitjacket2. Heck, I’ll take an old-school EDI transaction over SOAP any day."

SOAP? You lucky. It's 2013 and I'm still knee deep in RMI-IIOP sometimes. Wireshark with GIOP dissector helps, but I’ll take SOAP over CORBA any day:-).

Seriously: Your post summarized very well what I was thinking when reading the recent HTTP 2.0 posts. Full Ack.

Man, not having \n as a statement separator and having to pipe everything through xxd will definitely be a pain.

\n? or \r? or \r\n?

I think TLS/SSL is the elephant in the room here. A large part of the web is effectively already using binary protocol (HTTPS), and frankly I'd wish the part was even bigger.

Think of HTTP 2.0 as a successor for HTTPS instead of HTTP if it makes you feel better.

Wrong. An encrypted text session is different from a purely for binary session (encrypted or not). The former can be mangled from text by a generic encryption proxy, the latter needs a protocol-specific analyser.

I don't get it. Why does it matter if your tool just decrypts SSL or you use a tool that decrypts SSL and additionally unpacks the binary representation of HTTP 2.0?

The output is exactly the same - plain old HTTP.

The point is that HTTP may not be the be-all end-all protocol that we will ever want to use on the open web, and encouraging any potential new protocols to take HTTP's lead and use an arbitrary binary encoding that requires protocol-specific tools to make sense of, is maybe not a decision we should take lightly.

Binary vs. text is one dimension and it shouldn't be conflated with the overall alleged complexity of the HTTP/2.0 proposal. You can have robust and non-robust binary protocols.

I don't buy the "argument from visibility", for example.

HTTPS and DNS are both binary protocols and you need tools to parse them in order to get telnet-like visibility. Does that mean when it comes to those protocols "we are forced to rely on other tools to hide the underlying complexity and dumb things down to a point where they are manageable", as the author says?

I love these discussions about text vs. binary protocols. I especially love the part where people defend binary protocols because they are more "efficient" and don't eat as much bandwith or processing resources.

May I remind these nice people that most text-based protocols we rely on today were designed in a time when CPUs were orders of magnitude slower and modems ran at 2400 baud? Talk about processing and bandwidth resources...

The resources consumed by having to interpret a text-based protocol pale in comparison with the resources consumed by all the levels of abstraction that modern frameworks/language runtimes that we use today. But wait, these save developer time, so they are a good thing. So does having protocols that can be troubleshooted by a human with minimal tools.

It's 2013, not 1973, so I'd replace 7-bit with UTF-8 which is its natural extension.

In theory I agree.

"In theory, theory and practice are the same. In practice they are not" :(

But UTF-8 is too complicated for many shells! ;)

You may find this thing that we're working on relevant:

https://github.com/nudgepad/space

It's a very understandable and very powerful language that I think will at some point be extended to replace HTTP, amongst other things.

Amen, OP.

herge: You could also use the "data" program from Hobbit's netcat.

Marco claims that HTTP 2.0 is overly complex, but never gives any reasons why. He claims that text protocols are better than binary, but never really gives any reason for that either. Nostalgia for the days of telnet and gopher is not a reason.

Then he goes off to talk about SOAP, which he hates, because "the only way [he] know[s] to hand-debug a SOAP transaction is with a hammer and a straitjacket." But SOAP is text based, which directly contradicts his argument from earlier.

The other allegation here is that the new protocol "satisf[ies] what are essentially the needs of a small group of very influential players." Again, Marco just states this with no rationale at all. In fact, the opposite is true. By being more efficient, HTTP 2.0 will serve the needs of everyone. Should startups have to pay for 10 servers when they could have paid for 5 with HTTP 2.0? Apparently Marco thinks they should. It's easier for Google to pay for a few more servers than it is for Joe startup. This should be obvious to everyone, but apparently it's not.

The whole binary versus text thing is a complete red herring. If you have ever used wireshark, you know that it makes binary fields easy to read. Binary protocols are easier to develop and standardize. Pretty much every good developer realizes this, and the responses here reflect that. The idea that making something LOOK like (but not actually be) "human text" makes anything better is an idea we should have buried along with COBOL and other mistakes from the past. But I guess not.

SOAP is to REST as XML is to JSON. They're all technically 7-bit and notionally human-readable, but SOAP/XML are (well, can be) so verbose as to be near-impossible to parse by hand. Comparing the "Hello World"-style simple examples on Wikipedia is instructive:

REST

http://example.com/resources/item17

SOAP

<?xml version="1.0"?> <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"> <soap:Header> </soap:Header> <soap:Body> <m:GetStockPrice xmlns:m="http://www.example.org/stock"> <m:StockName>IBM</m:StockName> </m:GetStockPrice> </soap:Body> </soap:Envelope>

As someone who has been having to work with various SOAP APIs over the last month - amen.

Amen. "Human readable" is not the same thing as human readable.

As I said earlier: The idea that making something LOOK like (but not actually be) "human text" makes anything better is an idea we should have buried along with COBOL and other mistakes from the past. But I guess not.

The site crashes my phone browser.

     Dolphin Mini.
     HTC Wildfire.
     Android 2.2.2

I guess I'll check it out later.

Reply on news.ycombinator.com