Network protocols, sans I/O
sans-io.readthedocs.io
sans-io.readthedocs.io
May I dream that this + uvloop + asyncpg [0] and we can start to have something to build simple web services à la tornado that leverage python3.5?
[0] http://magic.io/blog/asyncpg-1m-rows-from-postgres-to-python...
I imagine it might be difficult sometimes depending on the protocol. IIRC TLS might sometimes require writes while reading.
OpenSSL exposes a "memory BIO", which is basically exactly this. You send in writes, and then emit data when you can, and in the meantime it buffers.
It's a bit more limited than that, but generally speaking it's not too bad.
Python has some of javas thinking through a standard convention on the method names for reading and writing (an interface!), but it seems common with departures from those simple rules when people get creative. From that perspective, an initiative like this is quite understandable and convenient. Nevertheless, I don't seem to feel the need in the same way in certain other languages.
From my Python years, I'd think that something built around mandatory read()/write() methods could do much the same.
But a File has no "send" and "recv". io.Reader and Writer apply to files, sockets, byte slices, strings, compositions of other Readers or Writers, HTTP request and response bodies, and anything else that strikes my fancy to implement the correct methods. Sockets appear to deliberately not have those methods, as there's this line in the docs: "Note that there are no methods read() or write(); use recv() and send() without flags argument instead."
There's nothing preventing Python from having the equivalent of Reader and Writer. It just doesn't, at least not with the standard library. In fact that's true of a lot of languages; there's nothing preventing that interface from existing, it just doesn't. This is a point in favor of getting the standard library right earlier rather than a point in favor of Go-the-language.
So again let me underline what I said before, that this is more a "standard library" problem than a language problem. Conceivably, Python could adopt Readers/Writers into the standard library, but it's late now. Python 2.0 didn't have the requisite stuff in the library to make the idea work, so modern-day penetration would be low. Go and other recent languages benefit from experience, and by putting these things in correctly from day one, can move on to the next problems we'll discover in standard libraries. :)
Sometimes people ask why we need new languages, and I think "get a fresh start on a standard library" is at least halfway to a valid answer. Imagine if Java the language stayed exactly the same, but we could reach into an alternate universe where a brand new batteries-included standard library was written based on the language as it is now. It would still be Java, but it would be a much better Java, with better interoperability between many currently-separate worlds through standardized, sensible interfaces and extension points, taking full advantage of lambdas, probably dropping the stupid synchronization stuff even if it is nominally in the language, etc. I think the end result is still something I wouldn't love, but it would be much better. But one can only dream about that.
Whether Python should hide that distinction is not clear, but it could.
The reason for this is basically that I/O and protocol logic are separate concerns, and whenever they start influencing each other too much they impose costs on each other.
The best example is actually testing. If your protocol code includes calls to Golang's `Reader`/`Writer` interface methods, it causes a few problems.
The easiest thing to see is that it causes testing problems. For example, for each call to `Reader`/`Writer` methods, in addition to testing all possible reads/writes (a protocol concern), you need to test all possible I/O failures (timeouts, closed connections, weird kernel problems) in order to actually cover the complete failure space.
However, if your code doesn't have reads/writes mixed in with protocol logic, your testing scenarios are much easier. Bytes just come in and go out. Reading/writing problems aren't an issue.
This is just basic separation of concerns stuff, but it really does help, even in languages with "blessed" I/O mechanisms.
Why not test with something that implements the reader or writer interface, but shuffles bytes around in memory? That should alleviate the testing explosion.
And whatever interface you do have must be putting bytes in our getting bytes out of the protocol layers. Why not call that interface reader or writer?
I'm not seeing the distinction.
The reason that just having an in-memory Reader or Writer doesn't solve the problem is that the failure modes don't match up. An in-memory reader/writer has basically no failure modes beyond ENOMEM. That's why in the no-I/O implementation, this is exactly what we use: write to an in-memory buffer.
Real I/O on the other hand has many failure modes. For an example, consider timeouts. If your parser does I/O, you need to test timeouts at every location that your parser does I/O. You need to confirm it handles those timeouts appropriately. And you need to decide what "appropriately" means here: do you retry? Do you abort? Do you attempt to unwind that state transition?
All of these are expansions of your state space. This means your protocol parser has to handle this combinatorial explosion of possible outcomes: at every point you have a Read/Write you need to be ready and prepared to handle all possible error conditions that can come out of that.
If your parser does no I/O, though, and only writes to buffers, this problem does not exist. That allows you to have two totally isolated sections of code: one part manipulates bytes in memory (the parser), and another bit is responsible for getting those bytes to and from the network. Each can be tested separately. If we need `n` tests for the no-I/O parser, and `m` tests for the I/O without parser, then to achieve equivalent test coverage your combined code requires `n * m` tests to achieve equivalent logical coverage of the possibility space.
Small, isolated components are good.
Because testing with a reader and writer interface lets you test errors too, but now you are talking about error recovery strategies (pull has to pass through or have smarts, push can know nothing).
I agree in many cases push has the advantages being discussed. I just wouldn't have called it no-I/O since that doesn't really have the right connotation.
The only real tip i have is to byte length encode variable length requests, because scanning strings sucks.
having a frame for messages for messages is nice, stick your fixed with stuff up front, and create fixed with length indicators. It sucks that you waste a little space with 0's, but avoiding the scan is pretty wonderful.
GET /foo HTTP/1.1
GET HTTP 01.10 0004/foo
Traditionally you'd have something like
HELO jfoutz hunter2
OK
REQEUEST 1
OK <bytes for resource 1>
REQUEST 2
OK <bytes for resource 2>
HELO jfoutz hunter2
REQUEST 1
REQUEST 1
OK
OK <bytes for resource 1>
OK <bytes for resource 2>
NO
NO
NO
This is really, really good advice to use whenever possible: it means clients can determine apriori how much data they need to read, and perhaps decide whether the length is valid and/or allocate sufficient memory. One of my annoyances with the traditional "almost-text-based" protocols like HTTP, FTP, SMTP, etc. is that parsing them is not trivial and often you need to keep reading until you hit the delimiter or reach an internal limit. In contrast, "read a length, then read length bytes" makes for simple and efficient implementation.
There are certainly cases when the amount of data cannot be determined ahead of time, and in those cases I'd suggest chunked-length-prefixing; delimiters are really a method of last resort.
There is a lot of "philosophy" about protocol design, which seems mostly sound. Although I wonder what happened to the protocol mentioned (BXXP or BEEP). I think HTTP was "good enough" as a universal application protocol?
This one also talks about HTTP being wildly successful beyond its original design parameters:
It'd be good to know what additional information you'd like on the "how". I think at a high level I addressed that in my talk, but if it didn't make it across or you needed more information I'd love to know what you need. Ideally I'll turn this into a blog post at some point so it'd be great to have an idea of what extra info is needed.
This might just be a symptom of so few libraries following this pattern. I'd love to see some specific examples of handling various kinds of protocols (not just HTTP) with this approach, to see how it addresses various kinds of protocol components. For instance: variable-length data structures with length prefixes, variable-length data structures with "number of elements in the following array" in the middle, variable-length data structures with a terminator, text structures that require parsing tokens, and so on. Right now, the main documentation for those kinds of patterns seems to be "hope HTTP has a similar pattern and read the corresponding code in h2 or h11".
I'd also love to see some reusable components that make it easier to build such protocol libraries.
In Python-land this is all pretty easy. For example, HTTP/2 is a protocol of the first kind ("variable-length data structures with length prefixes") at the framing layer, which is implemented in a Python packager called hyperframe. This uses a combination of the `struct` module and bytestring operations to achieve its results. A similar approach works for the second kind as well.
Basically, in Python this is almost always much easier because struct sizing and memory allocation isn't a concern like it is in a C-like language (though even there, dynamically sized structures and pointers are your friends).
But I agree, there is a lack of good discussion about "how do I actually do this?" I'd like to elaborate on that at some point for sure, because the reality is that it's remarkably simple.
I definitely don't want C anywhere near parsers for untrusted data, for so many reasons, this among them.
> But I agree, there is a lack of good discussion about "how do I actually do this?" I'd like to elaborate on that at some point for sure, because the reality is that it's remarkably simple.
Perhaps it would help to have some worked examples for some additional protocols?
Would you be interested in collaborating on a Python parser for some non-trivial data structures? I have a collection of such parsers as part of BITS (https://biosbits.org/) that really need reworking to decouple them from I/O, and I suspect the result would make a good article and/or conference talk.
There is also Construct which has a different syntax but is similar in many ways: http://construct.readthedocs.io/en/latest/index.html
Both suitcase/construct are definitely better suited for parsing binary protocols -- In my line of work, that limitation hasn't been a deal breaker. With suitcase, at least, I haven't done much work to optimize performance (mostly because if I cared, I wouldn't be using Python).