Identifying QUIC deliverables
mailarchive.ietf.org
mailarchive.ietf.org
Well, yeah, they have the same name. My first reaction to the headline was that lots of software isn't ready for http over udp.
As for the HTTP over UDP, indeed - there is a reason Chrome has the ability to race a TCP connection alongside QUIC opens :).
Everything on the web about QUIC is about UDP. It doesn't talk about HTTP/3 and /2 difference. /2 is already complex enough as it is.
Rebinding is an example of the latter. When you change networks (and thus addresses) in QUIC it can update the ongoing connection seamlessly.
HTTP-over-QUIC [2] specifies how to talk HTTP 2 over the QUIC protocol, with some adjustments like using "native" QUIC streams instead of multiplexing streams over a TCP stream, and adjusting the header compression to allow more out-of-order operations.
[1] https://datatracker.ietf.org/doc/draft-ietf-quic-transport/
This sounds a fascinating claim — is the idea that the programming model when implementing QUIC server applications will allow one to easily aggregate distinct server endpoints which automatically share or amortize the overhead of establishing encryption and congestion detection between all the logical streams used by the application across all the involved nodes?
This is a common way to describe congestion control which is a bit simple and reductive.
Given 2 endpoints on the network, you want to compute a congestion window for the entire flow rather than per-stream. Because 1) You can more accurately compute a single congestion window given aggregate stats from N streams rather than computing N congestion windows given stats from 1 stream each. 2) The congestion window is a shared resource across streams, which permits better network utilization
The other benefit of QUIC, coming from multiplexing streams over UDP rather than TCP, is that the "blast radius" arising from the consequences of a lost packet is better contained. It only impacts the stream that contained the packet; the rest of the flow can make progress uninhibited. So head-of-line blocking is not as severe.
Though maybe it requires way more future vision and planning then I expect.
on http2/spdy, they at least had the decency to publish something before forcing everyone into adoption
A few years ago, web sites and infrastructure had to be built to serve decade-old versions. Now it's reasonable to support only a few years-old versions, and this time frame is continuously shrinking as the current generation of evergreen auto-updating browsers takes more market share.
Though I don't like the bit about forcing adoption onto others, I do think that advancing the market to allow standards changes like this to happen in years instead of decades is a good thing.
they all already have ways to release security patches. firefox has a semi-secret/invisible system addon for that. so much that this is still the current way to release security patches and fix internet-breaking bugs.
rapid release for new features eas invented by microsoft with IE, to push the agenda on standards bodies. it moved so fast that nonody even remember IE5 or 5.5... and thats the reason google have rapid release. nothing to do with security, as proved on the 1st paragraph.
I'd also like to see updates to IMAP, SMTP and FTP to name a few.
Ftp is surprisingly common when integrating between parties, and this can be expensive to change for non technical reasons.
OTOH, AFAICT, SFTP has largely replaced FTP for the system-to-system role (though ad hoc HTTP-based protocols are also common.)
When users found out about all the issues, they preferred sticking to the old FTP, having a desktop sync app (eg. Google Drive), or using a WebUI for managing files (eg. Dropbox's website) etc.
The problem isn't that there are no standardized way of doing it. The problem is that the existing standardized way of doing it has no adoption, because it's not really optimal for current needs.
The "sandboxing" you're referring to is a serverside chroot, for what it's worth. And, of course, web servers have been doing that since NCSA httpd.
It was also possible for the client and server to fork off data handling processes for the data channel and continue to use the control connection without needing async poll/select loops.
Anyway, it wasn't insane. At the time anyway.
There is a well-established and widely-support IETF standard mechanism (consisting of several IETF standards) for that; it's called WebDAV.
> S3 is probably the closest thing to replacing FTP,
SFTP is much closer to replacing FTP, to the point that people often say FTP when they mean SFTP, which underlies most non-HTTPS enterprise integrations I've seen in the last decade. But SFTP isn't FTP (not even FTP-over-TLS, which is FTPS, which has much less use.)
> Ftp is surprisingly common when integrating between parties
SFTP is very common. FTP (including FTPS) is surprisingly common in the sense that any use of it is surprising given the well-established, battle tested, and superior in every way alternatives that are readily available.
No it doesn't. Don't fix what's not broke, and doubly so if the only motivation for the change is "well, Megacorp (c) says it's the bee's knees".
Saying that anything has no drawbacks is pretty much always going too far.
Yeah, the world's communication before the Internet existed wasn't exactly broken either, so let's not invent anything? Poor choice.
This is also why these platforms often use FTPS (FTP over TLS) instead of SFTP (SSH-based) – SFTP doesn't include any support for record-oriented files, only the stream-oriented files used on Unix and Windows.
Or you could just provide a simple API over HTTP, whether an actual REST system or a single endpoint one or two params with well defined inputs that can be accepted (a CGI, basically). Why bother formalizing some standard when the tools to handle this are so ubiquitous (Apache+$LANG on the server, cURL or wget on the client)?
> SFTP doesn't include any support for record-oriented files, only the stream-oriented files used on Unix and Windows.
That's because SFTP isn't really FTP (in the protocol sense) at all. It's just a specialized shell started after an SSH session/tunnel is created.
That it includes FTP in the name is really just marketing because they wanted to supersede the real FTP. In that respect, it makes sense for them to just cover what 99% of the users of FTP needed and stop.
Yes, I was assuming a more traditional client, which is probably a mistake on my part.
> What possible advantage could one get by replacing it with something hacked together with Apache/curl/wget?
Well, there's always the obvious one, which is firewalls are much easier to deal with, since there aren't two separate ports in use, so you get rid of a whole class of network and firewall errors that are quite common.
If possible, one should use extended passive mode (EPSV) over TLS. Then, we are just talking about two ports instead of one, without any of the connection tracking complexity on middle-boxes that active mode or non-extended passive mode can require (such as rewriting the IP address in the PASV command response to implement NAT). And then, you have to wonder, if you have to substantially change the software in use at the client and server (and possibly even write custom code, per your Apache+$LANG suggestion), are those significant changes really worth it just to save on one extra port open on the firewall?
Oh, sorry, I didn't make it clear before. I'm fully admitting that in the case of record-oriented file requirements, especially on the client side, FTPS probably doesn't have a better solution available.
I was just noting that a vastly simplified firewall configuration that both doesn't require multiple ports opened (or to somehow read the PORT/PASV or EPRT/EPSV commands and open ports dynamically) is simpler, which is an advantage. It doesn't necessarily make up to the record-oriented storage problems that would have to be dealt with in those cases though.
Now, I'm not familiar enough with either of those in such detail to be able to say which one is more complicated, but I have a suspicion.
SFTP also provides what is in effect a shell (or, charitably, a remote filesystem), baked into the protocol. For example, SFTP has SETSTAT, which lets you set file attributes. None of what SFTP allows you to do is particularly difficult in HTTP (some of them are made harder by browsers, but that argument doesn't cary any weight since SFTP already requires you to use something other than a browser), but at the same time, none of them need to be baked into a file transfer protocol, either.
But designing a HTTP API to implement the function that FTP gives you sounds like madness.
In your (Matasano/Latacora) experience, does SFTP register as a security risk / exposure, or does it check the box as long as they aren't using FTP/FTPS?
I'm mostly thinking of Erlang's `ssh_sftpd` application here, where your application can "drive" it by exposing a "file server" for it to talk to as if it was the filesystem.
Sometimes security doesn't matter, though.
Picture a "web-app" like an APT package repo host. (I know the reference impls of ALT hosts are essentially static-site-generators, but ignore that, and imagine an impl that's a web-app with dynamic business logic to expose the relevant routes.)
Such a web-app can be served over plain HTTP. Not even an MITM can exploit an APT client, since it looks for specific files on the server and then checks those files' signatures against a local GPG database.
What would be wrong with having this web-app also be an "SFTP-app"? Or an "FTP-app"? Or even a "telnet-app"?
In all these cases, it'd just be a virtual filesystem being exposed over a wire protocol where the security of the wire protocol is immaterial.
That's more in line with what I was picturing, here. You know how (public) S3 files can be retrieved over HTTP, but also over BitTorrent? Picture adding SFTP to that, as another option. That kind of API.
---
For the situations where the service would benefit from security, though, you can still just obviate the need for relying on SSH's security specifically.
For example, you can expose your SSH server only over a VPN. (I know a lot of people do this, though I personally find it to be more of a hassle than just correctly configuring SSH.)
Or you can expose your SSH server behind an stunnel instance, thereby reducing your configuration work down to just "setting up a TLS-terminating load-balancer", that can then be shared between HTTPS and SSH traffic.
---
Or, if it's just that you don't trust the Erlang SSH impl, you could just hack up OpenSSH to act as an "SSH terminator", forwarding the SSH-channel flows out of each SSH connection, as regular (auth-headered) TCP connections to your backend. Sort of like an fCGI server for SSH. I'm pretty sure the only reason nobody has done this yet is that no language (other than Erlang) has batteries-included support for speaking the server-side of SFTP.
Our SFTP server is written in go and exposes a VFS that has nothing to do with anything on the local file system. It doesn't even allow you to download the files you just uploaded in the same session.
Of course things like the recent libssh CVEs should make anyone exposing non-standard ssh services to the internet a bit nervous.
The biggest problem with FTP[S] is that it interacts poorly with NAT. So maybe the problem is IPv4 and the solution is IPv6.
It also interacts way too well with the NSA and ISP eavesdropping.
Passive mode FTP works fine with firewalls that allow outgoing connections. Even active mode can work using something like Port Control Protocol.
It's only a problem when the network is tightly locked down. And when everyone's answer to that is to use TLS/443 for everything, how is that better than just leaving the unprivileged outgoing ports open? You end up allowing everything either way, but with the everything-over-port-443 outcome you can't even selectively block things anymore.
> It also interacts way too well with the NSA and ISP eavesdropping.
You could say the same about HTTP, which is why they both support TLS.
- Marcus Ranum
I would really like to see each of these disappear to be replaced by new ways of doing things. The whole concept of e-mail needs a new thought. FTP should have already gone the way of the dodo if for nothing else the firewall issues its has.
FTP I agree - SFTP is an existing better option.
It may look like it's working from users' point of view but check how encoding is handled, it's a mess and then why is there still no widely deployed end to end encryption for such a core protocol?
Also DNS and domain is broken too. There's no point using UDP which only brings security problems and still no decent way to encrypt which domain you're querying through your browser and see how domain is governed by a structure which make crappy rules like who owns new TLD and you get some absurd "base" price for each domains instead of something more transparent and public. I wish if you write a book, you could easily have 'some-title.book' or when releasing a movie, you could get 'some-title.movie' but somehow it went wrong and no one can fix it.
Lack of improvements on the security on these points make me wonder if agencies just want to keep them dated so they can tap into people's communication easily instead of making the world a better place where everything is p2p encrypted.
For hiding hostnames eSNI is under development. Cloudflare with a recent Firefox nightly lets you see this for yourself.
(That said, DoH solves a very important, but different problem that is: I want to provide a service with authenticity assurances, but DNSSEC is broken for the users, what can I do? Not many people seem interested on providing that kind of service.)
(Email could use a better binary encoding for attachments than base64, though, since transports are basically 100% 8-bit safe, even if not binary safe. Usenet went with yEnc, which IETF balked at in what is a case of perfect being an enemy of the good).
I have seen plenty of web sites broken by it too, and it's a problem when moving files between Linux and Windows.
(And yes, things would be better if people standardized on binarybody instead of 7bitmime. unfortunately, the Microsoft server announces its support everywhere, but it's broken, so nobody can rely on that one extension (an inside parenthesis, there is a work-around that works everywhere, but it goes against the standard).)
Perfectionism & cert vendors / CA mafia.
I have been playing with workflows lately (in relation to agents) and wonder if that would be an e-mail replacement.
As a website operator, the hard part of HTTP/2 was getting TLS certificiates. As an end-user, I'm just happy that popular web pages load faster. And if most of the world still uses HTTP 1.1 I won’t care.
Making a datagram-based protocol available will open up a ton of possibilities for cool realtime apps over HTTP. Currently you’re stuck with WebSockets or WebRTC and for some applications these solutions aren’t good enough.
I can't think of many network applications you couldn't build on top of them.
While a lot of the modern web is great, it also has drawbacks that should be looked at very carefully: A simple HTTP page from yonder can still be rendered just fine. A modern page can't be rendered properly a year or so down the line. We need modern protocols to solve modern problems we've created such as huge pages with tons of links and back and forth traffic to many servers. While I think work needs to be done in both directions, I don't know what we really need to go so far as to push down yet another HTTP/x protocol this soon. HTTP/1 is going to be used for quite a while still, HTTP/2 solves a lot, and where it matters (ie: dealing with specialized servers) QUIC is already and has been in use.
Examples?
What I would E.g. agree upon is that http2 is a pain for exotic systems or E.g. microcontrollers, due to the window size and buffering requirements that the protocol imposes. But those systems are still free to stay with HTTP/1.1 for as long as they like. Nobody is going to remove support for it anytime soon.
One can create a non-conforming server in a few lines of code. The actual protocol has a horrendous number of corner cases and weirdness. E.g., determining a message body's size depends on whether content-length is sent, is it chunked, was the request a HEAD request, etc. There's also header folding (whereby a header can be split over multiple lines) though one has the out that the standard allows that to be optional (you can respond w/ 400). Trailers (additional headers after the body) and continuations (allows the client to request a potentially large upload, and the server can reject it prior to the client transmitting the body). Actually parsing headers beyond just a KV mapping, that is, trying to work with any of the actual data in a header is fraught with error: many of them can be specified multiple times, and can thus be in multiple occurrences of the header, or all in a single header but separated by commas, or some mix of the two. (Except cookie headers, those are special.) Some also allow a comma to appear in the value itself, meaning a parser that first splits on comma and then parses the resulting parts is broken.
These text protocols, while "simple" in the sense that a human can generally look at the on-the-wire data and parse with their eyeballs, require complex FSMs to parse correctly. (Complex, relative to an equivalent binary protocol with binary-encoded length-prefixed data or even a text protocol that didn't accommodate every random slightly different way of doing it.) Now, HTTP/2 also adds a lot of functionality above and beyond what HTTP/1 has, so that certainly eats up some of the gains in parsing simplicity (and I think it would be fair to say that the outcome is overall more complex).
I'd much rather have a binary protocol and good tooling. (And that's much more feasible with today's browser's devtools. I would not want to attempt this with the lame excuse for tooling IE 6 had back in the 90s.)
MIME's header encoding of non-ASCII characters is another abomination that I'd toss in this bucket. E.g.,
Subject: =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=
=?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=
Historically, HTTP has allowed field content with text in the
ISO-8859-1 charset [ISO-8859-1], supporting other charsets only
through use of [RFC2047] encoding. In practice, most HTTP header
field values use only a subset of the US-ASCII charset [USASCII].
Newly defined header fields SHOULD limit their field values to
US-ASCII octets. A recipient SHOULD treat other octets in field
content (obs-text) as opaque data.DNS zone files for example are specified incredibly poorly despite their apparent simplicity (multiple optional ambiguous fields, significant whitespace, poorly specified line-folding and paren semantics, DNS record-type dependent record value grammars, vaguely specified length limits, lexical includes, embedded parsing directives, it goes on...). You won't find any 2 implementations that behave the same.
It's typical to talk to engineers who haven't been through serious parser work and get responses like "well, it's just splitting a few strings isn't it?"
The biggest deliverable (to me at least) of QUIC is the formalism of a "session" layer which is transport binding agnostic: If you move to alternate underlying IP addresses, it can recover and continue.
Sessions were big in OSI. Internet applications/protocol stack basically ignored them, for reasons of NIH, complexity, place-and-time. But, Session layer is inherently useful
UDP? Schmoo Dee Pee. What matters to me, is a session layer with cryptography.
(ok. taste-testing the underlying fragmentation barriers, which was in an earlier QUIC, that was interesting because pMTU is broken)
That said, it doesn't seem like a good idea to me, since it's less of a clear protocol evolution. A variation of the http/2 name would IMHO make more sense, e.g. http/2q or something, since it's not changing the HTTP semantics.
HTTP/2 didn't change semantics, either. We literally only have one version of HTTP semantics, and a variety of transports for it.
Why? The core of HTTP/2 is defining how individual HTTP streams get multiplexed across a single TCP connection. That part is not applicable to QUIC over HTTP. There HTTP streams get mapped on top of QUIC streams. Some things might be similar between HTTP/2 and HTTP/QUIC mappings, but in the end they are separate definitions. The both define how HTTP semantics are carried over a different lower layer mechanism. So HTTP/3 or HTTP/QUIC seem reasonable.
I think it's funny that the Q in QUIC stands for Quick. Naming things is hard.
Some examples:
* GNU: GNU's Not Unix
* cURL: Curl URL Request Library
* WINE: WINE Is Not an Emulator
PHP3 was the first "real" version of PHP, it came out in 97, and wasn't called personal home page tools then. O'Reilly didn't have a PHP book until 99 if I remember rightly.
looks like they forced the Internet word in there to make it work...
maybe they need to make it work first
https://cloudplatform.googleblog.com/2018/06/Introducing-QUI...
Adoption of cloud gaming is accelerating. And we are already starting to see custom protocols in the wild
https://blog.parsecgaming.com/a-networking-protocol-built-fo...
Actually doesn't webRTC already have unreliable channels?
WebRTC has unreliable data channels, but they are based on sctp/dtls/udp. Making them work over quic streams might be possible, but also requires a new standardization effort.
If someone does it I would actually prefer an API outside of webrtc, since that one carries a lot of complexity for signaling and requires again a few extra protocols (ice, stun, turn, etc). An unreliable/unordered client to server protocol and api could potentially be much simpler.
The current focus right now is to deliver WebRTC 1.0, when this is done, we may reconsider it.
(I'm an editor of the document and am in favor of it).
Whats the benefit?
Surely we have an opportunity to split HTTP into a file delivery system and a socket-like streaming system?
Keep the web frontend culture contained where it is now, don't let them ruin everything else too.
If somebody still sold HTTP servers, I would look how this one was bribing the standardizing process. But it looks like people are doing it freely.
Really? I thought it was because they don't plan to release another major version of windows, and wanted the number to have a nice solid "10", rather than a "9"
To revert to the actual topic, to defend against this type of thing the QUIC binding for HTTP assigns an arbitrary non-contiguous group of identifiers as reserved to be ignored in various places, with the intent that some real systems will prod these once in a while, to discourage idiots hard-coding checks that will become obsolete.
I have seen code in the 2010s that did explicitly check for "Windows 9" as a means to check if it was running on Windows 95 or 98. That doesn't mean that the code would actually work on those systems (it could well be a prelude to saying "your system is too ancient, we don't support you"). But such kind of cruft tends to last a very, very long time without active maintenance.
Windows is already full of version APIs that lie to old programs.
It could also format the name differently. And people like to quote broken java code, but old java would just say "Windows NT (unknown)".
As a technical issue, it was small and easily avoided.
It was a marketing decision.
startsWith('windows 9') was never an issue
You can't avoid the one round trip on first connections, you pay that in QUIC too.
There is also a proposal to allow connection sharing for different domains, which would reduce the connection time to 0-rtt even if the user has never visited your site.
But right now, http2 adds complexity, a lot of frameworks are not compatible with it and must be fronted by a proxy that does, and the perf benefit is not obvious at all.
This doesn't matter for small sites, or for clients. But when you're serving 190Gb/s from a single box, paying 4x per byte really hurts.
Do you have specifics?
> It's about having a huge head on the on all the important internet standards and pushing their cloud interests
Is Google not supposed to be trying to improve the services they offer?
> It designed to work well at their scale on their network you connect to directly.
I don't see any reason why QUIC won't be perfectly usable on most networks and that is what the article was about. I'm not sure what standard Google is spearheading which is only going to be useful on networks their size and will someone harm other networks.
> The standard group should taking into small and medium provider but there not really.
Again, what is an example of this?
> Who has the resources to follow Google's multi-azimuth "standardisation" pace ?
I really don't see what the issue here is - Google is spearheading new standards. In doing so, it's working with the IETF to develop those standards. I'm not aware of any indication that its working in bad faith or subverting IETF processes. The IETF process is fairly open. No one can follow everything happening in technology all the time - too much is going on. It seems like you are saying that Google should stop or artificially slow its work with standards bodies so that outside observers unaffiliated with those bodies can follow more closely?
I assume the QUIC protocol also comes with some "happy accidents" that allows Google to more easily collect data on us, too.
This is just conspiracy theory nonsense. If QUIC has such a flaw, it has an open spec and you are free to go find it and point it out.
As for the happy accidents, none have been spotted so far, it is an open and collaborative project so feel free to check it yourself and chime in before its finalization.