QUIC is not quick enough over fast internet
arxiv.org
arxiv.org
We had instant internet in the late 90s, if you were lucky enough to have a fast connection. The pages were small and there were barely any javascript. You can still find such fast loading lightweight pages today and the experience is almost surreal.
It feels like the page has completely loaded before you even released the mousebutton.
If only the user experience were better it might have been tolerable but we didn't get that either.
It's a blast. It's faster and way more resilient. No more state desync between frontend and backend.
I admit there is a minimum of javascript (currently a few hundred lines) for convenience.
I'll add a bit more to add the illusion this is still a SPA.
I'll kill about 40k lines of React that way and about 20k lines of Kotlin.
I'll have to rewrite about 30k lines of backend code though.
Still, I love it.
But if your site works fast. Loads fast. With _a little_ JS that actually improves the functionality+usability in? I think that's completely fine. Minimal JS for the win.
I want the basic functionality to work without JS.
But we have a working application and users are not hating it and used to it.
We rely on modals heavily. And for that I added (custom) JS. It's way simpler than alternatives and some things we do are not even possible without JS/WASM (via JS apis to manipulate the DOM) today.
I am pragmatic.
But as you mentioned it, personally I also use NoScript a lot and if a site refuses to load without JS it's a hard sell to me if I don't know it already.
1. filtering a drop down list by typing rather than scrolling through lots of options to pick one 2. Rearranging items with drag and drop
The excessive stuff is requiring a whole lot of scripts and resources to load before you display a simple page of information.
A better example would be dynamically loading the list of options where it is very long and loading the entire list would make the page size much larger.
In a huge majority of cases I come across that is on the server. Some things really are client-side only though, think temporary state responding to user interactions.
Either way I also try really hard to make sure the UI is at least functional without JS. There are times that isn't possible, but those are pretty rare in my experience.
Even on the backend, now the golden goose is to sell microservices, via headless SaaS products connected via APIs, that certainly is going to perform.
However if those are the shovels people are going to buy, then those are the ones we have to stockpile, so is the IT world.
There are still good uses for micro services. Specific services can gain a lot from it, the list of those types of services/apps is pretty short in my experience though.
I got over this view and just finished the new version of my page. Raw HTML with some static-site-generator templating. The HTML size went down 90%, the JS usage went down 97% and build time is now 2s instead of 20s. The user experience is better and i get 30% more hits since the new version.
The web could be so nice of we used less of it.
SPA frameworks like Vue, React and Angular are ideal for web apps. Web apps and web sites are very different. For web apps, initial page load doesn't matter a lot and business requirements are often complex. For websites it's exactly the opposite. So if all you need is a static website with little to no interactivity, why did you choose a framework?
I look at this article and consider the result pretty much as expected. Why? Because it pushes the flow control out of the kernel (and possibly network adapters) into userspace. TCP has flow-control and sequencing. QUICK makes you manage that yourself (sort of).
Now there can be good reasons to do that. TCP congestion control is famously out-of-date with modern connection speeds, leading to newer algorithms like BRR [1] but it comes at a cost.
But here's my biggest takeaway from all that and it's something so rarely accounted for in network testing, testing Web applications and so on: latency.
Anyone who lives in Asia or Australia should relate to this. 100ms RTT latency can be devastating. It can take something that is completely responsive to utterly unusable. It slows down the bandwidth a connection can support (because of the windows) and make it less responsive to errors and congestion control efforts (both up and down).
I would strongly urge anyone testing a network or Web application to run tests where they randomly add 100ms to the latency [2].
My point in bringing this up is that the overhead of QUIC may not practically matter because your effective bandwidth over a single TCP connection (or QUICK stream) may be MUCH lower than your actual raw bandwidth. Put another way, 45% extra data may still be a win because managing your own congestion control might give you higher effective speed over between two parties.
[1]: https://atoonk.medium.com/tcp-bbr-exploring-tcp-congestion-c...
[2]: https://bencane.com/simulating-network-latency-for-testing-i...
It seems like it should never be the case that two parallel downloads will preform better than a single one to the same host.
CUBIC tries to increase bandwidth until it hits packet loss, then cuts bandwidth (to drain buffers a bit) and ramps up and hangs around close to the rate that led to loss, before it tries sending at a higher rate and filling up buffers again. Cubic is very sensitive to packet loss, which makes things particularly difficult on very high bandwidth links with moderate latency as you need very low rates of (non-congestion-related) loss to get that bandwidth.
BBR tries to do the thing you describe while also modelling buffers and trying to keep them empty. It goes through a cycle of sending at the estimated bandwidth, sending at a lower rate to see if buffers got full, and sending at a higher rate to see if that’s possible, and the second step can be somewhat harmful if you don’t need the advantages of BBR.
I think the main thing that tends to prevent the thing you talk about is flow control rather than congestion control. In particular, the sender needs a sufficiently large send buffer to store all unacked data (which can be a lot due to various kinds of ack-delaying) in case it needs to resend packets, and if you need to resend some then your send buffer would need to be twice as large to keep going. On the receive size, you need big enough buffers to be able to fill up those buffers from the network while waiting for an earlier packet to be retransmitted.
On a high-latency fast connection, those buffers need to be big to get full bandwidth, and that requires (a) growing a lot, which can take a lot of round-trips, and (b) being allowed by the operating system to grow big enough.
IIRC, servers were 64GB of ram and sendbufs were capped at 2MB. I was also dealing with a kernel deficiency that would leave the sendbuf allocated if the client disappeared in LAST_ACK. (This stems from a deficiency in the state description from the 1981 rfc written before my birth)
That’s not an inherent property of the QUIC protocol, it is just an implementation decision - one that was very necessary for QUIC to get off the ground, but now it exists, maybe it should be revisited? There is no technical obstacle to implementing QUIC in the kernel, and if the performance benefits are significant, almost surely someone is going to do it sooner or later.
Windows already has an in-kernel QUIC implementation (msquic.sys), used for SMB/CIFS and in-kernel HTTP. I don’t think it is accessible from user-space - I believe user-space code uses a separate copy of the same QUIC stack that runs in user-space (msquic.dll), but there is no reason in-principle why Microsoft couldn’t expose the kernel-mode implementation to user space
(And if someone can somehow coerce me into loading it, I have bigger problems.)
SCTP is essential for certain older telco protocols and in certain protocols developed for LTE it was added. End users probably don't use it much, but the harsware their connections are going through will speak SCTP at some level.
Am part way through adding NAT support for it to the firewall.
So yes, SCTP is under the covers getting a lot more use than it seems, still today. However all WebRTC implementations usually bring their own userspace libraries to implement SCTP, so they don't depend on the one from the OS.
> The performance gap between QUIC and kTLS may be attributed to:
- The absence of Generic Segmentation Offload (GSO) for QUIC.
- An additional data copy on the transmission (TX) path.
- Extra encryption required for header protection in QUIC.
- A longer header length for the stream data in QUIC.Inbound connections? You don't need to do anything other than make sure your fd limit is high and maybe not be ipv4 only and have too many users behind the same cgnat.
Outbound connections is harder, but hopefully you don't need millions of connections to the same destination, or if you do, hopefully they support ipv6.
When I ran millions of connections through HAproxy (bare tcp proxy, just some peaking to determine the upstream), I had to do a bunch of work to make it scale, but not because of port limits.
(Fun bonus story: I noticed significant drops in throughput when using battery on a MacBook. Something to do with the efficiency cores I assume.)
Secondly, quic does congestion control poorly (I was using quic-go so mileage may vary). No tuning really helped, and TCP streams would take more bandwidth if both were present.
Third, the APIs are weird man. So, quic itself has multiple streams, which makes it non-drop in replacement with TCP. However, the idea is to have HTTP/3 be drop-in replaceable at a higher level (which I can’t speak to because I didn’t do). But worth keeping in mind if you’re working on the stream level.
In conclusion I came out pretty much defeated but also with a newfound respect for all the optimizations and resilience of our old friend tcp. It’s really an amazing piece of tech. And it’s just there, for free, always provided by the OS. Even some of the main issues with tcp are not design faults but conservative/legacy defaults (buffer limits on Linux, Nagle, etc). I really just wish we could improve it instead of reinventing the wheel..
[1]: https://payload.app/
That sounds like the thread priority/QoS was incorrect, but it could be WiFi or something.
You might think you can send datagrams of up to 64KB, but due to limitations in how IP fragment reassembly works, you really must do your best to not allow IP fragmentation to occur, so 1472 is the largest in most circumstances.
syscall overhead is only on the order of 100-1000 ns. Even at a blistering per core memory bandwidth of 100 GB/s, just the single copy fundamentally needed to serialize 1 MB into network packets costs 10,000 ns.
The ~1,000 syscalls needed to transmit a 1 GB file would incur excess overhead of 1 ms versus 1 syscall per 1 GB.
That is at most a 10% overhead if the only thing your system call needs to do is copy the data. As in it takes 10,000 ns total to transmit 1,000 packets meaning you get 10 ns per packet to do all of your protocol segmentation and processing.
The benchmarks in the paper show that the total protocol execution time for a 1 GB file using TCP is 4 seconds. The syscall overhead for issuing 1,000 excess syscalls should thus be ~1/4000 or about 0.025% which is totally irrelevant.
The difference between the 4 second TCP number and the 8 second QUIC number can not be meaningfully traced back to excess syscalls if they were actually issuing max size sendmmsg calls. Hell, even if they did one syscall per packet that would still only account for a mere 1 second of the 4 second difference. It would be a stupid implementation for sure to have such unforced overhead, but even that would not be the actual cause of the performance discrepancy between TCP and QUIC in the produced benchmarks.
IIRC, Chrome's network simulation just applies a delay after a connection is established
It's a lot better than nothing, and doing it realistically would be a lot more work than what they've done, so I say this with all due respect to those who worked on it.
Really need something like ipfw, dummynet, tc etc to do it at the packet level
When I used to (try to) play online games in NZ a few years ago, RTT to US West servers sometimes exceeded 200ms.
DSL kills.
For them that don't know the difference. netquake was the original strict client server version of quake, you hit the forward key it sends that to the server and the server then sends back where you moved. quakeworld was the client side prediction enhancement that came later, you hit forward, the client moves you forwards and sends it to the server at the same time. and if there are differences it gets reconciled later.
For the most part client side prediction feels better to play. however when there are network problems, large amounts of lag, a lot of artifacts start to show up, rubberbanding, jumping around, hits that don't. Pure client server feels worse, every thing gets sluggish, and mushy but movement is a little more predictable and logical and can sort of be anticipated.
I have not played quake in 20 years but one thing I remember is at past 800ms of lag the lava felt magnetic, it would just suck you in, every time.
The main reason I made my original comment was to point out that the real numbers are more than double what the other commenter called “devastating” latency.
This implies that user space is slow. Yet, some(most?) of the fastest high-performance TCP/IP stacks are made in user space.
I truly hope the QUIC in Linux Kernel project [0] succeeds. I'm not looking forward to linking big HTTP/3 libraries to all applications.
Plenty of assumptions break down in such an environment and part of my work is to ensure that the user always knows that the app is really doing something and not just being unresponsive.
One of the things he highlighted was the higher CPU utilization of HTTP/3, to the point where CPU can limit throughput.
I wonder how much of this is due to the immaturity of the implementations, and how much this is inherit due to way QUIC was designed?
I never got the impression that it was intended to make all connections faster.
If viewed from that perspective, the tradeoffs make sense. Although I’m no expert and encourage someone with more knowledge to correct me.
- better behaviour under packet loss (you don’t need to read byte n before you can see byte n+1 like in tcp)
- better behaviour under client ip changes (which happen when switching between cellular data and wifi)
- moving various tricks for getting good latency and throughput in the real world into user space (things like pacing, bbr) and not leaving enough unencrypted information in packets for middleware boxes to get too funky
His testing has CPU-bound quiche at <200MB/s and nghttp2 was >900MB/s.
I wonder if the CPU was throttled.
Because if HTTP 3 impl took 4x CPU that could be interesting but not necessarily a big problem if the absolute value was very low to begin with.
Another feature you see for modern tcp-based servers is offloading tls to the hardware. I think this matters more for servers that may have many concurrent tcp streams to send. On Linux you can get this either with userspace networking or by doing ‘kernel tls’ which will offload to hardware if possible. That feature also exists for some funny stuff in Linux about breaking down a tcp stream into ‘messages’ which can be sent to different threads, though I don’t know if it allows eagerly passing some later messages when earlier packets were lost.
In an ideal World we would create a new TCP and TLS standard and replace and/or update all internet routers and hardware everywhere World Wide so that it is implemented with less CPU utilization ;)
It would have forced IPv6 migration immediately (no NAT) and forced endpoints to be secured with local firewalls and better software instead of middle boxes.
The Internet would be so much simpler, faster, and more capable. Peer to peer would be trivial. Everything would just work. Protocol innovation would be possible.
Of course tech is full of better roads not taken. We are prisoners of network effects and accidents of history freezing ugly hacks into place.
The problem with ipv6 in my understanding was that the transitional functions (nat-pt etc) were half baked and a new set had to be developed. It is possible that disruption would have occurred if that had to be done against an earlier address exhaustion date.
There's definitely some benefits to glean from a zero trust model, but putting a moat around your network still helps a lot and NAT is probably the best accidental security feature to ever exist. Half the cybersecurity problems we have are because the cloud model has normalized routing sensitive behavior out to the open Internet instead of private networks.
My middleboxes will happily be configured to continue to block any traffic that refuses to obey them. (QUIC and ECH inclusive.)
First of all, NAT is what saved the Internet from being forked. IPv6 transition was a pipe dream at the time it was first proposed, and the vast growth in consumers for ISPs that had just paid for expensive IPv4 boxes would never have resulted in them paying for far more expensive (at the time) IPv6 boxes, it would have resulted in much less growth, or other custom solutions, or even separate IPv4 networks in certain parts of the world. Or, if not, it would have resulted in tunneling all traffic over a protocol more amenable to middle boxes, such as HTTP, which would have been even worse than the NAT happening today.
Then, even though it was unintentional, NAT and CGNAT are what ended up protecting consumers from IP-level tracking. If we had transitioned from IPv4 directly to IPv6, without the decades of NAT, all tracking technology wouldn't have bothered with cookies and so on, we would have had the trivial IP tracking allowed by the one-IP-per-device vision. And with the entrenched tracking adware industry controlling a big part of the Internet and relying on tracking IPs, the privacy extensions to IPv6 (which, remember, came MUCH later in IPv6's life than the original vision for the transition) would never have happened.
I won't bother going into the other kinds of important use cases that other middle boxes support, that a hostile IPv4 would have prevented, causing even bigger problems. NAT is actually an excellent example of why IPs design decisions that allow middle boxes are a godsend, not a tragic mistake. Now hopefully we can phase out NAT in the coming years, as it's served its purpose and can honorably retire.
It’s possible that the cloud would not have been nearly as big as it has been.
The privacy benefits of NAT are minor to nonexistent. In most of the developed world most land connections get one effectively static V4 IP which is enough for tracking. Most tracking relies primarily on fingerprints, cookies, apps, federated login, embeds, and other methods anyway. IP is secondary, especially with the little spies in our pockets that are most people’s phones.
We'll see if this more direct communication actually happens as IPv6 becomes ubiquitous, but I for one doubt it. Especially since ISPs are not at all friendly to residential customers trying to run servers, often giving out dynamic prefixes or small subnets (/128s even!) even on IPv6. And I think the LTE network is decent evidence in support of my doubts: it was built from the ground up with IPv6-only internally, and there are no stable IP guarantees anywhere.
As to the privacy benefits, those are real and have made IP tracking almost useless. Your public IP, even in the developed world, very commonly changes daily or weekly. Even worse for trackers, when it does change, it changes to an IP that someone else was using.
This is false. Because of the inconsistency of NATs and other middle-boxes out there and the fact that many are broken, it's far less reliable. You end up having to relay some traffic, which imposes external cost that unlike a third party locator server isn't trivial. Now you're already losing the benefits of end-to-end connectivity.
Also if E2E is easy there are distributed algorithms for location like DHTs that can be implemented. With trivial end to end they're pretty easy and would be fast and reliable.
The way the Internet has developed has basically broken it for end to end connectivity, forcing everything into the cloud. That is far worse for privacy and autonomy (and cost, making everything a subscription) than IP tracking.
I think you're a little blinded by what is and unable to imagine an alternate path.
Evolution is very path dependent and small changes at one point make things massively different later. One less asteroid and we'd be warm blooded bird-reptile like things that laid eggs.
The much bigger problems are related to moderation, copyright enforcement, spam prevention, security. All of those are extremely hard if you don't have a centralized authority server.
Could Zoom have better quality more cheaply if it could easily do P2P connections for small meetings? Very likely. Could you make a fully distributed Zoom where anyone can call anyone else without a centralized authority server handling all calls? No, not without significant legal hurdles and effort on preventing malicious actors from spamming the network, from distributing illegal content, etc.
Also, back to middleboxes: not having NAT would not get rid of middleboxes. Even on IPv6, there will always be a stateful firewall blocking all outside connections to the internal network in any sane deployment, at least for home networks. And that firewall will probably be about as buggy as cheap NAT boxes are. And for corporate networks, you have all sorts of other middlemen critical to the security of the network, I clouding IDS and IPS systems, TLS listeners to protect from data e filtration etc. Those will interfere with your traffic far more than relatively regular NAT boxes would.
There's a difference between "better roads not taken", and "taking this road would require that most of our existing cars and roads be replaced, simultaneously".
Simply put, this isn't a road not taken, it's a road that didn't exist.
Haven't read the whole paper yet, but below 600 Mbit/s is implied as being "Slow Internet" in the intro.
Meanwhile, low-end computers ship with a dozen 10+Gbit class transceivers on USB, HDMI, Displayport, pretty much any external port except for ethernet, and twice that many on the PCIe backbone. But 10Gbit ethernet is still priced like it's made from unicorn blood.
My understanding is right around 10Gbps you start to hit limitations with the shielding/type of cable and power needed to transmit/send over Ethernet.
When I was looking to upgrade at home, I had to get expensive PoE+ injectors and splitters to power the switch in the closet (where there's no outlet) and 10Gbps SFP+ transceivers are like $10 for fiber or $40 for Ethernet. The Ethernet transceivers hit like 40-50C
Aging has it's upsides and downsides I guess.
Mine were over 90C, resulting in thermal shutdowns. I had to add an improvised heat exchanger to lower it down to ~70C: https://pics.archie.alex.net/share/U0G1yiWzShqOGXulwe1AetDjR...
But switches haven't really kept up. A simple unmanaged 5-port or 8-port 2.5GigE isn't too bad, but anything beyond that gets tricky. 5GigE switches don't seem to exist, and you're already paying $500 for a budget-brand 10GigE switch with basic VLAN support. You want PoE? Forget it.
The irony is that at 10Gbps fiber suddenly becomes quite attractive. A brand-new SFP+ NIC can be found for $30, with DACs only $5 (per side) and transceivers $30 or so. You can get an actually-decent switch from Mikrotik for less than $300.
Heck, you can even get brand-new dualport SFP28 NICs for $100, or as little as $25 on Ebay! Switch-wise you can get 16 ports of 25Gbps out of a $800 Mikrotik switch: not exactly cheap, but definitely within range for a very enthusiastic homelabber.
The only issue is that wiring your home for fiber is stupidly expensive, and you can't exactly use it to power access points either.
What do you mean by that? My home isnt wired for ethernet. I can buy 30m of CAT6 cable for £7, or 30m of fibre for £17. For a home use, that's a decent amount of cable, and even spending £100 on cabling will likely run cables to even the biggest of houses.
A rental service might help there, or a call-in service-- the 6 hours of drilling holes and pulling fibre can be done by yourself, and once it's all cut to rough length, bring out a guy who can fuse on 10 plugs in an hour for $150.
My ordinary home-centre electric drill and an affordable ~7mm masonry bit lets me drill a hole in stucco large enough to accept bare cables with a very narrow gap to worry about.
The price of modules was built into the first post mentioning fiber up here: https://news.ycombinator.com/item?id=41891559
The worry was that the actual fiber would be "stupid expensive", and nah it's fine.
That said;
> A rental service might help there, or a call-in service-- the 6 hours of drilling holes and pulling fibre can be done by yourself, and once it's all cut to rough length, bring out a guy who can fuse on 10 plugs in an hour for $150.
If you were paying someone to do it (rather than DIY) I'd wager the cost would be similar, as you're paying them for 6 hours of labour either way.
[0] https://www.cablemonkey.co.uk/fibre-optic-tool-kits-accessor...
If you decide you only need 50 meters, that reduces both power and cable requirements by a lot. Did we decide to ignore the easy solution in favor of stagnation?
If it's new construction or you already have everything ripped open it's less of an issue.
Instead we waited and waited before making slower versions of 10gig, and those are still very slow to roll out. Also 2.5gig and 5gig seem especially consumer-oriented, so for those users a cheap but half range 10gig would be all upside.
And 40gig can't reach 100m on any version of copper, so it's not like 100m is a sacred requirement.
In a perfect world we would start using fiber in consumer products that need to move that much bandwidth, but I think the standards bodies don't trust consumers with bend radiuses and dust management so instead we keep inventing new ways to torture copper wires.
For anything that wants 10Gbps lanes or less, copper is fine.
For ultra bandwidth, going fiber-only is a tempting idea.
We are already doing this. USB-C is explicitly designed to allow for cables with active electronics, including conversion to & from fiber. You could just buy an optical USB-C cable off Amazon, if you wanted to.
But bear in mind, standards like USB4 only support very short cables. It's impressive that USB4 can offer 40 Gbps - but it can only do so on 1m cables. On the other hand, 10 gigabit ethernet claims to go 100m on CAT6A.
You really can’t think of any major difference between 10G Ethernet and all of those other standards that might be responsible for the price difference?
Look at the supported lengths and cables. 10G Ethernet over copper can go an order of magnitude farther over relatively generic cables. Your USB-C or HDMI connections cannot go nearly as far and require significantly more tightly controlled cables and shielding.
That’s the difference. It’s not easy to accomplish what they did with 10G Ethernet over copper. They used a long list of tricks to squeeze every possible dB of SNR out of those cables. You pay for it with extremely complex transceivers that require significant die area and a laundry list of complex algorithms.
No matter, eventually USB4NET will work out of the box. The USB-IF is a clown show and they have tripped over their shoelaces every step of the way, but consumer Ethernet hasn't moved in 20 years so this horse race still has a clear favorite, lol.
> Look at the supported lengths and cables. … relatively generic cables. Your USB-C or HDMI connections cannot go nearly as far and require significantly more tightly controlled cables and shielding. … They used a long list of tricks to squeeze every possible dB of SNR out of those cables.
Meanwhile a 10g port on my home router will run over copper for far longer. Not that I’m a fan given the power use, fibre is much easier to deal with and will run for miles.
Gigabit ethernet is ~100MB/s transfer speed over copper wire or ~30MB/s over wireless accounting for overhead and degradation. That is more than fast enough for most people.
10gbit is seemingly made from unicorn blood and 2.5gbit is seeing limited adoption because there simply isn't demand for them outside of enterprise who have lots of unicorn blood in their banks.
It doesn't sound like there's a fundamental issue with the protocol.
It's simply not economical enough to lay fibre and put 5G masts everywhere (5G LTE bands covers less area due to being higher frequency, and so are also limited to being deployed in areas with a higher enough density to be economically justifiable).
Most importantly, it can be heavily over-provisioned for peanuts, so your cable is future-proof, and you will never have dig the same trenches again.
Copper only makes sense if you already have it.
But telcos have colossal copper networks, and they want to milk the last dollars from it before it has to be replaced, with digging and all. Hence price segmenting, with slower "copper" plans and premium "fiber" plans, obviously no matter if the building has fiber already.
Also, passive fiber interconnects have much higher losses than copper with RJ45s. This means you want to have no more than 2-3 connectors between pieces of active equipment, including from ISP to a building. This requires more careful planning, and this is why wiring past the apartment (or even office floor or a single-family house) level is usually copper Ethernet.
I grew up with 2400 baud modems as the super fast upgrade, so talk of multiple gigabits for consumers is blowing my mind a bit.
Basically, the old copper lines were replaced by the NBN, which is a government-owned corporation that sells wholesale networking to telcos. Essentially, the government has a monopoly, providing the last-mile fibre links. They use nested VLANs to provide layer-2 access to the consumer telcos.
Where it got complicated was that the right-wing government was in the pocket of Rupert Murdoch, who threatened them with negative press before an upcoming election. They bent over and grabbed their ankles like the good little Christian school boys they are, and torpedoed the NBN network technology to protect the incumbent Fox cable network. Instead of fibre going to all premises, the NBN ended up with a mix of technologies, most of which don't scale to gigabit. It also took longer and cost more, despite the government responsible saying they were making these cuts to "save taxpayer money".
Also for political reasons, they were rolling it out starting at the sparse rural areas and leaving the high-density CBD regions till last. This made it look bad, because if they spent $40K digging up the long rural dirt roads to every individual farmhouse, it obviously won't have much of a return on the taxpayer's investment... like it would have if deployed to areas with technology companies and their staff.
Some existing smaller telcos noticed that there was a loophole in the regulation that allowed them to connect the more lucrative tech-savvy customers to their own private fibre if it's within 2km of an existing line. Companies like TPG had the entire CBD and inner suburban regions of every major city already 100% covered by this radius, so they proceeded to leapfrog the NBN and roll out their own 100 Mbps fibre-to-the-building service half a decade ahead. I saw their unmarked white vans stealthily rolling out extra fibre at like 3am to extend their coverage area before anyone in the government noticed.
The funny part was that FttB uses VDSL2 boxes in the basement for the last 100m going up to apartments, but you can only have one per building because they use active cross-talk cancellation. So by the time the NBN eventually got around to wiring the CBD regions, they got to the apartments to discover that "oops, too late", private telcos had gotten there first!
There were lawsuits... which the government lost. After all, they wrote the legislation, they were just mad that they hadn't actually understood it.
Meanwhile, some other incumbent fibre providers that should have disappeared persisted like a stubborn cockroach infestation. I've just moved to an apartment serviced by OptiComm, which has 1.1 out of 5 stars on Google... which should tell you something. They even have a grey fibre box that looks identical to the NBNCo box except it's labelled LBNCo with the same font so that during a whirlwind apartment inspection you might not notice that you're not going to be on the same high-speed Internet as the rest of the country.
The history there is that Australia used to have a government run monopoly on telephone infrastructure and services (Telecom Australia), which was later privatised (and rebranded to Telstra). The privatisation left Telstra with a monopoly on the infrastructure, but also a requirement that they resell the last mile at a reasonable rate to allow for some competition.
So Australia already had an existing industry of ISPs that were already buying last mile access from someone else. The NBN was just a continuation of the existing status quo in that regard.
> They even have a grey fibre box that looks identical to the NBNCo box except it's labelled LBNCo with the same font
Early in my career I worked for one of those smaller telcos trying to race to get services into buildings before the NBN. I left around the time they were talking about introducing an LBNCo brand (only one of the reasons I left). At the time, they weren't part of Opticomm, but did partner with them in a few locations. If the brand is still around, I guess they must have been acquired at some point.
Sorry about your NBN!
I don't see a need for it yet though. I'm a really heavy user (it specialist with more than a hundred devices in my networks) and I really don't need it.
E.g.: 8K 60 fps video streaming benefits from data rates up to about 1 Gbps in a noticeable way, but that's at least a decade away form mainstream availability.
And internal network is 1 Gbit too. So it'll take ) and cost) more than just changing my subscription.
Also my TV is still 1080p lol
I’m not sure whether it’s possible, but could you theoretically offload large file downloads to HTTP/2 to get best of both worlds?
Yes, you can! You’d have your websites on servers that support HTTP/3 and your large files on HTTP/2 servers, similar to how people put certain files on CDNs. It might well be a great solution!
But such a latency issue isn't majorly increasing battery usage (compared to a CPU usage issue which would make CPUs boost). Nor is it an issue for server-to-server communication.
It basically "only" slows down high bandwidth transmissions on end user devices with (for 2024 standards) very high speed connection (if you take effective speeds from device to server, not speeds you where advertised to have bough and at best can get when the server owner has a direct pairing agreement with you network provider and a server in your region.....).
Doesn't mean the paper is worthless, browser should improve their impl. and it highlights it.
But the title of the paper is basically 100% click bait.
So "Not Quick Enough" is plain out wrong, it is fast enough.
The definition of "Fast Internet" misleading.
And even "QUIC" is misleading as it normally refers to the protocol while the benchmarked protocol is HTTP/3 over QUIC and the issue seem to be mainly in the implementations.
In 30 years it will be even faster. It would be silly to have to use older protocols to get line speed.
Enable http/3 + quic between client browser <> edge and restrict edge <> origin connections to http/2 or http/1
Cloudflare (as an example) only supports QUIC between client <> edge and doesn’t support it for connections to origin. Makes sense if the edge <> origin connection is reusable, stable, and “fast”.
https://developers.cloudflare.com/speed/optimization/protoco...
Or rather, not "Fast Internet"
QUIC is not quick enough over fast internet (acm.org)
https://news.ycombinator.com/item?id=41484991 (327 comments)
Brave/Vivaldi/Opera/etc: You should make a conscious choice.
This is an interesting hot take.
You’re also factually incorrect in a number of ways such as claiming that HTTP/2 was a Google project (it’s not and some of the poorly thought out ideas like push didn’t come from Google).
The fact of the matter is that other attempts at “next gen” protocols had taken place. Google is the only one that won out. Part of it is because they were one of the few properties that controlled enough web traffic to try something. Another is that they explicitly learned from mistakes that the academics had been doing and taken market effects into account (ie not requiring SW updates of middleware boxes). I’d say all things considered Internet connectivity is better that QUIC got standardized. Papers like this simply point to current inefficiencies of today’s implementation - those can be fixed. These aren’t intractable design flaws of the protocol itself.
But you seem to really hate Google as a starting point so that seems to color your opinion of anything they produce rather than engaging with the technical material in good faith.
It's just that their interests in certain aspects don't align with ours.
QUIC has some debatable properties, like mandatory encryption, or the use of UDP instead of being a protocol under IP like TCP, but there are good reasons for it, related to ossification.
Yes, Google pushed for it, but I think it deserves its approval as a standard. It is not perfect but it is practical, they don't want another IPv6 situation.
QUIC is also acknowledged as being quite different from the Google version, and incorporating input from many different people.
Could you expand more on why this seems like evidence that Google unilaterally dictating bad standards? None of the changes in protocol seem objectively wrong (except possibly Server Push).
Disclaimer: Work at Google on networking, but unrelated to QUIC and other protocol level stuff.
I guess I'm just generally disgusted in the way Google is poisoning the web in the worst way possible: By pushing ever more complex standards. Imagine the complexity of the web stack in 2050 if we continue to let Google run things. It's Microsoft's old embrace-extend-and-extinguish scheme taken to the next level.
In short: it's not you, it's your manager's manager's manager's manager's strategy that is messed up.
It literally is not.
Edit: I'm not the first person to make this comparison. Witness the Chrome section in this article:
https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...
What are they extending in this analogy? Http3 is not an extension of http. What are they extinguishing? There is no plan to get rid of http1/2, since you still need it in lots of networks that dont allow udp.
Additionally, its an open standard, with an rfc, and multiple competing implementations (including firefox and i believe experimental in safari). The entire point of embrace, extend, extinguish is that the extension is not well specified making it dufficult for competitors to implement. That is simply not what is happening here.
They have several thousand C++ browser engineers (and as many web standards people as they could get their hands on, early on). Combined with a dominant browser market share, this has let them dominate browser standards, and even internet protocols. They have abused this dominant position to eliminate all competitors except Apple and (so far) Mozilla. It's quite clever.
It's such a tired sad trope of people disaffected with the web because they can't implement it by themselves easily. I'm so exhausted by this anti-progress terrorism; the world's shared hypermedia should be rich and capable.
We also see lots of strong progress these days from newcomers like Ladybird, and Servo seems gearing up to be more browser like.
I think my message here is only hard to understand if your salary (or personal worth etc) depends on not understanding it. It's really not that complex.
Just because someone disagrees with you, doesn't mean they don't understand you.
However, if you think google is making standards unneccessarily complex, you should read some of the standards from the 2000s (e.g. SAML).
This is generally true of course, but here the complete non-engagement with parent's arguments shows either bad faith or actual lack of understanding. It's more likely to be the former, as the concept is not that difficult to grasp, and quite widely accepted. Heck, even the wikipedia page on EEE has chromium as an example.
I think this argument is reasonable, but QUIC isn't part of the problem.
But that's like all of them. Except edge but that was mostly dead before chrome came on the scene.
It seems like you are using embrace, extend, extinguish to just mean, "be succesful", but that's not what the term means. Being a market leader is not the same thing as embrace, extend, extinguish. Neither is putting competition out of business.
Assume that change X for the web is positive overall. Currently Google’s strategy is to implement in Chrome and collect data on usefulness, then propose a standard and have other people contribute to it.
That approach seems pretty optimal. How else would you do it?
In reality, there are perfectly valid reasons that motivate QUIC and HTTP/2 and I don’t think there is a reasonable argument that they are objectively bad. Now, for your personal use case, it might not be worth it, but that’s a different argument. The standards are built for the majority.
All systems have tradeoffs. Increased complexity is undesirable, but whether it is bad or not depends on the benefits. Just blanket making a statement that increasing complexity is bad, and the runaway effects of that in 2050 would be worse does not seem particularly useful.
[0] There are non-browser protocols that are based on H2 only, but since your complaint was explicitly about browsers, I know that's not what you had in mind.
It's not your fault, in case you were working on this. It was likely the result a strategy thing being decided at Google/Alphabet exec level.
Several thousand very competent C++ software engineers don't come cheap.
And it seems that you can't support either of those claims in any way. In fact, you're just pretending that you never made those comments at all, and have once again pivoted to a new grievance.
But the new grievance is equally nonsensical. HTTP/2 is not particularly complex, and nobody on either the server or browser side was forced to implement it. Only those who thought the minimal complexity was worth it needed to do it. Everyone else remained fully interoperable.
I'm not entirely sure where you're coming from here, to be honest. Like, is your belief that there are no possible tradeoffs here? Nothing can ever justify even such minor amounts of complexity, no matter how large the benefits are? Or do you accept that there are tradeoffs, and are "just" disagree with every developer who made a different call on this when choosing whether to support HTTP/2 in their (non-Google) browser or server?
HTTP/2 is not "particularly complex?" Come on! Do remember where we started.
> I'm not entirely sure where you're coming from here, to be honest. Like, is your belief that there are no possible tradeoffs here? Nothing can ever justify even such minor amounts of complexity, no matter how large the benefits are? Or do you accept that there are tradeoffs, and are "just" disagree with every developer who made a different call on this when choosing whether to support HTTP/2 in their (non-Google) browser or server?
"Such minor amounts of complexity". Ahem.
I believe there are tradeoffs. I don't believe that HTTP/2 met that tradeoff between complexity vs benefit. I do believe it benefitted Google.
Yes, HTTP/2 is not really complex as far as these things go. You just keep making that assertion as if it was self-evident, but it isn't. Like, can you maybe just name the parts you think are unnecessary complex? And then we can discuss just how complex they really are, and what the benefits are.
(Like, sure, having header compression is more complicated than not having it. But it's also an amazingly beneficial tradeoff, so it can't be what you had in mind.)
> I believe there are tradeoffs. I don't believe that HTTP/2 met that tradeoff between complexity vs benefit.
So why did Firefox implement it? Safari? Basically all the production level web servers? Google didn't force them to do it. The developers of all of that software had agency, evaluated the tradeoffs, and decided it was worth implementing. What makes you a better judge of the tradoffs than all of these non-Google entities?
- MS embrace extend extinguish
- Google is making the world complex
- Nth level manager is messed up
None of the above was connected to deliver a clear point, just thrusted into the comment to sound profound.
As long as the adverts arrive quickly the rest is immaterial.
Today I found the solution. Disable "Experimental QUIC Protocol" in Chrome settings.
This makes me kind of worried because I've had issues accessing wordpress.org for months. There was no indication that this was caused by QUIC. I just managed to realize it because there was QUIC-related error in devtools that appeared only sometimes.
I wonder what other websites are rendered inaccessible by this protocol and users have no idea what is causing it.
I didn’t look closely enough to see what their test system was to see if this is basic consumer systems or is still a problem for high performance desktops.
It's like buying the new 5G cell phone because it is X times faster than 4G even though 1) My 4G phone never actually ran at the full 4G speed and 2) The problem with any connection is almost never due to the line speed of my internet connection but a misbehaving DNS server/target website/connection Mux at my broadband provider. "But it's 5G"
Same thing cracks me up when people advertise "fibre broadband" for internet by showing people watching the TV like the wind is blowing in their hair, because that's how it works (not!). I used to stream on my 8Mb connection so 300Mb might be good for some things but I doubt I would notice much difference.
So it can’t function without it.
I wonder if this can be optimized at the protocol level by detecting usage patterns that look like large file transfers or very high bandwidth video streaming and swapping over to something less cpu intensive.
Or is this just a case of less hardware/OS level optimization of QUIC vs TCP because it's new?
QUIC is not quick enough over fast internet:
Maybe one advantage of HTTP/3 would be handling ip changes but I’m not sure this matters much because you can already resume downloads fine in HTTP/1.1 if the server supports range requests (which it very likely does for video)
But the parent asks
> Why do you need multiplexing when you are only downloading one (video) stream?
So that largely doesn't apply.
But throughput was never the compelling aspect of QUIC in the first place. It was always the reduced latency. A 1-RTT handshake including key/cert exchange is nothing to scoff at, and the 2-RTT request/response cycle that HTTP/3-over-QUIC offers means that I can load a blog page from a rinky-dink server on the other side of the world in < 500 ms. Look ma, no CDN!
QUIC has never been about "worse performance" to avoid TCP packet injection.
Anybody who cares about TCP packet injection is using crypto (IPSec/Wireguard). If performant crypto is needed there are appliances which do it at wirespeed.
I am unsure how this is a security flaw of TCP? Any middleman could block UDP packets too and get the same effect, or modify UDP packets in an attempt to cause the receiving application to crash.
In order to attack TCP, all you have to do is spy on a single packet (very easy) to learn the sequence number, then you can inject a wrench into the cogs and the endpoints will reject all legitimate traffic from each other.
Once the resource constraint is eliminared, you're no longer getting the benefit of that tradeoff but are paying the costs.
This paper needs multiple latency simulations, some packet loss and latency jitter to have any value.
Latency does not increase the chances of out of order packet arrival. Out of order packet arrival is usually caused by multipath or the equivalent inside a router if packets are handled by different stream processors (or the equivalent). Most routers and networks are designed to keep packets within a flow together to avoid exactly this problem.
However, it is fair to say that traversing more links and routers probably increases the chance of out of order packet delivery, so there's a correlation in some way with latency, but it's not really about the latency itself - you can get the same thing in a data center network.
But then again, being CPU-throttled isn't great for battery life, so perhaps there's a better way.
I don't really benefit much from http/3 or QUIC (I don't live in a remote area or host a cloud server) so I've already considered disabling either. A bandwidth cap this low makes a bigger impact than the tiny latency improvements.
Now it's back to headlines of HN.Seems like people all interested in this topic.
Without it's designed for use case all it does is slightly help mobile platforms that don't want to hold open a TCP connection (for energy use reasons) and bring in fragile "CA TLS"-only in an environment where cert lifetimes are trending down to single months (Apple etc latest proposal).
Such a internet connection becoming so low latency that the latency of receiver side processing becomes dominant is in practice not the most relevant. Sure theoretically you can hit it with e.g. 5G but in practice even with 5G many real world situations won't. Most importantly a slow down of such isn't necessary bad for Google and co. as it only add limited amounts on strain on their services, infrastructure, internet and is still fast enough for most users to not care for most Google and co. use cases.
Similar being slow due to receiver delays isn't necessary bad enough to cause user noticeable battery issues, on of the main reasons seem to many user<->kernel boundary crossings which are slow due to cache missues/ejections etc. but also don't boost your CPU clock (which is one of the main ways to drain your battery, besides the screen)
Also like the article mentions the main issue is sub optimal network stack usage in browsers (including Chrome) not necessary a fundamental issue in the protocol. Which brings us to inter service communication for Google and co. which doesn't use any of the tested network stacks but very highly optimized stacks. I mean it really would be surprising if such network stacks where slow as there had been exhaustive perf. testing during the design of QUIC.
I preference WebSockets over anything analogous to HTTP.
Commented edited because I mentioned performance conditions. Software developers tend to make unfounded assumptions/rebuttals of performance conditions they have not tested.
Have you ever read up on the technical details of QUIC? Every single of one of your bullets reflects a misunderstanding of QUIC's design.
Websockets are useful in some cases where you need to upgrade an HTTP connection to something more. Some people learn about websockets and then try to apply them to everything, everywhere. This seems to be one of those cases.
The current crop aside from WebSockets all seem to be born from taking a butcher knife to HTTP and hacking out everything that gets in the way of time to first byte. I don't think that's likely to produce anything worthwhile.
QUIC is a transport. HTTP can run on top of QUIC, but the way you’re equating QUIC and HTTP doesn’t make sense.
String headers and socket opening have nothing to do with the performance issues being discussed.
String headers aren’t even a performance issue at all. The amount of processing done for when the most excessive use of string headers is completely trivial relative to all of the other processing that goes into sending 1,000,000,000 bits per second (Gigabit) over the internet, which is the order of magnitude target being discussed.
I don’t think you understand what QUIC is or even the prior art in HTTP/2 that precedes these discussions of QUIC and HTTP/3.
That is universally incorrect. String instructions require parsing as strings are for humans and binary is for machines. There is performance overhead to string parsing always, and it is relatively trivial to perf. I have performance tested this in my own WebSocket and test automation applications. That performance difference scales in logarithmic fashion provided the quantity of messages to send/receive. I encourage you to run your own tests.
Just to pick at one point of this craziness, you think that communicating over web sockets does not involve round trips????
HTTP/3 is HTTP over QUIC. HTTP protocols v2 and onwards use binary headers. QUIC, by design, does 0-RTT handshakes.
> Yes, in theory UDP is faster than TCP but only when you completely abandon integrity
The point of QUIC, is that it enables application/userspace level reconstruction with UDP levels of performance. There’s no integrity being abandoned here: packets are free to arrive out of order, across independent sub-streams, and the protocol machinery puts them back together. QUIC also supports full bidirectional streams, so HTTP/3 also benefits from this directly. QUIC/HTTP3 also supports multiple streams per client with backpressure per substream.
Web-sockets are a pretty limited special case, built on-top of HTTP and TCP. You literally form the http connection and then upgrade it to web-sockets, it’s still TCP underneath.
Tl;Dr: your gripes are legitimate, but they refer to HTTP/1.1 at most, QUIC and HTTP/3 are far more sophisticated and performant protocols.
> A careful reading of RFC6455 only mentions the handshake and its response must be a static string resembling a header in style of RFC2616 (HTTP), but a single static string is not HTTP.
You're going to have to cite the paragraph, then, because that is most definitely not what RFC 6455 says. RFC 6455 says,
> The handshake consists of an HTTP Upgrade request, along with a list of required and optional header fields.
That's not "a single static string". You can't just say "are the first couple of bytes of the connection == SOME_STATIC", as that would not be a conforming implementation. (That would just be a custom protocol with its own custom upgrade-into-Websockets, as mentioned in the first paragraph, but if you're doing that, you might as well just ditch that and just start in Websockets.)
¹(i.e., I grant the RFC's "However, the design does not limit WebSocket to HTTP, and future implementations could use a simpler handshake", but making use of that to me that puts us solidly in "custom protocol" land, as conforming libraries won't interoperate.)
> You can't just say
I can say that, because I have my own working code that proves it cross browser and I have written perf tools to analyze it with numbers. One of my biggest learnings about software is to always conduct your own performance measurements because developers tend to be universally wrong about performance assumptions and when they are wrong they are frequently wrong by multiple orders of magnitude.
As far as custom implementation goes you gain many liberties after leaving the restrictions of the browser as there are some features you don’t need to execute the protocol and there are features of the protocol the browser does not use.
I never made any claim to the contrary.
> Therefore just achieve the handshake by any means and WebSockets will work correctly in the browser.
At which point you're parsing a decent chunk of HTTP.
> I can say that, because I have my own working code that proves it
Writing code doesn't prove anything; code can have bugs. According to the standard portion I quoted, your code is wrong. A conforming request isn't required to match.
> I have written perf tools to analyze it with numbers. One of my biggest learnings about software is to always conduct your own performance measurements because developers tend to be universally wrong about performance assumptions and when they are wrong they are frequently wrong by multiple orders of magnitude.
Performance has absolutely nothing to do with this.
Even if such an implementation appears to work today in browsers, this makes situations with a still-conforming UA damn near impossible to debug, and there's no guarantees made on header ordering, casing, etc. that would mean it would continue to work. Worse, non-conformant implementations like this are the sort of thing that result in ossification.
In the end an in house implementation that allows custom extensions is worth far more than any irrational unfounded fears. If in the future it doesn’t work then just fix the current approach to account for those future issues. In the meantime I can do things nobody else can because I have something nobody else is willing to write.
What’s interesting is that this entire thread is about performance concerns. If you raise a solution that people find unfamiliar all the fear and hostility comes out. To me such contrary behavior suggests performance, in general, isn’t a valid concern to most developers in comparison to comfort.
Current implementations have some disadvantages to TCP, but they are not inherent to the protocol, they just highlight the decades of work done to make TCP scale with network hardware.
Your points seem better directed at HTTP/3 than QUIC.