Investigating the impact of HTTP3 on network latency for search

My browser has made over 100,000 network requests so far today, and I expect yours has too. I'm sure at least some of those will be in the 99.99th percentile.

Source: I've worked on network protocols for video conferencing and have seen my share of corporate network horrors.

One potential solution that hasn't been discussed much here is leveraging AWS managed services like AWS CloudFront or AWS Global Accelerator. It's worth noting that Dropbox's website already uses AWS CloudFront, so they are already leveraging AWS in some capacity. Based on my cost calculations, using AWS CloudFront would cost around $40k a month, while AWS Global Accelerator would be around $22k a month.

As of August 22, 2022, AWS CloudFront supports terminating HTTP/3 in a Point of Presence (POP), which could potentially help with the latency issues Dropbox is facing. AWS Global Accelerator, on the other hand, is designed to improve the performance of applications by terminating UDP/TCP as close to users as possible then routing user traffic through the AWS global network infrastructure. This could help reduce latency by ensuring that user traffic is routed through the most optimal path, even if the user is located far from a Dropbox data center.

It's hard to estimate what the potential latency reduction of using e.g. AWS Global Accelerator is, especially at higher percentiles. However, using https://speedtest.globalaccelerator.aws/, and assuming symmetry, my connections to Asia are 35-40% lower latency.

Of course, there are trade-offs to consider when using managed services like AWS CloudFront and AWS Global Accelerator. While they can provide significant performance improvements, they also come with additional costs and potential vendor lock-in. However, given the scale of Dropbox's operations and the importance of providing a fast, reliable search experience for their users, it may be worth exploring these options further.

---

Cost estimates

Assumptions:

    1. Dropbox's peak traffic is 1,500 queries per second (QPS).
    2. Average data transfer per query is 100 KB.
    3. 50% of the traffic comes from North America, 25% from Europe, and 15% from Asia (remaining 10% from other regions, for pricing purposes put it into Asia's calculations).

1) AWS CloudFront cost estimation:

    Data transfer:
    - North America: 1,500 QPS * 0.5 * 100 KB * 60 seconds * 60 minutes * 24 hours * 30 days = 194.4 TB  
    - Europe: 1,500 QPS * 0.25 * 100 KB * 60 seconds * 60 minutes * 24 hours * 30 days = 97.2 TB  
    - Asia: 1,500 QPS * 0.25 * 100 KB * 60 seconds * 60 minutes * 24 hours * 30 days = 97.2 TB

    Data transfer cost:
    - North America: 194.4 TB * $0.085/GB = $16,524
    - Europe: 97.2 TB * $0.085/GB = $8,262
    - Asia: 97.2 TB * $0.120/GB = $11,664

    Total data transfer cost: $16,524 + $8,262 + $7,006 = $36,200

    HTTP requests:
    Total requests: 1,500 QPS * 60 seconds * 60 minutes * 24 hours * 30 days = 3,888,000,000

    Using the updated AWS CloudFront pricing (as of May 22, 2023):
    - HTTP requests cost: 3,888,000,000 * $0.0075/10,000 = $2,916

---

2) AWS Global Accelerator cost estimation:

    Data transfer:
    - Total data transfer: 194.4 TB (NA) + 97.2 TB (EU) + 97.2 TB (Asia) = 388.8 TB

    Using the updated AWS Global Accelerator pricing (as of May 22, 2023):
    - Data transfer cost (averaged across regions): 388.8 TB * $0.035/GB = $13,608
    - (Also need to add EC2 egress cost, 388.8 TB * $0.02/GB = $7,776
    - Total data transfer cost = $21,384

    Accelerator:
    - Assuming 1 accelerator with 2 endpoints (1 for HTTP/2 and 1 for HTTP/3)
    - Accelerator cost: 1 * $18/accelerator/day * 30 days = $540