HTTP vs. WebSockets: Which one is the fastest for Postgres queries at the edge?
neon.tech
neon.tech
You don't mention which version of HTTP you were using, which makes some of my comments here conjecture, but if I had to guess, I'd say you were using HTTP/1.1. Is that right?
It would really help if you could break down where the time is being spent. I'd love to know what fraction of the time is spent in processing, transit, querying the db, etc.
Did you consider HTTP/2 or 3? I would expect them to have lower setup overhead, and be more competitive with Websockets.
In particular, if you were using HTTP/2+, did you look into CONNECT proxying? It could replace Websockets while having lower connection setup overhead, albeit at the cost of a little payload unwrapping overhead in the proxy. But theoretically, this could offer both the faster startup of HTTP/2+, combined with the lower processing overhead for the db of a TCP stream. Comparing the graphs of 10 queries, it _appears_ that processing HTTP imposes a ~15 ms overhead over raw bytes. (Theoretically, the CONNECT proxy has to obey the host and port in the :authority pseudoheader, but in your case, the proxy would just ignore it, and hook up the edge function to whatever compute node it pleased.)
A key point of our WebSocket and http transports is to support environments where TCP connections aren't available. Are there environments that would let us send and receive binary data following an http CONNECT that wouldn't simply allow an ordinary TCP connection?
(HTTP/3 is ~90% the same as HTTP/2, just over a different transport layer, and ditching HTTP/2 prioritization mechanisms which you probably won't care about.)
> A key point of our WebSocket and http transports is to support environments where TCP connections aren't available. Are there environments that would let us send and receive binary data following an http CONNECT that wouldn't simply allow an ordinary TCP connection?
Based on the network diagram in your article, it looks like the edge functions speak HTTP to the proxy, upgrade that to WS, and then the proxy routes raw TCP to the conn pool/db.
If so, CONNECT would be the same. The edge function sends an HTTP CONNECT request, the proxy would set it up, and afterwards, both hops are using the same transports as with websockets. The edge function would send HTTP/2 DATA frames wrapping the raw bytes, the proxy would unwrap those bytes, and then route the bytes over TCP to the conn pool/db.
If I understand your setup correctly, HTTP/2 CONNECT would be applicable everywhere WS is.
The major advantage you'd get with HTTP/3 and QUIC is fewer round trips to start a conn, and no head-of-line blocking like with TCP (which might be irrelevant if the Pg protocol doesn't use simultaneous streams, dunno).
The disadvantage is you'd have to support QUIC in your proxy (and gmac said your server lib doesn't support it yet.)
Don't get hung up on HTTP/3 as the latest and greatest, HTTP/2 is probably still an improvement for you.
> Even when queries are sent from unrelated V8 isolates.
Heh, I've definitely written a few AWS Lambdas that exploited this sort of behavior. "Sure, this resource might not exist. But I'm going to check, just in case..."
We experimented with SELECT 1, so most of the latency is transit
I'm assuming the most likely difference is in Pg processing an HTTP request over processing raw bytes, but there are other explanations, too. (Older/slower code pathway for HTTP in proxy, different intermediate node behavior for HTTP vs TCP from proxy to db, etc. Much less likely, but not impossible.)
So there's technically an additional hop irrespective of the transport type, but not one that should add meaningfully to the latency, or that would make WebSockets slower than the others.
Do you miss the performance that you had when the DB was running inside the application process using SQLite? Are you at all sad that you had to give that up in order to outsource compliance to Azure?
A little bit, but it's honestly much better this way in our use case. For our specific business & application, going from 50uS to 5000uS feels like nothing. We were squandering 2+ orders of magnitude over JSON blobs in columns anyways. We really didn't need 50uS transactions for anything. What turned out to be far more important was one source of truth to worry about. We have a lot of installs and a lot of SQLite databases out there. Now we are looking at exactly one for the whole enterprise.
Moving to hosted DB forced cleaner schema design. No json blobs = law now because of IO scaling concerns. We can't afford to move 5 megabytes of JSON across the datacenter every time someone clicks a button anymore. Average per-user-action database hit has fallen by 4~5 orders of magnitude. Megabytes to hundreds of bytes. Overall perceived performance is actually much better because of these design constraints. We also try to get the web views composed using as few queries as possible (i.e. finally make the database do its damn job instead of lazily peddling JSON).
The edge is in general not the users browser.
So you can do a TLS/postgres connection instead of adding additional layers.
Sure you want to probably run a patched pg client/server which has various optimizations to reduce the number of round trips when opening a connection, but that also applies to any HTTP/WebSocket wrapper.
Sure, sometimes you can. But a key use-case for a serverless database is connecting from a serverless compute environment, and not all serverless compute environments support TCP connections.
That's why Neon offers this serverless driver, which supports WebSockets and now also http.
Postgres <> Neon HTTP Proxy <---HTTP---> Local Fake Postgres Server <--unix socket--> Agents
We need it for the open-source project windmill.dev and we will build this if no one else does it.
In case it's useful, I can clarify:
* Every Neon database is accessible over an ordinary TCP Postgres connection (as well as WebSockets and http).
* Our JavaScript serverless driver is based on node-postgres and is a drop-in replacement for it. So for JS, no rewriting is needed.
Assuming that doesn't help, do you specifically need http (rather than WebSockets), and a proxy that speaks the pg protocol?
If not, the simplest thing might be to (1) use Neon, or run a WebSocket proxy next to your own database (e.g. https://github.com/neondatabase/wsproxy); and (2) find/write a similar WebSocket proxy, but in reverse — i.e. one that listens out for TCP/socket connections and tunnels them over WebSockets — to run next to the agents.
If we were using javascript/typescript, then yes we would probably just be able to use that library.
> (2) find/write a similar WebSocket proxy, but in reverse — i.e. one that listens out for TCP/socket connections and tunnels them over WebSockets — to run next to the agents.
It's that part that I would love to see. My assumption is that this proxy next to my agents would need to speak pg protocol and then transcribe it to wss. If there is a simpler way, let me know :)
[1]: https://github.com/neondatabase/serverless [2]: https://github.com/MatteoGioioso/serverless-pg
[1] https://developers.cloudflare.com/workers/learning/integrati...
Neon is slow and flaky I am thankful you are making changes but strongmanning your technical choice when you are wrong is concerning as a user. I still use Neon due to not being able to move back to heroku. I also you keep making open source more.
WebSockets still seem to be stuck with TCP as rfc9220 [0] is not implemented I think.
The data is also offloaded to S3. Here is a diagram that describes the architecture: https://neon.tech/_next/image?url=https%3A%2F%2Fneon-hwp.dre...
But to crib from https://neon.tech, you might choose Neon because it's scalable, cost-effective, and easy to deploy and manage. Or you might choose it for its copy-on-write branching and point-in-time recovery, all open-source, and all on top of the solid, performant, feature-rich foundation that is Postgres. :)