Comment by illia-v | Hacker News Reader

Comment by illia-v | Hacker News Reader

Parent Full thread

> we contributed upstream to ensure these HTTP clients can route requests directly through the JavaScript `fetch` API in WebAssembly environments

Some context from an urllib3 maintainer:

urllib3 received and merged large contributions adding Pyodide/Emscripten support a few years ago, and later JSPI support, which is what made this work for Requests.

As far as I know, the funding for this work went to the external contributor who implemented it, not to the urllib3 maintainers. We reviewed and merged the changes, and the project is now responsible for maintaining the resulting backend.

This matters because the Emscripten backend is still considered experimental in urllib3, and is explicitly out of scope in our security policy.

CVE-2025-50182 is one example of the problems we've run into. urllib3's redirect controls did not have the expected behavior when requests were routed through `fetch`. There are potentially many more differences like this because browser/`fetch` networking semantics are quite different from urllib3's normal backend.

I'm glad the work was contributed upstream and is useful to Pyodide and Cloudflare. But I think there is a meaningful difference between funding a contribution to an upstream project and funding the upstream maintainers who have to support it afterwards.

I think this showcases the other issue I commented a few years ago [1].

Pyodide and Cloudflare don't use the real network stack for http requests (they patched the functions to use JS `fetch` underneath). They also patched Python's async event loop to use JS event loop.

This has a major downside: incompatibility issues. JS event loop is preemptive (async functions get called regardless of you calling await on them) while Python is lazy (async functions only execute when you await them).

In my belief, the network semantics should be preserved. When the behavior differs issues start arising.

[1] https://news.ycombinator.com/item?id=39907240

> JS event loop is preemptive

I think the word you want is "eager". Preemptive usually refers to things like signal handlers: when the signal is received the handler "preempts" normal execution without waiting for an explicit yield point.

In any case, with the WebLoop, Python coroutines stay lazy. The fundamental primitive a Python event loop needs to implement is call_later() which maps fairly cleanly to `setTimeout()`.

The reason we want to use the JS event loop is that the JS event loop is where all the actual I/O events in a JavaScript runtime happen. If you create a second event loop and run it, it will block actual IO on the JS event loop. So making uvloop work would be pointless.

> I think the word you want is "eager".

Yup, thanks for the correction! (and appreciate the extra insight!)

> Pyodide and Cloudflare don't use the real network stack for http requests

Pyodide in browsers _can't_ use the real network stack because of fundamental security principles of browsers. With direct networking you could get around the CORS restrictions.

Thanks to Gyeongjae Choi's work, Pyodide in Node/Cloudflare can use direct sockets. See this section of the blog post: https://blog.cloudflare.com/python-workers-ga/#using-postgre...

> I'm glad the work was contributed upstream and is useful to Pyodide and Cloudflare. But I think there is a meaningful difference between funding a contribution to an upstream project and funding the upstream maintainers who have to support it afterwards.

Any of the upstream maintainers consider blocking merge unless Cloudflare signs a support contract?

Not an easy process to navigate, but if you find an internal champion that can help make the case, maintenance can be funded. For critical deps, companies are generally down to budget some financial support if you ask and know how to work the process.

The Emscripten support has actually been merged in urllib3 for a while.

We intentionally keep that module experimental and don't actively maintain it in urllib3. The maintainers don't currently work on new features, bug fixes, or security hardening for it, and we explicitly say so in our security policy.

So this isn't really a case of blocking a merge until someone funds it. The code is already upstream; we just haven't taken on the ongoing maintenance of that backend.

If Cloudflare needs ongoing support for it, we'd be happy to discuss that with them.

It always seemed so strange to me that "native" http clients were so lousy or many cases even non-existent considering that that's half of development these days.

Reply on news.ycombinator.com