Show HN: Socketify.py: Http/Https and WebSockets servers for PyPy3 and Python3
github.com
github.com
Interesting results nevertheless.
If you look at it as a framework that minimises the networking overhead, then fine, it's an interesting piece of software.
If on the other hand you look at it like a "fast" web framework then things start to change and the discussion gets a bit more complicated.
So for example, you look at the source code of the applications being benchmarked (example: https://github.com/cirospaciari/socketify.py/blob/main/bench...) and you immediately see it's simply returning the string "hello world"). Which means that it's almost 100% of the time running in the fast path / best case.
My guess is that as soon as you start doing any kind of computation in the request handler in normal non-super-optimized python (trival example: validating some headers and/or checking some signatures as you would do with jwt tokens for example) then the python-vs-golang gap will start to go back to favour golang.
And then again, it boils down to what you're doing: anything io-intensive might benefit from the unetworking/uwebsocket beneath, anything cpu-intensive will benefit from the golang compiler producing native executable code.
Nice work anyways.
Pydantic is indeed really slow (https://ltworf.github.io/typedload/performance.html)
typedload is faster and written in pure python (unlike pydantic).
I know they are rewriting it in rust… but I don't know what will come out of it. After all my pure python implementation already manages to beat pydantic's binary, and when unions are in use, also apischema's binary.
If that is all one's application does, and can use your library in their organization/team, that's great. However a 2-3x performance boost for the parsing stage for a use case like an API call might not matter when that could be overshadowed by validation and/or upstream API calls. A realistic app would likely use a validation library like Pydantic's [2] to throw a custom typed-error that can be processed, e.g., localization, before returning it downstream.
[1] https://github.com/ltworf/typedload/blob/37c72837e0a8fd5f350...
Did you somehow miss all the other tests, even thought they are higher on the page? The important test is shown first: loading objects.
I'm not trying to benchmark my wifi or my disk. The IO time is not included there on purpose. Of course a bigger application that does other things wouldn't see this huge difference in performance. But I'm testing the performance of a library here.
typedload can use validators, but since it doesn't reimplement attrs/dataclass, I had no interest in testing those… it'd be a race between libraries I didn't write.
typedload's exceptions contain enough detail to tell the end user what went wrong.
when encountering errors in lists, typedload slows down due to keeping compatibility with python 3.7. However in my test with loading objects, despite the slowdown it remains faster than pydantic.
Not a Python expert, but could the Pydantic tests be possibly not realistic and/or misleading because they are using kwargs in __init__ [1] to parse the object instead of calling the parse_obj class method [2]? According to some PEPs [3], isn't Python creating a new dictionary for that parameter which would be included in the timing? That would be unfortunate if that accounted for the difference.
Something else I think about is if a performance test doesn't produce a side effect that is checked, a smart compiler or runtime could optimize the whole benchmark away. Or too easy for the CPU to do branch prediction, etc. I think I recall that happening to me in Java in the past, but probably not happened here in Python.
[1] https://github.com/ltworf/typedload/blob/37c72837e0a8fd5f350...
[2] https://docs.pydantic.dev/usage/models/#helper-functions
The kwargs thing is true… but I didn't design the API of pydantic. And it only happens on the small top level dictionary, not on all of them (unless it internally does it all the time).
Java has JIT, I agree that in that case keeping the output value is important. cpython isn't that smart. I honestly never tried to benchmark using pypy. I guess it could be interesting to try that.
I don't understand this this logic.
https://www.techempower.com/benchmarks/#section=test&runid=1...
Saying that Python is faster than Go with this as a proof looks like overreaching for Me. It only proves that wrapping C code in Python is fast. It’s an achievement, sure, but your app probably won’t be faster when you add thousands of Python lines between request and response.
I used https://github.com/TechEmpower/FrameworkBenchmarks and in the future i will write fortunes and other benchmarks with have more things going on!
> It only proves that wrapping C code in Python is fast.
Yes, it proves that for this specific use case Python is faster.
And it is not claimed anywhere. Besides, cpython is not really know to be fast, as it is an interpreter? Why would its implementation language matter much?
And if you are not making reference to a specific implementation then the comparison does not make any sense.
Then it kind of make sense to talk about " pure Python" (the interpreted one) and "wrapped C", doesn't it?
Good. Because otherwise people would be mislead to think that Python itself is fast. The Python plus C combo has opposite tradeoffs in many dimensions.
Yeah this test basically shows that Python backed by uWS is crazy fast, but is not an direct comparison to uWS to Go.
This test is just an troughput test, with is very useful to measure raw performance.
More tools like caching tools, a better database client etc is need to construct an complete scenario, and i'm working on it! (Maybe in 1 week or 2 weeks will be done)
https://github.com/TechEmpower/FrameworkBenchmarks
https://www.techempower.com/benchmarks/#section=test&runid=1...
Correction. It shows that uWS is crazy fast. Sure if python us used as a thin glue between something like "select username from user" and uWS or send file over uWS then the result is fast. Do something meaningful in Python itself an it will be slow.
This is not to diminish your work. Python is mostly glue anyways so in average the end result should be quite ok
It is positive. You've created very nice "Web Python". No arguments here.
It's kind of hopeless, Python still needs to fork per core to get any performance? So if you have 8 cores you're actually running 8 processes, so 8 DB pool etc ...
Running one process per core has been working well for scaling huge websites for decades at this point.
1,000,000 requests per second on what I think is an 8-core system (?) is 1/1,000,000 * 8 = 8us per request. 1,250,000 requests per second would be 6.4 us per request. Is this really your make or break issue? There's a set of people who can say "yes" to that. There's a set of people who think it is yes. The latter is much larger than the former.
(Although arguably the ones worst off are the ones for whom it is their make-or-break issue, but they don't realize it....)
I can't join the discord however since I was banned for unknown reasons and cannot create a new account without giving them my phone number, which I refuse to do.
But remember many great things get hated upon in the beginning, take dropbox as an example. People on HN thought it was a shitty idea and they seem to be doing just fine :)
https://www.techempower.com/benchmarks/#section=test&runid=1...
https://github.com/django/django
https://github.com/mopemope/meinheld
So django meinheld is basically saying that i used Django served by meinheld in that benchmark.
Without real problem-solving (business logic) written in Python, this is only lightweight wrapper on C/C++. When the number of Python code start growing in the hot path, then these blazing-amazing thruput numbers tremendously will be going down.
Real world benchmarks take more time to prepare, we get that, but let’s be honest: a bold claim _needs_ bold evidence.
For example: all the deno/bun benchmarks often use the Node.js frameworks in an unnecessarily slow way and don't measure apples to apples. (like using uWS in bun but not uWS for Node in the benchmark).
This one in particular just measures the speed of "hello world" in plaintext and JSON with very specific parameters.
A while back I implemented a game rules engine and MCTS in Torch (the Lua library). The training loop spent like half of its time running the rules engine. Writing it in Python would have been a disaster in comparison. To really make good use of the hardware and spend more time in the optimized machine learning code provided by libraries, one would have to write their rules engine in some language other than Lua or Python.
But I also think that writing Python and then dropping down into C or C++ for performance is perfectly valid and often a great idea. Of course, there's nothing wrong with just using a different language that's a sensible middle ground between Python and C (like Go). But hold on to the baby when that bathwater is being dumped: there's nothing wrong with writing a blend of Python and C++ for real uses.
This is the case for basically every programming language or performant library that exists. Yet, I never see this when discussions exists about node libraries or other languages for that matter. I mean you could argue that node itself is just a thin wrapper around fast C++ libraries.
Who the fuck cares if the request goes down to some compiled C++ library? I still write my logic in Python and get this benefit anyway. This is what makes it great.
The way this is titled, you know someone will read this as "Python can be faster than go" and influence their decision when it comes to performance ; that's actually why I clicked on that, thinking "how the hell did they achieve faster speed with Python".
If that is the case, that should influence the decision when it comes to performance. If performance is a very important deal in your app, you have to test this yourself so that you reach the performance you need.
Perhaps you can write C++ extensions to Python for the perf-heavy parts yourself instead of having to write the entire application in a language that takes much longer time to develop in?
Huh? It's the first comment for pretty much any of these stupid performance comparisons. Does someone really think node performance is great, or pick node because of its performance?
> I still write my logic in Python and get this benefit anyway.
That's part of the point. Writing your logic in Python is fine, but then your app performance isn't winning any comparisons. No one cares about the performance this article is explicitly talking about.
There is the 1% that are in a need of hard realtime performance, that thinks about GC performance as an issue that needs to be adressed. Sure I can get that for these people, these kind of discussions seems stupid and confusing since they have to develop in C/C++, Rust etc to get the pure performance they need. These people probably aren't in this thread because they know that Python is not a language for them and will never be.
Then there is the 99% crowd, which I am a part of, that could use slow-as-shit frameworks like Rails or Laravel because the performance is just a nice-to-have. These people, like myself, compares Python perf to perhaps PHP or Node because they are fast enough languages and in this kind of league a framework like OPs can make a huge difference. No matter what language you pick here you are basically always using something "beneth" it. Node has been picked by many because it's faster than for example Python and Ruby.
Then there is the people who are part of the 99% crowd but are just have to comment and point the needs of the 1% out because they are negative people who probably produce little themselves and have to hate on what other people create and distribute for free. This group is where most haters on HN belong in. :)
I started in PHP, when I went to Node in the early beginning I was deeply impressed by the great performance and nice features it had. Sure it was built upon C++ libraries but I am never gonna use C++ to write my web api. Still, I could use Node and get 2-3x AT LEAST the performance in all of my endpoints because of this and all my code was in javascript.
What I am trying to say is that I don't care if you pack my code and run it on a computer in Minecraft or your calculator, if it runs faster for very little to no pain on my side that is pure joy to me.
If you're a web developer, trying to pick a framework for websockets, you probably don't care that the Python framework is a libuv wrapper, while to Go framework is native.
So well done, I guess? I am really happy to see library authors taking performance seriously!
EDIT: This is assuming the benchmark is actually fair. I haven't looked at it, but it's not uncommon for benchmarks to be comparing apples and oranges.
Web developer dont care if the framework is written in native or it is an wrapper, developers just want something that works in a nice way :D
The benchmarks are in https://github.com/TechEmpower/FrameworkBenchmarks
EDIT: some preliminary results https://www.techempower.com/benchmarks/#section=test&runid=1...
So the concern is the title you used to get to the front page not the actual implementation.
well, a real world test would be both servers parsing 100kb json payload in each message
> EDIT: This is assuming the benchmark is actually fair. I haven't looked at it, but it's not uncommon for benchmarks to be comparing apples and oranges.
What is the point of this negativity? Why are you implying that the benchmarks might not be fair because you "haven't looked at it"?
It is as fast as the fastest C++ / Rust implementation.
I went back to Flask recently and keep finding everything I know is deprecated, or not best practice any more.
Then I read every week that some new framework is out that is better.
I'm tired of it. I end up spending more time learning new things than I do building new things! Maybe I should get off hn for a while
The thing is: You A) Don't need to learn the newest things if all you want is to create a cool app. And B) Won't get anything from learning about building apps without actually building them.
Maybe you do need to get off HN, but I recommend just browsing HN safely and knowing that there'll be time to learn these new things if you learn them through making cool stuff. If your priority is to make apps, these new frameworks shouldn't bother you so much. You can make a nice app just with Flask.
But I do feel old too. I used to think Angular was the hottest thing.. Now I hardly enjoy it lol but that's because my Angular-married colleagues use very outdated practices, which were "modern" a few years ago.
Whereas with art or math or writing projects I can come back to old notes decades later and basically pick up where I left off.
Anyway what worked fine yesterday isn't necessarily a bad choice today.
A few years ago I made a conscious decision that - at least during my remaining professional career - I will not switch to a new programming language anymore. Instead, I intend to keep specializing in Java. I embrace the fact that in 20 years I will be today's (Assembly|Fortran|C|...)-specialist. I do enjoy reading about other language features. I applaud the people at the cutting edge, experimenting with new stuff, and I am glad that Java is slowly incorporation some of those innovations. Every LTS release (2-3 years), I enjoy learning the curated set that Oracle's language architects decided to add.
This decision helped me tremendously with the fear of missing out on the latest and greatest.
"Any sufficiently large Flask project contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Django."
Just sticky with Django. It's stable, well-maintaned and includes most batteries out-of-the-box.
I would assume this is same case where - what ever Socketify is based on (pythons transport layer), is compared to FastHTTP. and mention of Fiber is click-bate.
The benchmarks are TechEmPower plaintext https://github.com/TechEmpower/FrameworkBenchmarks
preliminary results here: https://www.techempower.com/benchmarks/#section=test&runid=1...
[0] https://github.com/cirospaciari/socketify.py/tree/main/src/s...
src/socketify
├── __init__.py
├── __main__.py
├── asgi.py
├── cli.py
├── helpers.py
├── libsocketify_darwin_amd64.so
├── libsocketify_darwin_arm64.so
├── libsocketify_linux_amd64.so
├── libsocketify_windows_amd64.dll
├── loop.py
├── native
│ ├── Make.bat
│ ├── Makefile
│ ├── src
│ │ ├── libsocketify.cpp
│ │ └── libsocketify.h
│ └── uv_selector.txt
├── native.py
├── socketify.py
├── ssgi.py
├── status_codes.py
├── tasks.py
├── uv.py
├── uWebSockets
└── wsgi.py
make linux
cd ../
git add libsocketify_linux_amd64.soAnd it would have been the same with Python 2.7. This demonstrates yet again that the main value of Python has always been C-extensions.
The efforts of the old boys to add bloat in Python 3 (mostly written by other people of course, the old boys talk and rarely develop) have been misguided.
.NET were gaming their techempower benchmarks to advertise they were blazingly fast. Turns out, not so fast with real-world code.
Just as a piece of evidence, there are interpreted C++ (http://www.artificialworlds.net/wiki/IGCC/IGCC) and AOT Python (https://github.com/exaloop/codon) implementations.
When people refer to a language, most often they are using it as a synecdoche to refer to the whole language ecosystem and not just the the formal language definition.
If you look at the Python ecosystem, it is most definitely interpreted.
Or at least, that's how I'm hearing it from Google.
Just in case anyone else is getting it wrong like I was …
https://youtube.com/watch?v=v-n1vGeVIXo
(But really, https://youtu.be/u8_LDxZReTc)
I think it is fair to have a discussion about terminology and arguing with the actual meaning of terms. When there is a technical difference, it would be good to acknowledge that difference between terms. It does not hurt anyone to do that. One can still say something like: "When we communicate I will use the word x for meaning y, because it is shorter and more convenient."
Some languages have more of a proper spec than others, of course. If we look at for example Scheme dialects, then they often change their implementation for better conformance with the various rnrs standards. New Scheme dialects often early on try to conform very much to the standard to avoid issues later on.
Not yet it isn't. At least not for Python. I mean there are plenty of alternative implementations of Python but they aren't practical because they don't work with CPython modules (as far as I know anyway) so approximately nobody uses them. The de facto spec is "what CPython does".
I have run it with pandas+numpy and postgres+timescale without issue for the last year.
With Python, you don't even have to go "alternative" to use LLVM compiler infrastructure. You can use CPython + Numba and compile your kernels for both CPU and GPGPU on the go. Which is very practical.
Which is ironic, to compile a CUDA kernel in C++, you would have to use an alternative NVidia implementation of the language.
> general real world implementation
By this they mean the real-world-implementation that is in general use, which in this case is interpreted.
Yes, Python can be compiled and C++ can be interpreted. 99% of the time, they're not though.
When somebody says X language is interpreted, they mean that the most common and widely implementation is an interpreter. That's a useful bit of information to know about a language. "Correcting" them and saying that all languages can be interpreted OR compiled, doesn't really add anything of value to the discussion, other than to label yourself as a pedant.
On one hand, the conception from the 90s based on the most common Python implementation, on the other - a compiler that saves us tons of money. Which one adds value again?
That would be important in high-school debate club. In the real world, most people don't care about obscure edge cases and pedantry. In the industry vernacular, Python is an "interpreted language" and C++ is a "compiled language". Arguing against that in anything other than a highly (highly!) specialized context is just lighting candles in the wind.
We adopted it last year, and we don't rewrite code in C++ for GPGPU anymore. We save tons of effort on rewriting and even more on support.
How is this computer science?
I bet that's fun.
> There is no such thing as an "interpreted language"...
In the common vernacular, yes, there are.
All this is to say that a programming language these days is really the sum of its parts, not a spec.
E.g. many languages’ syntax is context-free, yet the languages themselves are almost always Turing-complete.
OTOH, interpreting C++ is actually a pain in the ass due to design choices it makes specifically regarding the semantics of translation units. You can see similar pain when trying to say, make a Go REPL: it's just not tuned for this.
It is of course technically possible to compile almost anything ahead-of-time. But, if you effectively wind up having to ship an interpreter or JIT into the resulting binary to run some code anyways, it's almost for naught. Both Python and JavaScript have eval. Python also has several other cases like Pickle where this can be an issue. PyPy being a JIT makes a lot of sense because it wants to be a drop-in replacement, and that makes the most sense for a language with these constraints. Codon can do AOT, but for practical reasons it is not nearly a drop-in replacement for CPython, just like the other Python AOT implementations.
A practical Python AOT is not compatible with loads of things that people associate with Python, like Django. PyPy is compatible with more, but even it is far from a drop-in today, and that is a problem.
> A language is just a set or rules and keywords. Everything you can fit into Backus–Naur form is already a language even if it doesn't have any implementation nor compiler neither interpreter.
This is objectively true. However, in practice, there are a finite number of available toolchains for any given language that exist today, and creating new ones is a non-trivial endeavor, especially production-quality ones. A language is just a set of rules and keywords. However, when people use Python and write Python, they are not merely writing Python to fit into those keywords and rules. They're writing Python to be executed and solve a problem, typically using CPython. Python is a language, but it's also an ecosystem.
In the same token, if something like Codon doesn't even support all of the things you can fit into Backus-Naur form about Python as it is in CPython and PyPy, can it even be called Python in this sterile technical sense?
My advice to you: Stop fighting that "misconception". See the other responses to back up the idea that Python is interpreted. The existence of a Python compiler is also not relevant here, as the linked project does not list that as a supported Python variant.
Let's assume that by some technicality you are not wrong. The "misconception" is so common that it is generally a fact in peoples mind. Rather than convince them, you are destroying your own credibility and probably making a fool of yourself in their eyes. And remember, I prefaced this with you not being wrong. It's worse for you if you are wrong. Just stop for your own good.
> Rather than convince them, you are destroying your own credibility and probably making a fool of yourself in their eyes.
I'd understand where you were coming from had the parent comment not explained what they meant and given examples.
Last year we adopted Numba for GPGPU programming. It is a Python compiler with LLVM back-end and full CUDA support. It generates the same code that NVCC (CUDA compiler for C++) does only it compiles Python code and not a C++ .cu dialect.
No one in our organization writes heterogeneous code in C++ anymore.
Sure there is. Even if it is technically possible to write an interpreter for a "compiled" language, or a compiler for an "interpreted" language, language design really pushes you towards one or the other. "Technically possible" can also represent a very real challenge, as that set of rules and keywords might make ahead-of-time compilation either impossible or pointless.
E.g. any language that exposes `eval` with an arbitrary argument will still need to ship an interpreter as part of the runtime even if you compile some of the code ahead of time. Monkey-patching is common in Python and Ruby, and is pretty hostile to compilation. Codon has several documented limitations around this. Perl 5 famously can't be statically parsed without evaluating the code as you go, which completely defeats any attempt at compiling it.
Inversely, You can write an interpreter for C++, but features like `consteval` and `constexpr` are meaningless without ahead-of-time compilation.
No they're not, `constexpr` and `consteval` indicate where you can use an expression, not when in the compilation/interpretation pipeline they need to be evaluated. The spec does not say that C++ has to be compiled, nor does the spec say "A compiler must execute `constexpr` expressions prior to entering `main()`".
If you think this is just being pedantic, well, that's kind of the point of a spec.
So when someone says "Python is an interpreted language" they are pleading to an implicit preexisting shared knowledge about the nature of the most commonly found usage of the concept.
Granted, when this shared knowledge doesn't match, that's where misunderstandings can happen, but that's the nature of human language. In this case I'd be hard pressed to think that any misunderstanding could have been possible for the immense majority of people reading this.