A Team at Microsoft Is Helping Make Python Faster | Hacker News Reader

A Team at Microsoft Is Helping Make Python Faster

devblogs.microsoft.com

41 pointspieterr26 comments

I think it's fairly well understood at this point that a big reason for Python's slow performance is the desire to keep the interpreter simple so that people can more easily contribute to it. I'd add that this has evidently been a great strategy; Python is beloved and its ecosystem is among the best of all languages. However, approaches should evolve over time and Python is at the point of maturity where people probably don't need to be hacking on the interpreter, and the inefficiencies are causing problems; for instance, in server costs. Some groups have even forked Python to implement their own optimisations, which fractures the ecosystem. Hence, the time has come to focus on performance, for the health of the language's community, as well as the social benefits. All this is to say that in my opinion, Python continues to be a well-managed project and it is still growing in a positive direction

Yeah but coming from someone who has read a lot of MS produced code I'm a little frightened lol. But if there is one thing MS knows it is compilers.

Microsoft is a giant collection of individuals, each with their own programming abilities. Specifically, Guido is a Microsoft employee, he's at the head of this team. And my understanding is that they're going to go the traditional PEP route to mainline their contributions.

Also Mark Shannon who drew up the original Faster Python plan that spurred this work, joined Microsoft to work on this.

I guess of course I should not expect to see IronPython mentioned under the "Microsoft’s Commitment to Python" section.

Doesn't surprise me unfortunately, any languages that target .NET that aren't C# or VB are ignored.

At this point all optimizations that make the codebase more complicated should be postponed. Finally someone made a good enough GIL-less patch a couple of versions back, this should be analysed and integrated ASAP so that we take advantage of today's multicore systems. MS's 25% speed improvement is nice but I an not sure it doesn't move things in a wrong direction.

Frankly speaking the target 5x improvement, if everything works out as desired, doesn’t change too much overall. Python is often magnitudes slower than other languages, oftentimes even other scripting languages. So for me at least it won’t change when I use python and when I decide to use/rewrite in another language.

Python is really good at gluing code together. I predict that Python will continue to see tremendous success if it can be fast enough that using it as glue is a no-brainer.

As an example, I maintain a SaaS with a backend written in Python. Only one third of the time in the backend is spent executing Python code, the rest is spent talking with remote services (database, third-parties...). Switching to a language an order of magnitude faster would barely change the perceived responsiveness of the backend.

On the other hand, I welcome any speed improvements in the interpreter because from my perspective it really is free performance.

IIRC, that 5x figure was assuming they could improve about 50% across four consecutive versions, as 1.5^4 ~= 5. I think the first stage improved things by 25% on average, so I wonder if that initial 5x figure should be updated to 2.5x, as 1.25^4 ~= 2.5.

I hope it does not become like node-chakracore and the "VM neutrality" and other farces intended to 3E node, but now with Python.

Or Microsoft R Open and MRAN, the 3E of R and CRAN.

It is a long way from feeling immediately Halloween or extend/… to now listening to a group hired by Microsoft working on such a major open source project without too much worrying. Cheers.

At this point isn’t pypy the peak effort on this front?

If things need to go faster it’s probably easier to either write it in go or rust, or write a small extension in C or Rust I’d think?

Embrace, extend, extinguish - https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

I remember that era quite well, especially as I naturally gravitated towards Linux and open source. It was a genuine concern at the time.

Seeing as that was over 25 years ago I have to wonder how much it holds true today. Looking at MS moves over the past decade (the .NET ecosystem, Mono, VS Code, Github, Azure, etc) I’m not that concerned.

Really? This is pretty clearly the "embrace" step. I can't help but see things like the inability to use passwords with github unless you use the github-specific cli tool as an attempt to extend.