Are you saying python never inlined functions before??
The explanation is that Python features heavy dynamism, meaning one can easily replace any part of the system at runtime (at any moment), including builtins, code objects, and even hook into the parser, AST, import mechanism, etc.
This renders inlining a dangerous exercice: say you inline the builtin len() function, how do you know it's not the intent of a code that runs later to replace it with a different implementation?
Now, there are ways to implement inlining, but they are not as straightforward as say, with a compiler, when you know nothing is going to change afterward.
So it's something that had be delayed.
I think progress could move forward by adding a compiler flag "assume no fully free function body replacement" you know just like in every other mainstream language.
E.G: functions are objects, and any object can be called as a function. And it's just a reference to "something callable" in the namespace mapping (literally a dict in most implementations), which is mutable by definition
Also a compiler flag would not help, since most users don't compile the python VM.
Now, you could put a runtime flag, that list all stuff that are created for the first time in a namespace, and refuse to allow reassignment.
It's possible, but it would break a LOT of things and prevent many patterns.
The last attempt was to put guards, and to assume no replacement, but if a replacement occurs, then at this moment we revert locally this assumption.
The process is refined at each iteration, but there is no turn-key solution as you seem to believe.
Ah yes, a compiler flag which literally says “break the language”, that sounds like a great feature which would be used a lot.
> assume no fully free function body replacement
There is no “body replacement”, the `len` builtin is looked up as a global in the module on every access, anyone can just replace the module’s global.
You might recall that there are still holdouts who haven't migrated to Python 3. Breaking changes are generally not a good idea in mature languages.
Although Perl 6 was meant to be a Python 3000 type thing, it was spun out into its own language (Raku). Perl 7 will continue the 5.x lineage, but with saner defaults. Thus, any code that’s around now should run, but the interpreter’s name might change.
(I think…I mostly keep up with Perl for nostalgia’s sake).
The problem for them is that Perl 6 actually had an official release (and IIRC, more than one) under the Perl 6 name; the rename of the language to Raku (which IIRC was originally the name of just one VM for running Perl 6 programs) came later. So anyone who managed to keep up with the latest release of the Perl language would have two huge breaking changes (Perl 5 to Perl 6 and Perl 6 to Perl 7), the second one undoing most of the changes of the first. (AFAIK, PHP avoided all that by deciding to go back before officially releasing PHP 6, so anyone who was following the latest release just jumped directly from PHP 5 to PHP 7, avoiding the breaking changes which had been planned for PHP 6.)
Also you should think in terms of Perl 5 -> 7. Raku is the language formally known as Perl 6. Perl 7 isn't undoing anything, it will be the successor to Perl 5. Perl 7 should have been 5.32 with saner defaults, but of course it's now going to take another 20 years of bikeshedding until this is happening.
Anyone keeping up with the latest release of the Perl language has had near zero breakage for decades. (Indeed Perl has a well deserved reputation for having an outstanding track record in this regard compared to almost all other mainstream PLs.) I personally see every likelihood Perl 7 will extend that track record, though of course my crystal ball prognostications are necessarily based purely on what I see.
No one using P6 or Raku had a huge breaking change from Perl 5. No one using P6/Raku will have another one going to Perl 7.
If you presume Raku and Perl are different languages you'll get the essence of what has actually happened so far, and seems likely to be more or less true for the rest of this decade at least.
The Perl community may be attempting to "fix" that mistake with Perl 7, but the damage has been done. I think Perl 6 fractured the community so hard and scared away so many users that both Perl 7 and Raku are likely to be minor languages for the foreseeable future.
If either of them has legs, I think it's likely to be Raku. Because Raku is at least an interesting language with a lot of really interesting, powerful language features.
Perl 5/7 by virtue of its history, is mostly just another dynamic scripting language, but one with a particularly unfriendly syntax. Aside from CPAN and inertia, I think there is relatively little reason to write new Perl 5/7 code when PHP, Python, Ruby, and JavaScript are out there.
There are more interesting things happening with Ruby and Python. In fact there are Python libs for obscure stuff like CAN-bus and ISO-TP. Want to talk to a vehicle ECU? It can be done with Python.
There's also lack of momentum and quite a lot of bikeshedding in the Perl community. There were attempts to modernise Perl by rurban and others, but they were met with unecessary resistence. Without community support they all ended up as one man shows. Perl is pretty much a dead end. You are hearing this from someone who still writes Perl code every day for work. My latest proof of concept was done in Ruby and it will probably end up as production code.
I wonder when it was the last time you tried Raku. Or compared it to a Perl script with Moose.
> plus every lib has to be rewritten from scratch, including the good and mature ones from Perl.
The good and mature ones from Perl can be used from Raku with Inline::Perl5.
Inline::Perl5 is quite an ugly last resort solution. And you still need a Perl intepreter as opposed to calling on C code from D where you just need the libs.
Also, when you're talking slow: why is it that any performant Perl module, actually has most of its logic written in XS (aka C)? So I think it's shows quite a bit of hutzpah to call Inline::Perl5 "an ugly last resort solution", whereas a *lot* of upstream CPAN modules rely on the hack that is XS to make them performant. To give you an example: the pure Raku version of Text::CSV is more than 2x as fast as the pure Perl version of Text::CSV.
They didn't decide to break over print() and cosmetic changes, it runs way deeper.
It's not unreasonable. It's just what the language makes easy and useful. It's still used much less than for example method aliasing in Ruby which has about the same result.
JavaScript doesn't do this, you can replace properties of window at will. I don't think Ruby does it, or Lua. PHP probably does it under some circumstances in modern versions but only because it's remarkably un-dynamic for an interpreted language.
This is normal fare for high-level interpreted languages, for better or worse.
But that means you have to add all sorts of dependency tracking, such that you are able to deoptimise any function affected by mutation on things which were optimised in or out.
This means your complexity increases very fast, very high, before you can have anything which actually works.
JavaScript too, any function can be dynamically changed at any time, but its function calls are 30x faster (roughly, I measured it just now with a for loop doing 100 million function calls, which took 0.5s in JS, 15s in python)
This is a realistic scenario if you want to run e.g. a custom statistics function on a large array in python so it can be annoying (and yes numpy can do things but that's no reason to keep the main language encouraging less readable code where you don't define separate functions)
Javascript has the benefit of having billions of dollars allocated to it from Google/Microsoft/Apple to hire dozen of amazing engineers full time over decades to work on it.
Python has existed since 1994, yet in 2011, the Python Software Fundation budget was less than $40K: https://pyfound.blogspot.com/2012/01/psf-grants-over-37000-t...
We are talking a difference of funding of 6 order of magnitudes. Not to mention one is a language that has to stand on its own, the second has an accidental monopoly on the most popular plateform in the world.
So it had to be delayed, unlike for JS.
Also, I'm pretty sure cpython devs should ask Google/OpenAI/MicrosoftAI funding, how many millions can they waste on useless projects while not improving the core bottleneck..
Your benchmark also measures iterators and boxed integers, which were not in the JavaScript version, so it's not clear how much of the difference is due to function call overhead.
(Of course, it certainly doesn't help Python's performance that there's no simple way to do loops without iterators and integers without boxing. It would be nice if a future version could optimize the abstractions away.)
My point was that the Python version of Aardwolf's function call benchmark [1] is definitely using iterators and boxing all numbers, so it's doing a lot more than just calling a function, and the Java equivalent would look something like this:
Long f(Integer x) {
return new Long(x.longValue() * x.longValue());
}
// ...
Double v = new Double(0.0);
Iterator<Integer> range = IntStream.range(0, 100000000).iterator();
while (range.hasNext()) {
Integer i = range.next();
v = new Double(v.doubleValue() + f(i).doubleValue());
}
> During a Python function call, Python will call an evaluating C function to interpret that function’s code. This effectively limits pure Python recursion to what’s safe for the C stack.
> In 3.11, when CPython detects Python code calling another Python function, it sets up a new frame, and “jumps” to the new code inside the new frame. This avoids calling the C interpreting function altogether.
> Most Python function calls now consume no C stack space. This speeds up most of such calls. In simple recursive functions like fibonacci or factorial, a 1.7x speedup was observed. This also means recursive functions can recurse significantly deeper (if the user increases the recursion limit). We measured a 1-3% improvement in pyperformance.
https://docs.python.org/3.11/whatsnew/3.11.html#inlined-pyth...
Sadly, it isn't true parallelism, more like green threads. And they didn't manage to kill the GIL.
Python's own stackframes are heap-allocated and chained (so each function has its own stack, in essence), so its use of a single unified allocation space for stack is already an implementation detail of the interpreter.