Hacker News Reader

Comment by BiteCode_dev | Hacker News Reader

Don't understand why you are being downvoted, coming from another language, it's an understandable surprise.

The explanation is that Python features heavy dynamism, meaning one can easily replace any part of the system at runtime (at any moment), including builtins, code objects, and even hook into the parser, AST, import mechanism, etc.

This renders inlining a dangerous exercice: say you inline the builtin len() function, how do you know it's not the intent of a code that runs later to replace it with a different implementation?

Now, there are ways to implement inlining, but they are not as straightforward as say, with a compiler, when you know nothing is going to change afterward.

So it's something that had be delayed.

View on HN

> The explanation is that Python features heavy dynamism

JavaScript too, any function can be dynamically changed at any time, but its function calls are 30x faster (roughly, I measured it just now with a for loop doing 100 million function calls, which took 0.5s in JS, 15s in python)

This is a realistic scenario if you want to run e.g. a custom statistics function on a large array in python so it can be annoying (and yes numpy can do things but that's no reason to keep the main language encouraging less readable code where you don't define separate functions)

Hence the "So it's something that had be delayed.", and not "this is something impossible".

Javascript has the benefit of having billions of dollars allocated to it from Google/Microsoft/Apple to hire dozen of amazing engineers full time over decades to work on it.

Python has existed since 1994, yet in 2011, the Python Software Fundation budget was less than $40K: https://pyfound.blogspot.com/2012/01/psf-grants-over-37000-t...

We are talking a difference of funding of 6 order of magnitudes. Not to mention one is a language that has to stand on its own, the second has an accidental monopoly on the most popular plateform in the world.

So it had to be delayed, unlike for JS.

> Python has existed since 1994, yet in 2011, the Python Software Fundation budget was less than $40K Exact, as usual it is mainly a question of human resources. Hence why I advocate for Python switching to GraalPython as the default runtime, it would enable much better performance in the next few years.

Also, I'm pretty sure cpython devs should ask Google/OpenAI/MicrosoftAI funding, how many millions can they waste on useless projects while not improving the core bottleneck..

You are asking for the rarest thing of all, a geek and volounteer worker that is also good at dealing with big entities and comfortable with managing money.

> its function calls are 30x faster (roughly, I measured it just now with a for loop doing 100 million function calls, which took 0.5s in JS, 15s in python)

Your benchmark also measures iterators and boxed integers, which were not in the JavaScript version, so it's not clear how much of the difference is due to function call overhead.

(Of course, it certainly doesn't help Python's performance that there's no simple way to do loops without iterators and integers without boxing. It would be nice if a future version could optimize the abstractions away.)

JS integers are value types unlike Java?!

On second thought, I'm not sure. Can you observe the number's object identity like you can in Python? Does JavaScript have to allocate a new Number object for each "i++"?

My point was that the Python version of Aardwolf's function call benchmark [1] is definitely using iterators and boxing all numbers, so it's doing a lot more than just calling a function, and the Java equivalent would look something like this:

    Long f(Integer x) {
        return new Long(x.longValue() * x.longValue());
    }

    // ...

    Double v = new Double(0.0);

    Iterator<Integer> range = IntStream.range(0, 100000000).iterator();
    while (range.hasNext()) {
        Integer i = range.next();
        v = new Double(v.doubleValue() + f(i).doubleValue());
    }

[1] https://news.ycombinator.com/item?id=31427506

> how do you know it's not the intent of a code that runs later to replace it with a different implementation?

I think progress could move forward by adding a compiler flag "assume no fully free function body replacement" you know just like in every other mainstream language.

It's not body replacement, it's function reference erasing. There is no way in Python to tell something has been replaced or not, since variables are dumb labels and a reference is the same kind for any object. In fact, there is no particular difference between 2 objects in python.

E.G: functions are objects, and any object can be called as a function. And it's just a reference to "something callable" in the namespace mapping (literally a dict in most implementations), which is mutable by definition

Also a compiler flag would not help, since most users don't compile the python VM.

Now, you could put a runtime flag, that list all stuff that are created for the first time in a namespace, and refuse to allow reassignment.

It's possible, but it would break a LOT of things and prevent many patterns.

The last attempt was to put guards, and to assume no replacement, but if a replacement occurs, then at this moment we revert locally this assumption.

The process is refined at each iteration, but there is no turn-key solution as you seem to believe.

> I think progress could move forward by adding a compiler flag "assume no fully free function body replacement" you know just like in every other mainstream language.

Ah yes, a compiler flag which literally says “break the language”, that sounds like a great feature which would be used a lot.

> assume no fully free function body replacement

There is no “body replacement”, the `len` builtin is looked up as a global in the module on every access, anyone can just replace the module’s global.

I'm just advocating for a compiler flag that says "do_reasonable things" just like in every other mainstream language. Python must free himself from those idiosyncrasies if it wants to stay relevant, migrations are not that hard especially since those patterns seems illegitimate or workaroundable (e.g. use overriding if you want to replace a function or do extension methods). At worst at least tag the concerned functions as dirty via an annotation which will tell cpython to locally deoptimize. I'm not arguing for an exceptional thing, just for conventionality and sanity.

> migrations are not that hard

You might recall that there are still holdouts who haven't migrated to Python 3. Breaking changes are generally not a good idea in mature languages.

One of our scientific collaborators is _literally_ of the view that "One day people will realise what a mistake python 3 was, and go back to Python 2" (which is a verbatim quote).

While this has happened with Perl (AFAIK, people realized what a mistake Perl 6 was, and there's a plan for Perl 7 to mostly go back to Perl 5, which makes it two breaking changes in a row), I see no signs of it happening with Python; instead, every new Python 3 release gets farther from Python 2.

I thought the Perl situation was a little more nuanced.

Although Perl 6 was meant to be a Python 3000 type thing, it was spun out into its own language (Raku). Perl 7 will continue the 5.x lineage, but with saner defaults. Thus, any code that’s around now should run, but the interpreter’s name might change.

(I think…I mostly keep up with Perl for nostalgia’s sake).

> Although Perl 6 was meant to be a Python 3000 type thing, it was spun out into its own language (Raku).

The problem for them is that Perl 6 actually had an official release (and IIRC, more than one) under the Perl 6 name; the rename of the language to Raku (which IIRC was originally the name of just one VM for running Perl 6 programs) came later. So anyone who managed to keep up with the latest release of the Perl language would have two huge breaking changes (Perl 5 to Perl 6 and Perl 6 to Perl 7), the second one undoing most of the changes of the first. (AFAIK, PHP avoided all that by deciding to go back before officially releasing PHP 6, so anyone who was following the latest release just jumped directly from PHP 5 to PHP 7, avoiding the breaking changes which had been planned for PHP 6.)

Incorrect, Raku(do) is not the VM. It's MoarVM (or JVM but that one is trailing behind). Rakudo is the reference implementation.

Also you should think in terms of Perl 5 -> 7. Raku is the language formally known as Perl 6. Perl 7 isn't undoing anything, it will be the successor to Perl 5. Perl 7 should have been 5.32 with saner defaults, but of course it's now going to take another 20 years of bikeshedding until this is happening.

https://www.perl.com/article/announcing-perl-7/

I get how you (and perhaps others) might think it's as you say. But I can tell you're not saying what you've said based on knowing it to be true but guessing it to be so. And while your guess isn't a surprising one given natural assumptions due to the names "Perl", "Perl 5", "Perl 6", and "Perl 7", it doesn't correspond to what has actually happened.

Anyone keeping up with the latest release of the Perl language has had near zero breakage for decades. (Indeed Perl has a well deserved reputation for having an outstanding track record in this regard compared to almost all other mainstream PLs.) I personally see every likelihood Perl 7 will extend that track record, though of course my crystal ball prognostications are necessarily based purely on what I see.

No one using P6 or Raku had a huge breaking change from Perl 5. No one using P6/Raku will have another one going to Perl 7.

If you presume Raku and Perl are different languages you'll get the essence of what has actually happened so far, and seems likely to be more or less true for the rest of this decade at least.

I don't think people considered Perl 6 a "mistake". It's more that they realized that it wasn't a successor to Perl 5 and is instead a separate language. The language itself isn't a mistake, it was treating it as the next version that was.

The Perl community may be attempting to "fix" that mistake with Perl 7, but the damage has been done. I think Perl 6 fractured the community so hard and scared away so many users that both Perl 7 and Raku are likely to be minor languages for the foreseeable future.

If either of them has legs, I think it's likely to be Raku. Because Raku is at least an interesting language with a lot of really interesting, powerful language features.

Perl 5/7 by virtue of its history, is mostly just another dynamic scripting language, but one with a particularly unfriendly syntax. Aside from CPAN and inertia, I think there is relatively little reason to write new Perl 5/7 code when PHP, Python, Ruby, and JavaScript are out there.

Raku is too slow for no obvious reason, plus every lib has to be rewritten from scratch, including the good and mature ones from Perl.

There are more interesting things happening with Ruby and Python. In fact there are Python libs for obscure stuff like CAN-bus and ISO-TP. Want to talk to a vehicle ECU? It can be done with Python.

There's also lack of momentum and quite a lot of bikeshedding in the Perl community. There were attempts to modernise Perl by rurban and others, but they were met with unecessary resistence. Without community support they all ended up as one man shows. Perl is pretty much a dead end. You are hearing this from someone who still writes Perl code every day for work. My latest proof of concept was done in Ruby and it will probably end up as production code.

> Raku is too slow for no obvious reason

I wonder when it was the last time you tried Raku. Or compared it to a Perl script with Moose.

> plus every lib has to be rewritten from scratch, including the good and mature ones from Perl.

The good and mature ones from Perl can be used from Raku with Inline::Perl5.

I've tried for several iterations until I finally gave up and moved on to Ruby, Racket etc. Moose is quite slow itself. I never use it or anything that depends on it. I use Moo or Mojo::Base instead.

Inline::Perl5 is quite an ugly last resort solution. And you still need a Perl intepreter as opposed to calling on C code from D where you just need the libs.

If you gave up on it, why still keep saying it is slow, when you have no current information about that?

Also, when you're talking slow: why is it that any performant Perl module, actually has most of its logic written in XS (aka C)? So I think it's shows quite a bit of hutzpah to call Inline::Perl5 "an ugly last resort solution", whereas a *lot* of upstream CPAN modules rely on the hack that is XS to make them performant. To give you an example: the pure Raku version of Text::CSV is more than 2x as fast as the pure Perl version of Text::CSV.

That mistake was sad. In the end they succeeded at creating a language that fixed most Perl problems and is actually quite awesome. But at the cost of catastrophic loss of mindshare.

It needed to be done; there were several choices in python 1/2 holding the language back, that all but necessitated breaking changes. Mostly with string/bytes/unicode handling. And the non-breaking route would have been a long term pain.

They didn't decide to break over print() and cosmetic changes, it runs way deeper.

I think you're missing how deep the function reference change goes in standard code. What about unittest mocks? Conditional module loading? Enabling/disabling tracing features? I'd bet there's some dynamic function reference assignment that happens during python console startup.

It's not unreasonable. It's just what the language makes easy and useful. It's still used much less than for example method aliasing in Ruby which has about the same result.

> just like in every other mainstream language

JavaScript doesn't do this, you can replace properties of window at will. I don't think Ruby does it, or Lua. PHP probably does it under some circumstances in modern versions but only because it's remarkably un-dynamic for an interpreted language.

This is normal fare for high-level interpreted languages, for better or worse.

and yet I'm pretty sure JS can inline just fine

It can, but it doesn't have a "do reasonable things" mode. Instead it notices when you do something unreasonable and undoes its optimizations.

Sure, so can pypy.

But that means you have to add all sorts of dependency tracking, such that you are able to deoptimise any function affected by mutation on things which were optimised in or out.

This means your complexity increases very fast, very high, before you can have anything which actually works.

If by "just fine" you mean, "with incredible engineering resources funded by a small number of large companies with deep pockets and with some performance cost to check that the optimization is still valid", yes.

Reply on news.ycombinator.com