Comment by jillesvangurp | Hacker News Reader

Comment by jillesvangurp | Hacker News Reader

Parent Full thread

Why does python have to be slow? Improvements over the last few releases have made it quite a bit faster. So that kind of counters that a bit. Apparently it didn't need to be quite as slow all along. Other languages can be fast. So, why not python?

I think with the GIL some people are overreacting: most python code is single threaded because of the GIL. So removing it doesn't actually break anything. The GIL was just making the use of threads kind of pointless. Removing it and making a lot of code thread safe benefits people who do want to use threads.

It's very simple. Either you did not care about performance anyway and nothing really changes for you. You'd need to add threading to your project to see any changes. Unless you do that, there's no practical reason to disable the GIL for you. Or to re-enable that once disabled becomes the default. If your python project doesn't spawn threads now, it won't matter to you either way. Your code won't have deadlocking threads because it has only 1 thread and there was never anything to do for the GIL anyway. For code like that compatibility issues would be fairly minimal.

If it does use threads, against most popular advise of that being quite pointless in python (because of the GIL), you might see some benefits and you might have to deal with some threading issues.

I don't see why a lot of packages would break. At best some of them would be not thread safe and it's probably a good idea to mark the ones that are thread safe as such in some way. Some nice package management challenge there. And probably you'd want to know which packages you can safely use.

> Why does python have to be slow?

Because the language's semantics promise that a bunch of insane stuff can happen at any time during the running of a program, including but not limited to the fields of classes changing at any time. Furthermore, they promise that their integers are aribtrary precision which are fundamentally slower to do operations with than fixed precision machine integers, etc.

The list of stuff like this goes on and on and on. You fundamentally just cannot compile most python programs to efficient machine code without making (sometimes subtle) changes to its semantics.

_________

> I don't see why a lot of packages would break. At best some of them would be not thread safe and it's probably a good idea to mark the ones that are thread safe as such in some way. Some nice package management challenge there. And probably you'd want to know which packages you can safely use.

They're not thread safe because it was semantically guaranteed to them that it was okay to write code that's not thread safe.

There are different definitions of slow, though. You might want arbitrary precision numbers but want it to be reasonable fast in that context.

I don't agree that it is "insane stuff", but I agree that Python is not where you go if you need super fast execution. It can be a great solution for "hack together something in a day that is correct, but maybe not fast", though. There are a lot of situations where that is, by far, the preferred solution.

There are ways to design languages to be dynamic while still being friendly to optimizing compilers. Typically what you want to do is promise that various things are dynamic, but then static within a single compilation context.

julia is a great example of a highly dynamic language which is still able to compile complicated programs to C-equivalent machine code. An older (and less performant but still quite fast) example of such a language is Common Lisp.

Python makes certain choices though that make this stuff pretty much impossible.

Not disputing it, but people don't pick Python because they need the fastest language, they pick it for friendly syntax and extensive and well-supported libraries. I loved Lisp, but none of the lisps have anything like Python's ecology. Julia, even less so.

People don't pick languages for language features, mostly. They pick them for their ecosystems -- the quality of libraries, compiler/runtime support, the network of humans you can ask questions of, etc.

> loved Lisp, but none of the lisps have anything like Python's ecology. Julia, even less so.

None of the lisps have anything close to julia's ecology in numerical computing at least. Can't really speak to other niches though.

> People don't pick languages for language features, mostly. They pick them for their ecosystems -- the quality of libraries, compiler/runtime support, the network of humans you can ask questions of, etc.

Sure. And that's why Python is both popular and slow.

I think that to a certain extent the quality of libraries can depend on language features.

> I loved Lisp, but none of the lisps have anything like Python's ecology.

Try Hissp (https://github.com/gilch). Lisp programming and Python's ecosystem.

Common Lisp is probably not a good point of comparison. It offers comparable (if not more) dynamism to Python and still remains fast (for most implementations). You can redefine class definitions and function definitions on the fly in a Common Lisp program and other than the obvious overhead of invoking those things the whole system remains fast.

Common lisp is in fact a good point of comparison once you look at how it's fast. The trick with Common Lisp is that they made a foundation of stuff that can actually be optimized pretty well by a compiler, and made that stuff exempt from being changed on the fly (or in some cases, just made the the compiler assume that they won't change on the fly even if they do, resulting in seg-faults unless you recompile code and re-generate data after changing stuff).

This is how Common Lisp people can claim that the language is both performant and flexible. The performant parts and the flexible parts are more disjoint than one might expect based on the way people talk about it.

But anyways, Common Lisp does manage to give a high degree of dynamism and performance to a point that it surely can be used for any of the dynamic stuff you'd want to do in Python, while also giving the possibility of writing high performance code.

Python did not do this, and so it'll be impossible for them to offer something like common lisp perf without breaking changes, or by just introducing a whole new set of alternatives to slow builtins like class, int, call, etc.

> > Why does python have to be slow?

> Because the language's semantics promise that a bunch of insane stuff can happen at any time during the running of a program, including but not limited to the fields of classes changing at any time.

You originally claimed Python is slow because of its semantics and then compare later to CL. CL has a very similar degree of dynamism and remains fast. That's what I'm saying makes for a poor comparison.

CL is a demonstration that Python, contrary to your initial claim, doesn't have to forfeit dynamism to become fast.

I think you're missing the point. Common Lisp is very dynamic yes, but it was designed in a very careful way to make sure that dynamism does not make an optimizing compiler impossible. That is not the case for Python.

Not all dynamism is the same, even if the end result can feel the same. Python has a particularly difficult brand of dynamism to deal with.

> CL has a very similar degree of dynamism and remains fast.

But not the dynamic parts remain "really" fast. Common Lisp introduced very early a lot of features to support optimizing compilers -> some of those reduce "dynamism". Code inlining (-> inline declarations), file compiler semantics, type declarations, optimization qualities (speed, compilation-speed, space, safety, debug, ...), stack allocation, tail call optimization, type inferencing, ...

> You can redefine class definitions and function definitions on the fly in a Common Lisp program and other than the obvious overhead of invoking those things the whole system remains fast.

You can also treat Julia as C and recompile vtables on the fly.

"hack together something in a day" JPM Athena trading platform had 35 million lines of code in 2019 with about 20k commits a week

> Because the language's semantics promise that a bunch of insane stuff can happen at any time during the running of a program, including but not limited to the fields of classes changing at any time. Furthermore, they promise that their integers are aribtrary precision which are fundamentally slower to do operations with than fixed precision machine integers, etc.

> The list of stuff like this goes on and on and on. You fundamentally just cannot compile most python programs to efficient machine code without making (sometimes subtle) changes to its semantics.

People said the same thing about JavaScript, where object prototypes can change at any time, dramatically changing everything. But it turned out JavaScript can run fast if you try hard enough. I suspect the same would be true for Python if a similar amount of resources was poured into it (probably with a similar "try the fast path, abort and fall back to a slow path in the extremely rare case that someone actually is doing funky metaprogramming" approach).

SELF, Smalltalk, Common Lisp are even crazier and have good performance, that has always been a common excuse, which is being proven false as proper resources are now being addressed into CPython.

And that while ignoring all the engineering that has gone into PyPy, largely ignored by the community.

Reply on news.ycombinator.com