Design of CPython’s Compiler
docs.python.org
docs.python.org
Do people just not care about performance? Is GIL locked performance just good enough? Do developers hate parens enough to justify the huge sacrifice in everything else?
Have you found some devastating performance problem with Python that you want to share? Because I'm not aware of one, as long as you don't try to write Java code in Python. If you want lots of concurrency for some special purpose, I can recommend gevent as a very nice option.
If I have a performance problem it's usually due to a tight inner loop and in the rare event that it can't be fixed based on judicious profiling, it is not really a huge problem to write a little code in C. I don't understand why I would have to switch to LISP to deal with a rare performance problem.
That means you are aware of one.
Anyways, I guess I have my answer, performance problems just aren't very common for many programmers. Keep in mind that profiling is only a solution to certain problems, due to Amdahl's law. To say that all interesting problems fit in that mould is disingenuous.
> it is not really a huge problem to write a little code in C.
Sure, but any language can do that. GCC is fast, but I thought we wanted to write code in something higher level? Or did we just give up?
Again - I am not aware of any devastating performance problem with Python - and in general I don't hear anything about the GIL except when someone is trashing Python from a list of talking points.
You suggest I'm being disingenuous by attributing to me the statement that "all interesting problems" are solved by profiling. But this is really dishonest, as I didn't say anything like that to begin with. Nobody was even discussing "all interesting problems".
The right way to deal with Python programs that perform poorly, in PRACTICE, is to start by measuring - and then focus your optimization effort on the parts of the code which are actually creating the problem. The vast majority of the time, there would be no need to switch off of Python, but even in those incredibly rare cases it is certainly possible to write a little function in C and call it a day.
Not to start by switching to LISP. LISP is not a miracle panacea. Ruby is not. Python is not.
For the purposes of optimizing highly performance-sensitive parts of a program, I prefer writing C to writing LISP. For the purpose of writing new software, I would in many cases choose Python and leave performance concerns to when they are actually demonstrably important, because I tend to get more-than-acceptable performance from Python in most cases. I like C just fine, I have written C for many years - in the event I am ever facing a problem which is magically impossible to solve in Python, I will not hesitate to use C. That is not "giving up". That is simply saying that there really is NOT any devastating reason why someone should avoid Python out of some evidence-free anxiety about performance. In the worst case, you can still use C to handle the nasty bits.
So I have NO reason to switch to LISP, least of all some perceived performance advantage due to a lot of hand-waving about the GIL. But anyone who wants to use it should feel free, I am not about to mount some kind of advocacy campaign against tools I don't happen to use.
No, the reason threads are used very heavily in Java (and elsewhere) is that we have hit the ceiling of single threaded performance in silicon based processors. If you want to scale, you add more processors. Right now, I can't write a shared address space, pure-Python, trivially parallel program that performs significantly [1] better on a 2011 8 core i7 than on a single core Pentium 4 from almost ten years ago.
That seems silly to me, but I understand that you can still build cool things with Python, and I'm not suggesting that you stop doing so. I am working with Python, too, and I love it. I was simply suggesting that Python is missing an opportunity to become a more powerful, more general purpose language.
[1] We do have to take into account some improvements in branch prediction, cache sizes, memory speeds, etc.
So where are they?
But most people don't like them.
This means you don't have as large of a following. Which means you don't have as many libraries. Which makes new programmers even more likely to choose python over lisp. Additionally python looks more like whatever languages people learn in intro CS classes. This further increases python's following....and its libraries...and it keeps snowballing.
And to answer your other questions, yeah, most people don't care about performance and GIL-locked performance is just good enough for what they're doing.
Most people these days are gluing things together and essentially doing plumbing (all of it single-threaded), and python is great for that.
Machines are fast enough these days that most tasks don't call for multiple cores/threads or even really efficient single-thread performance. It's just the way things are.
I don't know why lisp failed to become mainstream, but I wish the GNU vision of using guile as a universal extension language had succeeded. How different using computers would be today! Instead of the superficial changes in GNOME 3 and KDE 4, GNU would have a unified desktop vision, where all programs are as extensible as Emacs. We would have personal computing environments that evolve to handle our workflow.
Instead, we keep confusing simplicity with lack of extensibility for regular users. We keep creating incomplete scripting environments from scratch. No one writes one off Firefox extensions or OpenOffice plugins the way they do Emacs macros... But it didn't have to be this way. It's 2011 and we should all be scripting our computers, but instead we are slaves to software and its needs. We are forced to repeat more and more repetitive tasks as our programs improve.
This comment was automatically written by M-x commentator
This is a good companion article to that design doc - http://eli.thegreenplace.net/2010/06/30/python-internals-add...