I Am Worried About The Future Of Python
ironfroggy.com
ironfroggy.com
1) Python 3's backward incompatibility has made a mess of library support and has trapped us on Python 2.7.
2) memory usage is a problem on mobile for nontrivial Python programs; iOS isn't even a formally supported target.
3) Pythonic programs can't use multiple cores.
Now before you jump down my throat and tell me how bad a programmer I must be, etc, etc, let me provide some anticipatory clarifying comments:
1) Most libraries do now work in Python 3.3; we are waiting on gevent, pyasn1, and tlslite, and, of course, all our own stuff.
2) There's at least one fork of Python 3 on bitbucket with a bunch of promising memory-reducing changes (can't find it now).
3) Yes, I know about (multi)processing and that real threads are hard to work with.
Literally every few weeks we are confronted with a new problem caused by one of the above. The latest one: there is no WinRT port of Python yet, and the one Martin Loewis is working on is for Python 3.3. (Of course: Python 2.7 is barred from noncritical patches.) So we need to either make 2.7 work on WinRT or not ship a WinRT version. Windows 8 launches Oct 26; Python is frankly just not ready, and no one but Martin L and a few others seem to care (or even notice). (And yes, I know Win32 stuff still runs under Windows 8... but not on the ARM devices, and not in the UI-formerly-known-as-Metro.)
I wouldn't say I regret having chosen Python as the basis of a 250k-line system, because in retrospect it was still the best of the dynamic language choices. But I definitely feel like Python qua programming language has become a missed opportunity in many ways.
Nowadays if someone were to ask me what they should write a nontrivial new system in, I would probably recommend coding everything in JavaScript or C++11. This makes me sad, because Python's syntax and sensibility are unparalleled; I have never been as productive a coder as I have been coding in Python these last few years. And I've been coding in C dialects since, seriously, 1979 (on a Heathkit H89 with 64K of RAM!)
Part of the problem is that you could count on one hand the number of contributors who have worked on non-trivial Windows issues in CPython in the last 12 months (probably even 24). I think we're pretty aware of this, but there are still only 24 hours in a day.
In fact, my new SSD should be arriving today to complete the build of my new machine, and Martin's work on WinRT probably jumps ahead on my todo list.
As for the original article, I don't mind if Python's future isn't everywhere, but I find it sad that modern languages like Go and Rust don't copy more of Python's features (and my top contender for this is actually significant whitespace, because I find it greatly helps readability).
Erm... Not exactly. http://docs.python.org/whatsnew/2.7.html
"To help with porting to Python 3, several new features from the Python 3.x series have been included in 2.7"
It ships with Python 3.2 on device, and it can be used to develop actual apps with. Games are possible too, I know someone working on one right now.
You should check out our project's site at http://blackberry-py.microcode.ca/. I'm one of the lead contributors, and the project is still very much in its infancy.
I'm actually in San Jose right now for the BlackBerry Jam Americas conference if you'd like to meet up and discuss more.
OK, that's mean, but seriously the installed base of BlackBerry doesn't really inspire confidence.
In the time the community has spent dealing with/bickering about the transition, other languages have continued to forge ahead, while python just hasn't kept pace.
It hurts me to say that - I like python a lot, but lately it just doesn't seem to have it's mojo.
To take a really trivial example: the name "str" should not have been reused to mean "unicode", it either should have continued to mean "bytes" or it should have been dropped altogether.
Certainly, under no circumstance, should the name "unicode" have been removed, but it was; they also should not have removed the u'' syntax. Their goal wasn't just "fix things" it was "break old things".
The result is that it is nearly impossible to write code that runs on both Python 2.x and Python 3.x even when the semantics of the code have absolutely nothing to do with the things people really wanted to change.
Another great example of this is the exception syntax: they changed the way you catch exceptions to replace the "," with "as"; less confusing, sure, but again it means that you can't write code that works on both.
The answer to this was then a tool "2to3" which smoothed over some of the more obvious problems, but whose entire mission statement was flawed: to keep all code in Python 2, supporting Python 3 as a second-class citizen.
A lot of these changes, certainly the syntax ones, could easily have been made per-file: make new scripts .py3 or have a special header, or force old scripts to have a special command line argument or special load path.
At least then, you could mix and match new and old code without simply being stuck in the limbo that is Python 2.7... a limbo that didn't even help the cause, as you can't even write fully 2.7/3.x compatible code.
The result is that we still have tons of libraries that don't work. When they do, they often have less testing. When they are well tested, they often "took the opportunity" (just like Python itself, which set the bad expectation) to have different semantics, so you have to port your code anyway.
There was a carrot (better performance) to go with the stick.
Python 3 was pretty much all stick, unless you were a unicode fetishist.
The "except .. as" clause was backported to Python 2.6, which was released almost 4 years ago. If you don't need to support Python 2.5 then you can migrate to except..as right now. Python 2.6 added "from __future__ import print_function", and so on.
The goal was and is to make Python 2.7 a migration to a 3.x. People have and do write code which is usable, and unchanged, in both major version, which seems to contradict your statement that doing so isn't possible. And people continue to work to make that easier, while also advancing Python as a language.
Your proposal, to be able to "mix and match new and old code", does not work. At least, not without a lot more development effort behind it. The object models are different enough to make it not work. Remember, Python 2 has both old-style and new-style classes, while Python 3 does not. There's Python 2 code which probably still depends on Python 2 old-style classes, and will break. Which means you're left only being able to support the subset that the Python core developers, and those who wish to become core developers want to support. Which is what we have now.
This logic is circular: you are directly claiming that Python 3 could not mix and match support with Python 2 because Python 3 does not support mixing and matching features with those from Python 2. Yes: I realize that, and that is precisely what I am taking issue with.
The reality is that supporting many of these features is actually quite simple. Please realize before disagreeing: I develop compilers, and I have developed virtual machines. I know what kind of effort it takes to have parameterized grammars and internal transformations.
What really seems to happen with Python is that people get it in their heads "developers should not be using this old feature", and then it gets actively removed in order to clear the mental space for the features that should be used: a "there's only one way to do it" mentality.
However, there is code out there that is using that feature, and that code might not have a maintainer anymore; that code works, has been battle-hardened, and the only thing keeping anyone from using it is now some arbitrary syntax change that was pushed through.
For an example of a community that gets this: C. No one develops in K&R C anymore. However, there is still tons of code out there, even code inside of some core projects you use every day, that was written in K&R C and never needed to get serious modifications made.
That code still works, and still compiles with modern C compilers. While the syntax is quite different, it really does not take much time to make certain your compiler supports it, and you suddenly have access to all of this totally working code: this should be considered a "win".
Yet, something in the Python community seems to latch on to the idea that that situation is actually a "lose": that having support for old features is only something that should be done in Python 3.3 (which isn't even out yet) as a concession to some reality as opposed to out-of-the-gate in 3.0.
Can you help explain this one to me? Why is it possible for Python 2.7 to have all of these really cool "from __future__ import" features, but impossible for Python 3 to have just a couple "from __past__ import" features? The only answer I currently have is "stubborness".
(Also, on the exception-handling semantics, it is only this year that a lot of key projects, from Jython to App Engine, have supported anything but Python 2.5; I accept that it is nice that 2.6 supported it, but a lot of real-world/serious Python was stuck in 2.5, so if you want your library to be widely used you couldn't use that syntax. The sea-change on this seemed to happen this year, so this should increasingly not be an issue, but it is way way too late and could have been remedied by supporting that syntax with a switch.)
(The same kind of failure, for the record, is demonstrated in the decision to finally fix a few of these issues--the u'' syntax not being the only one--in Python 3.3. Python 3.3 is not even out yet; when it is, it will not be deployed in many places, even places where Python 3.2 already is, such as the numerous Ubuntu servers that are already deployed. If Python wants to be serious about helping developers and making Python 3 actually happen, they should backport that change to Python 3.2 so it can be pushed through major distributions; otherwise, it just delays the benefits by yet another year, if not longer.)
A saving grace is Python's historical emphasis on unit testing. That makes fore-porting things without horribly breaking them at least plausible. But still painful.
Get this into your head - Python has a different development model than C. There is no standards body, or requirements for backwards compatibility. Anyone who gets into Python thinking that there will be full backwards compatibility is fooling themselves.
Those who want backwards compatibility can do so on one of several ways: 1) keep using the old code and don't upgrade, 2) contribute time, effort, and funding to the core developer team in order to implement backwards compatibility, 3) fork and maintain your own version of the source, 4) make your own run-time hooks to convert old-style Python code to the new-style runtime, or 5) create a new version of Python which does have that backwards compatibility.
I know that REXX and a few other languages have made a point of backwards compatibility even as semantics change. They have different goals, and a different funding model than Python. If enough people disagree, then those people can take the existing Python code base and run with it. But that's not happening. (I think there are some unofficial patches with VC2010 support, mostly used for internal company builds.)
There's, what, a few score people working part time in Python code development? Even Guido works on it only half time. How do you expect that small of a team to pull off a clean 3.0 release without making mistakes? The 3.0 release had a new io system, which was a lot slower than 2.x. There were a lot of bugs, like unicode filenames, which had to be figured out, where previously it was the responsibility of each developer to figure out the Unicode issues.
"Can you help explain this one to me? Why is it ... impossible for Python 3 to have just a couple "from __past__ import" features?"
That's an easy one. Lack of resources. The Python code developers did not want to maintain both old-style and new-style class implementations. They used Python 3 as a way to get rid of old-style. As a consequence too, it means that I can stop putting in the silly 'object' in "class Spam(object):", which was the Python 2 way of indicating that I wanted a new-style class. If 5 people said that they were willing to support old-style classes, and showed that they were committed to doing so for many years, then there may have been a different decision.
You can see that even in the PEP 3000, written during the 2.6 days: "But I expect there to be at least one or two new 2.x releases even after 3.0 (final) has been released, probably well into 3.1 or 3.2. This will to some extent depend on community demand for continued 2.x support, acceptance and stability of 3.0, and volunteer stamina." (Emphasis mine.)
So many other things had deeper semantic changes, like dict.keys returning a view rather than a list, that a simple syntax change is trivial. I don't think a "from __past__ import comma_exceptions" would have had much benefit. I can be wrong, but that's exactly the thing that the 2to3 converter could handle, and there are patches to make distutils convert 2.x code automatically and import hooks to do that transformation at run-time. One such is pymolurus.blogspot.com/2012/04/pep-414-like-import-hook-for-python-32.html . It isn't a catch-all solution, but that approach could help some people.
BTW, as far as I can tell, clang does not support K&R C.
Personally, I don't even have skin in the game anymore: I am now using Clojure, and am piecemeal porting my code over. This is actually an easier time than the brutal experience I had dealing with Python's / operator change (which I adopted ahead of schedule, thinking it would help me: all it did was hurt me as my code no longer could be used on 2.5).
I thereby wonder: what are you defending? Do you think Python is succeeding? That in another couple months we are going to suddenly hear of a massive turnaround? That it will still be relevant in another two years when 3.3 is fully deployed everywhere and most libraries might be Python 3 compatible? What is your overall position?
I remember reading "Decline and Fall of the American Programmer." And "Rise and Resurrection of the American Programmer." It's a special kind of consultant who can get paid both coming and going.
Python is in decline? How do you reckon that? In my work, Python is the de factor high-level language for chemical informatics, molecular modeling, structure visualization, and the like. This is much improved over the last 10 years, and won't be dying in as short a time-span as two years in the future. No, most projects haven't migrated to Python 3. Yes, it will be annoying. So was porting client code from IRIX to Linux. That happens.
Really, you need to plan for it. And it sounds like you didn't plan for the division operator change. The ideas of that change first surfaced during user testing under the 1.x days, with Alice. That change decision was publicized in spring 2001, with PEP 238. The "from __future__ import division" was made available in Python 2.2. The runtime even has the -Q option so you can tell it to print warnings wherever there's a problem. The nice thing about this change is that it's purely a syntax issue which can be handled on a per-module basis, and mixed with modules which weren't compiled this way.
These all worked for 2.5, so what was the problem you experienced?
What I'm "defending" is my sense that Something is Wrong on the Internet. It sounds like you're doing things the wrong way. You expect certain behaviors which were never promised or implied, you don't read the update notices, and you don't use the tools which help you achieve the goals you want. No wonder you get so frustrated with Python! But I don't see what the Python developers could have done better, given the resources they had, and given that they care about goals that do not interest you.
You're going to run into the same problems with Clojure. Looking now, it seems that 1.3 broke code. Some things were removed, there were changes to the contrib namespace, and some third-party modules didn't work. Various places say it was a hassle to deal with. Sound familiar? How are you sure something like that will never happen again?
The article is called "I Am Worried About The Future Of Python" and the specific thread I chose to participate in directly described a way that the Python 3 schism was contributing to the concern (WinRT support). To quote the lead comment for this thread: "I definitely feel like Python qua programming language has become a missed opportunity in many ways".
I then provided my experience, as someone who has been developing for over 20 years in numerous languages (even designing them and teaching them in a university setting), of a unique set of challenges and mistakes that I see Python having run across--a perspective that one reader called "a brilliant if unpopular analysis"--and yet I'm now simply being dragged into the same defensive argument that comes up whenever the 2/3 topic is breached: to turn it into a very personal attack against me and my abilities led by highly inflammatory and insulting wording like "get this into your head", quite despite the fact that I am hardly in the minority with regards to these overall opinions.
Regardless, I am hereby so confused as to what conversation you feel you are participating in that I am just going to bow out: thank you, regardless, for taking the time to respond.
(Oh, I will respond to the semi-unrelated Clojure discussion, as I should probably make certain I don't inadvertently cause them problems: I do not use third-party libraries written in Clojure, as there is really no need given that all of Java is accessible; this already stops most of these problems. Also, the way Clojure 1.3 broke code is closer to the way Ruby 1.9 broke code--a change I stated elsewhere in this thread was a non-issue for the Ruby projects I'm involved in--than the way Python 3 broke code, and was in fact even less of an issue than that.)
I've been using Python now for 15+ years. I've read articles about bemoaning the lost opportunities of Python for almost as long. It needs static types otherwise all is lost. It needs tail recursion otherwise all it lost. It needs macros, it needs to get rid of explicit "self", it needs blocks, ... and so on. People thought for a while that Python development was proceeding too fast. There was the language moratorium so others could catch up. Now you say it's going too slow.
If you want to give me specific details about how Python is losing in popularity, please do. You haven't. Nothing I've seen tells me that there's anything more than the usual churn of people moving in between languages, of hipsters talking about the language de jour, of industry people insisting on certain trends. But what you've written describing your experiences with Python's 2-3 transition does not mesh with my own experiences and readings of the experiences of others. How did the change in division cause problems? Why weren't the existing mitigation mechanisms good enough for you?
However, I will defend my wording, as I think that is unrelated: the phrase "once you get it into your head that" (even with that direct "you", rephrased from "people get the idea into their heads", about others) is a statement about the virility of the idea that someone now believes; in contrast, "get this into your head" is a direct command given to a person used to indicate that they are being dense and thereby are unable (or unwilling) to understand the idea in question.
To make this distinction more clear, you can look at something generally considered positive: "once saurik gets it into his head that something should be built, he builds it" or "people get the idea in their heads that things should be built, and then they actually build them" are both fine; "get this into your head: once you determine something should be built, you should actually build it" is both harsh and patronizing: it is an entirely different concept for the sentence, used as an attack.
Again, very informative discussion, thanks.
Please be sure that I totally respect your work. Still, while I agree some of your points are right, I believe you're taking a shortcut here:
> For an example of a community that gets this: C. No one develops in K&R C anymore. However, there is still tons of code out there, even code inside of some core projects you use every day, that was written in K&R C and never needed to get serious modifications made.
The syntax evolved, but behind this syntax K&R C behaves in the exact same way, all the way up to the latest C revision. You can't say the same for old vs new classes, which behave in a not so subtly different way, and you can't say that either for the new str/old str, and indeed you propose a different name, which is perfectly legitimate, as they are different things.
The problem is also a vision of scale. I'd say the designed impact of python 3 breaking changes is not for next year, or even the next two years, but for the next 10-15 years. I think the decision came from the realization that there were retrospectively bad or overlooked design choices, and that they needed to be fixed, and thus cut out, now. The most critical things dropped from python 3 are actually stuff carried over from python 1. Python's design and philosophy is around a form of minimalism. The Zen of Python says:
Now is better than never.
Although never is often better than *right* now.
I'd say from anecdotal experience that every single python project I write today works on both 2.7 and 3.2. I sure wish python 3.2 received 2.6's from __future__ import unicode_literals, which would, along with 3.3, remove the biggest part of my headaches. The real problem, as stated, lies in the dependencies, and there's no denying that although you can compile K&R C today, such code was not written in a vacuum and certainly (recursively) relies on a bunch of libs that may or may not work today anymore (because of the code itself, the plaform, or the compiler evolving). It is also in plain sight today with Ruby, where a significant number of historical gems simply don't work anymore in 1.9.x (worse, some appear to work, and silently corrupt stuff, up to a point where it blows up. I'm looking at you mysql, which silently fails to transcode strings, the very situation Python introduced an "upfront error" breaking change).
The point I want to make is that the situation is simply not as clear cut as some want to make it. The short term cost is that we have a rocky transition and that we won't benefit from the new features right now, while the long term benefit will really be seen 10 years down the road. The question you seem to ask is: will Python still be relevant by then? To which none of us has an answer. If not, Python will have paid the price of its 9th zen statement: practicality beats purity.
Now I will return to the pending migration of three real world ruby 1.8 + rails 2.1 apps to ruby 1.9 + rails 3.2 (and hell was the migration of some other app from python 2.6 to 3.1 so much easier).
A lot of code operates in a regime where it takes strings as input, attempts to find substrings in those strings based on either pure ASCII literals or other strings also given for input, cuts ranges and substrings out of that string, and then generates more strings as output.
This is in the same way that much of that code was designed so it could accept either an str or a unicode, and as far as I can tell that same code should either work quite fine now taking either a bytes or a str; to the extent to which this is not the case, it could easily be made to be the case.
> ... and indeed you propose a different name, which is perfectly legitimate, as they are different things.
FWIW, I didn't "propose" the new name: "bytes" is the new name: that is what the old str type is now called in Python 3, and in fact if you are using Python 2.7 (even Python 2.6) "bytes" and "str" are aliases for each other, as even the people who designed Python believe they are equivalent for the purposes of portable code. Sadly, without keeping around "unicode" (and preferably just deprecating "str" but leaving it as a warning-laden alias for "bytes") that was pointless.
The reason being that the only place you could actually have compatibility between these types was Python 2.6+... except, most "serious Python" has been stuck in Python 2.5 until possibly (I think I see a sea-change) this year. Yes: that is four years during which these transition mechanisms were largely pointless: neither Python 2.6 nor Python 2.7 were actually deployable targets for major and serious libraries.
Personally, I found this surprising: I have been using Python 2.7 since it came out, and I have never had a difficulty upgrading within Python 2.x. However, time and again when I come across a large company using Python and I ask them "what version?" they say "2.5". Meanwhile, both Jython and App Engine were likewise stuck at that level.
That means to "remain relevant" the primary deployment target for most systems needed to be Python 2.5. The idea of supporting Python 3 or even Python 2.6/7 was largely seen as a hopefully-fun diversion at best, and a needless chore at worst.
Thereby, I will argue that what was really needed was actually not Python 2.7: all of the effort put into Python 2.7 with its support for bridging the gap should have been made to Python 3 in reverse. You can't tell me that supporting some of these things as the same time, at least in ways that make the remaining porting discrepancy "realistic", is possible, as Python 2.7 does it: it is obviously possible, as we have an existence proof.
Now, you can totally (and dalke does) tell me they didn't have time to do that to Python 3, but my argument is that all of the effort exerted into Python 2.7 would have been better spent doing the same thing to Python 3, but with the defaults reversed: I believe that the effort would have comparable, and the result would have been to avoid the last four years of hell.
> The short term cost is that we have a rocky transition and that we won't benefit from the new features right now, while the long term benefit will really be seen 10 years down the road. The question you seem to ask is: will Python still be relevant by then?
This "rocky transition", for people who actually use Python to develop libraries or applications, is simply "stagnation".
One of the things I found so weirdly surprising when I finally decided to "bite the bullet and move back to Java-land" was the amazing number of high-quality maintained libraries for relatively new concepts (even ones so new they were "buzzwords"): it was like a kid suddenly being thrown into a candy store.
Whereas I had spent forever, on numerous occasions, trying to piece together semi-working mechanisms out of multiple competing and seemingly now all disowned or disinterested Python libraries (such as for PDF editing), I suddenly had my pick of numerous fully maintained libraries.
This is not the state of the Python ecosystem when I entered, and I argue that the reason this changed is because of the Python 3 split: the result is that the people still sticking with Python during this "rocky transition" are putting themselves at a "practical" disadvantage for the idea of a "pure" language.
> If not, Python will have paid the price of its 9th zen statement: practicality beats purity.
Right: it seems like that tenant was thrown out the window with respect to the changes made in Python 3.
> Now I will return to the pending migration of three real world ruby 1.8 + rails 2.1 apps to ruby 1.9 + rails 3.2 (and hell was the migration of some other app from python 2.6 to 3.1 so much easier).
Is this difficulty caused by Ruby, however, or by Rails? I had had an application that used Sinatra on 1.8, and it took me about an hour to get it fully compatible with Ruby 1.9 (which I did years ago, at the first opportunity).
(Note: my application was handling Unicode data correctly both before and after change: something that would have been noticed immediately as much of the package information I used was in random languages from Russian to Chinese, and I am quite a stickler for having things encoded correctly in my database; yes: I did have to thwack it some on Ruby 1.8 to get it to work correctly, but the result after thwacking I believe required no modifications on Ruby 1.9, a result that I might argue indicates that most people having porting problems are actually fixing bugs, not porting.)
It is somewhat confusing that these two languages differ by a number and not by name, but the intention is somewhat clear. In the case of Python, they seriously seem to believe that everyone is going to "upgrade".
You can't really say the same thing about Python. If Python wants to remain in the game, I don't see how they can do it unless they are OK breaking backwards compatibility in another 6 years.
Python broke backwards compatibility for as simple a thing like a print statement. Nobody like to play stupid expensive migration games just because your language designer decided that print statement must be written a different way.
That is a silly argument. They didn't break compatibility because they wanted to change the way print is. They changed the print because they already had decided to break compatibility. Ie "lets break compatibility to change print" vs "while we are breaking stuff, lets fix the print too"
Technically Python 3 is superior, IMO, to Python 2.7 and barring any setbacks such as unported libraries (which are becoming increasingly scarce) I wouldn't start a new project in Python 2.7.
The reason libraries have been taking so long is because so many people are resistant to change that they hadn't begun making any serious efforts to port to Python 3 until 2011 by my estimation.
Python 2 is so ass-backwards and broken that I find it painful to use on large code bases. It conflates strings and byte streams, has pathetic Unicode support, and its libraries are rife with bit-rotting code. For the batteries to continue being useful they need to be updated once in a while. Have you seen urllib? It's atrocious by today's standards! Python needed an update and I think that the core team made the right choices.
The challenge has been porting and the core team knew that. They made some poor decisions in hind-sight such as 2to3 and dropping u"", but otherwise the porting story is not as painful as it seems. Cherrypy did it ages ago and Django is finally getting around to it. SQLAlchemy supports Python 3. There are fewer reasons not to switch than ever before and yet articles like this still come up to this day.
Python is only going to fall by the way-side if the developers of the language decide to stop using it. The language hasn't gotten worse by any stretch of the imagination. It needs more contributors to make it better. If you like it so much, port your libraries, contribute to python core, and stop complaining! It's been like five years people!
The big problem with Python was always predicting what any given piece of code would actually do. Considering the lack of static typing, pervasive use of exceptions for non-exceptional cases, and subtle behavior changes in the core language from version to version, you basically never knew what you were going to get. On the plus side, all the Python libraries I ever used were open source and the code was easy to access. On the minus side, I needed to access that code on a daily or even hourly basis. Getting something working on my own machine wasn't too hard. Getting something working on anyone else's computer was a challenge.
It seems that most Python developers now use virtualenv. This is a handy little tool that creates a small runtime environment for your Python program. Basically, it allows you to explicitly set the version of every library that you use.
I always viewed virtualenv as a sign of Python's failure. Having stable APIs is damn near impossible in Python, so let's just give up and use old versions for everything. Put the whole mess in an airtight chamber, and hope you never need to do anything with it in the future. With an attitude like this, it's no surprise that the adoption of Python 3 has been pushed off into the indefinite future.
Issues like the global interpreter lock (GIL) and lack of real GC for data structures with cycles just reinforce the impression of a better /bin/sh, not a serious language. Sadly, in some ways, Bourne shell is still the better language, because it got backwards compatibility right, has a bigger install base, and better startup time. It's also significantly less verbose for simple tasks. And really, if your task isn't simple-- you shouldn't be using a language that doesn't even properly support threads.
tl;dr: Overall, I think my interest in Python has come to an end. It's over, sorry.
> lack of real GC for data structures with cycles
This has not been true since (AFAIK) Python 2.0, which was released 12 years ago.
Sorta.
It seems that CPython still can't garbage collect objects in cycles if those objects have a __del__ method.
See http://www.electricmonk.nl/log/2008/07/07/python-destructor-...
Or see gc.garbage, from http://docs.python.org/release/2.7.3/library/gc.html
I've been using Python for almost 10 years, and mostly I've pretended that Python 3 doesn't exist so far :) That simplifies life in many ways.
I had no idea that people even ran Python on iOS, but if so it's cool that it even works!
It's not true that Pythonic programs can't use multiple cores. In many programs a lot of time is spent in C extensions (i.e. half the stdlib), where the GIL is released, so you can run in parallel fine in that case. Not to say you won't be bottlenecked, but don't write it off so quickly.
1) New ports (e.g., WinRT) are all going to be Python 3.X
2) All improvements (performance, memory size) are going in Python 3.X
Overall, Python 2.7 certainly works. But it's a wasting asset, and so is any codebase stuck on it.
People want to write games in Python (myself included). They really do. The problem is, try to get pyopengl/pyglew/pygame/pyogre/pysdl or whatever running on your development machine. Now try to make it work on five different machines, across Windows/OSX/Linux. Then try to create a reliable distribution scheme for it.
I've tried it. Even the first step is painful, often requiring intimate knowledge of C, autoconf/make/whatever build systems and the system you're targeting, jus tto get you going. It's such a pain I just end up writing stuff in C, and only maybe plug in the python stuff later on, if at all.
Really, the lowest levels like PyOpenGL should have been part of the official Python distribution, just like tkinter, even if they are thin ctypes-like wrappers.
That's how we've done it before with LUA and other scripting languages in game dev. You don't expose anything low level to the gameplay scripting beyond simple primitives. You certainly don't touch anything to do with OpenGL - even the C++ game code rarely goes that deep.
Sure, but it means much bigger swaths of C/C++ code and your game app is no longer a python program that imports some extension modules for performance critical tasks or third party lib bindings.
I've done both, and neither seemed all that difficult, but then again our scripting requirements were pretty primitive.
I've heard pretty bad things about pyogre though.
On the other hand, on other machines it was a breeze to set up.
from [0]:
brew install sdl sdl_image sdl_mixer sdl_ttf smpeg portmidi
pip install hg+http://bitbucket.org/pygame/pygame
Pyglet's master branch still uses Carbon, so you have to use a years-old experimental Cocoa branch, which won't be merged because it requires pyobjc[1]. This in turn requires pyobjc, which requires a little trickery to install:
export MACOSX_DEPLOYMENT_TARGET=10.6 # else it'll fail at finding some libs, and stubbornly tries to stay in 32-bit land
easy_install pyobjc-core==2.3 # else pyobjc fails to build this by itself
easy_install pyobjc==2.3 # finally
[1] https://groups.google.com/forum/?fromgroups=#!topic/pyglet-u...
We briefly had to make our own library for reasonable 2D performance.
Not sure if im being ignorant here but surely this is a problem for most server side languages? PHP, Ruby, Go, Perl would all suffer the same problems. Browser vendors only want to have JS client side for obvious reasons, this doesn't just affect python.
Js is clearly doing a fine job at client side programming, should we not be happy with that?
Why are people obsessed about creating a "jack of all trades" language instead of letting languages do what they do best?
Taken from the comments... "I suspect javascript will become like assembly and all languages targeting the browser platform will need to be compiled to javascript."
That sounds like hell to a front-end developer. Its bad enough learning coffeescript or other dialects if a company uses them, but other languages too?
Isn't that like saying that having many server side languages is hell to server-side developers?
And isn't it making JavaScript effectively a jack of all trades? You might want to run code on the browser that is better expressed in a different language. After all, the days of client-side code being just a few value checks and DOM manipulation are beginning to disappear.
I would so, so happily drop Javascript for just about ANYTHING else that works on IE+Chrome+Firefox+Safari.
IMO, Javascript is just awful, and I know I'm not alone with that opinion.
So Java Applets are forgotten already ... and they had such grand plans! I for one won't miss them at all.
I had that opinion at first, but actually Javascript is a pretty good (and even pretty) language if you stick to the good parts [1]. Having the code automatically checked by JSHint helps a lot.
[1] "JavaScript: The Good Parts" by Douglas Crockford
http://anongallery.org/img/2/0/javascript-the-good-parts-the...
Let's first see what satisfies these problem areas:
Mobile = [ObjectiveC, Java, C (native client)] (and maybe Javascript+HTML if you look at the "compile down"/"webview" solutions)
Browsers = [Javascript] (and _maybe_ C if you say Emscripten is actually a sane solution to problems but I'd argue it fits only a minority of cases)
Games = [ObjectiveC (iPhone games), Java, Lua (scripting/plugins), Flash/ActionScript, C, C++]
No single language fits across all of these (dare I call them) niches. Every single language other than Javascript fails the "Browser" test.
I'd argue that focusing on many of these niches is the wrong way to go. Trying to fight for a spot on the mobile front will be a losing battle as Python will always be a second class citizen or flat out denied. The game development front seems like a confusing goal -- how many devs picked up Lua from the plugin system? Additionally, Python isn't fast enough to be the base nor faster than Lua for the plugin system.
The metric that he says he's optimising for is "adoption". I just feel like he's working backwards and optimising for tiny use cases instead of the general case. Many Java classes at university are being replaced with Python for example, and I'd say that's more important for adoption than any of these areas (other than web).
The take away here is that since client side web programming can only be javascript, it pigeonholes a large chunk of development effort into a forced language. I can't wait for the day web browsers can leverage multiple DOM aware scripting languages. The fact the JS JIT is already tightly coupled with an html / css parser is already bad enough.
Face it: unless something miraculous happens (like Microsoft, Mozilla, Google and Opera all agreeing on a brand new language), javascript will likely remain the only choice for "in-browser" UI development. Unless the browser gets replaced wholesale, we're screwed like that.
As a language Dart is not exciting. It doesn't tout fancy monads or functional data structures and so on, but it looks basic, simple and average. Kind of like what Java would be if it was made for the web.
So there is at least on contender out there to replace JS.
Mozilla is working on Rust. A very exciting language that actually has many fancy features, some borrowed from Erlang. Not sure if that will ever make into an in-browser language, but who knows.
there are the compile-the-browser-component-to-javascript options like haxe and opa. haxe is even planning on compiling to mobile platforms.
Python's main competition in the future will be Dart and ES.Next(the next version of JavaScript)... and maybe even Kotlin which has a JavaScript backend.
"Why are people obsessed about creating a "jack of all trades" language instead of letting languages do what they do best?"
Sometimes these languages do things better than the original languages that had their niche.
Is there any indication that Dart developers want to move Dart outside of the web domain? Everything on their website is all about "HTML 5" and "web apps".
I can only speak for myself, but there is something extremely attractive about being able to do all of one project in a single language. A reduction in the overhead of context switches, perhaps? Fewer absent-minded semicolons in my python code? Correct me if I am wrong, but, isn't this a big part of the draw in Node.js?
Curious...
On the other hand, if you're working on a system that is almost exclusively python, it wouldn't make sense to use something only "very slightly sharper". E.g. calling out to perl to do some one-liner that needs 5 lines in python.
I'm not happy with that. Lots of people want to say that the Web will one day replace native apps and this is precisely one of the reasons I hope that never happens: with a native App I can do the whole thing in one language. With the web I have to deal with HTML, CSS, <server side language > and Javascript for the client side. At the very least I'd like to be able to use <server side language> on the client as well. Why must we all be forced to use Javascript? It's not even a particularly nice language IMO.
>That sounds like hell to a front-end developer. Its bad enough learning coffeescript or other dialects if a company uses them, but other languages too?
What? Right now all languages either use a VM or they compile to assembly. Has that been so bad? If you read server side code you might be required to learn literally any language. Why should the client be different?
My hope is that someone will come out with a VM browser plug in that lets me write in any language I want, compile to to a binary that runs on that VM, and this plug in eventually replaces Javascript as the defacto web language.
Stop holding the browser client side programming environment hostage!
As a full stack web developer I have to know multiple languages and constantly learn new ones. Why should front-end be any different?
Ironically, one of the most common accusation I hear when someone speaks out against server-side JS is an accusation of not willing to learn new things. Double standards.
I for one am not worried.
(The purpose of my original comment was because PyPy does work with third party libraries, especially if they are pure Python, and so I was asking if these were the sort of libraries that were being referred to.)
After all: If it's possible to get JS JITs to fly on mobile devices, which it is, why not Python? I really see no structural problems.
I see more people around me learning Python every day, because it is a useful language to get stuff done, has excellent bindings for almost every other library, and is pretty stable (ie, the language has been about the same for a long time).
Sure, very small scripts and hooks only do one thing one time, but usually those aren't the things that benefit from being heavily optimized in the first place, and could just as well be left interpreted (even better -- as interpreted code is more compact and thus better fits in the cache). The same holds for every other JIT-based language.
Overall, PyPy lefts expression as project that "gets fantastic things done" and I'm looking forward using it everywhere I can.
* smaller runtime foot print.
* faster runtime.
* simpler to embed
* the fewer batteries that are included are simpler to embed in your runtime. Python's libraries are often very tricky to embed.
I prefer python to lua, although they are both expressive and powerful languages, however the convenience of python isn't enough to trump the flexibility and speed of lua.
(edit formatting)
Whenever I hear someone complain about (lack of) Unicode support, my ears prick up. See, Unicode support is a many-headed beast, and almost no languages have very much of it, and most of what people naively think of as Unicode support (length, indexing into "characters", case conversion) doesn't really work when you take into account combining diacritics, ligatures, Turkish I, German S, etc.
I'm not especially familiar with Lua, but it transparently stores and compares UTF-8 strings, and there are even bindings for ICU. So what's missing from "Unicode support"? Script specifiers in regular expressions, perhaps? I'm asking out of genuine curiosity.
But no, it's fucking not OK. Lacking basic unicode handling means everyone handles unicode strings in different ways, so various libraries clutter in basic string capabilities. Sometimes this really piss me off.
Proper unicode support would include things like normalizing strings (Unicode has 4 different normal forms!) and testing for fuzzy equivalence. But last time I checked, the usual library for doing those things, libicu, was bigger than the whole Lua interpreter. So I can see why there isn't a lot of enthusiasm, especially if there isn't a real use case.
Thanks for this good news! The last time I tried Lua it didn't work. UTF-8 is ok, full Unicode is not necessary.
For better or for worse, I think there is pretty much 0 chance of Lua or for that matter Python showing up natively in browsers. There are too many technical and political things in the way.
But we already have good tools for compiling both of those languages into JavaScript. If for example the Python community made JavaScript a priority to compile to, we could make it run very well and integrate nicely with the browser environment.
Not if you write your own browser.
It required commenting a few things out, and swapping in the boehm gc for iOS port.
Of course, no native ui widgets available - but I used phonegap to do the ui in html5 and it called shedskin c++ for all the actual app logic. Phonegap supports connecting html5 -> native c++ via a plugin architecture. Works wonderfully.
High powered, efficient apps on the iPhone via python is a bit of paradise compared to apples own obj-c, I must say. Huge potential for shedskin adoption there if someone packaged things up this way for the general public..."
[1] http://groups.google.com/group/shedskin-discuss/browse_threa...
If you agree with the above quote, you are committing a Texas Sharpshooter Fallacy. We're not using Python for that, because the interpreter has a significant startup time, because Python performance is not stellar, etc. But it could be different.
I'm a novice programmer without commercial experience, but the statements in bold ring true to me: - Python is Expensive On Mobile - Python is Without a Door On Browsers - Python has Failed for Game Development
Yes, I would love to be able to use Python for games and mobile development too, but I do not expect to be able to use it for these things, just as I do not expect to be able to build web apps with Objective-C (even though some do).
For me, the power, elegance, philosophy, and community that sustain Python are enough to make it a worthwhile choice in a finite but important number of domains.
PHP is nice because you can throw it on any LAMP stack and it's interpreted, but Python runs much smoother for heavy data lifting.
Anyone predicting in 2003 that PHP would become less popular and/or suitable for the web would've turned out to be right. Yet, I bet anybody who might've made such a prediction just learned a new language (maybe even Python) and is still enjoying creating great software just like back then.
What languages are you considering that have a better story here? Note that better story has to mean more than "syntax-level support" -- I run highly concurrent services in Python right now using Twisted without language level support, and it's not always obvious in which ways language level support would make things nicer in ways that aren't already implemented (e.g generators + @inlineCallbacks).
2. You can't parallelize computation if it's written in Python. (Yeah, yeah, concurrency is different from parallelism, but there is a lot of overlap.) Even if it's written in a different language, the granularity is constrained due to the GIL.
This depends on the problem domain. A lot of the code I write is limited by memory bandwidth. The number of outstanding memory requests is limited by hardware, so saturating the memory bandwidth, especially when many active memory streams are needed, requires several cores to participate. A particularly useful approach now is for threads running on those cores sharing a particular cache level/prefetch unit to perform software "buddy prefetch" while working together on a traversal. The threads are weakly synchronized by memory dependencies, sharing both cache and bandwidth.
If you remove shared state, the threads each need their own ghost region and cannot share cache and bandwidth. I avoid shared state, especially shared mutable state, whenever possible, but there are still plenty of cases where it makes sense, especially at cache and NUMA domain granularity.
2. As I said above, you need to use multiple cores per memory bus to utilize the hardware bandwidth because there is a limited number of outstanding memory requests per core (or hardware thread). Remember that the max bandwidth realized by your application is bounded above by
num_pending_requests * payload_per_request / latency
For languages, number of programmers translates to: higher quality and number of libs, better tested, more work on the efficiency, compiler, more companies built around the ecosystem, more tools, more books, more tutorials, more open source projects in and with it, more bugfixes, new APIs with support for the language, new bindings etc.
It doesn't have to be "perfect for everything", but it has to be good in the kind of things that interest the most people people now. Scientific computing ain't it.
Much better to use a bad language with 100,000 programmers than a good one with 1,000, as people reluctant of JS in the past have found out.
Python strikes a very good balance between rapid development prototyping and something that is sustainable, and has an excellent level of readability that makes it outstanding for gathering patches from even non-professional programmers.
Most of the interesting parts of computing now cluster around web applications and web services, where the action happens on the server and Python is still reasonably well suited -- and in the servers behind them, where it's quite critical in Linux distributions. I write a ton of software for systems administrators and can't really see using anything else if I wanted the same contributor rate.
Sure, it doesn't run in the browser, but I can't see myself pining for the return of Java applets either. It all comes down to what types of applications you are writing.
If I was doing HFT, I'd clearly want something statically typed, fast, and functional. Those same properties are things I don't want in other applications. It's pretty much a screwdriver vs carburetor type of problem.
Yes, I'd love to see more investment in the runtime -- but I'm also reasonably fine with where it's at.
I've only been using Python for a few years and now I don't know how I managed to get anything done before. Python as a solid base and excels at certain things. Move on. Use the tool that fits.
As for the mobile hardware being low on power and battery life. Give it time. My computer 12 years ago had a difficult time doing anything when converting video. Now, I can play games while I convert a video and it uses less power/puts out less heat.
Patience.
Main drawback is a slow first-time start-up time as the package files are unpacked. Subsequent start-ups are fine in my experience. Once a kivy program is started, it runs like a charm on modern hardware.
I'm building a scaffolding for productivity apps in Kivy here: https://github.com/esbullington/vite-vite
You can try out the (very alpha quality) Android app build with the scaffolding here: https://github.com/esbullington/vite-vite/vitevite-0.1-debug...
I should have some more time to work on this next week, at which point I'll be cleaning up the code, adding more documentation, and adding a few features. Suggestions and pull requests are welcome.
Is the author's point that we're moving away from "big" games and towards less complex, "casual" games like are popular on mobile, and the latter aren't using python?
Here's a list of games--many of them AAA titles--using Lua: http://en.wikipedia.org/wiki/Category:Lua-scripted_video_gam...
Although he may indeed be confused about "dynamic," since dynamic dispatch has nothing to do with the above, and is supported by all OO languages.
Cython and Nuitka (among others) do this to varying degrees.
There are plenty of lisp compilers out there, but people still consider lisp a dynamic language.
A quote from Apple themselves:
"Compared to other object-oriented languages based on C, Objective-C is very dynamic. The compiler preserves a great deal of information about the objects themselves for use at runtime. Decisions that otherwise might be made at compile time can be postponed until the program is running. Dynamism gives Objective-C programs unusual flexibility and power."
https://developer.apple.com/library/mac/documentation/Cocoa/...
Looking now, "Appendix A of the Fortran 77 standard lists 24 things that are not backward compatible in moving from f66 and f77."
Please tell me how the chasm between Python 2.7 and Python 3.3 is significantly broader than the chasm between the aforementioned languages.
VB.NET is effectively a different language entirely. It's programming .NET with VB-like syntax, not the VB language running on .NET (in my mind, anyways)
What about Old S v.s New S? S v.s S4? S vs. R?
What about D1 and D2 of the D programming language?
Classic awk vs. nawk?
Honestly, I could go on. I think the person I'm responding to really doesn't know much about programming language histories.
There is a difference between a language rewrite and changing just enough of a language that it might have well been a rewrite.
The inevitable nature of changing requirements in programming mean, there is always going to be demands for new changes and new features in a programming language. Changes in Syntax and excess syntactical sugar many times come only at the expense of breaking backwards compatibility. What you are seeing now is a saturation with Python. Python is good, but people now want more. I bet you unless Python is ready for major backwards incompatible changes in another 5-6 years they are going to lose this battle against other languages. NodeJS, Go they are all going to eat Python's lunch. What you will see now is cool stuff getting built in some language like Go and will go viral every 15 days. People who had originally no interest will learn just because its getting famous. Architects who just want to sound different and distinct will use it, Slowly but sooner or later there will be job demands for Go. This is inevitable and going to happen. Suddenly people will find every reason under the sun to hate python and love go.
Python will increasingly find itself in the same situation Perl was some years back. And its fans will have to face the same trolling.
Its just your usual technology cycles. By the way I guess its time to start learning Go.
If you have some code that you would like to have working under Python 3, check out a little library called "six". With it, it is simple to maintain a single codebase that works with both Python versions (no translations necessary, forget the 2to3 tool).
For example, the Pyramid web framework (and many related libraries) have been successfully using this approach. This way one can support Python 2.6, 2.7 and 3.2+.
Together with "from future" imports (for Python 2.6 and 2.7), this is easy to understand, though "upgrading" the code may still take time.
At the top of every module of mine I have this:
from __future__ import (absolute_import, division, print_function, unicode_literals)
Python has been in decline for years, maybe 5-6 years now. Python3 definitely cemented a lot the current downward angle. Python isn't out and possibly never will be, but its design has gradually shifted away from the hacker community. And that means the momentum will consequently shift into other languages and environments. And that's fine. Each language will gather the following it deserves, in the ever-changing environment.
Just make use of the best tool for the job. In some cases that tool is not Python, as simple as that.
* good coders. ( bad coders can do much more damage. )
* you need to use cython (and maybe numpy).
* memory profiling is hard
* you need to search pypi alot, there is likely something there to help you.
* windows builds of extensions can be tricky.
I am also concerned about its future, the community seems in a "I know better than you" attitude and I don't see anything major coming, except pypy.
"http://www.python.org/ftp could bypass GFW to download it. Because of version 2.6.4, the sensitive number 'June 4' to government, so the GFW block the download page."
according to wpopenid on http://posted-stuff.blogspot.hk/2010/07/downloading-python-f...
The FTP site works for me. I visit Hong Kong often enough so not having access to the latest internet stuff isn't usually a problem for very long.
http://qt-project.org/wiki/PySide-Tutorials-by-Experience-Le...
https://github.com/pyjs/pyjs/wiki/GettingStarted
Needs more developers... (Hint)
Ahem...
http://golang.org/pkg/html/template/
It doesn't have its own equivalent of RoR or Django yet because most of the batteries needed for web development are built-in.
It seems that the web.go authors thinks that Go's template language batteries are of the wrong size.
What you're looking for is Revel. http://robfig.github.com/revel/
A very comprehensive list of the organizations using Go; I think it's kept reasonably up to date, but new ones are popping up on the mailing list from time to time.
For example, not to long ago we had the maintainer of pool.ntp.org asking some questions as he was rewriting some of the infrastructure for the pool from Perl to Go.