What Python Fixes
paulgraham.com
paulgraham.com
The language is well designed; the barrier to entry is low; the language allows for progressive discovery of its advanced features; it is powerful and flexible enough to allow an arbitrary combination of imperative, object oriented and functional programming; and the large, mature ecosystem of third party libraries means it can already do about 80% of just about any problem you can think to throw at it.
http://en.wikipedia.org/wiki/Just_intonation http://en.wikipedia.org/wiki/Equal_temperament http://en.wikipedia.org/wiki/Well_temperament
Basically, when you play music aloud naturally with your voice or acoustic instruments where you have complete control of the pitch, people tend towards using whole numbers for representing harmony, like 3:2, since it sounds the best. Unfortunately there are 12 notes in an octave in the Western music system so you end up with bad ratios if you want to use instruments which are tuned to play anything more than a single key. If you are in complete control of your pitch, like with voice or violin or other stringed instruments, the all of the players can adjust their intonation on the fly when the keys change, and still keep the nice whole number ratios.
Enter church organs, which are huge, expensive, and can't be tuned while they're being played. But they're very nice and impressive. So they were tuned to play in only one key -- C. That's why the keys representing the scale of C major on a keyboard are all white, and the other (less important) notes are smaller black keys. You won't be pressing these very often, better to get them out of the way.
Then along came some math people who calculated tables of compromises that ended up giving a pretty decent sound to harmonies, but was laid out in a way so that you could play different keys without having to adjust your tuning or intonation. There were different systems, but they were referred to as having temperament. Hence the name "Well-Tempered Clavier" in the famous composition for keyboard instruments, which made ample use of the newfound ability to change keys whenever one pleases.
So now there are a bunch of keyboard players who can suddenly play in any key they want with the new tuning systems. But the keyboard layout is still same one used for playing only in C. That all happened hundreds of years ago, but we're still stuck with the original layout for keyboards.
You know that sinking feeling when you crack open 3-year-old code written by your predecessor's predecessor and realize it's a load of garbage and that you'll never be able to fix the bug?
Yeah, well, with Python that doesn't happen. At least, not at all frequently.
Python forces you to spend a bit more time up-front and then saves you time each and every time you need to read, edit, or improve your code. People choose Python for the same reasons they choose good testing practices. Python's a mature language for mature developers who want to produce mature software.
Also if you browse some PEPS (and other sources) you will find that Python and Haskell's history are more intertwined that one would guess at first.
My guess is that "Less is More" in this situation. Python often does tend to read like pseudocode.
The problem with this, though, is that idiocy has seemingly infinite resources on its side. Occasionally, you will have an outlier who has vast holes in their understanding of programming and writes large amounts of jaw-dropping bad code and can do this even in simple elegant languages. In fact, there's one on the team I'm working with right now. (He was maneuvered onto the testing team to mitigate the damage.)
One interesting quirk of Python is that such code superficially resembles good code in everything except the length of functions and methods.
It's pervasive, so there isn't any single feature I could point to. Python embraces a general philosophy that starts from the observation that programmers spend much more time reading code then they spend time writing it [1]. Thus, most trade-offs between readability and writability should fall in favor of readability.
For instance, let's look at the whitespace thing. It's the first thing new Python users bump up against, and it's a source of perpetual controversy. There's a not-insignificant group of users who try Python, hit whitespace-for-control-flow, and leave in a huff. I won't argue that these people are wrong, but I will point out that it's ridiculously nice to have all the code you read always use the same control flow indentation conventions. Though Python's my primary language, I do more and more work in JavaScript theses days, and I could go on and on about my frustration trying to follow the flow of code written with varying intendation, brace, and semicolon styles.
This is encapsulated by the observation, from The Zen of Python, that "there should be one — and preferably only one — obvious way to do it" (http://www.python.org/dev/peps/pep-0020/). At its best, Python code eliminates any stylistic differences. This sounds like it's a stifling of creativity (and perhaps it is), but in practice it means maintaining someone else's code is just as easy as maintaining your own.
Take Django, for example: I can tell at a glance who's responsible for any given line of JavaScript in the framework, but it usually takes digging into SVN history to discover who's written some given line of Python [2].
Or look at Python's general lack of language features. There's a `while` loop but no `do ... while`. There's a `for` loop, but it's really just `foreach` and nothing else. There's `if/else`, but no `switch/case`. There's language-level primitives for basic types list, dicts and tuples, but not regexes, ranges, heredocs, or anything else more exotic. Python's a little language; I can keep it all in my head. I never have to run to the docs to find out what some obscure syntax does; there isn't any obscure syntax.
If I got to do green-fields development all of the time, I might very well choose a different language. I've certainly used languages that are easier to write. But I live in a world filled with old code, so I've tried to make sure my career involves code that sucks as little as possible.
[1] Indeed, I'd argue that reading code — especially other people's — is the most important skill a developer can have. I'd go so far as to argue that asking people to write code during interviews is pointless; we should be asking them to read it. But that's another show.
[2] I'm often surprised to find that the code was written by me, though I'm not sure if that's a praise of Python or a remark on my generally shitty memory…
[3] Well, until Python 3. Again, that's another show.
An excellent idea.
> [3] Well, until Python 3. Again, that's another show.
This doesn't seem to be referenced in the body of your comment. What are you referring to?
I immediately fell in love with significant whitespace. Since I was already indenting anyway, why not make it meaningful and skip the additional overhead of matched curly braces?
That's a bold statement. It happened to me just last week.
The difference from Perl is that common tasks are standardized ... stuff like OOP is built-in, so you don't have to chose or stay in touch with several frameworks for doing OOP, and you could care less about integration issues between modules that use different OOP systems.
It also has a clean syntax, which makes (good code, written by good developers) readable ... but that's not what bugs me about Perl ... after all, why should you read text in some language (natural or not) if you haven't bothered to learn it?
That's about it. Otherwise I'm beginning to hate Python's limitations.
1) (proper) anonymous code blocks ... it's the only sane way to have declarative APIs, although I've found for some tasks that Python's features (like decorators, meta-classes) are enough ... but not for all things I want to do
2) a limited form of macro support ... like, I want to get the syntax-tree out of a closure, instead of a reference to the compiled method ... to abstract the way some processing takes place. In Python you can workaround it with some magic (a la Django's ORM), but I love to be able to write Python syntax for those queries (e.g. something like Linq) ... and have for example a unified querying interface for collections of data
Is the anonymous bit necessary? Or asking another way: What about throw-away names? I find that even in Haskell I rarely use lambdas, but tend to name nearly all of my functions.
> That's a bold statement. It happened to me just last week.
Look, different strokes for different folks: I'm not trying to suggest that you should use Python; I'm explaining why I do.
That said, anecdote ≠ data. Your cited experience doesn't match up with what I've seen in a decade as a Python developer, and, more importantly, it doesn't match up with the data I've seen [1] suggesting that Python code takes 1/2 to 1/4 as much time to maintain as other comparable languages.
When I choose Python, it's because I know that the vast majority of my time is spent maintaining code (mine and other people's), not writing new code. I choose Python because, of all the languages I've tried in my career, it's the one that makes this task easiest.
[1] To forestal the obvious questions: no none of this data is publicly available (at least to my knowledge). I can share some rough details -- contact me privately if you want them -- but you'll pretty much have to either believe that I'm reliable or decide that I'm full of shit. I don't much care either way -- again, this is about my choices, not evangelism.
But regardless, if you haven't worked with badly written Python code that makes you want to switch career, that's just luck.
>> That's a bold statement. It happened to me just last week.
>Look, different strokes for different folks: I'm not trying to suggest that you should use Python; I'm explaining why I do.
You made a pretty definitive assertion, and when someone points out that they had an experience to the contrary you say "different strokes for different folks"? Seriously?
I like Python a lot. I've been using it for about 12 years now. My experience doesn't really line up with yours.
I've found that Python is great for writing small (<2000 loc) programs. However, every single time I've seen a large project written in Python it's ended up collapsing under its own weight.
I'm sure I'll get flamed for saying this, but I think part of the problem is Python's dynamic nature. Python is the dynamic language I have the most experience with, so perhaps I'm over-generalizing, but the problem I've seen with large Python projects is that they get to a point where a major refactoring is needed and it's impossible to pull it off because of the dynamic nature of the language. Perhaps with 100% test coverage things would be easier, but in the real world that's rare/nonexistent. (Also, I find that more tests are necessary in Python than in a static language because you end up having to test a lot of the same things your static language would have been proving for you.) In the bigger Python projects I've seen they eventually rewrote the whole system in a static language.
Python is not monkey-patched
Python is not multiple coding styles
Python is not enterprise
Python is not overly verbose
Python is not overly concise
Python is not lacking in libraries
Python is not a dialect
Python is not manual memory management
Python is not hard to learn
Python is not a roadblock to writing code quickly
And really the only important issue: Python is not a pain to use.
Python is not multiple coding styles - not entirely true, but probably true enough.
Python is not monkey-patched - again not entirely true, but maybe true enough.
Python is not overly concise - mostly true, though I've seen abuses of array slicing and passing/storing/calling functions.
Python is not overly verbose - pretty much at the mercy of whoever is writing code, but again, seems to be true enough.
Python seems to be pretty good or good enough in a lot of ways, and deserves credit for this. The far-sighted view will recognize it as a local maxima, however.
(A "perfect" language is like a perfect predator. A tireless, scentless, invisible, silent, indestructible, ultra-fast killing machine would probably wreck its environment. Considered in the context of evolutionary change, it's practically impossible for such a thing to come about.)
I would drop python in an instant of a better language came along. For me, the question isn't what it fixes, but what it succeeds at. It makes my professional life less painful and that's really the only reason I need.
edit: Also would like to not that the original article seemed fairly pointless. It's not really saying anything of substance.
The answer to 'Why Python?' is a lot better than what you're going to get for Ruby, Scheme, or the continued use of Perl. At least PHP has its execution story going for it (immediate mode) even if the language sucks ass.
'Why Arc' doesn't even have a polite answer at all. There's a log in pg's eye, and he's here picking at motes.
But Python is probably the best example in recent history of a language that made it big not because it was anything radically new or innovative, but because it has a great standard library and a solid, mature implementation.
There are few language features that would be hard to succinctly express in a line of pseudocode. ("If the implementation is hard to explain, it's a bad idea.")
Result: straightforward, algorithmic code looks like pseudocode, and any code that uses special language features really pops out -- if you see an at-sign or double-underscores, you know something Python-specific is happening.
Python to me is the new world where any immigrant has a niche for whatever he seeks.
That's good enough for me to not bother with any other scripting language.
I prefer using python for machine learning tho. I dislike the libraries written in matlab.
Is numpy+python faster than matlab?
The main reason I say matlab is better for engineers is that its documentation is beyond amazing, if you think of something you need you search in the help browser and there is a good chance it is there with substantial information (something that is much harder with the math/science python libraries) also if you know a functions name you just type: help function and get plenty of information which helps when you forget how to use something and need a quick lookup.
I have done some things with scipy but nothing more than solving some odes and I must say even with all the pains of matlab with writing separate files to solve odes and other such things plus some syntax grievances it is still the easiest solution for me.
I do know that I tried rewriting one of my C++ codes using scipy/numpy and it was about a hundred times slower for that particular task (which did happen to involve very large matrices, approaching the limits of memory).
The earlier solvers took strings with the name of a function whenever they needed a functional argument. Nowadays Matlab provides `function handles' to refer to a function without calling it. The syntax is @function_name, they need this special syntax, because just mentioning function_name itself in your code, calls your function without arguments.
Matlab even support proper closures with lexical scoping, now.
I do agree with all of these advantages of using Matlab in an environment where my data is already cleaned and normalized; I just rarely find myself in that position.
I do not know about I/O libraries. All my Matlab knowledge basically comes from helping out friends with Matlab problems, and reading the manual together with them.
The easiest way to install the suite of scientific computing Python packages on Windows may be PythonXY or Enthought EPD. Under Linux and OSX, users seem to have few problems.
If Python has fixed anything, it has been because of its roots as a teaching language, with a culture for treating the structure of the language as a tool. Blur the lines between syntax and semantics, and your only option is to create things that can be expressed in both realms at the same time.
I wouldn't actually call it a Lisp on its own. It is a derivative, a less directly powerful form. "It's a Lisp where..." in the same way that a unicycle is "a bicycle where..." or something.
But you could argue every language is a derivative of another, or another's functionality, or style, or...
To fairly call a language "a type of X" there honestly has to be a lot more in common with it.
> "It's a Lisp where..." in the same way that a unicycle is "a bicycle where..."
... it has wheels??!!
I'm sorry, it's not Lisp. It has several of Lisp's useful features, but it's likewise missing a large number of them; to the point that it's a real disservice to glibly compare it and conveniently ignores a lot of the power of Scheme, Common Lisp, and their true ancestors/siblings.
If you intentionally misread the very simple analogy that was made in order to give yourself the opportunity to make an inflammatory reply, then there is not much else I can say to sway you.
However, the comparison was not glib. I'm not conveniently ignoring anything. Perl was compared to Lisp when it first appeared, but it is not anymore. People compare JavaScript (mistakenly) to Scheme all of the time, and it is not really Scheme at all. Nor is Python actually Scheme or Lisp. It is an imitation of some of the features of Lisp, but specialized to some other purpose.
If you were replying in hubris with the intent of quashing the unqualified newbie who doesn't know anything about Lisp, allow me to offer some details.
Things Python lacks compared to Scheme: Continuations, tail call optimization, optional typing annotations (like in Bigloo etc.), direct AST construction, explicit metaprogramming, hygienic macros, formal specification (indeed, Python is not a language that can be parsed), lambdas with seq (Python lambdas are expressions only), a dynamic-wind solution for generators, and more.
Things Python has in common with Scheme: readily accessible FFI, rebindable syntax (as well as during runtime), garbage collection, eval, implicit metaprogramming, yielding, standard library with lists + associative maps + bytestring manipulation, extreme late binding as desired, symbol manipulation, lexical (and dynamic) scoping.
Or would you prefer to keep flailing around in anger?
Certainly it is; see the book Higher Order Perl, for example. If you've read SICP, HOP will seem familiar.
Hmm, have I missed something? What features are you talking about?
Clojure is curb stomping Lisps in applicability. I get lisp in general. I use Emacs to write Python. Lisp still can't get it done. And It is whatever people are doing this decade. Sure Lisps should be better, but it's always the fault of someone else that lisp hasn't taken over the world.
Everything else is love.
And I don't really buy the argument that Python is simple and clean. It has multiple inheritance, metaclasses, operator overloading, decorators and whatnot. Also the type system is broken, a deriving class can define an overriding method that has incompatible signature, rendering isinstance powerless (yes, I know you probably shouldn't use isinstance much anyway, but dammit, it should at least mean something).
Yes, but consider the competition. Python is a significant margin above average, but this says as much about the sorry state of computer languages in general as it says about Python.
Heck, English is considered an "easy to learn" language among human languages, and it's a mess!
Really the problem is with human beings.
(define x 3)
(let ((x 38))
(display x)
(newline))
38
(define x 3)
(begin (define x 38)
(display x)
(newline))
38
(display x) (newline)
38 ; we want 3 here
def f():
x = 1
def g():
x = 2 # creates new local variable x
# no way to assign new value to x in enclosing scope!I view this as a benefit - less accidental stamping on variables/etc. I wouldn't consider myself A+ #1 programmer by any stretch though. This sort of thing seems like it would enable too many 'side effects' though.
You can do this in Python too, but it requires a bit more work. Python is asymmetrical here; you can see a variable in an enclosing scope, and if it's a mutable object you can change it, but you cannot reassign the name.
http://shootout.alioth.debian.org/u64q/benchmark.php?test=al...
Should, and can Python be faster without sacrificing the speed of the programmer (including extension authors)? Yes, it can and will be, but never at the cost of productivity.
That this is a trade-off at all is a woefully outdated idea.
Sure; it's "easy" - but the cure can be worse than the disease for most of the python-using population. It's all a series of tradeoffs and compromises - in time, things like pypy and unladen swallow will change this (for python) but given both of those are pretty bleeding edge, I'd argue my initial point still stands.
Think about that; CPython's internal are old - sure, they've changed and been cleaned up and fixed up here and there, but it's still old. Obviously if it was written today, it might be done differently knowing what we know now.
So, yeah - it is excusable, given that it meets the original design goals of making C extensions dirt simply to write (really, they are dirt simple) and making the common, single threaded/single processor use case fast "enough".
Given the hindsight everyone has now - yeah, the GIL as it exists today is grossly suboptimal, and I don't know of anyone of rational mind or body who doesn't want it to go the way of the dodo, or at least be replaced with a cleaner, simpler and faster implementation. The latter is in progress in Python 3 - the former is a lot more difficult despite a lot of hand waving from non-committers and armchair interpreter designers.
Python's performance can (and is, and will) be improved over time, but as I said originally, it was never advertised as "the fastest" or "the most concurrent" language, those two things have never been a core "feature" of the language.
Which is what I also implied. C extensions are simple to use for Visualworks Smalltalk, whose development goes back in direct lineage to 1989. Again, why is the GIL such a problem?
Speaking as someone who loves Python and uses it all the time, but has spent a lot of time learning to love Cython in recent months. A nice feature of Cython (psychologically speaking, at least) that is missing in Python is the power to say that dammit, certain constants are constant, will never be changed, and can be inlined instead of having a hash table lookup every single time.
The sarcasm here is unwarranted, as is basically your entire paragraph. Why not just say: Python is slow ?