Python as an Alternative to Lisp or Java, Peter Norvig revisited
dr-josiah.blogspot.com
dr-josiah.blogspot.com
The ability of language X to solve a problem depends on:
1. The expressiveness of the syntax to capture algorithmic complexity. 2. The availability of tools to support the creation of the code. 3. The ability of the typical programmer, conversant in the language, to write the code. 4. The costs involved in maintaining the code over its lifetime.
Rendering all these dimensions into a single metric seems simplistic at best, undermining the argument as to which set of tools and technologies are deserving of more attention and ultimately plays into the hands of the people who wish to render programmers and programming into pure commodities.
One of the things I used to do when working in large teams was wander around and look to see if anyone was having trouble debugging. Sometimes you'd find someone who'd been stuck on a bug for several days. So what I'd do was I'd sit down with them, and start putting in all the 'fluff' like indenting their code, splitting lines with multiple variable declarations each onto their own line etc. Essentially putting in all the 'long cuts' that they'd taken 'short cuts' around in order to 'speed up' the programming.
Often either they or I would see the problem immediately.
These days of course there are tools that do the auto-formatting, but I still try to optimise for readability, simplicity and elegance in my code.
Back to the lines of code as a metric issue (tm) in that scenario where it takes 2x as long to write a line in language X, and that line does twice as much, it is probably at least twice as complex as the 'more verbose' language... and I think this implies that it will be harder to debug and maintain later on. So arbitrarily complex languages are optimal for some internet 'toss off' (one off, toss away) competition, but not necessarily optimal for long term efforts.
Some of these languages that allow you to redefine the language itself on the fly allow you to do 'cool stuff' (tm) but tend to attract people who think they are much cleverer than they actually are. I think it is a regional variation, but all the Ruby fans in my city scare me. The thought of what would happen if I'd spent days or weeks on a bug only to find out it was caused by someone else 'tweaking' one of the core classes and not bothering to tell anyone else... twitch
Code in a more productive (i.e. higher level) language is concise because it hides extra complexity you don't need to think about at the time, and is therefore simpler, not more complex (in the second sense).
In Python: for obj in seq: ...
In C++: std::base_class<template_type>::iterator obj; for (obj=seq.begin(); obj != seq.end(); seq++) {...}
The reduction in lines for Python doesn't make it more complicated, it makes it cleaner and easier to express ideas.
Another example of where this is true is automatic memory management. A line of Java code can do all the memory management that would take many lines in C. This is a mostly good thing. In this case, less lines = better.
By that, what I mean is this: if you give more thought to a program one tends to be able to say more and more but with less and less, and without making the code incomprehensible or complex.
Interestingly, when I show my code to a scheme veteran, he would often re-write it to make it even shorter and clearer. So I have a long way to go.
So line for line , yes it takes me more time to churn out scheme, but if I give the same amount of time to c++/java it doesnt compress as much. But, and here is the cool part, if it does, I can often trace it back to scheme way of thinking. This has happened a lot when programming with C++/STL.
One of the many nice things about Python is that whitespace - specifically, indenting - is significant.
Another problem with it is that I can't just comment out, say, an if, and have the code under it run automatically. I have to go through and reformat the business, leading to potential bugs. This also plays into refactoring as well.
Basically, I find the whitespace rule something that should have been left in the 60s with punch cards.
I am not a fan of Python. I consider Common Lisp and Perl to be fundamentally less ambiguous languages.
Interesting. In several years programming with Python, I've had exactly zero bugs based on whitespace being significant.
> I can't just comment out, say, an if
What I do is comment out the if statement and add another if statement under it, like so:
#if condition:
if 1 == 1: # FIXME temp code for testing
do_something()
I think this is debatable. After all - a function call compresses multiple lines into one as well.. And you want programmers to factor the repeated code into function calls.
The "dynamic redefinition of the language" is the same thing except at a different level. The problem is not with the lines of code per se - it is whether the semantic compression is worth the learning curve, and the "surprise" factor.
Think C++ templates, for example, with a "+" operator that would do something "non-intuitive". The "non-intuitiveness" may vary - and it is a fine art to not cross the line.
FWIW, I've done the exercice in Lua, ~ 1 hour and 107 lines (and for some reason a different order of outputs, which I did not want to debug since I went for a nice dinner with a couple of beers earlier today) - https://gist.github.com/752460
That's a good goal. One advantage of Python is that indentation etc. are required. You can't take a 'short cut' and write messy code, cause your code won't run.
This is such baloney.
I've written hundreds of thousands of lines of code in Algol-esque languages, and just about every last line (by which I mean > 99.9%, with possibly exceptions once in a blue moon where I accidentally hit tab before I compile or something like that, which just gets cleaned up the next time I reformat anyways...) of that was properly indented. It's just too easy not to do, and I'd fire, almost immediately, anyone that was checking in code that wasn't indented properly.
Really. Where are all these horrible developers that don't sanely indent their code? To hear the Python folks tell it, they're fucking up codebases all over the place with their carelessness, but I've never personally experienced any pain whatsoever over this. Certainly nothing that a simple CMD+i can fix...
Hell, different newline types interacting badly with version control have caused me so much more trouble than indentation, and Python's not immune to that. Nor is it immune to people's editors globally forcing tabs to spaces or vice versa.
It took me about 2 hours (compared to a range of 2 to 8.5 hours for the other Lisp programmers in the study, 3 to 25 for C/C++ and 4 to 63 for Java) and I ended up with 45 non-comment non-blank lines (compared with a range of 51 to 182 for Lisp, and 107 to 614 for the other languages).
Seems like the programmer, not the language, is the biggest determinant. If you have the Java programmer who wrote it in 4 hours, and the Lisp programmer who wrote it in 8.5 hours, you're better off writing it in Java!
If you want to write concise programs quickly, using the same language Peter Norvig uses might help, but being as good as Peter Norvig helps more.
It takes a different type of programmer to even want to venture to learn lisp in the first place.
I tend to not use separate URL mappers in controllers but have nested functions that define both logic and layout simultaneously. (Basically a tree structure of function definitions where the ending subpage elements are leaf nodes, and nodes render wrapper base templates on post traversal unless it was an ajax request). I can post some code if this doesn't make sense. Obviously not defining new variables nor using classes is extremely unpythonic and could pose difficulties for group work... minimum number of lines while fun isn't everything!
/78698/37395746288: Koch 8 Sud Hab Tirol
What did I miss in the rules?
The number you mention allows to get into the branch with the 'Koch o"d' - so no digit can be selected.
(I was reading the requirement as "If at a particular point no sequence of words from the dictionary can be inserted to complete the number, a single digit can be inserted" -- which still makes a lot more sense to me than the actual requirement!)
Sadly, no programming language could have helped me with this bug. :)
This is just wishful thinking.
Think in terms of the budget of whatever you're building - the python guys delivered in a quarter of the time it will take the Java guys.
It's all about solving lots of small problems quickly, imho.
So, yeah, that's probably why Google uses python extensively.
Readability means different things at different scales. Reading Python "in the small" is easy; it's looks like pseudocode. Once you start writing larger programs, between the 5k and 10k mark, the looseness of Python starts to become a burden. It can be very difficult to look at a piece of Python code in a large project and know exactly what it does. Dynamic typing means that a whole class of serious programming errors go unnoticed until runtime - and often only for a specific, uncommon input.
("Compulsory indentation" ?! Whomever works on a large project with others and doesn't format their code to the standard of the project should be taken out and shot. I'm only half-joking. Indentation has almost nothing to do with readability, though. It's just common sense.)
(I'm surprised to hear you say "there's only one way to do this" with regard to Python, as in my experience there are always several ways of doing anything, and it's not uncommon to see different/competing approaches used in a single project.)
I'm biased, but Go is a great choice for large-scale software development. It has the brevity of a scripting language, but being statically typed and compiled Go can be more reliable and efficient. It's also smaller and more consistent than any mainstream language. Because of this, you can typically look at a piece of Go code from any project and understand what's going on. This is because it's difficult to abuse Go in the same way you see done in the other four languages I've mentioned, and that is truly valuable at scale.
Um...it has EVERYTHING to do with readability. That is WHY it is just common sense.
The arguments I've read says that the "code scalability" is handled with more testing in the scripting languages. Is that wrong? I don't have statistics around, anyone got references?
(And yeah, indentation is mandatory. Implicit indentation often gets problems with copy/paste of code.)
The trick is that Python is a bit like special sauce. If your business is booming because of it's use, you are wary of expressing it as much, because it allows you to get so much more done so much more quickly than your competitors. If they find out, then they may have a chance to catch up.
It always blew me away seeing startups using JSP, .NET, etc., when starting out. Get it done first, then make it fast (if you need to).
YouTube itself is an interesting project from the inside, though perhaps not that much different from any multi-featured website. Everyone has their own favorite features, their own favorite parts of the site, which they think will become "the next big thing", despite having a few orders of magnitude to go in order to reach the top 5 (which is 99.9% of the traffic).
The chat/individual playlist thing was a cute feature, but it had maybe a few hundred concurrent users at any one time. If you consider that the recent Lonely Island video pulled about 3 million views in a 24 hour period, and was less than 1/333rd of YouTube's video views (using the publicly released 1 billion views/day number), you come to the realization that any code that isn't being used heavily, is effectively an unused feature that offers you nothing, and even worse, can't earn the site any money in ad revenue.
Heck, I saw micro-communities as "the next big thing" on YouTube, and saw how poorly the original YouTube Groups fulfilled that. I envisioned small-medium sized forums where people would post, rate, and watch videos. I pushed and pushed and pushed until I got a little under a year to work on it, releasing Groups 2.0 in August/September of 2009. I watched how it was being used, got rid of some superfluous features, and pushed out an optional new version for people to use, which became the default on my birthday as my parting gift. Turns out that some code that I was relying on working from another part of the site had broken by another engineer, so they manually made it the default and only version a week or two after I left.
Sadly, YouTube Groups is no more, having been turned down on December 1, 2010, code deleted a couple weeks later. A friend and former coworker who had maintained Groups after I left had pointed me off to a WWE fan group with over 100k video posts and a few hundred thousand members (which would be a huge success anywhere else). Since no one was really willing to truly maintain and improve it (I can't blame them, there were some nasty hacks in there), and it wasn't driving as much traffic as even one Lonely Island video, they tossed it. It probably didn't help that I'd ruffled some feathers with the ways I'd pushed for Groups; I'm surprised it lasted even that long. Ah well, many lessons learned, many good memories.
I must say that it sounds great Youtube's culture allows features to be "turned down... code deleted a couple weeks later". You don't always get that.
The guy who ultimately deleted my code had taken on maintenance of at least two large pieces of code that 3-4 engineers (including myself) had written over the course of a year and a half, none of whom are still with Google/YouTube. Turning Groups down and deleting the code simplified his job immensely, of that I am sure.
As about the indentation, the inherent nature of mandatory indentation significantly improves code reading quality which will allow someone new coming in to work with "100,000 to 1 million" lines of code an easier time blending in.
This would give one the best of both worlds: fast prototyping for initial development, and type safety, added information when refactoring, and additional optimization when the code is mature.
I love Python and haven't touched Java in years but my colleagues who use Java seem to write programs that are maybe around 3 times longer (this is a ballpark figure).
I get the feeling that programmers that produce gobs of Java code are probably going to do the same thing in Python. A good Fortran programmer can write Fortran in any language :).
I'm definitely going to use this definition of 'verboten' from now on. I like your style. Fuck the dictionary!
Also, to save you the work:
ver·bo·ten
forbidden, as by law; prohibited.
Nonetheless, functional programming is not the encourage common style in Python: there are no data structures available, many methods have side effects, it's hard to isolate mutable values from mutable. Not saying it's right or wrong, it's just different.
I should also note that Common Lisp isn't all that functional compared to other languages: I'd rate it being just slightly more functional than Ruby. The reason it appeals to some Lispers is due to simple syntax, data structure literals and dynamic typing. These features are common now, are useful for experimental programming (of the sort that frequently happens in AI and ML work) but they originated with Lisp. Lack of syntax, presence of macros and resulting ease of metalinguistic abstraction are not present in Python, but not every Lisp(er) used them: some preferred functional style (e.g., never using loop macro and using higher order functions and closures as means of abstraction), some preferred macros and DSLs, some stayed away from both.
Java bashing is healthy: while I personally don't mind Java, many developers were forced into writing Java (having previously worked in Smalltalk, Lisp, C++, Perl, and others) as the language was heavily marketed to managers. Java hatred and bashing is thus a well understood reaction (although the culprit for verbosity are awfully horrid libraries such as Spring and J2EE: alternatives like Guice, for example, are much less verbose). However, JVM is now home to Scala, a much stronger candidate for an OO/functional hybrid language than Python (that's not to say Python isn't an elegant and powerful language with many advantages, especially when it comes to tools/automation development).
myList = [[1, 2], [3, 4], [5, 6]]
for elem in myList:
del elem[0]
map(lambda elem: del elem[0], myList)
map(lambda elem: print elem, myList)
function makeCounter() {
var value = 0;
return function() {
return value ++;
}
}
> c = makeCounter();
> c();
0
> c();
1
... and so forth
def makeCounter():
value = 0
def count():
nonlocal value
tmp, value = value, value + 1
return tmp
return count
Of course, Haskell has neither state nor statements, so the above examples don't literally show why Haskell's lambdas are necessarily superior, but the point isn't that Python disallows assignment in lambdas; it's that Python's lambdas can only express a subset of possible functions, while JavaScript and Haskell (and Scheme &al.) don't have that limitation.
my_list = [[1, 2], [3, 4], [5, 6]]
my_other_list = [ elem[1:] for elem in my_list ]
As for your second example, I would consider using "yield".
The issues with Python's lambdas really come up when you start trying to do things the Haskell-esque way—which is to say, the super-functional category-theory-on-the-brain way—which can be the right solution to certain problems (e.g. monadic parsers) but tends to be difficult to express in a way that's pleasant, non-hacky, and Pythonic. But I am of the personal opinion that Python's design choices make sense and I don't want to rail against it for not being Haskell or Ruby; I just wanted to show how anonymous functions in particular differ from language to language.
I side step the lack of closures by using named locals with the keyword arguments when defining inner functions. However, for numeric types, this doesn't really work:
def makeCounter():
value = 0
def count(value=value):
tmp, value = value, value + 1
return tmp
return count
inc = makeCounter()
print inc()
print inc()
def set_adder():
s = set()
def add(item, s=s):
s.add(item)
return s
return add
s = set_adder()
print s(1)
print s(2)
t = set_adder()
print t(3)
print t(4)
$ python mk.py
set([1])
set([1, 2])
set([3])
set([3, 4]) def set_adder():
s = set()
def add(item):
s.add(item)
return s
return add
def make_counter():
value = [0]
def count():
temp, = value
value[0] += 1
return temp
return count
I think basically what the parent comment is saying is that it's much easier to pass functions as arguments to other functions in Haskell than it is in python. They call them high order functions.
After you read that, hopefully you'll start passing around functions as arguments inside another functions. It's tremendously fun and changes your way of thinking about code.
And this little answer in StackOverFlow summarizes what lambdas can do and how you can use it in python: http://stackoverflow.com/questions/890128/python-lambda-why/...
The simplest explanation is think of your program functions like mathematical functions that spit out values, with that in mind you can start sending in functions as arguments. A very brief e.g from the link:
def addition(n): return lambda x: x + n f = addition(3) f(4) # is 7
here, f is actually binded to a function - the addition function from above (because it returned a function called lambda).
In my opinion there are 3 things that help with maintenance: * automated tests (to see what breaks if you change something) * refactoring tools (helps in keeping code clean) * domain driven design (helps in mapping a business concept to a piece of code)
These are all things you could do in Python as well.
Inexperienced programmers write sub-optimal (or plain weird) code. Once they learn their tools, they will write better code. That's not surprising at all.