This is just wishful thinking.
Think in terms of the budget of whatever you're building - the python guys delivered in a quarter of the time it will take the Java guys.
It's all about solving lots of small problems quickly, imho.
So, yeah, that's probably why Google uses python extensively.
YouTube itself is an interesting project from the inside, though perhaps not that much different from any multi-featured website. Everyone has their own favorite features, their own favorite parts of the site, which they think will become "the next big thing", despite having a few orders of magnitude to go in order to reach the top 5 (which is 99.9% of the traffic).
The chat/individual playlist thing was a cute feature, but it had maybe a few hundred concurrent users at any one time. If you consider that the recent Lonely Island video pulled about 3 million views in a 24 hour period, and was less than 1/333rd of YouTube's video views (using the publicly released 1 billion views/day number), you come to the realization that any code that isn't being used heavily, is effectively an unused feature that offers you nothing, and even worse, can't earn the site any money in ad revenue.
Heck, I saw micro-communities as "the next big thing" on YouTube, and saw how poorly the original YouTube Groups fulfilled that. I envisioned small-medium sized forums where people would post, rate, and watch videos. I pushed and pushed and pushed until I got a little under a year to work on it, releasing Groups 2.0 in August/September of 2009. I watched how it was being used, got rid of some superfluous features, and pushed out an optional new version for people to use, which became the default on my birthday as my parting gift. Turns out that some code that I was relying on working from another part of the site had broken by another engineer, so they manually made it the default and only version a week or two after I left.
Sadly, YouTube Groups is no more, having been turned down on December 1, 2010, code deleted a couple weeks later. A friend and former coworker who had maintained Groups after I left had pointed me off to a WWE fan group with over 100k video posts and a few hundred thousand members (which would be a huge success anywhere else). Since no one was really willing to truly maintain and improve it (I can't blame them, there were some nasty hacks in there), and it wasn't driving as much traffic as even one Lonely Island video, they tossed it. It probably didn't help that I'd ruffled some feathers with the ways I'd pushed for Groups; I'm surprised it lasted even that long. Ah well, many lessons learned, many good memories.
I must say that it sounds great Youtube's culture allows features to be "turned down... code deleted a couple weeks later". You don't always get that.
The guy who ultimately deleted my code had taken on maintenance of at least two large pieces of code that 3-4 engineers (including myself) had written over the course of a year and a half, none of whom are still with Google/YouTube. Turning Groups down and deleting the code simplified his job immensely, of that I am sure.
The trick is that Python is a bit like special sauce. If your business is booming because of it's use, you are wary of expressing it as much, because it allows you to get so much more done so much more quickly than your competitors. If they find out, then they may have a chance to catch up.
It always blew me away seeing startups using JSP, .NET, etc., when starting out. Get it done first, then make it fast (if you need to).
As about the indentation, the inherent nature of mandatory indentation significantly improves code reading quality which will allow someone new coming in to work with "100,000 to 1 million" lines of code an easier time blending in.
Readability means different things at different scales. Reading Python "in the small" is easy; it's looks like pseudocode. Once you start writing larger programs, between the 5k and 10k mark, the looseness of Python starts to become a burden. It can be very difficult to look at a piece of Python code in a large project and know exactly what it does. Dynamic typing means that a whole class of serious programming errors go unnoticed until runtime - and often only for a specific, uncommon input.
("Compulsory indentation" ?! Whomever works on a large project with others and doesn't format their code to the standard of the project should be taken out and shot. I'm only half-joking. Indentation has almost nothing to do with readability, though. It's just common sense.)
(I'm surprised to hear you say "there's only one way to do this" with regard to Python, as in my experience there are always several ways of doing anything, and it's not uncommon to see different/competing approaches used in a single project.)
I'm biased, but Go is a great choice for large-scale software development. It has the brevity of a scripting language, but being statically typed and compiled Go can be more reliable and efficient. It's also smaller and more consistent than any mainstream language. Because of this, you can typically look at a piece of Go code from any project and understand what's going on. This is because it's difficult to abuse Go in the same way you see done in the other four languages I've mentioned, and that is truly valuable at scale.
Um...it has EVERYTHING to do with readability. That is WHY it is just common sense.
The arguments I've read says that the "code scalability" is handled with more testing in the scripting languages. Is that wrong? I don't have statistics around, anyone got references?
(And yeah, indentation is mandatory. Implicit indentation often gets problems with copy/paste of code.)
This would give one the best of both worlds: fast prototyping for initial development, and type safety, added information when refactoring, and additional optimization when the code is mature.
I love Python and haven't touched Java in years but my colleagues who use Java seem to write programs that are maybe around 3 times longer (this is a ballpark figure).
I get the feeling that programmers that produce gobs of Java code are probably going to do the same thing in Python. A good Fortran programmer can write Fortran in any language :).
After you read that, hopefully you'll start passing around functions as arguments inside another functions. It's tremendously fun and changes your way of thinking about code.
And this little answer in StackOverFlow summarizes what lambdas can do and how you can use it in python: http://stackoverflow.com/questions/890128/python-lambda-why/...
The simplest explanation is think of your program functions like mathematical functions that spit out values, with that in mind you can start sending in functions as arguments. A very brief e.g from the link:
def addition(n): return lambda x: x + n f = addition(3) f(4) # is 7
here, f is actually binded to a function - the addition function from above (because it returned a function called lambda).
I think basically what the parent comment is saying is that it's much easier to pass functions as arguments to other functions in Haskell than it is in python. They call them high order functions.
myList = [[1, 2], [3, 4], [5, 6]]
for elem in myList:
del elem[0]
map(lambda elem: del elem[0], myList)
map(lambda elem: print elem, myList)
function makeCounter() {
var value = 0;
return function() {
return value ++;
}
}
> c = makeCounter();
> c();
0
> c();
1
... and so forth
def makeCounter():
value = 0
def count():
nonlocal value
tmp, value = value, value + 1
return tmp
return count
Of course, Haskell has neither state nor statements, so the above examples don't literally show why Haskell's lambdas are necessarily superior, but the point isn't that Python disallows assignment in lambdas; it's that Python's lambdas can only express a subset of possible functions, while JavaScript and Haskell (and Scheme &al.) don't have that limitation.
my_list = [[1, 2], [3, 4], [5, 6]]
my_other_list = [ elem[1:] for elem in my_list ]
As for your second example, I would consider using "yield".
The issues with Python's lambdas really come up when you start trying to do things the Haskell-esque way—which is to say, the super-functional category-theory-on-the-brain way—which can be the right solution to certain problems (e.g. monadic parsers) but tends to be difficult to express in a way that's pleasant, non-hacky, and Pythonic. But I am of the personal opinion that Python's design choices make sense and I don't want to rail against it for not being Haskell or Ruby; I just wanted to show how anonymous functions in particular differ from language to language.
I side step the lack of closures by using named locals with the keyword arguments when defining inner functions. However, for numeric types, this doesn't really work:
def makeCounter():
value = 0
def count(value=value):
tmp, value = value, value + 1
return tmp
return count
inc = makeCounter()
print inc()
print inc()
def set_adder():
s = set()
def add(item, s=s):
s.add(item)
return s
return add
s = set_adder()
print s(1)
print s(2)
t = set_adder()
print t(3)
print t(4)
$ python mk.py
set([1])
set([1, 2])
set([3])
set([3, 4]) def set_adder():
s = set()
def add(item):
s.add(item)
return s
return add
def make_counter():
value = [0]
def count():
temp, = value
value[0] += 1
return temp
return count
Nonetheless, functional programming is not the encourage common style in Python: there are no data structures available, many methods have side effects, it's hard to isolate mutable values from mutable. Not saying it's right or wrong, it's just different.
I should also note that Common Lisp isn't all that functional compared to other languages: I'd rate it being just slightly more functional than Ruby. The reason it appeals to some Lispers is due to simple syntax, data structure literals and dynamic typing. These features are common now, are useful for experimental programming (of the sort that frequently happens in AI and ML work) but they originated with Lisp. Lack of syntax, presence of macros and resulting ease of metalinguistic abstraction are not present in Python, but not every Lisp(er) used them: some preferred functional style (e.g., never using loop macro and using higher order functions and closures as means of abstraction), some preferred macros and DSLs, some stayed away from both.
Java bashing is healthy: while I personally don't mind Java, many developers were forced into writing Java (having previously worked in Smalltalk, Lisp, C++, Perl, and others) as the language was heavily marketed to managers. Java hatred and bashing is thus a well understood reaction (although the culprit for verbosity are awfully horrid libraries such as Spring and J2EE: alternatives like Guice, for example, are much less verbose). However, JVM is now home to Scala, a much stronger candidate for an OO/functional hybrid language than Python (that's not to say Python isn't an elegant and powerful language with many advantages, especially when it comes to tools/automation development).
I'm definitely going to use this definition of 'verboten' from now on. I like your style. Fuck the dictionary!
Also, to save you the work:
ver·bo·ten
forbidden, as by law; prohibited.
Inexperienced programmers write sub-optimal (or plain weird) code. Once they learn their tools, they will write better code. That's not surprising at all.
In my opinion there are 3 things that help with maintenance: * automated tests (to see what breaks if you change something) * refactoring tools (helps in keeping code clean) * domain driven design (helps in mapping a business concept to a piece of code)
These are all things you could do in Python as well.