Hacker News Reader

http://news.efinancialcareers.com/us-en/173476/investment-ba...

"...There are around 5,000 developers using [Python] at Bank of America, .. there are close to 10 million lines of Python code .. and we got close to 3,000 commits a day."

IMHO, it would be scary with any language.

Is this different for Python over any dynamic language?

This seems to be a better argument for statically compiled languages over dynamic languages rather than an argument against Python.

That's a good point, and perhaps it is. The majority of my experience with dynamic languages is probably in Python.

One thing that makes a difference between Python and other commonly used dynamic languages (here referring to JavaScript / ECMAScript and Lua) is that Python is dramatically more complex out of the box. This is both in terms of language features and Python's batteries-included standard libraries. For example, the amount of overloading that you can do in Python is very impressive, but the result is that common operations aren't necessarily predictable or dependable without more context (not that this isn't a problem in static languages - looking at you C++, but at least in the other languages you get type information to help out and some checking at compile time).

Funny, my experience has been that large/ complex python code is actually easier to follow than JavaScript because of the line noise that JS requires. Side by side, the same code will usually be easier to read in Python (or Ruby.)

Same goes for Java, although in a slightly different sense. I've seen non-trivial Java projects balloon quickly where you have dozens or hundreds of classes & interfaces with no clear sense of structure. High line noise, lots of boilerplate. Sure it compiles but at the end of the day it's just as likely to hit a NPE so you have to rely on functional and unit tests regardless of static/dynamic/compiled/interpreted.

Obviously a lot of it comes down to who's writing the code. For me, a language that lets me succinctly express my intentions with minimal cruft is what wins.

I totally agree that it comes down to who the code is written by. I've done both python and java, and in python I would go so far as to define interfaces as a form of documentation in my python code so people who use it know what I expect out of their classes which want to interact with my object. It doesn't require the class to explicitly implement the interface but they can if it makes sense (fairly similar to Zope Interfaces I guess). Unfortunately, I don't see this a lot in most python applications and you're left searching around through calls to figure out exactly what is required of the object being passed in. Look at a lot of popular libraries -- you are still left searching through code just to see what can possibly be returned because they aren't always the same type -- also, exceptions aren't part of the function or method definition, which means if you don't document your exceptions, I'm up shit creek. I think a lot of the problem around the python ecosystem, again, that I've seen, is people just don't follow best practices -- documentation, it's a core component if python development -- It's one of the arguments they use against static typing. Personally, I think python developers need to be much more strict about development practices than java developers. I think it's a lot easier to make bad python code than it is to make bad java code. The same goes for C, which I used to also do as well. To be a good C developer, you have to be very strict and structured (however, for somewhat different reasons). With great power comes great responsibility.

I've seen shit java code as well. A lot of bad java code usually revolves around things not being modular or not having some form of consistent development patterns or not breaking methods down into simpler sub-problems -- I think documentation isn't as important as it is in python for the fact that I know exactly what is being returned, what exceptions can be thrown, and what exactly needs to be passed in just by looking at a method. In terms of the business logic associated with the class, that still needs to be documented.

That said, I understand why python is generally used at startups -- it allows fast initial development where at startups, time is critical. Long term development really relies a lot on the teams ability to make structured decisions and organize their code, which is a difficult task in any language.

Most of these runtime errors rarely make their way to production and are easily identified and rectified in cases when they do slip through.

More restricted languages, e.g. Java, do have advantages in terms of forcing developers to do some more self-documenting and better opportunities for tools to help with syntax highlighting, refactoring, and warning about those errors you mention. But those errors are usually caught easily even with basic test procedures and almost always if there's a half-decent automated testing regimen.

A lot of it comes down to, who's writing this big project? If developers are talented, there's a lot to be said for the freedom granted by a language like Python or Ruby. If developers are more mediocre, the forcing function of a language like Java may help to produce a "boring" and larger but understandable code base.

>If developers are talented, there's a lot to be said for the freedom granted by a language like Python or Ruby. If developers are more mediocre, the forcing function of a language like Java may help to produce a "boring" and larger but understandable code base.

This is a bit of a false dichotomy.

Much like I don't think lisp is popular for its functional-ness or whatnot, I don't think python or ruby are popular for their dynamic properties. My guess is that 90% of it is just how the code ends up looking. And with type inference getting further along, I think we'll see more people realizing that typing is a great sanity test to have in your code (especially in big codebases). See the popularity of Go in pythonic circles.

I have never been shown any non-tiny piece of robust code that was easier to deal with because of a language's dynamic nature. I have been on this Earth a bit less time than others, but my gut feeling is that it doesn't actually exist.

(note: my day job involves a lot of python, and I love the language for many things. I am looking forward to optional type annotations, though, as are many of my coworkers)

This is an argument for unit tests, not for a statically typed language.

Type errors tend to be very few and far between compared to logic errors.

Multiple processes in most web applications are best handled by a pre-forking webserver in front of something like mod_wsgi, and using an engine like Celery for asynchronous and long-running operations on the backend, often launched with something like supervisor.

I do take issue with some parts of the article, namely saying there is a good type system (there's not, but it's all fine, duck typing, etc) and that twisted is a good framework.

It is definitely true however that Python is a great fit for all kinds of serious applications, but with all things, it takes discipline.

What is the benefit to using a dynamically typed language if you have to spend all your time writing tests and verifying the static properties of your code?

The idea is that writing code + writing tests in a dynamically typed language is easier and faster just writing code in a statically typed language. (Of course it depends on the languages in question, the programmers and a number of other factors)

Mainly the boilerplate other languages force upon you. Whenever i look at Java Code i start to chuckle.

Its so much more information that you need to load into your head.

To quote a picture from the post: https://www.paypal-engineering.com/wordpress/wp-content/uplo...

And its not like you need static typing everywhere just in some places its better to enforce it for your own sanity. Type hinting looks like a good way to solve this problem in python.

That image isn't really a fair comparison (in that C++ / Java aren't the only statically typed languages.)

I imagine a statically typed Python dialect would only have an extra 15 or so lines, and the benefits would be numerous.

I love python as much as anyone, but having to keep a reference manual handy just to use someone's library because they had the audacity to use a variable is not my idea of a fun time.

EDIT: I'm probably exaggerating too much.

Would I use Python in a large, complicated, multi-developer project? Yes, I would. And my only real complaint would probably be that I like static typing so much that I'd miss it.

> I imagine a statically typed Python dialect would only have an extra 15 or so lines

Actually you can have a statically typed Python if you want. Have a look at http://docs.cython.org/src/quickstart/cythonize.html

All type declarations are optional. It compiles modules to versions completely interoperable with the rest of Python code. They're importable, behave as you'd expect, etc.

Consider Haskell. It's about as anti-boilerplate as they come. And with automatic type inference.

I've never had to spend all my time doing that. I am making the case that type errors (passing a hash instead of an int, etc) are NOT one of the major causes of errors in software development.

I'm apt to screw up other things, but that's not one of them.

> I am making the case that type errors are NOT one of the major causes of errors in software development.

That's symptom of a bad typing system. I'm with a moderately sized Haskell codebase right now and type errors represent the biggest share of errors on it by a huge margin - several times bigger than runtime bugs.

Haskell makes programming into a duel with the type system specifiers.

I prefer to think of it as a dialectic.

Yes, it does. That duel replaces a chase on unknown and strange runtime bugs.

Sometimes it's a win, other times it isn't.

Shouldn't you be writing tests anyway?

NB I'm not being flippant - I've been using Python for a couple of years and appreciate the lack of boilerplate compared to many other languages and if there are "type" problems my unit tests, which I will be writing anyway, find them. Using an IDE like PyCharm does a good job of warning you about type problems at development time so I don't have a lot of run time type problems.

Yeah, you should, but you don't have to check trivial things like "is this a list?"

Well, I wouldn't write a unit test for that - I would write a unit test for the high level observable behavior and if I get it wrong that something is a list then it'll break and my unit test will fail.

I think that's closer to functional testing than unit testing though...

> This is an argument for unit tests, not for a statically typed language.

Static typing is like unit tests that take no maintenance.

> Type errors tend to be very few and far between compared to logic errors.

False distinction. If you just take your python code and write it line-by-line in another language, sure, you won't gain much. But if you actually work with the language, you turn logic errors into type errors. See e.g. http://spin.atomicobject.com/2014/12/09/typed-language-tdd-p... , and note that that's a lot more verbose than it would be in a modern language with type inference, higher kinds and so on.

>Static typing is like unit tests that take no maintenance.

And makes your code 20% longer, which makes for a lot of extra fun not-really-so-cost-free-after-all maintenance.

I found the complaint that it finds bugs earlier (at compilation time!) ironic given that, in my experience, running a suite of unit tests takes less time in python than, say, compiling does in Java, never mind compiling and running tests.

> And makes your code 20% longer, which makes for a lot of extra fun not-really-so-cost-free-after-all maintenance.

Just 20%? Sounds like a good deal, considering those that brag that brag that "over half of our code base are just tests!". I don't know if that kind of thing is fashionable or widespread any more, though.

> And makes your code 20% longer

Not necessarily true, especially if you are using a language with type-inference or you are able to encode your logic in types.

Additionally - what do you think unit tests are? They aren't additional code you have to write?

I'm not someone who thinks types completely reduce the need for testing, but I absolutely do not get why dynamic language fans are like "Uhg, I HATE having to write types", but then end up basically reimplementing a type system in a much more verbose testing framework.

>Not necessarily true, especially if you are using a language with type-inference or you are able to encode your logic in types.

I agree that it is not necessarily true, but the languages which manage to squeeze in static typing and still end up with programs that are shorter and sweeter than python's are all fairly niche right now, which carries its own set of problems.

>Additionally - what do you think unit tests are? They aren't additional code you have to write?

Never denied it for a second. There aren't any languages which don't require unit testing though, and there probably never will be. Let's not pretend otherwise.

>I'm not someone who thinks types completely reduce the need for testing, but I absolutely do not get why dynamic language fans are like "Uhg, I HATE having to write types", but then end up basically reimplementing a type system in a much more verbose testing framework.

I've never done this.

I'd wager that the amount of code I have to write, including tests, is less than in all other practical languages. Often much less. That is very valuable.

> I'd wager that the amount of code I have to write, including tests, is less than in all other practical languages. Often much less. That is very valuable.

It would take a very gerrymandered definition of "practical" to say that Python counts but F# doesn't, and I'm pretty confident F# would win that comparison for most problems. (If you'll allow me Scala, which is my language of choice and the one I use full-time at my job, I'm very confident it would win the comparison for the vast majority of problems)

F# might, but I doubt scala would.

And as someone who has coded both Python and Ruby on the job, and most recently Scala, I find the amount of time I'm debugging runtime errors much less in Scala, which is very valuable.

I also don't feel the amount of code I am writing to be all that significantly larger than what I was writing in Python/Ruby.

I would guess a well done scala project would be at least one order of magnitude shorter than a comparable python project. You just can't get the same level of abstractions without a type system to help you out.

> And makes your code 20% longer, which makes for a lot of extra fun not-really-so-cost-free-after-all maintenance.

Less than that, in my experience. Certainly a lot less than the amount of tests and documentation you'd need to make up for the absence of types.

> I found the complaint that it finds bugs earlier (at compilation time!) ironic given that, in my experience, running a suite of unit tests takes less time in python than, say, compiling does in Java, never mind compiling and running tests.

You need to work with the language. Is a python test cycle faster than a full rebuild? Probably. Is it faster than the time between making a change and seeing the red underline in one's IDE? Absolutely not, in my experience.

>Less than that, in my experience. Certainly a lot less than the amount of tests and documentation you'd need to make up for the absence of types.

Certainly more, in my experience.

>You need to work with the language. Is a python test cycle faster than a full rebuild? Probably.

By an order of magnitude.

> Is it faster than the time between making a change and seeing the red underline in one's IDE?

It can be that quick, yes. I have a unit test watcher that reruns them every time a file save in the project is detected (using watchdog/epoll), and I get the results back in seconds.

That has the added advantage of detecting more than just trivial type errors. It catches logic errors too.

> And makes your code 20% longer

One thing I've noticed on larger dynamic language projects: Developers tend to adopt a very defensive strategy in terms of validating inputs, to ensure it is clear a bug is not in their component. This includes a lot of what are essentially runtime type checks.

This is not necessarily a bad thing, because it ultimately makes the code more reliable. But it is "more code" which needs to be maintained, and eliminates the much of the advantage of using a dynamic type system in the first place.

> Static typing is like unit tests that take no maintenance.

Um, no. At best they are like a tiny fraction (and trivial fraction at that) of the unittests required. AND static typing often requires maintenance (as all code does) including initial burden of creating them in first place.

> This is an argument for unit tests, not for a statically typed language.

> Type errors tend to be very few and far between compared to logic errors.

The divide between type errors and logic errors is entirely dependent on the language and the discipline/inclination of the developers. In some languages, what you might think of as "logic errors" can easily be moved to "type errors". And in that regard, the difference between what is unit testable and statically typeable is also entirely dependent on the language and the discipline/inclination of the developer. So what might have to be expressed as unit tests in one language, might be checked by the static type system in another language.

But I can understand how some people might think that static type systems are only for catching things like "error: expected addition of integers, but got attempt at adding an integer with a FireTruck". ;)