Hacker News Reader

Comment by pranjalv123 | Hacker News Reader

My biggest problem with Python is that projects larger than a given size tend to become unmaintainable rather quickly.

This is in large part because of the lack of strong typing and type annotations; if you aren't the only author or can't keep the codebase in your head, it takes real effort to figure out what a function does. Even the type annotations provided in a language like Java or C++ make this task much easier, not to mention languages with real strong type systems like Haskell.

That's not to say that building large systems in Python is impossible, but it takes a lot more effort and documentation that it would in other languages.

View on HN

Python 3 actually does let you annotate function arguments, although it's with an arbitrary expression rather than explicitly being a type. So this is valid:

    def haul(item: Haulable, *vargs: PackAnimal) -> Distance:

As is this:

    def thing(something: "A string!") -> "does stuff":

It's also not explicitly checked, although if you felt like it you could probably write a metaclass to enforce type checking. (Though I imagine to do so would be rather un-pythonic)

This is a significant problem for me as well, coming from C++ and Haskell. Even just going through and making a few changes here and there, there’s no way for me to know without extensive testing whether my edit was even remotely correct beyond a cursory syntax check.

I’ve never had to deal with testing all that heavily in statically typed languages, because I can rely on the type system to do a lot of very helpful static analysis. The lack of basic static knowledge about a Python program is frustrating and discomfiting, and ultimately discourages me from using the language for anything serious.

+1 on the static type checking problem.. i am beginning to feel very annoyed by the same... have found that writing and maintaining unit test cases help.. but are a PITA nevertheless...

Well, as I see it, more or less every error should be expressible as a type error. In order to actually do that, you’d need full dependent types, and that’s a huge language-design “now you have two problems” feature—compiler authors are understandably uncomfortable with undecidable, non-inferable type systems.

Anyway, Haskell and even C++ used intelligently can tell you a lot about your program at the outset, so ultimately your tests can be more focused.

> more or less every error should be expressible as a type error. Am not sure i agree.. i would like to believe it's true though.... >Haskell have just begun to poke around, like the currying + function composition.. will keep poking...

Yep, I agree with you - although every error should be expressible as a type error, languages like Coq where this is actually possible make writing usable code pretty difficult. I think there's a pretty wide range of middle ground, ranging from C++ to Haskell, where you get the benefit of static types without making it really difficult to write working code.

> I’ve never had to deal with testing all that heavily in statically typed languages,

Lack of tests means bugs now, or later when you're refactoring. They're pretty essential in static languages, too, although you can certainly lean on the compiler for many things.

Anecdotally, my bugs usually aren't caused by having passed in the wrong kind of an object.

Well yeah, to be fair, neither are mine. It’s just irritating to be running a prototype thinking everything is more or less fine, then SPOOM goes a runtime error because you didn’t properly change a method invocation, which could have been caught by a static check and saved you some time. The more tools I have to help me attain my actual goal of writing software that does stuff without spooming, the better.

I find the focus on dynamic typing a bit fascinating: you don't see people focus so much on it on a discussion about LISP. [EDIT: I meant to say that this most likely reveals something else is at work in the particular case of python].

When a C program compiles, it really does not tell you much about whether it will work or not. Maybe it is a matter of application domains, but I tend to actually spend more time testing C parts of the code compared to the python code. But, in proportion, I spend more time testing in python than in C, because I spend so much less time doing things in python than in C.

Haskell is different, because types can be used to enforced constraints in a much better fashion than C/C++/Java/etc…

The one part when simple typing systems is useful is refactoring. Doing so in a dynamically typed language is difficult (but possible, as shown by smalltalk, which is supposedly even more dynamic than python).

Lisp’s other useful features just make the particulars of the type system less of an issue. When writing prototypes, I move things around and refactor a lot. Python makes this easy at the cost of certainty about whether it was done right. I don’t write a lot of Lisp, but in my experience refactoring there is quite different: I’m not editing methods and classes, I’m factoring functions and macros. If I were one for supposing, I’d suppose the OO philosophy of Python forces you to create interfaces before you necessarily know that they’re correct, so you end up refactoring them, and there’s poor support for that when no static type system has your back.

This is already a more interesting discusion. I cannot comment on Lisp, as I have too little experience with it. When you say python forces you to create interfaces, I am not sure I understand. For me, programming is mostly about creating interfaces (as the interface part of API), python or not. When I refactor some C code, static typing does not help me that much, as what's difficult in C is lifetime's object (e.g. ref counting in C extensions), things like that.

What is true is that it is easy to write code that is impossible to refactor, because it is full of globals, or "stringified API" (e.g. using a dict instead of objects when the set of keys is supposed to be fixed). But I am afraid people writing this kind of code would write bad code in any language, or are at least very inexperienced in the language.

"I find the focus on dynamic typing a bit fascinating: you don't see people focus so much on it on a discussion about LISP. "

Interesting point.Thinking back on the Clojure discussions here over the last two years i don't recall many issues with readability/maintainability due to the lack of a static types system.

Are there any really big multi programmer clojure code bases yet though ?

Just to be clear, Python is strongly typed but not statically typed. But I definitively agree with your comment. The startup I'm working for is Python based, but I'm currently learning Haskell, because I'd like to take advantage of a good type system for my larger efforts.

Aren't you concerned about unpredictable memory requirements?

To be honest I don't have enough experience with lazily evaluated languages to be concerned about it yet.

I must respectfully disagree. I work with web-based systems and find the dynamic typing of Python to be its biggest strength.

The ability to just throw another property onto an object before sending it to the template-engine saves so much engineering work, it totally overcomes the downsides of not annotating required types.

I can see where it may be an issue, but I think the documentation should focus on what function calls need to be supported, as opposed to type-checking i.e. "This function takes an object with the read() and update() methods."

"I must respectfully disagree. I work with web-based systems"

Web-based systems tend toward smallness and/or flatness, with fairly straight execution paths through the system. You're not going to run into the limitations being discussed.

Yeah and big systems architects should really learn from that. A bunch of small services connected by well defined interfaces is the flipping bomb!

When possible, I totally agree. It's great when you can get it. Most of the work I do is just like what you describe. Sometimes it's not, and that's when I really feel the utility of types.

That's essentially how Go handles polymorphism. http://golang.org/doc/go_spec.html#Interface_types

I think the proper way to do this is through the use of interfaces (Java), abstract superclasses (C++), or typeclasses (Haskell, etc.). These give you the benefits of polymorphism, e.g. you can pass anything that implements "read" to a function that processes input in a given way, but also gives you the compile time typechecking that helps you avoid bugs.

Using interfaces is nice, in theory, but I find them cumbersome in practice.

Say you have a function that takes a IAmReadable object, with the method read(). You implement it, and all is well with the world.

Then, you encounter a function that also uses read(), but requires an IAmAlsoReadable object. Again, this function only uses read(), but IAmAlsoReadable requires you to implement random(), silliness() and 4 other methods that you'll never use.

Java's interfaces require you to implement all of those methods, even if it's just to stub them out (or return 0, 0f, or null). Blame poorly designed interfaces, but ultimately we have to live with them. And yes, you can get your IDE to generate them, but that's just working round the basic problem which is that the type system sucks.

Personally, I think the answer, if you're using a static language, is to analyse the code to see what methods are actually required. This would reduce the interface system to merely providing "hints", but would keep you from cluttering up your code with meaningless method definitions.

That's my two cents on the matter, at least. I'm sure someone more qualified will point out a glaring hole in my logic :-)

Yup. This is also the case with Ruby, and is one of the challenges when trying to dive in to a large unfamiliar project. For example, when something in Rails doesn't behave correctly and you flip deep into a source file, it's not always easy to track down exactly what kind of arguments a method expects. Often you'll have to breakpoint inside the method just to inspect what's around.

I find that it's a problem with most languages. Beyond a certain point a project will start to need infrastructure, documentation, code style guidelines and tests - especially when there's more than one developer.

My personal preference is for building a large project as multiple smaller ones, or else leaning on other libraries as much as possible (eg. Numpy, Scipy, pygame)

An alternative viewpoint is that Python makes it so you GET to that point a lot quicker, since your iterative development cycle is so much faster, No compiles, no need to refactor a bunch of stuff to accommodate minor data structure changes...

I agree and I was worried I was the only who would say it :)

Working on big legacy code base without proper unit testing and strong style checks is probably hard in any case. With Python the idea is that e are grownups. If someone accesses private methods or changes the type of an argument without proper checking, it's his problem.

Reply on news.ycombinator.com