Hacker News Reader

Comment by silentbicycle | Hacker News Reader

I wonder how much of this is still true. Python was a very compact language a decade ago (when I started using it), but gradually acquired bloat (perhaps because people switching from Java demanded it - decorators* ? tuples AND dicts AND lists?). I later switched to Lua, which on the whole is very similar, but has constraints that have made it stay compact.

* Seriously - downvoters, explain. What do decorators actually add that wouldn't come free with non-broken lambdas? I've tried to make sure I'm not missing something, but they seem like a gratuitous workaround.

    @dec
    def somefunc(x, y):
        blabla

vs.

    somefunc = dec(fun (x, y)
        blabla)

View on HN

You complain that Python has acquired bloat in the past decade, and appear to be citing the presence of tuples, dicts, and lists as an example of this. Since tuples, dicts, and lists have existed in Python for well over a decade, I don't quite see how they can be an example of acquired bloat.

As for decorators...well.

    @dec
    def somefunc():
        pass

is almost precisely identical to

    def somefunc():
        pass
    somefunc = dec(somefunc)

In other words, decorators are pure syntactic sugar. I suppose you could call that "bloat", although I'd rather strongly disagree. There's nothing at all wrong with making common patterns easier to write, and I've written plenty of code which has benefited from access to decorators.

What really puzzles me is that it seems that what you really want is simply added syntax for multi-line anonymous functions...in other words, syntactic sugar. I suppose that "bloat" is any sugar that doesn't come in your preferred flavor.

My complaint mixed two things - one, I'm not convinced that decorators are a special-case "feature" that would come for free with better lambdas. Two, does Python need tuples, dicts, AND lists? (I know it's had them for a long time.)

Python seemed really refreshingly clean compared to Perl and C++, the languages I'd been using previously. I still think it's a decent language, but Python is to Perl and Lua is to Python along those lines. I've moved on.

I'm probably still just annoyed about Guido's position on tail-call optimization, though.

Python's lambda is not broken, it's just that the notion of a (progn) from Lisp doesn't exist in Python, so Python's lambda is a lot harder to use because for anything more complicated then 1 expression, you'd have to define a fully named function somewhere. For the most part this isn't an issue because you probably wouldn't want to have more then 2 or 3 expressions in a lambda anyway.

Granted, a lot of languages like Ruby, Lua, ML and even CoffeeScript have arguably even cleaner syntax then Python, but none has as vast a community and quality libraries that mirror Python. In choosing a language, the language itself matters, but the ecosystem equally matters IMHO.

> Granted, a lot of languages like Ruby, Lua, ML and even CoffeeScript have arguably even cleaner syntax then Python

But Python has still the cleanest standard API. At least for me, that's what made me switch from Ruby to Python in the past.

They're also much cleaner in languages with expression-oriented rather than statement-oriented semantics -- languages where the result of a function is implicitly the last value in it, without needing a return statement.

But you can nest function definitions, so in practice the lack of a multi line lambda just means just prior to where you use it you'd just have a "def f ...".

> Two, does Python need tuples, dicts, AND lists?

While I agree that tuples and lists could be merged, I don't think that dicts should be merges with them, too.

When I first looked at PHP I found the concept of having a unified "array" type (for arrays and hash tables) very clever. However, I quickly changed my mind.

Now I think that the array/hash unification is one of the biggest design mistakes in PHP as it provides no benefit (both have completely different use cases), but causes a lot of trouble.

A similar argument goes for ECMA Script (aka JavaScript).

It seems to work well for Lua, though it has an explicit optimization so that dicts with consecutive integer keys are automatically stored internally as an array.

> does Python need tuples, dicts, AND lists?

Pretty much yes. dicts are something completely separate from list-like types. Those should not be mixed.

lists and tuples depend on your preference really, but I appreciate them being available. There are many places where you want to pass an immutable sequence of objects and the list is just not right... Sure we could get rid of them, but what would be the gain there?

Generally I will use lists for fast and loose vectors, in unoptimized code, and tuples for more firmly "typed" data, or performance enhancements.

I don't need more and I don't need less: the two seem well-balanced for me.

As someone who abhorred Perl, Ruby looks a little frightening. Python looks and feels clean, especially for blocks of data.

In OO though, self.whatever appears more frequently than I'd like.

Decorators and lambdas are completely different and separate topics, one CAN exist without the others.

A decorator is a function that takes a function, optionally some parameters and return a function. What this tells you is that a decorator is a special instance of a higher-order function, that takes a (partially applied) function, fudge around with its internals and return that same function, fully applied. When I said function, they can be named of course, which is the case for most of the Python functions.

Your second example would accomplish the same thing, but let's change it to something like this:

  somefunc = dec(def (x, y):
      blabla)

However, this proposal instead of introducing a new language syntactic feature, you would have to change how the interpreter parses and compiles function argument, namely it'll need to understand a def function definition without a name as an expression. The interpreter would also need to assign an internal name to the lambda and compile it to a function object.

That's not a trivial change in the grammar of the language. However, I'd agree that having that instead of a Python lambda would make a lot of HO functional style programming a lot more fluid.

I've implemented a Python-inspired syntax for JavaScript which supports multi-line lambdas. This is what it looks like:

http://bitbucket.org/wkornewald/pyvascript/src/tip/pyvascrip...

It would look better with syntax highlighting, but I'm not sure if the code stays readable enough with multi-line lambdas.

It also becomes more ugly once you want to use lambdas within dicts or lists: http://bitbucket.org/wkornewald/pyvascript/src/tip/tests/tes...

The problem here is that

return x,

could be either a lambda returning the tuple (x,) or the dict/list item separator, so the separator must be put on a separate line.

I'm sure something like this was already proposed, so what were Guido's objections against it? I suppose it's not possible to make multi-line lambas readable enough?

Maybe we need more powerful IDEs that go beyond editing text in a "stupid" text field and instead work directly on the AST and use other visual hints like boxes around functions/lambdas, etc.

I just took a quick scan at PEP 318 and it seems multi-line lambdas would violate a list of design goals.

  * not needlessly complicate secondary support tools such as language-sensitive editors and other "toy parser tools out there [12]"
  * allow future compilers to optimize for decorators. With the hope of a JIT compiler for Python coming into existence at some point this tends to require the syntax for decorators to come before the function definition

I'd suppose the ambiguity you mentioned above would play a role in the rejection of this syntax too.

Also, there's this thing called a class decorator now. To accomplish the same thing using a prefix notation function application syntax would yield some ugly monster like this:

  decor(class C:
      def __init__(self):
          # 200 more lines of code...
  )

Bad idea. The commenter of this thread seems to think Python has acquired some bloat since the article was written, which is true but probably to a much lesser degree syntactically then some might think. If you've watch GvR's Google Talk videos on Py3k, you'd notice most of the cruft in Python 2.x is actually in the library and the data model. There are only a couple small changes to the grammar in Py3k to rectify multiple except clauses, print statements, strings and removed old style classes. Regardless, Py2.x is still a very readable language as a whole.

I'm talking about orthogonality, not syntactic cruft per se.

Hmm, Python had tuples, lists, dicts and classes from the very beginning (version 0.9.1). Much bloat was tacked on after the simple class model was deprecated in favor of new-style classes... class methods, static methods, metaclasses, descriptors, properties, decorators, you name it. Then there were additions like list comprehensions and generators (although those are actually useful).

IMHO, Python's indentation-based syntax is a great way to make (and keep) the language readable, but at the same time it's an impediment to language flexibility. In a more flexible language, you could have implemented most (or all) of the aforementioned features yourself. (This sounds like something a Lisper would say, but it would work in other languages as well, like Io and Ruby.)

Decorators are just syntactic sugar. Are you arguing against syntactic sugar and perhaps for a slimmer "purer" syntax? If so, I think that's a valid point (though I happen to disagree in this case).

All syntactic sugar actually adds nothing: we don't need +=, ++, or even C-esque array indexing. However, they often provide easier to read/write/maintain source code.

As for Python decorators, one easy example of why they're neat is in Django: you've got your view functions, and if you want to restrict access to a view to only authenticated users, you add the line "@login_required" right about the function definition. I happen to think it looks nice.

> Decorators are just syntactic sugar. Are you arguing against syntactic sugar and perhaps for a slimmer "purer" syntax? If so, I think that's a valid point (though I happen to disagree in this case).

I am in favour of semantic sugar, I just don't think it should be part of the language. Languages should be small, and have features that enable syntactic sugar to be written in the language. These features might include macros (like in Lisp, or at a pinch C) or overloaded operators (as in C++ or Python).

Regarding decorators, I don't see that they bring much value. The decorator syntax is no shorter than doing it without decorators, so if you remove the decorator syntax you're making Python a smaller language which makes it easier to learn and understand.

Macros (and their more hygienic cousins) are not always a good idea. They make it very hard to understand what the rest of the program does, unless you can grep the whole thing. Lispers will violently disagree with me here.

Regarding decorators, no they don't seem to be that useful. They do bloat the documentation, if not the language; and you don't ever really need them.

> Macros (and their more hygienic cousins) are not always a good idea. They make it very hard to understand what the rest of the program does, unless you can grep the whole thing.

So does object-oriented programming, more often than not.

I'm not sure that's a sufficient argument, especially if there's no other way to do static metaprogramming in a language.

Though decorators functional identically do just passing a function to another function which returns a function, the decorator syntax is uniquely useful. For framework like Django, it lets you "tag" a function with a single line at the top. I think this greatly enhances readability over the alternative.

Didn't downvote you, but the difference is that lambdas can only be executed (i.e. applied to something) while decorators let you analyze the code of the function and do something else other than just executing it.

How is this so? Decorators are just syntactic sugar for defining a function then replacing it with the result of calling another function (the decorator) on it.

The decorator can do introspection on the function object. This project uses the idea to execute (some) Python code on the GPU:

http://code.google.com/p/copperhead/

Which being able to wrap the function definition in a function call could accomplish. Unfortunately that's not the direction the Python team wants to take the language.

> The decorator can do introspection on the function object.

so can normal (i.e. not using decorators) code after the the function, e.g.:

   def myFunction(x):
      blah
   doSomethingWith(myFunction)

Here, doSomethingWith() can introspect on myFunction()

Ah, yes - of course decorators do not fundamentally increase the expressive power of the language (and neither do lambdas, obviously). It's all just a matter of convenience and elegance, which are in the eye of the beholder anyway...

or, more usually one of these:

    def somefunc(x, y):
        blabla
    dec(somefunc)

    def somefunc(x, y):
        blabla
    somefunc = dec(somefunc)

The rationale for decorators, AIUI, is that if somefunc() is long, then putting the decorator at the end makes it easier to overlook, whereas putting it at the top is harder to overlook. Personally, I never use them; if I want the functionality of decorators, I write code immediately after the function, e.g.:

   def add(a, b):
      r = a.getValue() + b.getValue()
      return sm.UInteger(r)
   notify(add, "ii", "i", "+", cascade=True)

BTW, lists, tuples and dicts have been in Python since forever (at least as far back as 0.9.1)

I'm surprised no one has mentioned yet what I regard as the key reason decorators are useful: they maintain the property that you can always find the definition of a function named "foo" by textually searching for "def foo". Lots of alternate syntaxes break that.

Don't know about decorators. The real issues with Python that I find working at Red Hat (unfortunately Python and Ruby are popular choices here):

- generally poor choices for implementation: inefficient storage of values

- 64 bit version approximately doubles the size of all data

- interpreted, no code generator so it's very slow

- lack of type safety is a constant source of bugs that just wouldn't appear in other languages

- whitespace layout screws up patches silently

Edit: don't downvote, comment on what I have to say.

- 64 bit issue is going to be the same for all languages. Twice the pointer size, twice the data size (unless you're using a very specific type)

- No code generators? You could complain that they're not good enough, don't support some specific extensions, don't play nice with threading, or something else, but there are many versions of python which work as compilers. Just google a bit.

- Lack of type safety is there by design. If you want type checking, why are you even looking at duck-typed scripting languages? It's a basic design decision, not something that could be changed / improved.

- Why is whitespace even an issue in patches? If anyone submits a patch, they should use exactly the same style as the rest of the file. If they don't, patch should be rejected. The same thing happens in projects where indentation doesn't really matter. Also, it's hard to "silently" screw up code covered by tests. Any big enough project should have those...

Thanks for replying.

The 64 bit issue is definitely not the "same for all languages". Programs get slightly bigger, but don't double in size as happens for Python programs. Most Linux 64 bit C programs will be using 32 bit 'int'. Python consumes 24 bytes to store any single int on 64 bit.

Code generators: I mean ones which anyone uses.

Lack of type safety: it may well be "by design", but it's a dumb design which causes lots of problems. Search Bugzilla for Python tracebacks and you'll see hundreds of bugs caused by lack of basic compile-time type checks. It's easy for a compiler to check that a function you're calling requires 3 arguments of type foo, bar and baz, and not doing that simply causes lots of errors.

The whitespace issue is also real: if/else changes its meaning completely if you're not hyper-careful about how you apply patches (especially when you have to make manual fixes to patches to get them to apply).

I've been doing python and ruby full time for the past 8 years. In that time I can count on one hand the number of bugs which got into production because of a lack of static typing. The gain in productivity from using dynamic languages in that time far far exceeds the cost of those few bugs.

Well Red Hat have suffered from thousands and thousands of such bugs. Don't believe me? Check for yourself:

http://www.google.co.uk/search?q=site%3Abugzilla.redhat.com+...

(There are many more searches of this type you can do).

RedHat clearly needs better developers.

There are a few conclusions that we can draw.

Red hat engineers suck (I know this isn't true)

I'm a genius (this isn't true either)

Different types of software have different needs and we shouldn't generalize about all software development based on our particular niche. Writing code that runs on other peoples systems is much harder than writing code that will run on servers I control. Python and ruby are great for the latter. I've never done the former at the scale of redhat. Writing code with a lot of collaborators is harder than writing it with a few collaborators.

What language do you prefer for your type of work?

OCaml

you don't get more bugs in python vs statically typed languages, you just get different types of bugs

also - there are code parsers like pylint that help you sanity check your source code in that manner.

speed - python is slow. We use python in scientific computing in extremely CPU intensive problems because python makes it very easy to control C and C++ layers that do the underlying work. that power in an interactive environment, complete with good visualization tools is great for science

I'm a huge proponent of dynamic typing, but I don't think your statement about bugs is true.

If I typo something in ruby, I will not know unless that code path gets executed through a test. This would never happen in a statically typed language.

However I think the productivity benefits of dynamic typing far outweighs this risk in my experience.

Leaving the other points which are valid in some situations and not in others...

> It's easy for a compiler to check that a function you're calling requires 3 arguments of type foo, bar and baz

No, it's not. It's actually impossible to determine, since you can reassign any function at runtime. Essentially you can do:

    self.some_func = one_implementation
    ...
    self.some_func = another_implementation

Those two implementations don't have to have the same types, can accept any number of parameters, can be wrapped in something else, etc. etc. Compiler cannot figure it out ahead of time.

On the other hand, it opens a lot of new possibilities that are not available in compile-time type checks. For example mocking objects in other modules:

    def test_some_other_module(self):
      tested_module.SpecificClass = Mock()
      tested_module.SomeFactory.getBlah(1,2,3)
      self.assertTrue(tested_module.SpecificClass.called_with([1,2,3]))
      reload tested_module

Whether it introduces other bugs is a different discussion... Sure, it's worse than Haskell where things tend to work the first time they compile, but on the other hand I get more annoyance from Java-style types than they're worth in practice. YMMV

To say that dynamic typing is dumb is rather misguided. I mean you may not like it, but it is very powerful, and a lot of languages are based on this principle (not just python and ruby).

The argument that there are hundreds of bugs because of it does not mean anything - you make the wrong assumption that everyhing else is equal. Of course, if dynamic typing was only about not checking types, it would be dumb. But it also gives you flexibility and avoids a lot whole lot of other kind of bugs, so you are trading one kind of bugs with another kind. For me, complaining about dynamic typing is like saying that exception cause a lot of bugs compared to error codes, because your application crashes if you don't catch exception. It is true in some sense, but the argument is rather stupid.

As for memory consumption: running an application in 64 bits, even if programmed in C, often requires much more memory, at least statistically speaking and in my experience. For example, running a simple ubuntu desktop in 64 bits takes easily 50 % more memory than in 32 bits (and it is not because of python :) ). Int, floating points are indeed the same, but structures, pointers and alignement requirements almost inherently increase memory in a significant way.

The problem with Red Hat is it's stuck on Python 2.4. Altho' the hassle of persuading the SAs to install ActiveState too was what led me to OCaml, so it's not all bad :-D

Red Hat are deploying Python 3 right now ...

... which is another saga. Python 3 is "close" but incompatible with Python 2.x, and the way around this is to have two installed copies of everything. Sheesh, brilliant plan.

(In case it's not obvious, I really dislike Python).

I'd like to have seen /usr/sbin/python (this is the one the OS uses itself) and /usr/bin/python (this one's for users). So you could patch them independently without straying outside of "official" RPMs.

I LOVE this idea. Do any mainstream distros do this now?

Not that I'm aware of. In practice we leave /usr/bin/python untouched (on 2.4) and install ActivePython in /opt. That makes the estate slightly less manageable because now Red Hat isn't doing all the integration and maintenance for us (and RH costs a fortune, as in more than Windows) but what can you do? People want to use 2.6 features at the very minimum.

I really like python, and FWIW, I find the whole python 3 saga a disaster. The small improvements really did not justify breaking backward compatibility.

Your complains sound more about dynamic languages in general than Python.

I think you can go too far in keeping a language compact. If there are common patterns in programming then they should be captured in the language. It makes code easier to write and understand.

Reply on news.ycombinator.com