Python and the Principle of Least Astonishment | Hacker News Reader

Python and the Principle of Least Astonishment

lucumr.pocoo.org

207 pointsmattyb50 comments

Differentiating properties and methods from internal methods is as simple as using another char, like dots and colons. So box.length would not be the same as box:length, that way you could assign all names you want without fear of collision. Poor choices in language design will always haunt you til the end of times.

That being said, python is my favorite language right now and the only complain I have is about underscores which I try to avoid.

Now, GO being a new language, I can't really understand why it does not implement methods for primitive types like "hello".ToUpper(), instead we have to import "strings" and call strings.ToUpper("hello").

Easier for the compiler but harder for the programmer has never been my mantra.

I'm not sure about different namespaces for special methods. Python's object system is so self-referential and so transparent that it'd be surprising.

Go devs are of the opinion that what behaves like a function should be a function, with no exception. It's also the reason they don't do operator overloading, which I think is just dumb.

Would you mean this to be a namespace which only the Python implementation could access? For example, if the protocol "it:next" is added (which is equivalent to next(it)) then is there any way to support that syntax in an older version? With next() as a builtin it's easy; try: next except NameError, and in case of exception, implement the function yourself. But if I can't implement it myself then there's no good migration path.

With your particular choice of ":", then what would "d={a:length()}" be? Currently it's a 1-element set. What would "words[middle:length()]" do?

"is there any way to support that syntax in an older version"

Nop, that is exactly what I meant with decisions haunting you forever. You better start a new language than radically change the current implementation.

With regard to my particular choice of ':' there are three solutions:

1. pick any of the 100k unicode chars for internal methods but imperative is to differentiate them to freely use attributes and methods as pleased.

2. force the compiler and the coder to disambiguate the possible same use as internal method: first assign then use n=a:length(), d={n}

3. design the language to use .. as slicer and {a=b} as dictionary assignment.

Again, we are talking about design decisions for a new language, not to change the core of an existing language which is not an easy task. That is exactly why language design is not an exact science an very few make it to stardom.

My point is that it isn't "as simple as using another char." #1 doesn't work if you want things visible on most keyboards in the world. #2 is counter to a design principle (which Python uses) that humans are less good at resolving ambiguity than people (see "template <template <int>>" in C++ pre 2011), and #3 as a specific case means problems getting methods of floats, as in "1..hex()". Of course Python has that already with "1 .bit_length()" being different than "1.bit_length()".

I agree that seemingly minor decisions can have a big impact in the future of a language. What I disagree with is that this choice of a special syntax for built-in methods is "simple", and I believe that doing it yields a language not meant for stardom.

how is ':' less onerous than underscores i.e. '.__'?

Sounds like you just have irrational hatred of underscore (or possibly love of colon?)

btw having two or more access operators '.', ':' and distinguishing between "internal/language" and "user" methods is a poor language design choice over having a standard but unenforced __ means "special" and "internal/language" and "user" all being coequal in semantics, syntax, rights and responsibilities.

"how is ':' less onerous than underscores i.e. '.__'?"

There is no difference at all, just one char vs five if we count ending double-underscores too, and more visual noise. They both accomplish the same.

Being both the same, what difference do you think there is between, in one hand .init() and .__init__(), and on the other hand .init() and :init()

Just syntactic sugar, I rather pick my colon ;-)

So... Python has len() because it would be too hard to standardize on .len(). Any by the way, len() calls .__len__(), on which we have standardized.

Got it.

The better explanation is that there are a series of fallback actions for all of the top level functions that delegate to the .__whatever__ methods.

For example, if you write a class that quacks like a collection that has a length (it implements .__len__) Python can use that to get a value for bool() even if you didn't implement .__nonzero__.

When people inquire about the len() function it is mainly because it is (or at least looks like) a global function rather than an object method. In short, people wonder why you have to call len(x) instead of x.len(). This is a valid concern seeing that Python's standard library is, by and large, object oriented (c.f. string methods like "upper" or list methods like "append" or file methods like "read").

Neither the article nor your comment has a good explanation of why that is so. The article only harps on about why it's good that it's not called length() or getLength() or size() or getSize(), implying that len() is somehow the god-given name for this feature. And the fallback mechanism you refer to could just as well be implemented if len() was a method of the root class (object) rather than a global method.

I like Python a lot but this is one that I never quite understood and I believe there isn't really a good reason for it, I think it was just a historical accident. Might as well admit it rather than trying to come up with half baked justifications after the fact.

Guido on the subject: http://mail.python.org/pipermail/python-3000/2006-November/0...

tldr - He thought it more intuitive.

Very interesting, thanks for the link! I don't feel like arguing with the father of the language and this story is already on its way out of the HN front page so I'll leave it at that. If anything it's comforting to know that such a core language feature didn't come about by accident.

It's a bit dated, but the Python Cookbook does a great job of teaching you practical examples of Python if you already know other languages and want to quickly grok what is "pythonic." It starts with string crunching and hits just about every other general sysadmin use case, covering most of the standard library along the way.

May I know why it is a bit dated? because I'm thinking to buy the book and the year is 2005, which is good enough for me since 2005 means Python 2.5 at least (or even more).

The 2nd edition of Python Cookbook doesn't cover Python 2.5. It covers language features that were there till Python 2.4. (http://oreilly.com/catalog/9780596007973) The 3rd edition of Python Cookbook is expected to appear by end of this year. http://dabeaz.blogspot.com/2010/12/oreilly-python-cookbook-p...

The goal in language design is not "least astonishment" but inferability. And, while Python may not meet the least astonishment goal for new users coming from another language, Python is remarkably inferable and internally consistent.

For extra fun....

  a = 5
  def print_a():
      print a
  # Prints 5
  print_a()
  
  def print_and_assign_a():
      print a
      a = 2
  # raises UnboundLocalError
  print_and_assign_a()
  
  class PrintOnInitSet(set):
      def __init__(self, *args, **kwargs):
          print "init!"
          set.__init__(self, *args, **kwargs)
  # Creates a PrintOnInitSet and prints "init!"
  a = PrintOnInitSet([1,2])
  # Creates a PrintOnInitSet and prints "init!"
  b = PrintOnInitSet([3,4])
  # Creates a PrintOnInitSet and prints nothing
  c = a | b

Well, your first example makes sense. It ensures you're always referring to the same-scoped 'a' throughout your function. FWIW if you said 'global a' at the start of 'print_and_assign_a', Python wouldn't have a problem:

    >>> a = 5
    >>> def print_and_assign_a():
    ...     global a
    ...     print(a)
    ...     a = 2
    ... 
    >>> print_and_assign_a()
    5
    >>> print_and_assign_a()
    2

Your second example, however, seems to show an implementation detail of Python leaking out, and is probably a bug. Union starts off by making a copy of 'a' and adds the elements of 'b' to it, rather than creating a new object and adding the elements of both.

On a hunch that PyPy's implementation would be less special-cased, I installed PyPy and and it actually does print "Init!" on "c = a | b". So, I vote bug in CPython rather than an inconsistency in the language.

One more question arises, however. Should "a = set([1,2])" be equivalent to "a = set(); a.add(1); a.add(2)"? I overloaded 'add' on your PrintOnInitSet and it's not:

    >>>> a = PrintOnInitSet(); a.add(1); a.add(2)
    init!
    Called add with item: 1
    Called add with item: 2
    >>>> a = PrintOnInitSet([1,2])
    init!

The 'print_and_assign_a' case comes up more often with a in an enclosing local scope, so you couldn't really get it to work prior to the introduction of 'nonlocal' in Python 3. The best you could do was something silly like

  a = [5]
  def print_and_assign_a():
      print(a[0])
      a[0]=2

It's cool that the second case works in other implementations. I hadn't thought to test that.

Yes, 'nonlocal' definitely closed a hole in the language.

> Should "a = set([1,2])" be equivalent to "a = set(); a.add(1); a.add(2)"?

Why should it be? Adding elements one-by-one is slow and shouldn't be forced on the language model level.

Overall it's a good article, but I don't fully understand his complaint about decorators, and I think he may not understand them. @foo and @foo() intentionally mean very different things. I don't see how you could "add a parameter to a previously parameter-less decorator" without drastically changing the meaning of the whole thing.

They may be tricky to wrap your head around, but there's nothing too surprising about decorators.

> I think he may not understand them.

I won't think so. He codes a lot, in general, and in Python. I have read some of his posts and code, and he has a deep understanding of Python and programming in general.

His complain was about @foo and @foo() requires different decorator implementations. If @foo by default meant @foo(), that makes introducing parameters at a later time a bit more straightforward.

I am not arguing about it being a valid expectation. I am just explaining what I think he meant.

If I have the following code:

    @auth_required
    def some_view():
        pass

And later I decide to change that decorator to be parameterized:

    @auth_required("basic")
    def some_view():
        pass

I have now created a massive amount of pain for myself if I decide to define auth_required() such that it defaults its argument to "basic", unless I force people to call it as @auth_required(), because of the differing callable signatures.

Some people feel @auth_required() is ugly. That's all.

> Some people feel @auth_required() is ugly. That's all.

The biggest issue here is backwards compatibility. Nowadays I just make a separate decorator that accepts arguments and keep the old one around unchanged. I learned my lesson :)

You may also use a simple wrapper to create decorators which would automatically define @decorator and @decorator().

I could also aim with a bazooka at my foot. Been there done that, lesson learned :-)

Apparently there's something pretty big here I'm not getting. If you add a parameter to auth_required, of course you're causing a huge amount of pain because you're totally changing what auth_required means. It's going from being directly applied to some_view to generating a function that is applied to some_view. If every decorator was automatically called, every decorator would have to return a function. Among other consequences, you wouldn't be able to use property as a decorator.

Even if it's inconvenient, python's behavior here is still not surprising. It's perfectly consistent, and more general than automatically adding parentheses.

This is a nice article, as a non-Python programmer who's dabbled a bit in order to read some random Python code, it gave me some appreciation for the "Pythonic"-way.

I have to say though, that the thing that astonished me the most about Python is the Java-like "closures". I sort of thought it would be like Ruby, which I thought was closer to Scheme and JavaScript, and then I realized that Ruby isn't quite like them either. (I know Ruby is close with its lambdas and blocks, but it's not intuitive to me.)

This isn't necessarily to the detriment of Python (or Ruby) but just something I observed when trying to explain Java's lack of closures to someone who only knows Python.

I have only a trivial amount of experience experience with Scheme, could you explain how its closures are different from Python's?

I haven't used Java a lot either, but from what I have done Python's function seems more similar to JavaScript than to Java.

> I have only a trivial amount of experience experience with Scheme, could you explain how its closures are different from Python's?

In pre Python 3, the closed over value isn't mutable unless it's a reference to a mutable object.

        def counter(num):
            def foo():
                num += 1
                return num
            return foo

    c = counter(5)
    c()

This won't work because you can't mutate the closed variable `start`.

This would work in languages with proper closures(Ruby, Perl, Scheme...).

Here is how you do it in Ruby:

    def counter(n)
        lambda { n += 1 }
    end
    c = counter(5)
    c[] # returns 6
    c.call() # Alternate syntax. returns 7
    c[] # returns 8

The above python will work in Python 3 if the closed variable is declared `nonlocal`.

    def counter(num):
       def foo():
           nonlocal num
           num += 1
           return num
       return foo

Or you can have workarounds in pre Python3.

    def counter(num):
       def foo():
           foo.num += 1
           return foo.num
       foo.num = num
       return foo

I dunno if I'd say that's an issue of Python not having proper closures, or just that before you could say 'nonlocal' there was no way to refer to the outer scope, since the only scopes you could refer to were 'local' and 'global'.

Python has had proper closures since version 2.2, which is quite old at this point. You can verify this using the following code:

    def func():
        x = []
        def func2():
            x.append(1)
            return x
        return func2
    closed = func(); closed(); assert closed() == [1,1]
    closed = func(); closed(); assert closed() == [1,1]

That's what I was referring to as "Java closures".

Python closes over names, not objects. This is consistent with the rest of Python, which treats names as references to objects rather than variables which hold objects. I do agree that it's confusing for people used to closures and used to languages with different semantics.

In Perl, for reference:

  use strict; use warnings;

  sub counter {
        my $n = $_[0];
        return sub { ++$n }
  }

  my $c = counter( 5 );
  say $c->();
  say $c->();

And its also handy that you can create a closure without needing to define a subroutine:

  my $c = do {
      my $n = 5;
      sub { ++$n }
  };

Python has Java like closures? No comprende.

Java's closures are a pain as they need inner classes to emulate closures - Python's are a lot easier, hands down.

Java needs the local variables to be final for it to close over them. Python doesn't but you can't directly mutate it. Won't you agree in that sense the statement holds some truth?

> then I realized that Ruby isn't quite like them either.

I am curious. How does Ruby's closure differ from Scheme's?

To be honest, I'm not really sure. I think it's a superficial difference. When I put this into a Ruby 1.9.2 REPL

    def foo (n)
        lambda {|i| n += i } end

I expected to be able to do this:

    acc = foo(0)
    acc(1)

But instead have to do

    acc.call(1)

Maybe there's another way...

Edit: oops you pointed out this example in another comment, sorry! Is there no way for the returned lambda to "feel" like a regular function?

you should read Armin's other posts about Python and webdev, if you haven't already:

http://lucumr.pocoo.org/tags/python/

Written by someone who actually knows Python (which is rare for articles like this).

The actual surprises section is great.

Personally, I love that joining is ' '.join(list), since PHP (e.g.) works the same way, in that the first argument to implode() is the string.

It being consistent with PHP is pretty much the same thing as being inconsistent.

"I was quite fond of Raymond Hettinger's talk about what makes Python unique because he showed a bunch of small examples that almost exclusively used the good parts of the standard library and hardly any user written logic."

Is this talk available anywhere?

I'm not certain but I think it was one of these

http://ep2011.europython.eu/conference/speakers/raymond-hett...

Maybe this http://ep2011.europython.eu/conference/talks/what-makes-pyth...

I miss something like the Principle of Least Astonishment for design.

Why every time there's a revamp of a website the new design is more astonishing and less clear? Why the search button in Google Calendar is now blue?

Why new design is usually a tease on our muscle memory?

Reply on news.ycombinator.com