Python and the power of 'first class' everything | Hacker News Reader

Python and the power of 'first class' everything

itworld.com

33 pointsabennett24 comments

Of course, not everything is actually first-class in Python.

There's a whole bunch of statements — simple (assert, pass, del, break, continue, import, raise, …) and compound (if, while, for, with, try) — which definitely don't qualify

The list is shorter than that, though.

How we'd override "pass" boggles my mind. Same for break and continue, and mostly raise (though you have control over the exception hierarchy).

del can be modified to some extent by changing the "del" double-underscore method of the target. import can be modified in several ways, which is how the zipimport is implemented (not an atomic capability). if, while, and try are effectively atomic, but "for" loops can be redefined by defining an iterator, and with is specified entirely in terms of an interface that allows you to affect what it does too.

Personally, I don't think the right way to phrase it is as "everything is first-class in Python", though. I see it as "when the language designer wants to put a capability into the language, like, say, importing from zip files, instead of hard coding it into the interpreter, an interface is coded into the interpreter that anybody can use", which unfortunately I don't have a snappy word on hand for. Looking over the history of Python you can even see this evolving, with the increasing openness of "import" and metaclasses being the most obvious examples.

"Everything is first class" would lead you to something like Lisp macros, which Python eschews. (For better and for worse.) However, I for one would wish more languages would learn the Python lesson above; it's so frustrating to me when I see a language implement something like "import from a different source" but still give you no way to use this new capability yourself. If it's convenient and powerful to you as a language designer, it'll be even more convenient and powerful in the hands of all your users.

Io has no keywords; everything is a message. This includes return, break and continue.

Yes, but late-binding everything is not free, or even always a clear win - it's a major language design trade-off, with implications for both performance and compile-time analysis. What happens to existing code if someone changes the meaning of 'return', for example?

You catch changes in the meaning of return that break things because you write tests and monitor your processes (business and OS) so that you know when something goes wrong.

I've never changed the meaning of return, but I easily added a new construct called returnIfError. Instead of having to type the following over and over:

  possibleError := attemptSomething
  if(possibleError, return(possibleError))

I can now type:

  trySomething returnIfError

The power granted by quick iteration of ideas is almost always worth the performance and compile time analysis trade off for the things that I work on.

> How we'd override "pass" boggles my mind.

An expression that does nothing. Putting `''` in a function is equivalent to `pass` in that it lets the parser do its stuff. `pass` could be a builtin defined as `None` and the behavior would be the same as today, without a keyword.

> Same for break and continue

I'm not saying I know how to replace everything (though I'd probably just remove break and continue, they're nice from time to time but not exactly essential), just that there's a lot of stuff which isn't first-class in Python.

> if, while, and try are effectively atomic, but "for" loops can be redefined by defining an iterator, and with is specified entirely in terms of an interface that allows you to affect what it does too.

I don't get what you're trying to say here.

For loops aren't as magical as "break" or "continue". They are just a particular client for iterators. C has hard-coded for loops. Python's for loops do almost anything you want or need them to do, and indeed, coming at them from a C point of view is really missing the point. Anything conforming to the iteration protocol can be used in a for loop [1], and indeed, if you're feeling evil, code in the for loop can even manipulate the iterator as it goes. Not recommended, but it would work.

"with" is a special statement, yes, but the specialness is all stuff you can access from a first-class point of view, up to and including dynamically creating a class that implements the "with" behavior you want, then using that with "with". Yes, you can't create your own arbitrary special-purpose statements the way you can in Lisp, but neither is it true that "with" is cast in stone such that you can't do anything with it. Write anything that conforms to the "with" protocol [2] and it'll work.

Also "first class" in the sense that you can program them are . (the attribute retrieval operator) and [] (usually the dict-lookup operator, but it can be rewritten to do anything you like for a given object).

Mind you, I'm not saying that Python is as flexible as Lisp, it clearly isn't. But it does not lead to understanding to say that "with" has no "first-class" aspects to it. It occupies a middle-ground between C and Lisp. Python has chosen this approach with due deliberation. You might still not like it after you understand it, but let us damn it for what it is, not for what it isn't.

[1]: http://docs.python.org/library/stdtypes.html#typeiter

[2]: http://docs.python.org/reference/datamodel.html#with-stateme...

You're very cute, but I know what Python is and how it works thank you very much, I've been coding with and in it for years. Doesn't change anything: Python's statements are not first-class objects.

And lisp's got nothing to do with that either.

> You might still not like it after you understand it, but let us damn it for what it is, not for what it isn't.

You… completely missed the point?

>> How we'd override "pass" boggles my mind.

> An expression that does nothing. Putting `''` in a function is equivalent to `pass` in that it lets the parser do its stuff. `pass` could be a builtin defined as `None` and the behavior would be the same as today, without a keyword.

The net behaviour would be the same but actually referencing None in the function body increases the refcount of None and immediately decreases it again.

  >>> def foo(): pass # no function body, after all
  ... 
  >>> def bar(): None # reference to None
  ... 
  >>> def frob(): "" # docstring
  ... 
  >>> import dis
  >>> dis.dis(foo)
    1           0 LOAD_CONST               0 (None)
                3 RETURN_VALUE        
  >>> dis.dis(bar) # incref, decref
    1           0 LOAD_CONST               0 (None)
                3 POP_TOP             
                4 LOAD_CONST               0 (None)
                7 RETURN_VALUE        
  >>> dis.dis(frob) # see "1 (None)" instead of "0 (None)"
    1           0 LOAD_CONST               1 (None)
                3 RETURN_VALUE
  >>> foo.__doc__, frob.__doc__
  (None, '')

I admit the difference is merely an academical one but saying pass is not special is not true.

>Same for break and continue, and mostly raise

These are all escape continuations.

>which unfortunately I don't have a snappy word on hand for

Abstractions?

Missing: first-class syntax.

and first-class anonymous functions

unrestricted first-class anonymous functions.

Python already has first-class anonymous functions (`lambda`), but highly restricted ones.

In that case, C has highly restricted first class functions. "restricted first class" is a contradiction, IMO. First class with restrictions is not first class.

The restriction in Python is not on the first-classeness but on the anonymity.

What do you mean?

Lisp macros.

Whether Python would actually be improved by macros is debatable, though.

People should look at Smalltalk if they want "first class" everything.

first-class almost everything. I don't think "^" is a first-class element in Smalltalk (though I might be wrong)

Mh, I have to say, this is far less impressive if you think in interfaces everywhere (or haskells type classes everywhere)... This is just something like:

interface TypeConstructor<T,V> { T buildType(V value); }

List<TypeConstructor<T,V>>;

or something like this.

Is this a joke? That's nowhere near as simple.

Haskell is making explicit (and inferring - you don't need to type it!) what Python implicitly assumes will probably work, but checking whether it's actually correct.

The terminology is math-y to the point of exclusiveness because of Haskell's close association with academia* , but there's a point after which it's simpler to make the compiler check than to rewrite the same kinds of tests over and over and over. Type systems are for expressing your requirements in a way that the compiler can do the tests once, at compile time. Computers are good at automating tedious and repetitive work, y'know?

It's too bad most of the people discussing how these things work seem hell-bent on using Greek letters and other insular terminology for everything. Static typing is kind of a wash for smaller projects (I'd just use Lua or Python for something that will only be a few pages of code), but once you start worrying about maintenance or testing changing interfaces between modules, the immediate feedback from a good static type system really comes in handy.

OCaml is a pretty great language, but has some major usability warts and not that much written about it in English. (And Haskell's "avoid success at all costs" motto is...telling.)

Yeah, I understand that, but the examples were supposed to be impressive in their simplicity. You can mix types in an array even in C, but the code you'd need to write to do so makes it unattractive.

If the comment was "But you don't get any type checking" instead of "My language can do this too", then it'd make perfect sense.

My point was: Yes, you can cram differently typed values into the same array, but this won't make sense unless all elements in the array share a common interface to access them. If this is not the case, all the dynamicness is useless, because you cannot do anything with the elements inside the array.

And once I arrived at this important realization, all the discussion of 'yay, I am SO dynamic' kind of fell over, because a lot of things reverted to the same principle of interfaces and programming to interfaces.

Reply on news.ycombinator.com