Comprehensions in Python the Jedi way

I think chaining syntax from e.g. Elixir gives you more flexibility than map/filter/reduce defined on objects too. Big fan of that, though it does interact oddly with elixirs optional parentheses for function calls (which is a mistake of the language imo)

  [p for p in range(2,100) if all(p%q!=0 for q in range(2,p))]

  range(2,100).filter(p => range(2,p).map(q => p%q!=0).all())

I hate this design approach deep in my soul. It makes for very brittle code that creates lots of backward compatibility issues. If you're working on some legacy code that has some nonsense like

    foo.get_status().dispatch_handler().log_error().close()

In that example, say someone changes `foo.get_status()` to return some new kind of "status" object, and it alters the `dispatch_handler` and so on. Of course one can implement this in a way where the chain of downstream calls doesn't break, but the point isn't so much that, through huge engineering effort it is possible, but rather that it is extremely brittle and adds a layer of complexity that's not needed.

It's just so much better to write something like:

    dispatch_result = run_dispatcher(foo.get_status())
    log_error(dispatch_result)

`run_dispatcher` in my example would be a hell of a lot easier to unit test and throw some mocked example class into for debugging or refactoring than if it is `some_class.run_dispatcher` ... and then if `some_class` has child classes that specialize the behavior, you're just hosed.

The problem is composability. People think that the fluent interface makes things composable because from some arbitrary point in the middle of the chain of calls, they have easy attribute-like access to the next operation they want to do. This artificially feels easy and convenient.

But contrast this to a functional language like Haskell, where none of these things need to be member functions of an object, and hence the context of the object doesn't have to be created at any point in the fluent chain. Then you can write something even better:

    (close . logError . dispatchHandler . getStatus) foo

    let statusDispatchLog = (close . logError . dispatchHandler . getStatus)

To do the same thing in a fluent interface, we need a horrible lambda or a whole new function definition, exactly because the fluent interface is only sweeping the composability issues under the rug.

    statusDispatchLog = lambda x: x.get_status().dispatch_handler().log_error().close()

In Python this is particularly a shame because functions are first class objects. Of course, you can write helper functions / decorators that sort of do function composition (if you're willing to throw away useful argument signatures), or you can use flaky hacks like the common Infix pattern in Python, and then live with ugly "<< . >>" or "|.|" misleading syntax.

It always makes me sad that Python lacks an extremely short function composition infix operator that provides some information about the function signatures of the functions being composed.

Because not even a comprehension can help you when you need to do the fluent interface stuff in Python.

    [x.h().f().g() for x in some_iterator]

    map(g.f.h, someIterator)

    [(g.f.h) x | x <- someIterator]

    [g(f(h(x))) for x in some_iterator]

http://python-history.blogspot.com/2010/06/from-list-compreh...

The PEP is silent on the issue:

https://www.python.org/dev/peps/pep-0202/

I guess mail list spelunking would probably reveal some conversations about how to handle it.

The original concept was that it provided a more concise way to write loops so that:

   t = [expr(x, y) for x in s1 for y in s2]

   _t = []
   for x in s1:
       for y in s2
           _t.append(expr(x, y))
   t = _t
   del _t

The question of whether to expose the loop induction variable didn't get much discussion until I proposed Generator Expressions in PEP 279 and decided to give them the behavior of hiding the loop induction variables so that the behavior would match that of a normal unrolled generator function.

When set and dict comprehensions (displays) came along afterwards, they were given the latter behavior because there was precedent, because there was a mechanism to implement that precedent, and to provide a short-cut for then common practice of creating dicts and sets with generator comprehensions:

    s = {expr(x) for x in t}

    s = set(expr(x) for x in t)

The current state in Python 3 has the advantage of being consistent between all four variants and matches how mathematicians treat bound and free variables.

There are some disadvantages as well. List comprehensions can no longer be cleanly explained as being equivalent to the unrolled version. The disassembly is harder to explore be cause you need to drill into the internal code object. Tracing the execution with PDB is no fun because you go up and down the stack. It is more difficult to explain scoping -- formerly, all you had was locals/globals/builtins, but now we have locals/nonlocals/globals/builtins plus variables bound in list comps, genexps, set/dict displays plus exception instances that are only visible inside the except-block.

  def palindrome(s):
      return s == s[::-1]

Also, there are objects that can be reversed() that are not sequences.

http://stackoverflow.com/questions/931092/reverse-a-string-i...

Don't know what a better term might be, but it's good to be aware of the distinction.

Thanks, that's a sane way to explain the order. Up to right now, it was always "the opposite of what you'd expect", which was a memory rule that always failed me.

And for those few lines of code you've traded the ability for new programmers to understand it easily. To me that's a net loss.

I am rather fond of dictionary comprehensions as mentioned in the article:

    colors = [jedi['lightsaber_color'] for jedi in jedis]
    frequencies = {color: colors.count(color) for color in set(colors)}

    print(frequencies)
    # {'green': 6, 'red': 5, 'blue': 6}

    coordinates = (star.x, star.y, star.z for star in star_map)