Python List Comprehensions Explained Visually
treyhunner.com
treyhunner.com
A website like VisuAlgo (http://visualgo.net/) is a great example of exampling something visually should mean
Sorry for the misleading title!
E.g.
cartesian_product = [(a,b) for a in [1,2,3] for b in ['a','b','c']]
You think what now?
I've actually implemented typing exercises that are straight "copy this image" as one of the assignments I give students each week. Now, students have my lecture, the demo done in class, AND a typing exercise before trying to implement something like, say, a collision detection algorithm.
I get the underlying idea, that you "feel" and understand the code more if you type it, but holy hell would that be annoying !
- If found a boilerplate HTML layout and wanted to insert some stuff into it, I would use copy-paste.
- If I were editing a function to enqueue data, and I wanted to send it to two queues, I would re-type the 2nd enqueue command to avoid blindly copying over an inappropriate parameter, which is the kind of thing a tired mind might gloss over.
It's primarily a method to ensure the reader 'learns' by mimicking versus blindly copying. Annoying to a seasoned programmer, but its a great way for students to get extra practice.
> Do Not Copy-Paste
> You must type each of these exercises in, manually. If you copy and paste, you might as well not even do them. The point of these exercises is to train your hands, your brain, and your mind in how to read, write, and see code. If you copy-paste, you are cheating yourself out of the effectiveness of the lessons.
(from the intro of "Learn Python The Hard Way" ( http://learnpythonthehardway.org/book/intro.html#do-not-copy... ))
Overall, I think the right answer is to use the appropriate format and let people sort themselves out.
For example:
inserts = [a + c + b for a, b in splits for c in alphabet]
inserts = []
for a, b in splits:
for c in alphabet:
inserts.append(a + c + b)
inserts = splits.reduce([]) do |x, (a, b)|
alphabet.each{|c| x << a + c + b }
x
end
inserts = [ a + c + b
for a, b in splits
for c in alphabet ]
That's way too confusing and hard to parse. List comprehensions are great for simple operations, but once you start nesting, you're just being a jerk to the next dev that has to read your code.
Code should be clear first, elegant second.
I disagree: code should be correct before it is clear. And because it's so easy to mess up a for loop (for me at least), I choose list comprehensions where reasonable.
> you're just being a jerk to the next dev that has to read your code
That's not very charitable to either party. You're assuming that the motive of the author is to be a jerk, and you're assuming that the reader won't understand. If I had a dev on my team who couldn't read a list comprehension, I'd (a) wonder how they were hired, and then (b) teach them.
for row in [[i*j for i in range(1, 8)] for j in some_list if j % 2 == 0]:
some_op(row)
for j in some_list:
if j % 2 == 0:
row = [ i * j for i in range(1,8) ]
some_op(row)
Of course, I'm a rubyist, so I'd prefer:
some_list.select { |j| j % 2 == 0 }
.map { |j| (1..8).map { |i| i * j } }
.map { |row| some_op(row) }
Haskell list comprehensions are a bit easier to parse because they have symbolic delimiters, the fact that Haskell is naturally more terse, and because you can always check the type of the list
[ [i*j | i <-[1..8]] | j <- [1..4], j % 2 == 0 ]1. Longer than ~80-120 characters
2. Can't be explained in 1 comment line
Should be written as a nested for loop IMO.
Of I think 95% of the list comprehensions I wrote have been refactored to for loop by myself while debugging or extension the code later. So now I just never use them anymore, except for throwaway code.
List comprehensions are much easier to read just because they're so much shorter : there's less to remember while reading it.
This is an issue one always hits when you try to implement stuff that's more complex. List comprehensions help because they let you talk directly about more complex sets. They raise the level of abstraction. It's hard to get comfortable with but it really helps.
the difference in our points of view might be that I work on a code base with some ten million lines of code. Anything in there that is not purely flat stupid python, anything clever in fact, will be an annoyance some day. For example, someone started to use a "clever" `template.render(locals())` 5 years ago, and now we have no way to tell where the variables come from.
Inside a list comprehension, how do you print? branch? comment? emit logs? (I know you "can" do that, but then it becomes a mess not worth it). Moreover, if in you 5 line long-list comprehension, if there is an error, will the traceback tell you where it is?
My perspective is that in large codebases I have far more trouble remembering function names and what they do than I ever have trouble decoding statements. So the increased code size, long functions and ... that comes from those long list construction statements and loops always seems to take more out of me than difficult list or dictionary comprehensions. Also there are very much fewer places for bugs to hide.
There's also the additional argument that list comprehensions are more efficient because they don't actually construct the list. They generate iterators. It is not a huge difference like in Haskell but it definitely helps.
And there are special cases where you have to use list comprehensions : infinite lists can work as a comprehension, yet the equivalent code to generate them is pages long.
This is Python after all.
Is it not normal to do that? I always lay out the nested ones, and even single comprehensions if the conditional is complex.
splits.flat_map do |(a, b)|
alphabet.map{|c| a + b + c}
end
doubled_odds = []
for n in numbers:
if n % 2 == 1:
doubled_odds.append(n * 2)
doubled_odds = [n * 2 for n in numbers if n % 2 == 1]
for n in numbers: if n % 2 == 1: doubled_odds.append(n * 2)
The one line list comprehension is as much readable for people familiar with Python as the variant with the explicit loop.
It is not uncommon to use several list comprehensions in a function. If you rewrite them as explicit loops then it may increase the line count 4 times e.g., it can convert a simple 3-lines function that could be understood at a glance into 12 lines function that you have to read line by line to understand -- it is easy to keep track of 3 values on 3 lines but it is not clear whether it is true for 12 lines.
odd_numbers = range(1, limit, 2)
doubled_odd_numbers = [n * 2 for n in odd_numbers] L = []
for x in X:
for y in Y:
L.append(x*y)
L = [x*y for x in X for y in Y]
The specification of the inner loop should always be most proximate to the body, not last independent of the location of the body.
"All names longer than 4 characters in the list of cities?"
all(len(name) > 4 for name in cities)
"Sum of x-squared for x from 10 to 20."
sum(x**2 for x in range(10, 20))The parts in the comprehension
[*wanted* *iteration(s?)* *condition?*]
*iterations(s?)*
*condition?*
*append wanted*
One of my favourite comprehensions is of the sort
[item for item in items in for i in (0, 1)]
list(items) * 2
[item for item in items in for i in (0, 1) if items[0] != 'a']
If you have something written as a simple for loop, there's a real danger that you're moving in the wrong direction by translating it into a list comprehension.
Why are you even doing this? This isn't an acceptable way to write code.
> Plus it's so verbose and horizontal, half the time if you want descriptive names you're gonna write something so verbose it doesn't even make it that much more readable.
If your list comprehension is more than 80 characters wide, it's not because list comprehensions are bad, it's because you did something too complicated. Pull it into a function and name it so people know what the hell you're doing.
I disagree with this thesis, although the post is otherwise a very nice guide to list comprehensions. Special syntax for specific types indicates a failure of the language to be generic enough.
List comprehensions anoint lists as a special thing. You don't get to play with list comprehensions, unless you're using the type that the language has decided to let you play with. If you decide to use a different type for some reason, you... can't.
map and filter can be part of a typeclass/interface/protocol (in Python these would just be informal), so you can use them on arbitrary types. If you want to switch your list type, it should just work.
I write a lot of Swift, and I'm constantly frustrated that optional chaining (?.) and exceptions are special things. I can't implement ?. for my type (say, a Result). Only the language creators can use it. Somewhat confusingly, ?? is a thing I can implement.
In Python's case, I think that list comprehensions are necessary because the language's support for first-class closures is poor.
You can use generator expressions and pass it to your custom type's constructor.
That's one reason why I like Scala's comprehensions; they have the conciseness of list comprehensions, but are more generic.
https://www.python.org/dev/peps/pep-0289/
You can pass them into anything that expects an iterable. That's pretty much the same since iterable types will frequently consume an iterable in their constructor.
For example, here's a "tuple comprehension" (really just a generator expression passed to a tuple):
>>> tuple(x for x in [1, 2, 3])
Same thing passed to list and set constructors (which you'd never do because we have the special comprehension syntax for those:
>>> list(x for x in [1, 2, 3])
>>> set(x for x in [1, 2, 3])
I explained these in a very slightly different way in the linked webinar I linked in the post: https://youtu.be/u-mhKtC1Xh4?t=35m05s
>>> x = (a*a for a in range(3))
>>> type(x)
<type 'generator'>I do this all the time (for set) because it works even in python 2.6. Also dict((key, value) for foo in bar). In fact, I think it was a bad idea to add special comprehension syntax for sets and dicts.
I work on a project that's written mostly in Python; on occasion, other coworkers have to edit/work with my code and they're not used to Python at all, so I find myself writing for loops instead of list comprehensions for clarity.
When writing actual code, I tend to stick with regular for loops.
I think animating the transformation from a normal loop to a list comprehension is a great way to show how the syntax translates between the two forms. Very awesome and comprehensive post.
However, from a practical point of view I think it is great introduction for procedural programmers to begin using list comprehensions.
arrays/lists (like maps/hashes, though with a more narrow restriction on the indices) are (or at least are isomorphic to) sets of pairs of (index, element).
They carry information about order which does not exist in multisets/bags.