Hacker News Reader

Comment by StavrosK | Hacker News Reader

Sure, there are many ways to do it, but I think the author was going for speed here.

I'd be surprised if my way is slower. Any time you unpack into a function such as zip() python has to create an intermediary list to store all the results before calling the function.

Exactly, I would be very surprised if the zip version was faster. One of the first steps to optimization in python is moving everything you can to generators and using of itertools.

The OP's question of is this genius or bad is clear in that regard: it is bad, due to not being the proper optimization direction, but it is interesting.

I would be very surprised if it weren't. The zip version has the hot path written in C, the other version has it written in Python with a bunch of exception handling/list assignment code, etc. The original method is just zip(iterator, iterator, iterator), which is probably as fast as anything can be.

That is a common misconception. Moving to iterators adds a function call while list creation in C is quite fast. Every case has to be tested for performance.

I always thought the reason iterators are used in preference to lists was due to the memory advantages, not the performance.

There's also flexibility improvements to do with lazy evaluation, and it makes the Twisted guys not hate you.

The memory advantages are key to performance when you have a non-trivially sized dataset, in my experience.

In my experience, building a non-trivially transformed sequence with the builtin C-backed list and list comprehensions vs iterators and generators, the iterators and generators win.

Maybe I'm doing crazy stuff, though!

You could always use izip:

http://docs.python.org/2/library/itertools.html#itertools.iz...

I was a bit wrong about that. For some reason I imagined that there would be quite a few args being passed to zip, when in fact there are just the three iterators. It does create a temporary list, but it's so small it's negligible. Using izip wouldn't really change anything.

Did you test and time the itertools version?

I did not, but I might now.

EDIT: See update above.

Reply on news.ycombinator.com