Hacker News Reader

Comment by zmmmmm | Hacker News Reader

In case there are Pythonistas here who will indulge a question: one thing I've never figured out how to do elegantly ("idiomatically", I suppose) is create a list of predetermined size initialized with a constant(usually zeros). I always end up with something kludgey like:

    x = [0 for i in xrange(0..100)]

which seems so roundabout it can't possibly be the right solution.

View on HN

    x = [0] * 100

Thanks! At first I wondered about efficiency but it seems like I can allocate lists with tens of millions of elements this way without any spike in CPU so it is obviously implemented efficiently. I knew there would be a better way :-)

This is much better than my own comment - I never even knew about that syntax!

Is it efficient though?

My guess is that it's optimized, but I don't know. Semantics-wise, it's consistent with list addition.

    >>> [0] + [0] + [0]
    [0, 0, 0]
    >>> [0] * 3
    [0, 0, 0]

I just read, that syntax is sugar for the list.extend() method so it's still making copies of the list - I wouldn't use it for a list containing complex objects; but that is very idiomatic for simple use cases (like this one?).

It isn't syntax - help(list.__mul__)

Just make sure that when you do this, you're not initializing it with mutable objects, e.g. lists.

    x = [[0]] * 2
    print x[0], x[1] # prints [0] [0]
    x[0].append(1)
    print x[0], x[1] # prints [0, 1] [0, 1]

That's because the objects inside the repeated list are simply references to the same object internally.

Yeah, that is kludgy (you're creating two lists now!).

First thing I would do is ask why I need that; I've never come across a case where I needed a pre-built list of items all initialized to a common or empty value - if you're doing that, I would explore some sort of custom generative process (building the items of your list as you need them). But, just because I can't think of a good use case for this (where I couldn't use generative recursion or iteration) doesn't mean you haven't. Weigh what you are doing.

I would do this one of two ways: create a function that will produce an object that will generatively build the list (like xrange, but pass an "initializer" value, like myxrange(0, 100, initialize=0)). See the source for Python's xrange to do this it should be quite easy.

Or do this in a while loop (flat, easy to understand, and you aren't being too naughty by producing more than once list):

    ls  = []
    cnt = 0
    
    while cnt <= 100:
      ls.append(0)
      cnt += 1

Alternatively you can do the above with a for loop but you would have to iterate over a sequence (which means creating a list to do it); the while loop is going to be the most efficacious way of doing what you want I think, excluding building a custom xrange style implementation.

Actually [x for x in xrange(..)] will produce only one list. In Python 2.X, range() returns a list while xrange() an iterable.

If you just need an iterable, itertools.repeat(0, 100) will do the trick.

I suppose I didn't say it correctly, you're still maintaining two objects though (xrange the iterable, and the new list being built by the comprehension). The "optimization" I suppose is a moot point because Python's handling of lists is pretty solid and this isn't the 90's anymore, but at least with the while loop you've only got one list object and an integer (which is faster, I would think, to increment than it is to produce the next sequence in the iterable object?).

Anywho, splitting hairs. Goladus had the most helpful comment (I even learned something).

This is way premature optimization, and it's based on a lot of assumptions. Did you know that, on CPython, your integer increment creates a new object each time (outside of the range -5 to 255 or so, IIRC)? Now, xrange might do its own increment, so let's call it even on object creation.

If either xrange or list comprehensions are implemented in C instead of Python, do you still think your version will run quicker? What do you think the likelihood of either or both of these being the case is on your Python implementation? How many name lookups and function calls do you think each version does? Do you think you should find out before writing longer and more complicated code to attempt to out-perform it?

On my machine, your implementation performed ~3 times slower than the naive idiomatic "[0 for _ in xrange(100)]" and closer to 4 times slower when I bumped the list size up to 20000. And your version was ~32 times slower than "[0] * 100" and around 60 times slower when I bumped the list size up to 20000.

So please, don't optimize without measuring and instead just write idiomatic code the first time.

The code, for reference:

    def mk_list_1(size):
        ls  = []
        cnt = 0

        while cnt <= size:
            ls.append(0)
            cnt += 1

        return ls


    def mk_list_2(size):
        return [0 for i in xrange(size)]

    def mk_list_3(size):
        return [0] * size

    from timeit import timeit
    args = {
        "number":1000000,
        "setup":"from __main__ import mk_list_1, mk_list_2, mk_list_3"
    }
    print "Executing %i runs:" % args["number"]
    print "mk_list_1 took %i s" % timeit('mk_list_1(100)', **args)
    print "mk_list_2 took %i s" % timeit('mk_list_2(100)', **args)
    print "mk_list_3 took %i s" % timeit('mk_list_3(100)', **args)

Output on my machine:

  Executing 1000000 runs:
  mk_list_1 took 32 s
  mk_list_2 took 10 s
  mk_list_3 took 1 s

Fair enough, thanks for the comment.

Reply on news.ycombinator.com