Hacker News Reader

Comment by jacobsimon | Hacker News Reader

Python has some unusual performance behaviors. IIRC you can also speed up the performance of your program a lot by assigning intermediate variables instead of referencing properties, for example:

[A.b[i] for i in range(100)]

is a lot slower than:

B = A.b

[B[i] for i in range (100)]

View on HN

Unusual is a strange qualifier here. Removing a dereference leading to a speedup is probably one of the few almost universal optimizations.

People do it in C# a lot with Count and Length on Lists and Arrays and it ruins array bounds check elision.

It is something that compilers do for you, in this context, in most programming environments.

Unfortunately, Python being hopelessly dynamic, `A.b` could involve executing arbitrary code and side effects depending on what `A` is. So hoisting `A_b = A.b` out of the loop may produce different results and thus can't be done automatically without some kind-of incredibly (impossibly?) smart static analysis :(

(though perhaps a JIT could perform this optimization for the usual, non-surprising path)

It isn't the removed dereference that speeds it up though, it is placing the pointer into the local scope, which is searched before the global scope.

Properties are not the same thing as attributes though. If `A.b` is a property (has a __get__ method associated with it), then this could easily break your code because `A.b` is supposed to change values. If it's an attribute that stays the same, then yeah I'll grant you that it could speed things up if it's being accessed frequently in a loop. There's not really any way for Python to tell if it can be optimized statically.

https://ideone.com/4R0suA

I'm not sure about other implementations, but "optimizing" CPython ends up being optimizing against counter-intuitive interpreter internals rather than time-complexity of the code.

For example, in CPython 3.6,

  n = 0
  d = 100
  for i in range(10**6):
      n += i
      if n >= d:
          n %= d

is slower than

  n = (n + i) % d

This counter to lower level languages, where dividing by a variable is costly, and the CPU can predict the pipeline to be false most of the time in the conditional and thus skip it.

> can predict the pipeline to be false most of the time in the conditional and thus skip it.

I think there's a bug in the code then. It will be skipped a few times, but for i between 100 and 10^6, the condition is guaranteed true every time.

Reply on news.ycombinator.com