Common Python Mistakes
toptal.com
toptal.com
Python supports optional function arguments and allows default values to be
specified for any optional argument.
it can lead to some confusion when specifying an expression as the default value
for an optional function argument.
the bar argument is initialized to its default (i.e., an empty list)
only the first time that foo() is called
class variables are internally handled as dictionaries
In "Common Mistake #5", he uses both a lambda and array index based looping, neither of which are particularly Pythonic. A better example of where this is a problem in otherwise Pythonic code would be good.
In "Common Mistake #6" he uses a lambda in a list comprehension -- for an article of mistakes mostly made by Python beginners, this is going to make it tough to follow the example.
In "Common Mistake #7", he describes "recursive imports" where he means "circular imports".
In "Common Mistake #8" he refers repeatedly to "stdlib" where he means the Python Standard Library. Someone is going to read that and try to "import stdlib".
[Toptal blog editor]
Again, thanks for the attention to detail. We've fixed this as well.
Because it states it right near the top of the article.
3 is a really easy mistake to make for anyone (which is why the syntax was changed).
6, 7, 9 and 10 are more obscure, and where I really appreciate this article -- and they definitely can be issues for more experienced Python devs.
My use case is when interviewing candidates I often ask them to rate themselves on a scale of 1-5 in the languages they know, and then ask them increasingly 'tricky' questions in each language to get a feel for how their "personal" scale aligns to their real knowledge. This works fine if we have an overlap of several languages, but in the case where I know nothing or very little of one of the languages they know I lose that data point.
I find it valuable to know what a "I am a 1 at X" vs "I am a 3 X" vs "I am a 5 at X" means to them, since I've found little correlation between how harshly someone rates themselves and their true ability. Sometimes self-rated 5s are really 5s by my book, sometimes self-rated 3s are really 5s by my book, and sometimes self-rated 5s are really 2s by my book. So I want to know how "my scale" translates to "their scale". If it was more formalized I'd go as far as to get a "confidence quotient" for a person as self-critical and self-confident people can be fantastic engineers or horrible engineers.
Does anyone else do this process when interviewing?
Everyone wins!
Everyone wins!
If such a resource indeed already exists, that is a bad reason not to link us to it.
If such a resource does not already exist, what you say is a bad reason to dissuade someone (who might otherwise be inclined to) from building it.
It's a pretty bad reason to argue against its existence, all around.
These would be interesting research areas for instrumenting IDEs / other eco-system tools to collect some of this data. (I'm sure there is already some work in some of these areas and would appreciate names or links to high-quality reviews.)
>>> l = ["a",
... "b",
... "c"
... "d"]
>>> l
['a', 'b', 'cd']>>> l = ["A", "c""d"] >>> l ['A', 'cd']
Imho, adding a comma after each list element is a good practice. You can easily swap them, add more, and never run into a an issue you describe:
foo = [
"a",
"bc",
"def", # comma here, too
]"Explicit is better than implicit"
It seems someone thought along similar lines and took action: http://legacy.python.org/dev/peps/pep-3126/ but the issue got rejected.
odd, because we have triple-quotes for that, don't we?
With triple quotes you get a string with newlines and indentation in it. While you can not indent the following lines it looks ugly, and you can't do anything about newlines.
Take a look at how F# handles the issue: http://stackoverflow.com/a/14599828
And CoffeeScript: http://coffeescript.org/#strings (apparently implemented relatively recently (https://github.com/jashkenas/coffeescript/issues/3229) and borrowed from LiveScript.
There are other language which support multiline strings (Here docs) with indents stripped via means of syntax, like YAML (with |), Racket (which doesn't do dedenting, but being language it is it's very easy to add) and many shells (with <<-). Python doesn't have this feature, and parse-time string literals concatenation serves this purpose.
Of course, you can do something like:
foo = """bar
indented at first
and after newline
"""
textwrap.dedent(foo)
class A():
def __init__(self):
self._x = 0
@property
def x(self):
return self._x
@x.setter
def x(self, new_value):
self._x = new_value
a = A()
print a._x # 0
print a.x # 0
a.x = 4
print a.x # 4
print a._x # 0 wait WTF?!
def x(self, new_value):
self._x = x
return x,
Also, as opposed to one of his examples, if you are using python 2.7, declare your exception blocks as:
except (FooException, BarException) as e:
No? I do that occasionally, e.g.:
x, y = 5, 6 (x, y) = (5, 6)
print x,
print y
x = ,
x = (,)
>>> x=(,)
File "<stdin>", line 1
x=(,)
^
SyntaxError: invalid syntax> Empty tuples are constructed by an empty pair of parentheses;
And it works fine
>>> x = ()
>>> x
()
Programming languages are meant to be read as well as written, and someone relatively new to Python (and many who have used the language for a long time) is certain to get confused about the difference between:
return [lambda x, i=i : i * x for i in range(5)]
return [lambda x : i * x for i in range(5)]Being more explicit and less hacky can well be combined with staying true to functional style:
from functools import partial
mul = lambda x, y: x*y # could use int.__mul__, too
multipliers = [partial(mul, n) for n in range(5)]
It would have a big ugly 'return' in it and be a few characters longer, but it would work the same, so I don't see what lambda brings to it.
I have an issue with that statement. No languages are inherently "compiled" or "interpreted", that's a property of the implementation.
If we are talking about CPython here, Python code is compiled to bytecode which is then interpreted. Not unlike Java - with the difference that the main implementation has a JIT and afaik, Python's does not.
But that's CPython. What about PyPy? It has a JIT.
Anyway, all of those terms do communicate something, even if tomorrow they may wrongly describe the language.
A language and it's implementation are usually designed at the same time. Compiled or interpreted will affect design choices that go into the language. While additional implementations may follow, it can be hard/impossible to design a compiler (machine code, not byte code) for a language that was designed to be interpreted without dropping features (ie eval).
It may be more correct to say 'Python was designed to be interpreted' than 'Python is interpreted'
I think this just illustrates the fuzziness of these definitions. A compiler is just a piece of code; you can embed it into another piece of code and run it whenever necessary. Maybe in the world of shrink-wrapped desktop software there was a sharp distinction between AOT compiled languages and interpreted ones, but we haven't lived in that world for a couple decades now.
Of course, this doesn't address issues of incremental evaluation which often requires additional semantics for compiled languages.
> You can run C++ or Java with run-time type-checking, but it doesn't conform the to spec.
I'm not sure what you mean by "conforming with the spec". You can opt out of type checking by using only `object` or `void*`. But golang has a mode where you run a go file directly and a mode where you generate a binary. There are no traditional interpreted languages anymore (or at least only very few). And what we call "compiled" languages today are not actually compiled ones. The way Java runs is closer to how JavaScript runs than to how C runs. It's only about the interface the offer to developers.
That said, for more precise discussions, what you pointed out is valid and important. One of the early questions I ask in a programming interview is to explain some high-level differences between two languages they're familiar with, which is often Java and Python. One of the common responses I get is that Java compiles to bytecode which is executed by a VM, while Python is interpreted. Of course, I point out that CPython is also compiled to bytecode and executed by a VM.
class Bag(object):
def __init__(self, items=[]):
self.items = items
def add_item(self, item):
# check the item is valid
self.items.append(item)
bag1 = Bag()
bag1.add('an item')
bag2 = Bag()
print(bag2.items) initial = ['first item']
bag3 = Bag(initial)
bag3.add_item('second item')
print(initial)
This actually happens when the function is defined, not when it's called the first time.
Mildly excessive? /s
def create_multipliers():
def multiplier(i):
return lambda x: i*x
return [multiplier(i) for i in range(5)]
for multiplier in create_multipliers():
print multiplier(2)
Why do you think so? I think that both cases seem consistent, or rather, correct (and therefore this example should not be treated as a common Python mistake), because x is not assigned a value anywhere in class C, and C inherits from A, so it should be clear to anyone knowing OOP and inheritance, that C's x is the same as A's x. (And the same holds true for inherited methods.) Even the OP says that in the post:
>In other words, C doesn’t have its own x property, independent of A.
So this variable is neither completely shared across classes and their subclasses (per Smalltalk class variables), nor completely independent across classes and their subclasses (per Smalltalk class instance variable), but instead its [in]dependence alters based upon whether (and where) you assign values to it.
While I can understand that in terms of the dictionary mechanism used to implement it, from my point of view it's just weird behaviour.
I suppose I just prefer the idea that the meaning of assigning a value to a variable should be "assign this value to the variable", rather than "alter the inheritance behaviour of my class such that mutable state is stored in it where it wasn't stored before, and then assign this value to the variable."
Not knowing the type of x is unrelated to question of where x's value is stored, or whether x's value will be stored somewhere else after we've assigned a new value to it.
(edit: replaced "an expression" with "a statement")
>>> x = 1
>>> def a():
...: print(x)
...:
>>> def b():
...: x = 2
...: print(x)
...:
>>> def c():
...: print(x)
...:
>>> a(), b(), c()
1
2
1
>>> x = 3
>>> a(), b(), c()
3
2
3
So from that point of view, it comes down to whether we expect that an inherited class variable really is just some variable in an outer scope that we can shadow with a local variable of the same name (per your example), or whether we expect that inheritance provides some stronger notion of ownership of the inherited variable.
I dislike the former case, largely because I dislike the idea that the location at which a variable is stored can appear to change merely by assigning to it. But then, I dislike Python's implicit declaration of local variables for exactly the same reason. So you're right, there IS some consistency there. ;-)
Edit: Added some extra blank lines because lines were getting joined together.
# class_variables.py
class A(object):
x = 1
pass
pass
print A.x, B.x, C.x
print id(A.x), id(B.x), id(C.x)
B.x = 2
print "After B.x = 2, A.x, B.x, C.x and their ids:"
print A.x, B.x, C.x
print id(A.x), id(B.x), id(C.x)
A.x = 3
print "After A.x = 3, A.x, B.x, C.x and their ids:"
print A.x, B.x, C.x
print id(A.x), id(B.x), id(C.x)
And the output:
>python class_variables.py
Initially: A.x, B.x, C.x and their ids:
1 1 1
30519808 30519808 30519808
After B.x = 2: A.x, B.x, C.x and their ids:
1 2 1
30519808 30519796 30519808
After A.x = 3: A.x, B.x, C.x and their ids:
3 2 3
30519784 30519796 30519784
The example solved this by properly using import mymodule, although this might cause some more problem if your design is wrong, as see in the example. Calling f() from the module ("library") code itself is a very bad idea. Instead one should do this:
a.py:
import b
def f():
return b.x
import a
x = 1
def g():
print a.f()
import a
a.f() >>> def foo(bar=None):
... if not bar:
... bar = []
... bar.append("baz")
... return bar
...
>>> bar = []
>>> foo(bar)
["baz"]
>>> bar
[] bar = list(bar) if bar else []numbers = [n for n in range(10)]
this should be: range(10)
Though, really, even there, while the list comprehension works, its kind of an awkward construction to use that instead of:
numbers = list(range(10)) # mypackage/__init__.py
from .settings import settings
from mypackage import settings
"when the default value for a function argument is an expression, the expression is evaluated only once"
I would explain the behavior he shows as due to the default value being mutable. I don't see an expression there, just an empty list used as a default.
def f(now=datetime.datetime.now()):
...
import datetime, time
def foo2():
def foo(a=datetime.datetime.now()):
time.sleep(1)
return a
return foo()
for n in range(10): print foo2()
2014-05-08 11:23:01.642871
2014-05-08 11:23:02.644276
2014-05-08 11:23:03.644579
2014-05-08 11:23:04.645146
2014-05-08 11:23:05.646328
2014-05-08 11:23:06.647572
2014-05-08 11:23:07.647904
2014-05-08 11:23:08.648213
2014-05-08 11:23:09.648973
2014-05-08 11:23:10.649742
If you define several functions at different times, the default argument will be evaluated each time you define a new function. You'll have the same behaviour if you keep reassigning lambdas at the same function name, or if you keep edditing the globals.
You are misunderstanding how dynamic Python is. And, yes, the part about "module load time" was a simplification.
In Python, variables are references (pointers). `[]` gets evaluated on import time, and returns a pointer to an object (an empty list). This object is then further mutated on the body on the next calls because if you don't pass the parameter, it still points to the same object.
def get_analytics_data(options = {})
options = options.merge({'ids' => GAReadonly.configuration.id })
def foo a, b=a.succ
b
end
foo(1)
=>
2Is #6 really called 'late binding'? That seems like the wrong term.
for i, x in enumerate(numbers):
if odd(x):
del numbers[i] def foo2():
def foo(a=[]):
a.append('ba')
return a
return foo()
print foo2()
print foo2()
print foo2()
print foo2()
print foo2()
['ba']
['ba']
['ba']
['ba']
['ba']Your example is pretty contrived and doesn't illustrate what he was pointing out, as you're creating a new function foo every time foo2 is called, and only calling it once.