Python: Common Newbie Mistakes, Part 1
blog.amir.rachum.com
blog.amir.rachum.com
That functionality really should be changed for the next major python release, and have default evaluate each time the function is called - which is how 99.9% of people using it expect it to work.
There's plenty of code out there that rely on default values being assigned a mutable value and changing it -- even in a major point release -- would be far worse off. Python's been around for 20 years; it's too late to make a change like that.
It IS a "wart" in that you need to be aware of it to know that it's an issue, but once you are, it is an obvious thing to spot.
Make lists immutable you mean? Like tuple()? Edit: Or force re-evaluation on every function call (could be pretty slow)?
The best practice is to just use a tuple (immutable) instead of a list to begin with. You can always do mylist = list(defaulttuple) inside your function first thing if you really want to work with it as a list.
Also, both pylint and pep8, two python static analyzers, will all warn you about any mutable default arguments you provide. Many Python developers have these hooked into their editor or precommit hooks to catch things like this (among others).
Lists are largely for homogeneous collections, whereas tuples are heterogeneous. When you mix them, you abuse what they signal to the reader.
Python doesn't do much for type safety. Either accept the pitfalls of mutability as a tradeoff, or subclass list to make a safer type.
In Python, lists and tuples mean almost exactly the same thing. That's how Python is implemented. Those two things have "totally different" semantics in many other languages, not so in Python. As so, using the one over the other in many cases is purely stylistic matter, not the matter of semantics... unless it's about mutability, of course.
If they're as different as you and grandparent suggest, then please list those differences, and mean in Python. As I said, in OCaml Lists and tuples are completely different types, starting with the fact that tuples are not iterables (!). The same is true in Erlang, where you need to explicitly convert a tuple to list to get all the elements of a tuple. Both list and tuple are iterable in Python, however.
I just see much more similarities between lists and tuples in Python than differences, to the point that I wouldn't mind calling tuples an immutable_list or frozenlist, or something like that.
What? What makes you think that?
As I've also seen, a good rephrasing is that lists have order whereas tuples have structure. Yet another mnemonic I've heard that seems apt is that tuples are appropriate where a named-tuple could also conceivably be used.
However, don’t treat Guido’s statement as a mandate!
"Yes, taking Guido too seriously can have that effect on people <1.1 wink> The trick is in knowing when Guido is proscribing dogmatic law, and when he is describing his intent when he designed feature(tte)s of Python, or his point of view at this moment, or how he wished people would see it, or reacting to what he had for lunch. He's just human, you know, and no more infallible than the best of us." Thomas Wouters March 12, 2003 in python-dev
For example, consider object.__class__.__bases__. That's a tuple of homogenous data. It's not of a fixed length, and the base class at a given position has no intrinsic name, other than "first", "second", etc.
Do you think the use of tuple here is a bug? Should it be a list instead?
GvR aside, even the documentation on python.org mentions this: http://docs.python.org/2/tutorial/datastructures.html#tuples...
Example of some typical tuples: (1,'A') (2,'B') (3,'C')
In Python, of course lists can be heterogenous, so you could have a list like [1, 'A', 2, 'B', 3, 'C'] but it would be an atypical/un-idiomatic use case. A list of tuples or possibly a dict would make more sense for that.
With a language that uses duck typing (so that the meaningful "types" of objects are often not their explicit classes but the implicit -- both in definition and in association with particular objects -- interfaces expected by the code which accesses them), this ambiguity is even more pronounced.
Yes, the list of [1,'A',2,'B',3,'C'] is really after all just a list of [Object, Object, Object, Object, Object, Object]. And the list itself is also just an Object.
But the explicit classes of the elements of the list are different ([int,str,int,str,int,str]), and often that actually does matter for the use case, because int and str have different interfaces, and "adding" ints means something different than "adding" strs for example. It really just depends on what you're going to do with that data, and I think Guido was also talking in terms of the idiomatic use cases for the different data structures.
Well, yeah. I'd be kind of stunned if he wasn't. I was making a note qualifying the examples that were given in isolation, not questioning the basic premise.
> Regardless of inheritance, interfaces, and duck typing, the explicit classes of the objects still do matter.
Whether or not that's true depends on the use case; the really relevant sense of homogenous or heterogenous that distinguishes the idiomatic use of tuples vs. lists is contextual, and is more about the interfaces that the consuming code expects the members to provide than the explicit classes per se.
There's nothing unidiomatic in Python about using a list for a series of items of different explicit classes that are going to have the same operations (supported by a common interface they all share) performed on them.
This. In particular, tuples are for collections whose length is fixed and meaningful, and for which you could give names to each field. Lists are for collections whose length doesn't correspond to anything in particular and which you could imagine growing and shrinking.
As an aside, django makes this mistake and confuses people: using tuples where semantically they should be lists, presumably out of a desire for immutability.
Do believe that the use of a tuple here is a bug, like you do for Django?
The __bases__ can be re-assigned. Assignment to an instance is guaranteed atomic in Python. Some of the list operations are not. Thread-safe guarantees might be another reason for restricting list's mutability.
It is probably also true (generally speaking) that people tend to use lists when the cardinality is not fixed and tuples otherwise.
No, that's not good advice. Tuples are not just immutable lists. See below.
def print_now(now=time.time()):
print now
If you're worried about speed, you could make a class/instance variable, and then set the default to that variable. That's what 'default' is actually doing in the current implementation - and it sounds unexpected in the extreme.
Now personally I hardly every use default values, but to me, when I see this - > I see no reason why anybody would not want the default to evaluate every time
It means you should have put it in the method.
No, it's not "some other language", it's every other language. Why? Because the behavior being described isn't really a default. It's more like a static variable.
Why would I ever want to have the value from when I first launched the program?! Your argument that "this is how python does it" is not a very good one. Why should I have put it in the method? I want a default value! The reason for having the default value on a function in the first place is to allow me to make functions work with a default value. If this doesn't work, the feature is useless. As I said in the first post, the only reason the feature is usable at all is because, luckily, people nearly always use only numbers or strings for default values.
EDIT: Plus your observation that you rarely use default values is probably for a good reason: they don't really work. Maybe if they worked, you'd use them?
Honestly, no, it's probably because I don't write methods to be generic enough.
Ignoring that,
>argument that "this is how python does it" is not a very good one
Except that if you're writing Python you have to work by Pythons rules, so yes it is.
> The reason for having the default value on a function in the first place is to allow me to make functions work with a default value
Then in python, make sure you are using datatypes that are passed by value, not by reference. That is the core of the problem and it bites everyone at some point in every language because the rules of passing by reference or value are different in a great number of languages.
def foo(name=[]):
In python, is not saying if I didn't pass something make name an empty list, it says 'if name is not defined in this scope, create an empty list.' But with Pythons rules a name[] was created in that scope when the function was defined. It was the programmers mistake for assuming otherwise.
Once again, we are writing Python here, Pythons rules are the only ones that matter. If you are getting unexpected results because you're not following the rules, the issue is yours. Python is not called C#, or Java, or C or Ruby or whatever because it is not C# or Java or whatever.
Both ways have their advantages, python (and the python implementations) just fell one way instead of another.
http://stackoverflow.com/questions/1132941/least-astonishmen...
current behavior is totally fine. it is well defined that somelist.append(x) appends x to somelist inplace.
>>> class A(object):
... def method(self, a): pass
...
>>> A().method()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: method() takes exactly 2 arguments (1 given)
>>> class C:
... def m(self,a):
... pass
...
>>> C().m()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: m() missing 1 required positional argument: 'a'
>>>I actually figured it out by printing out the id of the object being modified in my code, and when seeing it was the same had to really scratch my head. Default args are initialized when the function (or similarly, class) is defined, not when executed.
But overall, this is a problem involving documentation, design and naming, and your own mindset.
Ideally you would never really be put in the position of not knowing, because it would be clearly documented. Of course, we all know how well that tends to work.
Second, well-designed systems with good naming practices will make it fairly obvious most of the time what sort of arguments are expected, and what you can expect to get back. (This is related to the next point.)
Finally, coming from a "static" language, your brain just isn't trained to perform its own type inference and deal with duck typing. This is mostly a matter of practice, you get better at it. But if you aren't already familiar with various kinds of typing, be sure to do some reading. A better understanding of the differences will help make sense of what you're seeing.
def print_now(now=time.time):
print now() def print_now(now=None):
now = now or time.time()
print nowEdit:
if now is None:
now = time.time()import mock
def print_now(now=mock.sentinel.now): now = now if now != mock.sentinel.now else time.time()
_sentinel = object()
def print_now(now=_sentinel):
if now is _sentinel:
now = time.time()
...I was wrong, and now I know better. Thank you.
def foo(**kwargs):
numbers = kwargs.pop('numbers', [])
numbers.append(9)
print numbersAlso, I really don't like the decision to make Python 2 disobey the system locale and use ascii as the default encoding. Lots of Python programs out there have broken i18n for no good reason because of that (nobody starts teaching python by telling you to use .decode('utf-8')/.encode('utf-8') when you pipe data, and by the time you realize what's wrong, you often already have code out there.)
def f(x=()): x = list(x) ...
def g(y=(())) y = dict(y) ...
f() is the pattern for lists, g() is for dicts. Note that you can pass in regular lists to f() and regular dicts to g().
>>> arrays = [[]] * 5
>>> print arrays
... [[], [], [], [], []]
>>> arrays[0].append(1)
>>> print arrays
... [[1], [1], [1], [1], [1]]If you use a variable it's easier to understand:
a = []
arrays = [a] * 5
On the topic of the bug in OP, would a Python newbie expect the similarly confusing behavior caused by interactions between (im)mutability and function arguments being passed as a "reference"?
Using OP's foo():
>>> a = [1,2,3]
>>> foo(a)
[1, 2, 3, 9]
>>> a
[1, 2, 3, 9]
>>> def bar(n):
n += 1
print(n)
>>> b = 3
>>> bar(b)
4
>>> b
3 def foo(num=[0]):
num[0] = num[0] + 1
return num[0] class Foo:
def __init__(self):
self.n = 0
def __call__(self):
self.n += 1
return self.n
foo = Foo()