October 19, 2007

Note to self: default parameter values are mutable in Python

Just hit a somewhat unexpected behaviour in Python code. What would the following code snippet print, what do you say ?
def foo(x = []):
x.append("bar")
print x

foo()
foo()
If you think about the def statement as a declaration, the answer is obvious - it should print
[ 'bar' ]
[ 'bar' ]
but in fact it prints
[ 'bar' ]
[ 'bar', 'bar' ]
Why ? Because in Python def is an executable statement, which means that the list of arguments for the method to be created (x) and most importantly their default values ([]) are themselves nothing but arguments to def. Something like this:
foo = def(x, default_x)
and when this gets executed, default_x is bound to something that at that point evaluates to an empty list, but remains mutable. Then, whenever the created foo is executed, the append method modifies the contents of default_x - effectively the "default value" of x.

This sounds strange, but is clearly documented in the language reference, quote from http://docs.python.org/ref/function.html

Default parameter values are evaluated when the function definition is executed. This means that the expression is evaluated once, when the function is defined, and that that same "pre-computed" value is used for each call. This is especially important to understand when a default parameter is a mutable object, such as a list or a dictionary: if the function modifies the object (e.g. by appending an item to a list), the default value is in effect modified.
The spec suggests using None for all the default parameters, but you should have no problem using any immutable objects as well. For example:
def foo(x = None):
print (x or []) + [ "bar" ]
where x is None, or
def foo(x = ()):
# oops, no append method
print x + ( "bar", )
Phew, I've been lucky using None's so far...

2 comments:

Paddy3118 said...

Hi Dmitry,
If you are using None as a sentinel value then it is best to test if x is None rather than if x is True as if someone calls the function with another value that tests as True , such as a null string or a [] then the wrong code could be executed.

- Paddy.

Dmitry Dvoinikov said...

I agree with you, None is a good candidate for a sentinel precisely because it stands out and can be explicitly tested for. And I tend to use explicit check if x is None rather than a shortcut if x as well.

I still believe immutable values such as strings, ints or booleans can be used as default values when appropriate in a declarative kind of way.