Python scoping..

wat-py.txt

Some notes on the python scope:

Python has a weird scoping problem, which stems from its intention to create a nice
surface syntax and perhaps to satisfy the imperative programming paradigm. This
affects the semantics of the Python programs in unintuitive ways.

Consider the following code:

A. 

>>> def foo():
...   x = 42
...   def bar():
...     return x
...   return bar
... 
>>> f = foo()
>>> f()
42
>>> 

So far so good.

Consider this second program:

B.

>>> def foo():
...   x = 0
...   def bar():
...     x = 1
...     return x
...   return bar
... 
>>> foo()
<function bar at 0x7f9de1479af0>
>>> f = foo()
>>> f()
1
>>>

In this program B, in the inner function, bar(), we declare a local variable 
(with the same name, x) and this creates a new binding for x in the inner 
function, bar().

Now, consider a third program:

C. 

>>> def foo():
...   x = 0
...   def bar():
...     x += 1
...     return x
...   return bar
... 
>>> f = foo()
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in bar
UnboundLocalError: local variable 'x' referenced before assignment
>>>

So, if we are trying to update a variable from the outer scope, we get an error. WAT?
One way to explain this behaviour is by desugaring x+= 1 into: "Look up the value of x in
the environment; then create an assignmet x <- (value of x in the env) + 1". But since
we lookedup the value of x *and* we had an assignment in the same scope later, we get
an UnBoundLocalError.

The AST dump of the statement "x += 1" looks like this:
{"body": [{"nodetype": "AugAssign", 
           "target": {"nodetype": "Name", 
                      "ctx": {"nodetype": "Store"}, 
                      "id": "x"}, 
           "value": {"nodetype": "Num", 
                     "n": 1}, 
           "op": {"nodetype": "Add"}}], 
 "nodetype": "Module"}

So, the body has the nodetype called "AddAssign" (for '+=' operation). "Assign" (Program B) works, 
but "AddAssign" don't.

A variant of the same behaviour.

>>> def foo():
...   x = 4
...   def bar():
...     print(x)
...   bar()
... 
>>> foo()
4
>>> def foo():
...   x = 4
...   def bar():
...     print(x)
...     x = 44
...     print(x)
...   bar()
... 
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 7, in foo
  File "<stdin>", line 4, in bar
UnboundLocalError: local variable 'x' referenced before assignment
>>> 

WTF?

Now, python 3.x adds a new syntax called 'nonlocal' that allows 
changing the value of a variable defined in the outer scope from 
inside.

>>> def foo():
...   x = 4
...   def bar():
...     nonlocal x
...     print(x)
...     x = 44
...     print(x)
...   bar()
...   print(x)
... 
>>> foo()
4
44
44
>>>

Another very weird behaviour:

>>> def foo(x):
...   def bar():
...     x = x
...     return x
...   return bar
... 
>>> f = foo(42)
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in bar
UnboundLocalError: local variable 'x' referenced before assignment
>>> 

One would have expected the right side x (in the 'x = x' expression inside bar()) 
to refer to the x from the outside scope (argument of foo()). But instead, 
Python chose to complain saying that local variable 'x' is referenced before 
assignment. In a way this leaks some implementation details. *Probably* the 
interpreter is scanning the body for local variable creation, marking those
variable symbols as local and then does the lookup. (I haven't looked up
CPython code to verify if this assumption is correct.)

Now, some sane behaviour:

>>> def foo(x):
...   x = 10
...   return x
... 
>>> foo(20)
10

.. and another sane behaviour.

>>> def foo(x):
...   def bar():
...     return x
...   return bar
... 
>>> f = foo(10)
>>> f()
10
>>> 

.. and another.

>>> def foo():
...   def bar():
...     x = 42
...     return x
...   print (x)
...   return bar
... 
>>> foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in foo
NameError: global name 'x' is not defined
>>> 

Here is another weird example:

def foo(x):
    def bar(y):
        return x + y
    x = x + 1
    return bar

>>> inc = foo(1)
>>> print(inc(3))
5
>>> 

To summarize:

- If x is a local variable and is bound in the local environment, we lookup and return its value from the environment.
- if x is a local variable and is not bound in the local environment, an "UnBoundLocalError" is returned.
- if x is not a local variable but is bound in the enclosing environment, we lookup and return its value.
- otherwise we consider the variable as unbound and return "NameError".