Skip to content

Instantly share code, notes, and snippets.

@jbasko
Last active April 10, 2018 04:57
Show Gist options
  • Save jbasko/31c70457e0875b1ca68432039b892445 to your computer and use it in GitHub Desktop.
Save jbasko/31c70457e0875b1ca68432039b892445 to your computer and use it in GitHub Desktop.
This doesn't do what you think it does
def filter1(r):
return r % 3 == 0
def filter2(r):
return r % 5 == 0
def apply_all(records, *filters):
for f in filters:
records = (r for r in records if f(r))
return records
# This prints [0, 5, 10, 15, 20, 25, 30, 35]
# NOT [0, 15, 30]
print(list(apply_all(range(40), filter1, filter2)))
@jarnold-timeout
Copy link

Nice one!

To see why this is happening, first replace the generator expression with its moral equivalent:

def apply_all(records, *filters):
    for f in filters:
        def __gen(recs):
            for r in recs:
                if f(r):
                    yield r
        records = __gen(recs)
    return records

Notice that __gen is a closure. In particular, its reference to f is bound to the variable f of apply_all. So when __gen is executed, the value it sees for f is whatever is contained in that variable at the time of execution.

Finally, note that the expression, for f in filters: updates the f variable of apply_all. It does not create a "new" variable at each iteration, even though we sometimes think of it that way.

Putting this together, when you call apply_all(range(40), filter1, filter2) the result is a nested chain of generators, each referring to the single variable, f that was set in the apply_all invocation. The value of that variable is the last value that was assigned to it, which is filter2.

Here's a slightly simpler example of the same phenomenon:

def adders(*values):
    res = []
    for v in values:
        def f(x):
            return x + v
        res.append(f)
    return res

(a100, a200) = adders(100, 200)

# This prints 201, not 101:
print(a100(1))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment