How lazy evaluation can bite you

beware-lazy-eval.md

beware-lazy-eval.R

jenny Tue Mar 29 12:29:36 2016

I tweeted a link to my Draw the rest of the owl write-up and got some free code review. Konrad Rudolph pointed out that I really should use force() on n and VERB inside the function factory, to make sure I can't get burned by lazy evaluation. Reading more on that drew my attention to the function operators section of Advanced R, which is clearly relevant to my problem: "A function operator is a function that takes one (or more) functions as input and returns a function as output." Oops.

So I re-read function operators and my head exploded a little. It is probably just as well that I worked off the dorky power() example, which is much easier to understand and got me most of the way.

But what exacty is the issue with lazy evaluation and force()?

My non-deterministic VERB will just be frustrating here, so I'm going back to simple examples.

call_me_n <- function(f, n = 3) {
  function(...) for (i in seq_len(n)) f(...)
}
a_fun <- function() cat("calling a_fun()!\n")
b_fun <- function() cat("calling b_fun()!\n")

I set n and enclosed_fun in the global environment. Then I apply call_me_n() to them.

n <- 2
enclosed_fun <- a_fun
a_2 <- call_me_n(enclosed_fun, n)
a_2()
#> calling a_fun()!
#> calling a_fun()!

Now I change n and enclosed_fun in the global environment and call my_fun() again. I am expecting horrible things to happen to me. Namely, I expect to see b_fun() executed 4 times.

n <- 4
enclosed_fun <- b_fun
a_2()
#> calling a_fun()!
#> calling a_fun()!

And I do not. Why not?

Now Kevin Ushey explains: The problem here is with promises -- when you call a_2() the first time, the promises passed in (n, enclosed_fun) get forced, and so a_2 gets its own copies of those guys.

Therefore to see the problem I need to apply my function factory but redefine n and/or enclosed_fun before actually invoking the thing it has produced.

n
#> [1] 4
enclosed_fun
#> function() cat("calling b_fun()!\n")
b_4 <- call_me_n(enclosed_fun, n)

BUT WAIT! Before I actually call b_4(), I redefine n and enclosed_fun:

(n <- 3)
#> [1] 3
(enclosed_fun <- a_fun)
#> function() cat("calling a_fun()!\n")
b_4()
#> calling a_fun()!
#> calling a_fun()!
#> calling a_fun()!

There we go! I have finally correctly done the wrong thing.

How do we fix this? force() inside the factory.

call_me_n <- function(f, n = 3) {
  force(f)
  force(n)
  function(...) for (i in seq_len(n)) f(...)
}
n
#> [1] 3
enclosed_fun
#> function() cat("calling a_fun()!\n")
a_3 <- call_me_n(enclosed_fun, n)
(n <- 6)
#> [1] 6
(enclosed_fun <- b_fun)
#> function() cat("calling b_fun()!\n")
a_3()
#> calling a_fun()!
#> calling a_fun()!
#> calling a_fun()!

This time a_3() honors the values of n and enclosed_fun upon its definition, as opposed to their current values in the global environment.

# gistr::gist_create("beware-lazy-eval.R",
#                    description = "How lazy evaluation can bite you",
#                    public = FALSE, knit = TRUE, include_source = TRUE)

beware-lazy-eval.R

#' ---
#' output: github_document
#' ---

#+ setup, include = FALSE
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  error = TRUE
)

#' I tweeted a link to my [Draw the rest of the
#' owl](http://stat545-ubc.github.io/bit007_draw-the-rest-of-the-owl.html)
#' write-up and got some free code review. [Konrad Rudolph pointed
#' out](https://twitter.com/klmr/status/714698697351815170) that I really should
#' use `force()` on `n` and `VERB` inside the function factory, to make sure I
#' can't get burned by lazy evaluation. Reading more on that drew my attention
#' to the [function operators](http://adv-r.had.co.nz/Function-operators.html)
#' section of [Advanced R](http://adv-r.had.co.nz), which is clearly relevant to
#' my problem: "A function operator is a function that takes one (or more)
#' functions as input and returns a function as output." Oops.
#'
#' So I re-read [function
#' operators](http://adv-r.had.co.nz/Function-operators.html) and my head
#' exploded a little. It is probably just as well that I worked off the dorky
#' `power()` example, which is much easier to understand and got me most of
#' the way.
#'
#' But what exacty is the issue with lazy evaluation and `force()`?
#'
#' My non-deterministic VERB will just be frustrating here, so I'm going back to
#' simple examples.

#+ factory-and-fxns-to-enclose
call_me_n <- function(f, n = 3) {
  function(...) for (i in seq_len(n)) f(...)
}
a_fun <- function() cat("calling a_fun()!\n")
b_fun <- function() cat("calling b_fun()!\n")

#' I set `n` and `enclosed_fun` in the global environment. Then I apply
#' `call_me_n()` to them.
n <- 2
enclosed_fun <- a_fun
a_2 <- call_me_n(enclosed_fun, n)
a_2()

#' Now I change `n` and `enclosed_fun` in the global environment and call
#' `my_fun()` again. I am expecting horrible things to happen to me. Namely, I
#' expect to see `b_fun()` executed 4 times.
n <- 4
enclosed_fun <- b_fun
a_2()
#' And I do not. Why not?
#'
#' Now [Kevin Ushey
#' explains](https://gist.github.com/jennybc/65c577f98c2bad7e2b3d0ccb773dfaf8#gistcomment-1736926):
#' The problem here is with promises -- when you call `a_2()` the first time,
#' the promises passed in (`n`, `enclosed_fun`) get forced, and so `a_2` gets
#' its own copies of those guys.
#'
#' Therefore to see the problem I need to apply my function factory but redefine
#' `n` and/or `enclosed_fun` before actually invoking the thing it has produced.
n
enclosed_fun
b_4 <- call_me_n(enclosed_fun, n)

#' BUT WAIT! Before I actually call `b_4()`, I redefine `n` and `enclosed_fun`:
(n <- 3)
(enclosed_fun <- a_fun)
b_4()
#' There we go! I have finally correctly done the wrong thing.
#'
#' How do we fix this? `force()` inside the factory.
call_me_n <- function(f, n = 3) {
  force(f)
  force(n)
  function(...) for (i in seq_len(n)) f(...)
}
n
enclosed_fun
a_3 <- call_me_n(enclosed_fun, n)
(n <- 6)
(enclosed_fun <- b_fun)
a_3()
#' This time `a_3()` honors the values of `n` and `enclosed_fun` upon its
#' definition, as opposed to their current values in the global environment.

# gistr::gist_create("beware-lazy-eval.R",
#                    description = "How lazy evaluation can bite you",
#                    public = FALSE, knit = TRUE, include_source = TRUE)

The problem here is with promises -- when you call my_fun() the first time, the promises passed in (n, enclosed_fun) get forced, and so my_fun gets its own copies of those guys (with a_fun). If you don't force those promises (ie, don't call my_fun() and later redefine n and enclosed_fun, then you'll see b_fun() getting called.

You can avoid this kind of problem by forcing the promises on creation, e.g.

call_me_n <- function(f, n = 3) {
  force(f)
  force(n)
  function(...) for (i in seq_len(n)) f(...)
}

In other words, given the code:

foo <- function(promise) {
   function() { print(promise) }
}

Since promise isn't actually evaluated in foo's body (it only appears in the body of a nested function; those expressions don't get evaluated), promise won't actually get assigned a value until foo is called. You can convince yourself with e.g.

foo <- function(promise) {
  function() { print(promise) }
}

f <- foo(n) # no 'n' in scope, yet this succeeds as 'n' is never evaluated
f() # ouch!
n <- 100 # define it now
f() # evaluates (with a warning)