attempt-law-violation.md

How does attempt violate the laws?

The IO scaladoc mentions that the attempt function can be used to violate the functor laws. This seems like an odd claim, especially since one would expect such a violation would be a bug to be fixed! Specifically:

Materializes any sequenced exceptions into value space, where they may be handled. This is analogous to the catch clause in try/catch. Please note that there are some impure implications which arise from observing caught exceptions. It is possible to violate the monad laws (and indeed, the functor laws) by using this function! Uh... don't do that.

So how does this happen exactly? Conspicuously, IO passes the functor laws (as well as all of the other laws) in its unit test suite, so either the functor laws must be under-specified, the scaladoc is wrong, or something very fishy is going on.

The answer is something of a mix of the three. Here is the law (in Haskell syntax) which is violated:

fmap f xs = xs >>= return . f

What this is saying, in essence, is that map is the same as flatMap composed with pure. That's very intuitive. In fact, it's pretty close to the definition of map on IO:

  final def map[B](f: A => B): IO[B] = this match {
    case Pure(a) => try Pure(f(a)) catch { case NonFatal(t) => Fail(t) }
    case Fail(t) => Fail(t)
    case _ => flatMap(f.andThen(Pure(_)))
  }

Cool. So where is the problem? Well, the issues start to surface if you map with a function that throws an exception. For example:

IO.pure(42).map(_ => throw new Exception)

The above is equivalent to IO.fail(new Exception). Now, remember that laws are defined in terms of a notion of substitutability. The thing on the left side of the = needs to be syntactically substitutable for the thing on the right side, regardless of what else you find around either expression. So we can apply this substitution check to a context which includes unsafeRunSync(), just to see if we can observe a difference:

val ioa = IO.pure(42)
ioa.map(_ => throw new Exception).unsafeRunSync()               // => exception thrown!
ioa.flatMap(_ => IO.pure(throw new Exception)).unsafeRunSync()  // => exception thrown!

So far, so good. Everything seems to be substitutable. We can even push this a bit further and leverage referential transparency (an important assumption in all of this!) to factor out some code which is getting a little bulky:

val ioa = IO.pure(42)
ioa.map(_ => throw new Exception).unsafeRunSync()  // => exception thrown!

val iop = IO.pure(throw new Exception)             // => exception thrown!
ioa.flatMap(_ => iop).unsafeRunSync()

Now, we have to be careful here, because exceptions are involved, but these two programs do still produce the same result! An exception is thrown in each case. The fact that the second program throws an exception on the first line, while the first program throws an exception on the second line is sort of irrelevant. In either case, the program crashes.

This changes though if we add attempt:

val ioa = IO.pure(42)
ioa.map(_ => throw new Exception).attempt.unsafeRunSync()  // => Left(new Exception)

val iop = IO.pure(throw new Exception)                     // => exception thrown!
ioa.flatMap(_ => iop).attempt.unsafeRunSync()

Now we see a difference. And that's bad.

So either referential transparency is violated here, or the laws are violated, or maybe some mixture of the two. Either way, it's not good.

Exceptions are bad. You can make a pretty convincing argument that a function which throws exception is not really a "function" at all. Unfortunately, as IO is designed to work even in multi-threaded contexts, aggressively catching exceptions and thread-shifting them as data is simply a necessary reality. You literally cannot do useful multi-threaded programming on the JVM without this feature. So we catch exceptions in map and flatMap. Without attempt, this is basically fine, because you can't observe the fact that we're catching them. But attempt allows you to "peak behind the curtain", in a way, and see which exceptions we've caught and which ones have slipped through the net. And in this way, we can observe a difference between these two "substitutable" forms.

In general, this is an example of pragmatism over purity. Either removing the attempt function (thus eliminating observability of exceptions) or removing the exception handling on map/flatMap would bring IO into compliance, but would also cause problems for practical multi-threaded applications. Prior work (such as scalaz's Task) has also made this same design tradeoff, for the same reasons.