Skip to content

Instantly share code, notes, and snippets.

@LukaJCB
Created September 8, 2019 19:11
Show Gist options
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save LukaJCB/4440ad2bc351f1d8e98601f247ec4c02 to your computer and use it in GitHub Desktop.
Save LukaJCB/4440ad2bc351f1d8e98601f247ec4c02 to your computer and use it in GitHub Desktop.
Bifunctor extensions

Uniting the bifunctor and monofunctor hierarchies

With the rise of so-called bifunctor IO types, such as ZIO, questions have naturally arisen of how one could leverage the cats-effect type classes to make use of this new power. So far suggestions have mostly focused on duplicating the existing hierarchy into two distinct branches, one parameterized over F[_] and another parameterized over F[_, _]. To me this is not a great situation, as now library maintainers would have to write code for both of these hierarchies or choose one and leave the other one in the dust.

Instead we should find a way to unite the two shapes under a single hierarchy. This is a draft on how to enable this unification using polymorphic function types in Dotty.

Let's begin with the standard Monad and Bifunctor classes:

trait Monad[F[_]] {
  def (fa: F[A]) flatMap[A, B](f: A => F[B]): F[B]
  def (a: A) pure[A]: F[A]
  def (fa: F[A]) map[A, B](f: A => B): F[B]
}

trait Bifunctor[F[_, _]] {
  def (fa: F[A, B]) bimap[A, B, C, D](f: A => C)(g: B => D): F[C, D]
}

Now the interesting bit, is when we want to have a Monad for a type of shape F[E, A]. Naively we could duplicate the hierarchy in place and create new typeclasses for Bimonad, Biapplicative, BiflatMap etc., but I'd like to demonstrate a different route.

Basically something like a Bimonad[F[_, _]] is just a forall E. Monad[F[E, ?]] and in Dotty we can actually encode this!

type Bimonad[F[_, _]] = [E] => () => Monad[[A] =>> F[E, A]]

This might look a bit confusing at first, but it will look better future versions of dotty once partially applying types becomes a feature (akin to what kind-projector does with ?/*) Now we can write functions using abstract bifunctor shapes:

def bar[F[_, _]: Bimonad: Bifunctor](fa: F[Throwable, Int]): F[String, Int] = {
  implicit def F[E]: Monad[[A] =>> F[E, A]] = implicitly[Bimonad[F]][E]()

  fa.flatMap(_.pure).bimap(_.toString)(identity).flatMap(_.pure)
}

As you can see here we change the error type from Throwable to String, but are still able to call flatMap afterwards. This is not something we can do without polymorphic function types.

Note here, that the implicit def F is needed because of a restriction in Dotty at the moment, hopefully it will be resolved soon! You can track the progress of that ticket here: lampepfl/dotty-feature-requests#66

So far so good, we can use both operators like flatMap that come from the monofunctor hierarchy as well as operations like bimap from the bifunctor hierarchy. The interesting bit is what comes next, when we talk about type classes like MonadError and Bracket:

trait MonadError[F[_], E] extends Monad[F] {
  def (e: E) raiseError[A]: F[A]
  def (fa: F[A]) handle[A](f: E => F[A]): F[A] 
}

Here we want the error type E to be the left side of our bifunctor, so a BimonadError would look like this:

type BimonadError[F[_, _]] = [E] => () => MonadError[[A] =>> F[E, A], E]

Now we'd able to write functions like this:

def bar[F[_, _]: BimonadError: Bifunctor](fa: F[Throwable, Int]): F[String, String] = {
  implicit def F[E]: MonadError[[A] =>> F[E, A], E] = implicitly[BimonadError[F]][E]()

  fa.handle(t => 2.pure).bimap(_.toString)(_ * 3).flatMap(n => n.toString.pure)
}

Pretty neat, if you ask me! It's easy to extend this to Bracket as well as it's already polymorphic in the error type and has the same shape as MonadError. If we did the same thing for Concurrent and other type classes in cats-effect, we could make everything bifunctor friendly without duplicating the hierarchy at all.

For a baseline I think this is already pretty good. It might need a few extra type classes such as this one I came up with a year ago:

trait ErrorControl[F[_, _]] extends Bifunctor[F] {
  def (fa: F[A, B]) controlError[A, B, C, D](f: Either[A, B] => F[C, D]): F[C, D]
}

Which would mean we could recover from errors and let that be shown in the type signatures themselves. Potential usage could look like this:

def baz[F[_, _]: ErrorControl: BimonadError](fa: F[Throwable, Int]): F[String, Int] = {
  implicit def F[E]: MonadError[[A] =>> F[E, A], E] = implicitly[BimonadError[F]][E]()
  fa.controlError { 
    case Left(t) => 3.pure
    case Right(n) => (n + 1).toString.raiseError
  }
}

Anyways that's all for now, I hope we can find a way to make this more ergonomic and usable so that we can enjoy the best of both worlds in the future. I'm very excited to see what's going to happen and what new and interresting stuff we can come up with. If you have any comments, concerns or suggestions, please do leave a comment :)

@neko-kai
Copy link

Okay, the primary reason why I didn't go forward much with quantification-based type classes is because V e. MonadError[F[e, ?], e] is not sufficient to define a type-changing def catchAll(fa: F[E, A])(f: E => F[Nothing, A]): F[Nothing, A]. MonadError's
handleErrorWith(fa: F[A])(f: E => F[A]): F[A]) constrains F to be the same on the lhs and rhs, there's no way to overrule this, save a cast. That means I still have to define custom typeclasses for type-changing operations.
Also, cats' syntax does not allow for variance in flatMap and other basic operations – quantification is sufficient to allow for variance, so I don't need to define new classes, just new syntax.

So, given the value proposition that:

  1. Bifunctor hierarchy still needs new custom typeclasses for type-changing ops
  2. Bifunctor hierarchy still needs custom syntax for variance on monofunctor typeclass ops
  3. The only upside is that bifunctor implicits are compatible with monofunctor implicits – but this can also be achieved via shims

At this point just having a parallel hierarchy with implicit conversions back to monofunctor typeclasses seemed way better to me than trying to reuse the existing hierarchy and having to introduce more typeclasses and convoluted machinery in trying to do so.

@LukaJCB
Copy link
Author

LukaJCB commented Sep 11, 2019

@neko-kai, I totally understand your concern in this regard. I think you're right that we need something like the catchAll function and my proposal was to introduce something along the lines of ErrorControl defined above (which would allows you to define catchAll). AFAICT however, this is pretty much the only function in the cats-effect type class hierarchy that needs this sort of error-type-changing behaviour.

@LukaJCB
Copy link
Author

LukaJCB commented Sep 11, 2019

Okay, now that I went through the hierarchy you propose, I see two other functions:

  • pure of course returns Nothing on the left hand side, I feel like we could add that to a potential ErrorControl type class.
  • guarantee expects the finalizer to return an F[Nothing, Unit], which is IMO a good point, but also might overconstrain it a bit. I feel like there might be some cases where it could be okay for a finalizer not to succeed. (which of course you could just attempt.void, so I guess the point isn't all that strong)

I think we can mitigate these two fairly easily without duplicating everything. I'm not sure how some downstream libraries could make use of them which is a shame. Should they also duplicate their abstractions? I.e. having both fs2.Stream2[F[_, _], A] and fs2.Stream[F[_], A]`? I'm not yet fully convinced we want to do that 🤔

@neko-kai
Copy link

neko-kai commented Sep 11, 2019

@LukaJCB

I think we can mitigate these two fairly easily without duplicating everything.

The bigger problem that caused me to abandon this having to duplicate basic syntax like flatMap, bracket etc. to enable variance. Also increased compile-times caused by using shapeless. Having to add additional type classes on top of existing cats type classes makes it even worse...

I'm not sure how some downstream libraries could make use of them which is a shame. Should they also duplicate their abstractions? I.e. having both fs2.Stream2[F[_, ], A] and fs2.Stream[F[], A]`? I'm not yet fully convinced we want to do that 🤔

We use fs2.Stream just fine with an abstract [F[+_, +_]: BIO], I didn't really miss any type-changing operations in fs2, although you may look at ZStream and check the type-changing operations there.

hierarchy you propose

It's over a year in production with more than 150k lines of proprietary code written with Bifunctor TF over BIO* constraints, I can't say it's super battle-tested, but it's not in proposal stage :P

@LukaJCB
Copy link
Author

LukaJCB commented Sep 13, 2019

@neko-kai

The bigger problem that caused me to abandon this having to duplicate basic syntax like flatMap, bracket etc. to enable variance. Also increased compile-times caused by using shapeless. Having to add additional type classes on top of existing cats type classes makes it even worse...

That's a fair point about variance, not sure how to best deal with that.

We use fs2.Stream just fine with an abstract [F[+, +]: BIO], I didn't really miss any type-changing operations in fs2, although you may look at ZStream and check the type-changing operations there.

Sorry I meant it not from a user perspective, but from a library developer perspective. If libraries like fs2 and http4s, only use the monofunctor hierarchy, is it really useful to have these duplications? I'm not quite convinced. I wonder if there's a way to go from X[_[_, _]] to Y[_[_]] by fixing Throwable, I tried for a while, but couldn't quite get it.

It's over a year in production with more than 150k lines of proprietary code written with Bifunctor TF over BIO* constraints, I can't say it's super battle-tested, but it's not in proposal stage :P

Sorry, I didn't mean to say it's in a proposal stage, I only meant that it's a proposal for cats-effect 3. I'm glad to hear that it's been working out well in production. I would love to try it some day soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment