Skip to content

Instantly share code, notes, and snippets.

@johnhungerford
Last active September 8, 2024 00:46
Show Gist options
  • Save johnhungerford/cc22eb5b23c7407aa45479a845a7ead8 to your computer and use it in GitHub Desktop.
Save johnhungerford/cc22eb5b23c7407aa45479a845a7ead8 to your computer and use it in GitHub Desktop.
ZIO-like dependency injection using implicit resolution

ZIO-like dependency injection using implicit resolution

Daniel Ciocîrlan recently published a video showcasing a dependency-injection (DI) approach developed by Martin Odersky that uses Scala's implicit resolution to wire dependencies automatically. (See also his reddit post.)

The basic pattern for defining services in Odersky's approach is as follows:

class Service(using Provider[(Dep1, Dep2, Dep3)])
  def someMethod():
    provided[Dep1].doSomething()
    provided[Dep2].doSomethingElse()

We can now construct a Service instance simply by calling Service() as long as implicit Providers for Dep1, Dep2, and Dep3 are in scope. If another service needs Service, we could easily define a Provider[Service]. We can even provide it globally by adding the following to the companion object:

object Service:
  given default(using Provider[(Dep1, Dep2, Dep3)]) = provide(Provider())

As u/Krever indicated in a comment, however, this approach makes it awkward to separate the DI framework from service definitions. Here is what it would look like to separate DI from the service itself:

class Service(dep1: Dep1, dep2: Dep2, dep3: Dep3)])
  def someMethod():
    dep1.doSomething()
    dep2.doSomethingElse()
    
object Service:
  given default(using Provider[(Dep1, Dep2, Dep3)]): Provider[Service] =
    provide(Provider(provided[Dep1], provided[Dep2), provided[Dep3])
    
  object providers:
    given test(using Provider[Dep1]): Provider[Service] =
      provide(Provider(provided[Dep1], Dep2(/* custom conf */), Dep3(/* custom conf */))

Providing generic instances can get pretty verbose, especially when we start providing alternate instances for different contexts.

ZIO layers

The approach taken by ZIO is to separate constructors used for DI from the services they construct into a new type called ZLayer. DI works in ZIO by using a macro to automatically compose these layers, or constructors, in such a way as to provided the desired type while satisfying all dependencies. One of the best thing about layers is that it easy to provide several different layers for the same service and choose the one you want for each when constructing the application.

The basic pattern for defining services in ZIO looks like this:

final case class Service(dep1: Dep1, dep2: Dep2, dep3: Dep3):
  def someMethod():
    dep1.doSomething()
    dep2.doSomethingElse()

object Service:
  val live = ZLayer.fromFunction(Service.apply)
  val test = ZLayer.fromFunction { (dep1: Dep1) =>
    Service(dep1, Dep2(/* custom conf */), Dep3(/* custom conf */))
  }

In the above example, the live layer will have a dependency on Dep1, Dep2, and Dep3, whereas test will only have a dependency on Dep1 (Dep2 and Dep3 are constructed manually). The dependencies for each layer are tracked in their types, which in the above case are inferred (another advantage).

ZLayers using implicit resolution?

The code included in this gist demonstrates a way to combine the ZIO approach with Odersky's implicit Provider approach. Rather than simply providing a value, a Provider now represents a constructor, which tracks the dependencies (i.e., the constructor parameters) at the type-level. Provider[R, A] represents a constructor of A that needs R, where R is either a dependency or a tuple of dependencies. We now also include a second type Provided which represents a value that has been provided -- either directly or via evaluation one or more Providers. The mechanics of this DI framework thus lie in the implicit resolution of a Provided instance from given Provider and/or Provided instances.

This approach now allows us to define our services and DI instances in a similar way to ZIO with less boilerplate:

import di.*

final case class Service(dep1: Dep1, dep2: Dep2, dep3: Dep3):
  def someMethod():
    dep1.doSomething()
    dep2.doSomethingElse()

object Service:
  given default: Provider[(Dep1, Dep2, Dep3), Service] =
    provideConstructor(Service.apply)
  
  object providers:
    given test: Provider[Dep1, Service] = provideConstructor { (dep1: Dep1) =>
      Service(dep1, Dep2(/* custom conf */), Dep3(/* custom conf */))
    }

There are a couple important differences from the ZIO pattern, all of which follow from the fact that providers, unlike layers, are not simply values but given instances. One benefit of this is that we are able to provide a single default given at the top level of our application which will be used automatically when we call provided[Service] without needing to import anything. We can override the default by importing Service.provided.test in the scope where we call provided[Service].

The second difference is that since we are defining our Providers as given instances we must explicitly annotate their types. While not the end of the world, this is an ergonomic sacrifice. ZIO makes good use of Scala's type inference to allow tracking complicated dependencies without having to fuss with boilerplate. In the ZIO example above, for instance, you can change the constructor parameters of Service and the changes to live's dependency type will be inferred. In our version, changing the types of Service's parameters would require rewriting the type annotation on default.

Other missing ZLayer features

Since ZIO provides what is in my view the best DI framework available, it's worth pointing out a few things it offers that this one doesn't.

  1. Error messages. ZIO's provide macro will display very nicely formatted error messages explaining which layers are missing which dependencies. It will also tell you which layers provide ambiguous (conflicting) dependencies. When using implicit resolution, we depend on built-in compiler error messages with only minimal customization possible (e.g., implicitNotFound and ambiguousImplicit annotations).
  2. Effects. ZLayer allows you to include effects in your dependencies constructors, which is important for initializing and scoping resources needed by services. Provider/Provided certainly supports effectful construction, but not using any powerful context like Future, ZIO, or IO.
  3. Composability. ZLayers can be composed manually using combinators like >> and ++, allowing the user build layers from one another without having to redefine a constructor. For instance, to make a test layer for Service from the above example, you might want to define it as:
    val test = (Dep2.test ++ Dep3.test) >> Service.live
    
    This will use the test layers from Dep2 and Dep3 to satisfy those dependencies while still requiring a layer for Dep1. It would be possible, though complicated, to implement this for Provider, but doing so would requiring explicitly annotating the resulting provider instance with the dependencies of Dep1 and Dep2 which would offset a lot of the convenience.

Conclusion

By using Providers modeled after ZLayer, it is easier to separate the DI mechanism from our service definitions and provide alternate versions of the same dependency for different contexts. Nevertheless, there is still a fair amount missing that we get in mature DI frameworks like ZIO's.

//> using scala 3.3.3
//> using jvm 21
import scala.annotation.{experimental, implicitNotFound}
import scala.compiletime.{error, erasedValue, summonInline}
object di:
// Provides a dependency of type A given dependencies encoded by type R
sealed case class Provider[-R, +A] private[di] (constructor: R => A):
// Memoization is necessary so that constructor is run only once per
// dependency
private var memo: A | Null = null
private[di] def apply(value: R): A =
if memo == null then
val result = constructor(value)
memo = result
result
else memo.asInstanceOf[A]
// Create a provider from a function. Uses transparent inline + compile-time
// utilities to resolve types correctly.
transparent inline def provideConstructor[F](inline construct: F): Provider[Nothing, Any] =
inline construct match
case f0: Function0[b] => Provider[Any, b]((_: Any) => f0())
case f1: Function1[a, b] => Provider(f1)
case f2: Function2[a1, a2, b] => Provider(f2.tupled)
case f3: Function3[a1, a2, a3, b] => Provider(f3.tupled)
case f4: Function4[a1, a2, a3, a4, b] => Provider(f4.tupled)
case f5: Function5[a1, a2, a3, a4, a5, b] => Provider(f5.tupled)
case f6: Function6[a1, a2, a3, a4, a5, a6, b] => Provider(f6.tupled)
case f7: Function7[a1, a2, a3, a4, a5, a6, a7, b] => Provider(f7.tupled)
case f8: Function8[a1, a2, a3, a4, a5, a6, a7, a8, b] => Provider(f8.tupled)
case f9: Function9[a1, a2, a3, a4, a5, a6, a7, a8, a9, b] => Provider(f9.tupled)
case f10: Function10[a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, b] => Provider(f10.tupled)
case f11: Function11[a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, b] => Provider(f11.tupled)
case f12: Function12[a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, b] => Provider(f12.tupled)
// Could go up to 22, or find more generic approach
end provideConstructor
// A provided dependency of type A
opaque type Provided[A] = A
// Provide a simple value
def provide[A](value: A): Provided[A] = value
// Provide a value lazily
def provideSuspended[A](value: => A): Provider[Any, A] =
Provider((_: Any) => value)
// Retrieve a dependency of type A by resolving an implicit Provided[A] instance
def provided[A](
using
@implicitNotFound("Unable to provide a value of type ${A}. Make sure any dependencies are provided.")
pr: Provided[A],
): A = pr
trait LowPriorityProvided:
given providedFromProvider[R, A](using lyr: Provider[R, A], apr: Provided[R]): Provided[A] =
lyr(apr)
object Provided extends LowPriorityProvided:
given providedNonEmptyTuple[A, T <: Tuple](using apr: Provided[A], npr: Provided[T]): Provided[A *: T] =
apr *: npr
given providedEmptyTuple: Provided[EmptyTuple] = EmptyTuple
given providedFromTrivialProvider[A](using pr: Provider[Any, A]): Provided[A] =
pr(())
end di
//////////////////////////////////////////
// SERVICE DEFINITIONS WITH PROVIDERS //
//////////////////////////////////////////
final case class Service1(int: Int, bool: Boolean)
object Service1:
// Provider instance that will be used by default because it's at top level in the
// companion object. Can use Provided here too.
given default: di.Provider[(Int, Boolean), Service1] =
di.provideConstructor(Service1.apply)
final case class Service2(str: String)
object Service2:
given default: di.Provider[Service1, Service2] =
di.provideConstructor((service1: Service1) => Service2(s"${service1.int} - ${service1.bool}"))
object providers:
// Given Provided instance that can be imported explicitly to override
// default instance. Can use Provider here too.
given test: di.Provided[Service2] =
di.provide(Service2(s"TEST (no dependencies!)"))
final case class Service3(service1: Service1, service2: Service2)
object Service3:
given default: di.Provider[(Service1, Service2), Service3] =
di.provideConstructor(Service3.apply)
//////////////////////////////////////
// CONSTRUCT AND USE DEPENDENCIES //
//////////////////////////////////////
object Main:
// Three ways to provide a zero-dependency type (needed by Service1):
// 1: Provide directly with a value (eagerly evaluated)
given di.Provided[String] = di.provide("hi")
// 2: Provide with a suspended value (lazily evaluated)
given di.Provider[Any, Int] = di.provideSuspended {
println("Performing side-effect...") // This should only run once
23
}
// 3: Provide with a Function0 (lazily evaluated)
given di.Provider[Any, Boolean] = di.provideConstructor(() => false)
// Uncomment this import to inject a test version of Service2
// import Service2.providers.test
def main(args: Array[String]): Unit =
// Resolve Service3 dependency from Provided/Provider instances
val service3 = di.provided[Service3]
println(service3)
@tewecske
Copy link

tewecske commented Sep 2, 2024

Is this doable with Scala 2.13?

@johnhungerford
Copy link
Author

Is this doable with Scala 2.13?

Should be doable, but would have to be translated.

@tewecske
Copy link

tewecske commented Sep 6, 2024

I kind of made it work, but I'm not sure if it's good or not. First I overcomplicated it because I misunderstood the errors, then I reverted and I basically just had to change the memo, overload the provideConstructor because there is no transient inline and convert the opaque type to a case class because I couldn't make it work with a simple type alias. If you have time can you check if it could be improved in any way?
https://scastie.scala-lang.org/0MgWvH8qReiHiHq5msQ66g

@johnhungerford
Copy link
Author

That looks right to me. The overloaded provideConstructor is cleaner than the version I had anyways. Are you planning on using this? If so I can recommend one improvement: the memoization should really be scoped. If you call provided in one part of your application with one set of dependencies and you call it in another place with another set of dependencies, you could end up with the wrong provided value in one of those two places.

If you're interested I could adapt your 2.13 code in a new gist to show a good approach for dealing with this.

@tewecske
Copy link

tewecske commented Sep 7, 2024

Thanks for checking! I will probably not use the Scala 2 version as we are migrating to Scala 3 anyway. I wanted to do something like ZIO ZLayers when I saw your post and got excited, I was curious to understand how this works! I checked Kyo as well and the Layer implementation is in a separate kyo-prelude module but it's for Scala 3 only. I will probably test that too sometime in the future.

The scoping is only needed if any of the services is not a "singleton", right? I think ZIO has the same (or similar) problem. If you want to use the same Service implementation but with different configuration you can't really do that because it's all type based and not name/id based. So If I want to do something like this I have to create differently named subclasses of the Service. Not a big deal and rarely comes up but sometimes it does.

@johnhungerford
Copy link
Author

johnhungerford commented Sep 8, 2024

Scoping is needed if you need to use different implementations of the same types in different places. This is not a problem in the ZIO and Kyo because their the memoization happens only when the layer is actually constructed which in both cases is lazy. In the case of Provider, the construction is eager and the memoization is global. Here is what the edge case would look like:

// Service1.scala
trait Service1:
  def doSomething(): Unit

case class Service1Live():
  def doSomething(): Unit = ???

object Service1Test:
  def doSomething(): Unit = ???

object Service1:
  given Provider[Any, Service1] = provideSuspended(Service1Live())

  object Test:
    given Provided[Service1] = provideValue(Service1Test)

// Service2.scala
case class Service2(service1: Service1):
  def doSomethingElse(): Unit = ???

object Service2:
  given Provider[Service1, Service2] = provideConstructor(Service2.apply)

// Main.scala
object Main:
  val service2 = provided[Service2]
    
  def main(args: Array[String]): Unit =
    service2.doSomethingElse()

// Service2Test.scala
object Service2Test extends SomeTestSuite:
  import Service1.Test.given

  def testServiceTwo():
    val service2 = provided[Service2]
    service2.doSomethingElse()
    assert ???     

In this case, the line val service2 in Service2Test might have the same value as val service2 in Main, because they are both evaluated when their respective objects are constructed (which will be prior to anything else running), and they will both be generated by the given Provider[Service1, Service2] in object Service2.

This wouldn't happen in ZIO or Kyo.

@tewecske
Copy link

tewecske commented Sep 8, 2024

Ah, ok! Then it's a different problem what I described.
I checked ZLayer and it also has a Scoped MemoMap for caching. I don't quite understand yet how it works but will look into it, thanks! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment