At Infi, we started our first Scala project (link in Dutch) in mid-2016. When it became clear that Scala might be one of the technologies used in the project, I jumped at the chance to be part of it, because I'm always eager to learn new tech, and doing a project in a functional programming language was already near the top of my professional wish list.
As always when learning new technology, I like to push the envelope to see where things start to break down. I think that's a nice way to get to know the limits of that technology. As it turns out, Scala is a powerful language, with a strong type system that lets you use many advanced concepts I won't detail here (eg. type classes, high-level abstractions like the ones in the Typeclassopedia with the help of scalaz or Cats, generic programming with shapeless and much more).
The thing I want to discuss here is a common annoyance: mapping one type to another and the amount of boilerplate it takes to do so. How to get rid of that silly, boring-to-write mapping code?
Some programming languages already have standard library functionality that limits the amount of boilerplate you need for type-mapping, especially the dynamically-typed languages (JavaScript and Python make it fairly easy). In other languages you may have to resort to libraries like AutoMapper or use runtime reflection.
Here's how I tried to solve that problem in Scala.
It might be familiar: you have two unrelated composite data types that share some fields with the same name and type, and you need to map an instance of the first type to an instance of the second type. For example:
<script src="https://gist.github.com/SteefH/c2da047022e6de1642d0df698f1fe487.js?file=1 - the problem - 1.scala"></script>case class Foo(integerField: Int)
case class Bar(integerField: Int)Say you want to get a Bar instance from a Foo instance. This one is simple, right?
// using named arguments for explicitness
val foo = Foo(integerField = 1)
val bar = Bar(integerField = foo.integerField)All clear, no doubt what's happening here. But as your types get more fields, more mapping code needs to be written:
case class Foo(integerField: Int, stringField: String)
case class Bar(integerField: Int, stringField: String)
val foo = Foo(integerField = 1, stringField = "quux")
val bar = Bar(integerField = foo.integerField, stringField = foo.stringField)Also, when more types get added that you want to map to and from, you'll get more lines that are a variation of the last one in the code above. Yuck.
In short: the amount of mapping code you need to write depends on two things:
- The size of your data types (let's call this the quality axis)
- The number of data types (the quantity axis)
So... In keeping things DRY, you don't want to repeat the above kind of mapping code at each location you want to get a Bar from a Foo. You might create something like the Scala equivalent of a C# extension method that keeps all the boring boilerplate code in one place, separated from Foo, Bar and other future types to map:
// mappings.scala or some other generically-named file
package nl.infi.thingamabob.data
package object mappings {
// map Foo to Bar, ooooh yeah
implicit class MapFooToBarSyntax(val foo: Foo) extends AnyVal {
def toBar: Bar = Bar(integerField = foo.integerField, stringField = foo.stringField)
}
}
// DoodadService.scala
trait DoodadService {
def ohPleaseFooGiveMeAFlurpz(foo: Foo): Flurpz = {
import nl.infi.thingamabob.data.mappings._
val bar = foo.toBar // wheee
??? // I'll get me a Flurpz, I just don't know how yet
}
}You probably guessed it: as the number of classes that need mapping increases, the nl.infi.thingamabob.data.mappings package will grow over time, potentially becoming a maintenance burden. There might be a better way...
Until now, we wrote mapping code with built-in Scala features, but it does not scale well along either axis, quality or quantity. There might be some generic solution, right?
At this point I started thinking of using an external library, but to keep myself challenged, a library that already does all the mapping was not something I was looking for. A quick search for such a library didn't give me much anyway, apart from maybe this Stack Overflow question. So time to get back to typing.
I already mentioned the shapeless library in the introduction. It is described as Generic programming for Scala, so that promises a nice base for a generic solution to the boilerplate problem.
I won't dive deeply into the magical world of shapeless, but one of the features the library offers is converting Scala case class instances to HList instances. A HList is a datatype for heterogenous lists of fixed length, meaning you can have a list containing elements of different types. For instance:
val fooList: Int :: String :: HNil = 1 :: "quux" :: HNil // HNil is to HList what Nil is to ListThese HList instances can be used to represent arguments to a function. Since a case class constructor is a function, you can use HList instances as a projection of a case class constructor call, and vice versa. The shapeless library offers this through the use of Generic:
import shapeless.Generic
case class Foo(intField: Int, stringField: String)
case class Bar(intField: Int, stringField: String)
val fooGeneric = Generic[Foo]
val barGeneric = Generic[Bar]
val foo = Foo(1, "quux")
// OK, cool, give me the arguments passed to the constructor call that created foo
val fooArgs = fooGeneric.to(foo)
// Now give me a Bar from those arguments
val bar = barGeneric.from(fooArgs) // Bar(1, "quux")Okay, that's nice. But one thing, field names are ignored. Not of much use when you want to map case class fields based on name. The code below will work, but does not give the desired result; the fields are mapped regardless of field name:
import shapeless.Generic
case class Foo(intField: Int, stringField: String)
case class Baz(thisIsNotAnIntFieldOhButActuallyItIs: Int, struungFuuld: String)
val fooGeneric = Generic[Foo]
val bazGeneric = Generic[Baz]
val foo = Foo(1, "quux")
// OK, cool, give me the arguments passed to the constructor call that created foo
val fooArgs = fooGeneric.to(foo)
// Now give me a bar from those arguments
val baz = bazGeneric.from(fooArgs) // Baz(1, "quux")And another thing, it requires the two separate case classes to have fields of the same type in the same order. So this will not work:
import shapeless.Generic
case class Foo(intField: Int, stringField: String)
case class Bar(stringField: String, intField: Int) // My field are swapped
val fooGeneric = Generic[Foo]
val barGeneric = Generic[Bar]
val foo = Foo(1, "quux")
// OK, cool, give me the arguments passed to the constructor call that created foo
val fooArgs = fooGeneric.to(foo)
// Now give me a Bar from those arguments
val bar = barGeneric.from(fooArgs)It will fail during compilation:
// taken from an ammonite session
cmd30.sc:1: type mismatch;
found : $sess.cmd26.fooGeneric.Repr
(which expands to) shapeless.::[Int,shapeless.::[String,shapeless.HNil]]
required: $sess.cmd27.barGeneric.Repr
(which expands to) shapeless.::[String,shapeless.::[Int,shapeless.HNil]]
val bar = barGeneric.from(fooArgs)
^
Compilation FailedSee those expanded types in the compilation error? The HList representation of Foo is Int :: String :: HNil while the one for Bar is String :: Int :: HNil. Type mismatch. Compilation failed. Sad face :(
There's another tool in the shapeless box that's a better candidate for name-based field mapping, called LabelledGeneric. What it does is almost the same as Generic, but with the added nicety of tagging a field name to the element types in the HList representation of case class fields. It does so through some macro magic that inspects the case class type during compilation. You can use it like this:
import shapeless.LabelledGeneric
case class Foo(intField: Int, stringField: String)
case class Bar(stringField: String, intField: Int) // My field are swapped
val fooGeneric = LabelledGeneric[Foo]
val barGeneric = LabelledGeneric[Bar]But still, when the code continues like below...
val foo = Foo(1, "quux")
// OK, cool, give me the arguments passed to the constructor call that created foo
val fooArgs = fooGeneric.to(foo)
// Now give me a Bar from those arguments
val bar = barGeneric.from(fooArgs)Failure. Very bad.
// taken from an ammonite session
cmd38.sc:1: type mismatch;
found : $sess.cmd34.fooGeneric.Repr
(which expands to) shapeless.::[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("intField")],Int],shapeless.::[String with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("stringField")],String],shapeless.HNil]]
required: $sess.cmd35.barGeneric.Repr
(which expands to) shapeless.::[String with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("stringField")],String],shapeless.::[Int with shapeless.labelled.KeyTag[Symbol with shapeless.tag.Tagged[String("intField")],Int],shapeless.HNil]]
val bar = barGeneric.from(fooArgs)
^
Compilation FailedType mismatch. Compilation failed. Sad face :(
What to do? Well...
Ideally, for mapping objects, I'd like to have a nice syntax. Something like Foo(1, "quux").mappedTo[Bar]. It should be available not only for Foo instances, but for other case classes too. Sounds like a task for an implicit class:
package nl.infi.thingamabob.data
package object mapping {
implicit class MappedToSyntax[Source](val sourceInstance: Source) extends AnyVal {
def mappedTo[Target] = ??? // uhmm....
}
}Now, what is needed to map one case class instance to an instance of another case class? Maybe this should be the way to go:
- Get a
HListrepresentation of theSourceobject throughLabelledGeneric[Source], let's call thissourceHList. It will contain the data, but also information about the field names the separate data elements belong to. - With the help of
LabelledGeneric[Target], get the type of theHListrepresentation of the target case class, let's call thisTargetRepr. - For each component type in
TargetRepr, look up the corresponding element insourceHlist, and build a value of typeTargetRepr. Let's call that valuetargetHList. - Get a fresh, shiny
Targetobject throughLabelledGeneric[Target].from(targetHList).
Sounds simple enough:
import shapeless.LabelledGeneric
case class Foo(intField: Int, stringField: String)
case class Bar(stringField: String, intField: Int) // My field are swapped
val foo = Foo(1, "quux")
val fooGeneric = LabelledGeneric[Foo]
val barGeneric = LabelledGeneric[Bar]
val sourceHList = fooGeneric.to(foo) // Step 1
type TargetRepr = barGeneric.Repr // Step 2. This is the type the LabelledGeneric converts to and from
val targetHList = ??? // Step 3... Something something sourceHList, TargetRepr
val target = barGeneric.from(targetHList) // Step 4So it's
- Get source representation
- Get target representation type
- ???
- Profit
The tough part in step 3 is that it involves traversing a HList type.
Traversing a list value is simple. Below is a useless function that copies a list in a contrived way. You'd never need something like that in application code (unless you get paid by the line).
def copyList[A](as: List[A]): List[A] = as match {
case Nil => Nil
case a :: tail => a :: copyList(tail)
}This is the bread and butter of functional programming in Scala: pattern matching and recursion. Can we use pattern matching and recursion at the type level, at least for shapeless.HList types? Turns out we can:
import shapeless.{HList, ::, HNil}
trait HListCopier[Elements] {
// Abstraction of the different cases in the mapList example
def apply(elements: Elements): Elements
}
object HListCopier {
// equivalent to case Nil => ... in the copyList example
implicit def hNilCopier: HListCopier[HNil] = new HListCopier[HNil] {
def apply(elements: HNil): HNil = HNil
}
// equivalent to case a :: rest => ... in the copyList example
implicit def nonEmptyHListCopier[A, Rest <: HList](implicit restCopier: HListCopier[Rest]): HListCopier[A :: Rest] =
new HListCopier[A :: Rest] {
def apply(elements: A :: Rest): A :: Rest = elements.head :: restCopier(elements.tail)
}
}
object HListCopy {
// This is the equivalent of copyList
def copyHList[Source <: HList](source: Source)(implicit hListCopier: HListCopier[Source]): Source =
hListCopier(source)
}Whew... It has generics, implicits, and it's a lot more to type than that copyList() function. And in the end it doesn't even do anything useful! I'll give an overview of what happens when you use the code.
HListCopy.copyHList() can be called with a HList instance:
val hlist = 1 :: "quux" :: HNil // Int :: String :: HNil
val copy = HListCopy.copyHList(hlist)When the compiler encounters the call to .copyHList(), it notices the absence of the hListCopier argument in the call. Since it's an implicit argument, it will try to find the best candidate for that argument with the specified type of HListCopier[Source].
In this case, it will look for an instance of HListCopier[Int :: String :: HNil]. The method the compiler uses to find that implicit is discussed in more depth than I will on Stack Overflow, but it boils down to inserting a call to either HListCopier.nonEmptyHListCopier[A, Rest] or HListCopier.hNilCopier. HListCopier.hNilCopier will return a HListCopier[HNil] which doesn't match the type of HListCopier[Int :: String :: HNil], so the compiler will try HListCopier.nonEmptyHListCopier[A, Rest]. That one promises to return a HListCopier[A :: Rest] which is generic over both the head and the tail of the HList type, and possibly matches HListCopier[Int :: String :: HNil]. After this first implicit lookup, the code will be equivalent to this:
val hlist = 1 :: "quux" :: HNil
val copy = HListCopy.copyHList[Int :: String :: HNil](hlist)(
HListCopier.nonEmptyHListCopier[Int, String :: HNil]
)HListCopier.nonEmptyHListCopier[A, Rest] in turn has its own implicit argument, asking for a HListCopier[Rest], which in this case expands to HListCopier[String :: HNil]. Again, this matches with a call to HListCopier.nonEmptyHListCopier[String, HNil]:
val hlist = 1 :: "quux" :: HNil
val copy = HListCopy.copyHList[Int :: String :: HNil](hlist)(
HListCopier.nonEmptyHListCopier[Int, String :: HNil](
HListCopier.nonEmptyHListCopier[String, HNil]
)
)Now for the next implicit lookup, the implicit argument for HListCopier[Rest] expands to HListCopier[HNil]. We have HListCopier.hNilCopier which promises to return exactly that, so it will be called:
val hlist = 1 :: "quux" :: HNil
val copy = HListCopy.copyHList[Int :: String :: HNil](hlist)(
HListCopier.nonEmptyHListCopier[Int, String :: HNil](
HListCopier.nonEmptyHListCopier[String, HNil](
HListCopier.hNilCopier
)
)
)At this point, there are no more implicits to look for, and the resulting code typechecks and runs fine.
That's a lot of work for emulating a call to Predef.identity()...
But it offers a starting point for the actual field name-based mapper we have as our goal.
Here it is:
import shapeless.{HList, ::, HNil, LabelledGeneric}
import shapeless.labelled.{FieldType, field}
import shapeless.ops.record.Selector
trait FieldsSelector[SourceHList, TargetHList] {
def apply(source: SourceHList): TargetHList
}
object FieldsSelector {
implicit def hNilFieldsSelector[SourceHList]: FieldsSelector[SourceHList, HNil] = new FieldsSelector[SourceHList, HNil] {
def apply(dontCare: SourceHList): HNil = HNil
}
implicit def hListFieldsSelector[Value, Key, TargetHListTail <: HList, SourceHList <: HList](
implicit
restFieldsSelect: FieldsSelector[SourceHList, TargetHListTail],
select: Selector.Aux[SourceHList, Key, Value] // select the value of type Value labelled with Key from a HList of type Source
): FieldsSelector[SourceHList, FieldType[Key, Value] :: TargetHListTail] =
new FieldsSelector[SourceHList, FieldType[Key, Value] :: TargetHListTail] {
def apply(source: SourceHList): FieldType[Key, Value] :: TargetHListTail =
field[Key](select(source)) :: restFieldsSelect(source)
}
}
trait CaseClassMap[Source, Target] {
def apply(source: Source): Target
}
object CaseClassMap {
implicit def caseClassMap[Source, Target, SourceRepr <: HList, TargetRepr <: HList](
implicit
sourceGen: LabelledGeneric.Aux[Source, SourceRepr],
targetGen: LabelledGeneric.Aux[Target, TargetRepr],
labelledHListMapper: FieldsSelector[SourceRepr, TargetRepr]
): CaseClassMap[Source, Target] = new CaseClassMap[Source, Target] {
def apply(source: Source): Target = targetGen.from(labelledHListMapper(sourceGen.to(source)))
}
}
object MappedToSyntax {
implicit class MappedToSyntax[Source](val source: Source) {
def mappedTo[Target](implicit ccMap: CaseClassMap[Source, Target]): Target = ccMap(source)
}
}Now let's use it:
import MappedToSyntax._
case class Foo(intValue: Int, stringValue: String)
case class Bar(stringValue: String, intValue: Int)
Foo(1, "quux").mappedTo[Bar] // Bar("quux", 1)It compiles, and does what we want, awesome.
But what if we try to use it in a way it isn't supposed to be used?
import MappedToSyntax._
case class Foo(intField: Int, stringField: String)
case class Baz(thisIsNotAnIntFieldOhButActuallyItIs: Int, struungFuuld: String)
Foo(1, "quux").mappedTo[Baz]Compilation fails, as it should:
Error:(57, 25) could not find implicit value for parameter ccMap: CaseClassMap[Foo,Baz]
Foo(1, "quux").mappedTo[Baz]
^