Mixing overloads and default parameters in Scala

mixing-overloads-default-params.md

Mixing overloads and default parameters in Scala

The problem

There are some situations that arise where you want default arguments in object apply methods, but you also want to overload the apply. For instance, in GeoTrellis, we have an S3LayerWriter which allows you to write an RDD of rasters out of Amazon's S3 storage backend. In order to operate, it needs an AttributeStore, which is the type responsible for reading and writing metadata. A simplified (not real) signature of the attribute store looks like

case class AttributeStore(bucket: String, prefix: String)

An S3LayerWriter also takes some options, like whether or not to clobber an existing layer with that id, or if the key index will be one to one with the elements of the RDD. A simplified version of an S3LayerWriter might be:

class S3LayerWriter(attributeStore: AttributeStore, clobber: Boolean, oneToOne: Boolean) {
  def write(): Unit = println(s"S3LayerWriter $attributeStore $clobber $oneToOne")
}

And the companion object's apply method might look like this:

object S3LayerWriter {
  def apply(attributeStore: AttributeStore, clobber: Boolean, oneToOne: Boolean): S3LayerWriter =
    new S3LayerWriter(attributeStore, clobber, oneToOne)
}

However, there's some sane defaults to clobber and oneToOne, so let's put those in:

object S3LayerWriter {
  def apply(attributeStore: AttributeStore, clobber: Boolean = true, oneToOne: Boolean = false): S3LayerWriter =
    new S3LayerWriter(attributeStore, clobber, oneToOne)
}

In developing the API, you may find it useful to have the S3LayerWriter construct the attribute store itself, just passing in the bucket and prefix parameters. To do that, you might want to overload apply like this:

object S3LayerWriter {
  def apply(attributeStore: AttributeStore, clobber: Boolean = true, oneToOne: Boolean = false): S3LayerWriter =
    new S3LayerWriter(attributeStore, clobber, oneToOne)

  def apply(bucket: String, prefix: String, clobber: Boolean = true, oneToOne: Boolean = false): S3LayerWriter =
    apply(AttributeStore(bucket, prefix), clobber, oneToOne)
}

Seems like a simple enough solution. However, this is not allowed.

Overloads and Default Parameters

Your compiler might say that the above code is ok. I have had similar code compile, that is until I try to package the code into a JAR, at which point you'll get the compiler message

Multiple overloaded alternatives of method define default arguments

It turns out you just can't do it. It's baked into the language that you can't have overloaded methods that specify default arguments.

What to do?

You could code a lot of overloads:

object S3LayerWriter {
  def apply(attributeStore: AttributeStore, clobber: Boolean, oneToOne: Boolean): S3LayerWriter =
    new S3LayerWriter(attributeStore, clobber, oneToOne)

  def apply(attributeStore: AttributeStore, clobber: Boolean): S3LayerWriter =
    apply(attributeStore, true, false)

  def apply(attributeStore: AttributeStore): S3LayerWriter =
    apply(attributeStore, true)

  def apply(bucket: String, prefix: String, clobber: Boolean, oneToOne: Boolean): S3LayerWriter =
    apply(AttributeStore(bucket, prefix), clobber, oneToOne)

  def apply(bucket: String, prefix: String, clobber: Boolean): S3LayerWriter =
    apply(AttributeStore(bucket, prefix), clobber, false)

  def apply(bucket: String, prefix: String): S3LayerWriter =
    apply(AttributeStore(bucket, prefix), true, false)
}

We exploded out to 6 methods, and this doesn't even cover all the permutations of possible parameters.

We could create an Options class, with a default:

object S3LayerWriter {
  case class Options(clobber: Boolean = true, oneToOne: Boolean = false)
  object Options {
    def DEFAULT = Options()
  }

  def apply(attributeStore: AttributeStore, options: Options): S3LayerWriter =
    new S3LayerWriter(attributeStore, clobber, oneToOne)

  def apply(attributeStore: AttributeStore): S3LayerWriter =
    apply(attributeStore, Options.DEFAULT)

  def apply(bucket: String, prefix: String, options: Options): S3LayerWriter =
    apply(AttributeStore(bucket, prefix), options)

  def apply(bucket: String, prefix: String, clobber: Boolean): S3LayerWriter =
    apply(bucket, prefix, Options.DEFAULT)
}

This cuts down on the number of overloads, but still feels awkward and undiscoverable.

You might reach for the Magnet Pattern, which is a great solution to some issues with method overloading in Scala. However, it specifically does not work with default parameters.

Proposed solution: Curried Defaults Pattern (working title)

My proposed solution is to recognize that there's normally two groups of parameters in these types of cases: the necessary parameters (and various ways to construct them), and optional parameters. If we split those groups of parameter, we would get something like this:

object S3LayerWriter {
  def apply(attributeStore: AttributeStore)(clobber: Boolean = true, oneToOne: Boolean = false): S3LayerWriter =
    new S3LayerWriter(attributeStore, clobber, oneToOne)

  def apply(bucket: String, prefix: String)(clobber: Boolean = true, oneToOne: Boolean = false): S3LayerWriter =
    apply(AttributeStore(bucket, prefix))(clobber, oneToOne)
}

but that will still give us the multiple overloaded alternatives of method apply define default arguments error. Instead, if we define all of our optional parameters in an inner class's apply method, and have the constructor parameters be the necessary parameter group, we can define the object apply methods like this:

object S3LayerWriter {
  class Apply(attributeStore: AttributeStore) {
    def apply(clobber: Boolean = true, oneToOne: Boolean = false): S3LayerWriter =
      new S3LayerWriter(attributeStore, clobber, oneToOne)
  }

  def apply(attributeStore: AttributeStore): Apply =
    new Apply(attributeStore)

  def apply(bucket: String, prefix: String): Apply =
    new Apply(AttributeStore(bucket, prefix))
}

This will compile and work. One last touch is to create an implicit conversion from Apply to S3LayerWriter so that calling the object apply with no second parameter list (not even an empty parameter list (), which would work) will result in something that we can treat like an S3LayerWriter:

object S3LayerWriter {
  class Apply(attributeStore: AttributeStore) {
    def apply(clobber: Boolean = true, oneToOne: Boolean = false): S3LayerWriter =
      new S3LayerWriter(attributeStore, clobber, oneToOne)
  }
  implicit def applyToCall(a: Apply): S3LayerWriter = a()

  def apply(attributeStore: AttributeStore): Apply =
    new Apply(attributeStore)

  def apply(bucket: String, prefix: String): Apply =
    new Apply(AttributeStore(bucket, prefix))
}

This pattern allows you to call all the iterations of the apply method you might want:

S3LayerWriter("foo", "bar").write()
S3LayerWriter("foo", "bar")(oneToOne = true).write()
val attributeStore = AttributeStore("foo2", "bar2")
S3LayerWriter(attributeStore)(clobber = false).write()
S3LayerWriter(attributeStore)(true, false).write()

Cons

This pattern forces you to separate out the required parameters and the optional parameters out into separate parameter lists, which is a restriction on how you design your API.

Discoverability might be an issue; I haven't looked at the scala docs generated by this sort of method, but they are probably not as easy to navigate as a simple method call with one parameter list.

This also does not work if the apply method takes any implicit parameters.