Skip to content

Instantly share code, notes, and snippets.

@mbloms
Last active December 20, 2020 22:35
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mbloms/60951d4e12d523fa0e7e925b04e4ef0d to your computer and use it in GitHub Desktop.
Save mbloms/60951d4e12d523fa0e7e925b04e4ef0d to your computer and use it in GitHub Desktop.

Pre-SIP: Transparent members for Opaque types

I think opaque type aliases is one of Scala 3's most interesting features. Many use cases for it has already been discussed. One oppertunity that I think would be quite useful, while at the same time being a very good fit with the rest of the language, is the ability to expose members of the underlying type as members on the opaque type.

For example: The current imlementation of IArray uses extension methods to export methods from Array:

extension (arr: IArray[Byte]) def apply(n: Int): Byte = arr.asInstanceOf[Array[Byte]].apply(n)
...
extension [T <: Object](arr: IArray[T]) def apply (n: Int): T = arr.asInstanceOf[Array[T]].apply(n)
extension [T](arr: IArray[T]) def apply (n: Int): T = arr.asInstanceOf[Array[T]].apply(n)

...

extension (arr: IArray[Byte]) def length: Int = arr.asInstanceOf[Array[Byte]].length
...
extension (arr: IArray[Object]) def length: Int = arr.asInstanceOf[Array[Object]].length
extension [T](arr: IArray[T]) def length: Int = arr.asInstanceOf[Array[T]].length

Given that re-exporting methods on the underlying type is probably a quite common use case, I think exposing a subset of the underlying type would be very useful.

People have proposed the ability to export methods before, but I've not seen a bigger discussion about it:

https://contributors.scala-lang.org/t/having-another-go-at-exports-pre-sip/2982/22?u=mbloms https://contributors.scala-lang.org/t/proposal-for-opaque-type-aliases/2947/54?u=mbloms

Syntax is always hard, for now I'm borrowing the new syntax @odersky introduced for givens/structural instances. Using that, a concrete way to extend opaque types to support this could look like this:

opaque type Nat with {
  def +(x: Nat): Nat
} <: Int = Int

opaque type IArray[+T] with {
  def apply(i: Nat): T
  def clone(): IArray[T]
  def length: Nat
} = Array[_ <: T]

Of course, the transparent members can't override the implementation of underlying members, but they should be able to override the type signatures in a compatible way.

The overriding type signature of a transparent member must be compatible with:

  • The underlying type as seen from inside the scope of the definition
  • The lower bound as seen from outside the scope of the definition
  • The upper bound as seen from outside the scope of the definition

One way to guarantee this could be that given the definition

opaque type T with {def m(x: A): B} >: Lo <: Hi = Rep
  • There must be a member definition of m in Rep where the erasure of m in T matches the erasure of m in Rep
  • There must be a matching member definition of m in Lo, and m in Lo must subsume the definition of m in T
  • If there is a matching definition of m in Hi, then m in T must subsume the definition of m in Hi

Where the definition of matching and subsumes is taken from the 5.1.3 and 3.5.2 respectively of the Scala Language Specification.

Join of types with transparent members

The fact that transparent members correspond to real members could be used to preserve them in joins:

trait C
  def children: List[C]

object opaques:
  opaque type A {
    def children: List[A]
  } = C
  opaque type B {
    def children: List[B]
  } = C

import opaques._

def x: A & B = ...
def xs: List[A & B] = x.children

def y: A | B = ...
def ys: List[A | B] = y.children

Normally, the join of A and B would be Any since A and B is unrelated to each other. With transparent members, the fact that children in A and B is owned by the same base class is known on the outside. In other words the ownership of a transparent method is transparent.

Using my own notation I express the join of A and B like this: The join of A | B is {this: opaque C => def children: List[A | B]}

Motivating examples

At first, it might not be obvious why this would be useful, but I would say there are

Subproposal - support export clauses:

Something something blah blah I want this:

opaque type IArray[+T] = Array[_ <: T] with {
  export this.{apply,clone,length}
}

And also this:

object Defs {
  opaque type Name = String

  extension (n: Name) {
    export n.length
  }
  def newName(s: String): Name = s
}
@ciuncan
Copy link

ciuncan commented Dec 7, 2020

About the supporting export clauses within extension blocks for opaque types, let me put the concern that @smarter commented here from gitter.im/lampepfl/dotty:

I think the main problem is that very often you'll have stuff like class Foo { def append(x: Foo): Foo = ... }, so if you make an opaque type around that and export append, the exported method will take and return a Foo instead of your opaque type
so if export was allowed it could be more confusing than helpful

My reasoning for asking whether we could use export clauses for opaque types was to see if would selectively delegate some of the methods of underlying type without having to write all the wirings explicitly. We could just say export n.{length, charAt} to expose two methods and keep everything else encapsulated.

As @smarter mentioned in his comment, of course if those methods use the underlying type (in return or parameter positions) we would be leaking the abstraction that opaque type created. First solution comes to mind is to transform the extension method definition so that occurrences of underlying types in the extension method signature are replaced with the opaque type, but I imagine that could cause problems with variance (or maybe not, just thinking aloud).

@mbloms
Copy link
Author

mbloms commented Dec 7, 2020

I think the problem with export clauses in general. For instance consider this example which do not use opaque types:
https://gist.github.com/mbloms/31f8c8c32cfb0dec2d636d5f55ad9499

Before I realised Builder has the mapResult method, my first thought was simply to make an anonymous builder and export methods from ArrayBuilder. If it wasn't for mapResult I would have had to redefine every method in Builder that return this explicitly.

Perhaps in this special case, this could be solved by attaching signatures in the export clause, but that would not match existing syntax. In any case, there are definitely cases where you don't necessarily want to blindly replace the original type with the opaque version.

@ciuncan
Copy link

ciuncan commented Dec 7, 2020

I see what you mean. Indeed each usage of export must be checked if anything leaks or not, or has an appropriate/desired signature as a surface.

@dcsobral
Copy link

I think it's important to focus on what the most common use case for transparent members will be, and IArray isn't it. What I see as the most common use case -- and, of course, I might be wrong about it -- is addressing primitive obsession. It's replacing String and the AnyVal subclasses with opaque types without performance impact or the drudgery of implementing forwarding methods.

But that also means curating where the backing type is replaced on method type signatures, and where it is preserved and, for that matter, where the type signature needs to be changed outright. Some examples:

  • contains on String takes a CharSequence, but it makes more sense for opaque types to take themselves instead.
  • /(x: Int): Int on Int should become /(x: Opaque): Opaque, but /(x: Double): Double (etc) should not be exported at all.
  • >>(n: Int): Int on Int should become >>(n: Int): Opaque.
  • Any methods shouldn't be overloaded at all, which I suppose is the status quo.

Given that, I think the most common use cases can solved by the library instead. Provide OpaqueInt, OpaqueString et cetera traits covering the "primitives" (String being the outlier) with carefully curated method forwarding and that should take care of most use cases without any added complexity to the language.

@mbloms
Copy link
Author

mbloms commented Dec 18, 2020

How would this library solution be used in practice? Is OpaqueString a trait or is it some kind of annotation?

Anyway: personally I don't really see why it's a problem to explicitly define the type signature of contains of an opaque alias of String for instance. Isn't it clearer to have it explicitly written out?

What I'm mostly concerned is:

  • using extension methods as forwarders incur some overhead
  • they aren't part of the type so they don't work for structural types
  • joins are weird: OpaqueInt | Int only have the members from Any
  • they are more boilerplate than they have to be
  • other stuff I can't think of at the moment

Getting rid of boilerplate is not the main objective here, it's a secondary desirable side effect.

I want the semantics of this proposal before I can start building the library

@dcsobral
Copy link

OpaqueString would be a trait or class. It would be used like this:

object Name extends OpaqueString
type Name = Name.Type

It would provide all of String methods with adjustments like contains(other: Name): Boolean instead of contains(other: CharSequence): Boolean. And, of course, it would not be a String, nor will its methods accept or return String unless appropriate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment