Trying out workarounds for SI-2712: we are running into it when trying to generically extend the FlowOps / FlowOpsMat types in Akka Streams’ Scala DSL.

Trial2712.scala

import scala.annotation.unchecked.uncheckedVariance

object Trial2712 {
  trait NotUsed
  import language.higherKinds

  /*
   * The following are simplified versions of FlowOps / FlowOpsMat and their
   * concrete subtypes.
   */

  trait FO[+Out, +Mat] {
    type Repr[+O] <: FO[O, Mat] {
      type Repr[+OO] = FO.this.Repr[OO]
    }
    def map[T](f: Out => T): Repr[T] = ???
  }

  trait FOM[+Out, +Mat] extends FO[Out, Mat] {
    type Repr[+O] <: FOM[O, Mat] {
      type Repr[+OO] = FOM.this.Repr[OO]
      type ReprMat[+OO, +MM] = FOM.this.ReprMat[OO, MM]
    }
    type ReprMat[+O, +M] <: FOM[O, M] {
      type Repr[+OO] = FOM.this.ReprMat[OO, M @uncheckedVariance]
      type ReprMat[+OO, +MM] = FOM.this.ReprMat[OO, MM]
    }
    def mapMat[T](f: Mat => T): ReprMat[Out, T] = ???
  }

  class Source[+O, +M] extends FOM[O, M] {
    type Repr[+OO] = Source[OO, M @uncheckedVariance]
    type ReprMat[+OO, +MM] = Source[OO, MM]
  }

  class Flow[-I, +O, +M] extends FOM[O, M] {
    type Repr[+OO] = Flow[I @uncheckedVariance, OO, M @uncheckedVariance]
    type ReprMat[+OO, +MM] = Flow[I @uncheckedVariance, OO, MM]
  }

  class SubSource[+O, +M] extends FO[O, M] {
    type Repr[+OO] = SubSource[OO, M @uncheckedVariance]
  }

  class SubFlow[-I, +O, +M] extends FO[O, M] {
    type Repr[+OO] = SubFlow[I @uncheckedVariance, OO, M @uncheckedVariance]
  }

  /*
   * These are helpers to work around SI-2712 which could be provided by Akka Streams.
   */

  trait U[X] {
    type Out
    type Mat
    type R[+o] <: FO[o, Mat] {
      type Repr[+O] = R[O]
    }
    def ch(x: X): R[Out]
  }

  implicit def uOne[O, M, TC[+o, +m] <: FO[o, m] { type Repr[+O] = TC[O, m @uncheckedVariance] }] =
    new U[TC[O, M]] {
      type Out = O
      type Mat = M
      type R[+O] = TC[O, Mat]
      def ch(x: TC[O, M]) = x
    }

  implicit def uTwo[I, O, M, TC[i, +o, +m] <: FO[o, m] { type Repr[+O] = TC[i, O, m @uncheckedVariance] }] =
    new U[TC[I, O, M]] {
      type Out = O
      type Mat = M
      type R[+O] = TC[I, O, Mat]
      def ch(x: TC[I, O, M]) = x
    }

  trait Umat[X] {
    type Out
    type Mat
    type R[+o, +m] <: FOM[o, m] {
      type Repr[+O] = R[O, m @uncheckedVariance]
      type ReprMat[+O, +M] = R[O, M]
    }
    def ch(x: X): R[Out, Mat]
  }

  implicit def uOneMat[O, M, TC[+o, +m] <: FOM[o, m] { type Repr[+O] = TC[O, m @uncheckedVariance]; type ReprMat[+O, +M] = TC[O, M] }] =
    new Umat[TC[O, M]] {
      type Out = O
      type Mat = M
      type R[+o, +m] = TC[o, m]
      def ch(x: TC[O, M]) = x
    }

  implicit def uTwoMat[I, O, M, TC[i, +o, +m] <: FOM[o, m] { type Repr[+O] = TC[i, O, m @uncheckedVariance]; type ReprMat[+O, +M] = TC[i, O, M] }] =
    new Umat[TC[I, O, M]] {
      type Out = O
      type Mat = M
      type R[+o, +m] = TC[I, o, m]
      def ch(x: TC[I, O, M]) = x
    }

  /*
   * This is how client code can now use the helpers to write extension methods that
   * work across all FlowOps subtypes.
   */
  implicit class x[X](val x: X) extends AnyVal {
    def xx(i: Int)(implicit u: U[X]) = u.ch(x).map(identity).map(identity)
    def xy(implicit u: U[X]) = u.ch(x).map(_.toString).map(identity)
    def m[T1, T2](f: T1 => T2)(implicit u: Umat[X] { type Mat <: T1 }) =
      u.ch(x).mapMat(f)
  }

  /*
   * These don’t work as intended: it seems impossible to get the types
   * out there. This results in that an expression like
   *
   *     f.mm(_ => "buh")
   *
   * might compile, but its returned type will not longer unify with
   * Flow et al. It seems that the path-dependent type leaks out in the
   * form of an existential.
   */
  class Y[X](val x: X)(implicit val u: Umat[X]) {
    def mm[T](f: u.Mat => T) = u.ch(x).mapMat(f)
    def xz[T](f: u.Out => T) = u.ch(x).map(f)
  }
  implicit def y[X](x: X)(implicit u: Umat[X]) = new Y(x)

  /*
   * All these compile successfully. Type ascriptions to lambda arguments are unfortunately
   * required, the attempt presented in class Y above does not work.
   */
  val s1 = new Source[Int, NotUsed].xx(12).m((_: NotUsed) => "").map(_ + 12)
  val s2: Source[Int, String] = s1

  val f1 = new Flow[Int, Int, NotUsed].xx(12)
  val f2: Flow[Int, Int, NotUsed] = f1

  val f3 = f1.m((_: NotUsed) => 42).xy
  val f4: Flow[Int, String, Int] = f3

  val subs1 = new SubSource[Int, NotUsed].xy
  val subs2: SubSource[String, NotUsed] = subs1

  val subf1 = new SubFlow[Int, Int, NotUsed].xy
  val subf2: SubFlow[Int, String, NotUsed] = subf1
}

forgot to link to Daniel’s gist that I used for inspiration

Can't compile it in scala 2.11.7

first had to define

type NotUsed = Unit

then scalac complains about: (see comment in code):

  trait FOM[+Out, +Mat] extends FO[Out, Mat] {
    // this part seems weird for scalac (is Repr really needed?)
    type Repr[+O] <: FOM[O, Mat] {
      type Repr[+OO] = FOM.this.ReprMat[OO, Mat @uncheckedVariance]
      type ReprMat[+OO, +MM] = FOM.this.ReprMat[OO, MM]
    }
    type ReprMat[+O, +M] <: FOM[O, M] {
      type Repr[+OO] = FOM.this.ReprMat[OO, M @uncheckedVariance]
      type ReprMat[+OO, +MM] = FOM.this.ReprMat[OO, MM]
    }
    def mapMat[T](f: Mat => T): ReprMat[Out, T] = ???
  }

and

  trait U[X] {
    type Out
    type Mat
    type R[+o] <: FO[o, Mat] /**is the following {} needed?*/ {
      type Repr[+O] = R[O]
    }
    def ch(x: X): R[Out]
  }

Ah, right, I used akka.NotUsed in there. But apart from that Scala 2.11.7 should compile this (it does for me).

Update: I was fooled somehow, fixing it.

The refinements are necessary in order to be able to fluently chain DSL operators without losing the resulting types. Without them you’d need to manually flatten Repr#Repr#Repr chains, and with them only reasonable implementations are permitted as a bonus.

fixed compilation errors and removed Akka dependency

ok I see why you have to do that... wan't even aware it was possible to write that in this way and make it compile ;)

You're encountering the exact issues that Miles has been trying to solve (and others & me too)...

in UMat, you do not manipulate a R[_] but a R[_, _] and as soon as you try to fix one of the type params to obtain a R[_], scalac isn't able to unify types and fails...

The solution of Miles is to force scalac to "reify" types R[_] using Unapply (as you've done) plus Aux technique (to clearly reify types) plus a macro to brute-force scalac to become aware of type unification.
Have a look at Generic1 in shapeless, this is where those tricks come from...

That's what I've done in a simpler way in Precepte too https://github.com/MfgLabs/precepte/blob/master/precepte-core-cats/src/main/scala/HackSI2712.scala#L39 ... I know what I'm looking for so the macro is much simpler...
This could work in your case too as you know you're manipulating Source/Flow/Sink...

But if scalac could do that in a generic way, it would help so many of us that now use higher-kinds a lot in their code :D