Skip to content

Instantly share code, notes, and snippets.

@deusaquilus
Created June 25, 2018 03:20
Show Gist options
  • Save deusaquilus/51ae4a832760d6a6b0199f14b9ad8c71 to your computer and use it in GitHub Desktop.
Save deusaquilus/51ae4a832760d6a6b0199f14b9ad8c71 to your computer and use it in GitHub Desktop.
Type Member in Spark Report
// Sample report base-class:
trait Report {
type T
def apply(runtimeConf:MyConf)(encoder:spark.sql.Encoder[T]):Unit
}
// Sample implementation:
class OrdersReport {
type T = OrderRecord
def runReport(runtimeConf:MyConf)(encoder:spark.sql.Encoder[OrderRecord]):Unit
}
// Record type that the report uses
case class OrderRecord(...)
object RunReports {
def main(args:Array[String]):Unit = {
// I want to do this:
val reports = List(new OrdersReport, new SalesReport)
reports.find(_.condition == something).get.runReport
// ... but it throws an exception (encoder cannot be found)
// So maybe write a wrapper like this:
class Wrapper[R <: Report](rpt:R)(implicit enc:Encoder[rpt.T]) {
def runDefer(runtimeConf:MyConf) = rpt.runReport(exchange)
}
// ... and do this:
val reports = List(new Wrapper(new OrdersReport), new Wrapper(new SalesReport))
reports.find(_.condition == something).get.runDefer
//... but that doesn't work either (still 'encoder cannot be found')
// Maybe try the aux pattern:
type Aux[TE] = Report{type T = TE}
class Wrapper[TEA](aux:Aux[TEA])(implicit enc:Encoder[TEA]) {
def run(runtimeConf:MyConf) = aux.runReport(exchange)
}
val reports = List(new Wrapper(new OrdersReport), new Wrapper(new SalesReport))
// ... still no luck!
}
}
@deusaquilus
Copy link
Author

deusaquilus commented Jun 25, 2018

Actually my production class Report can't be an object because it takes input arguments! Nooooooo! Vortex of un-unifiability is sucking me in!

@dreadedsoftware
Copy link

Report doesn't need to be stable, just its Instances!

@deusaquilus
Copy link
Author

How can Report instances be objects if they need to take constructor arguments? I.e. what I actually have is:

trait Report {
  type T
  def runReport(runtimeConf:MyConf)(encoder:spark.sql.Encoder[OrderRecord]):Unit
}

abstract class BaseReport(someArgs:SomeCaseClass) extends Report
class OrdersReport(someArgs:SomeCaseClass) extends BaseReport(someArgs)
// How do I make OrdersReport stable?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment