Created
June 25, 2018 03:20
-
-
Save deusaquilus/51ae4a832760d6a6b0199f14b9ad8c71 to your computer and use it in GitHub Desktop.
Type Member in Spark Report
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Sample report base-class: | |
trait Report { | |
type T | |
def apply(runtimeConf:MyConf)(encoder:spark.sql.Encoder[T]):Unit | |
} | |
// Sample implementation: | |
class OrdersReport { | |
type T = OrderRecord | |
def runReport(runtimeConf:MyConf)(encoder:spark.sql.Encoder[OrderRecord]):Unit | |
} | |
// Record type that the report uses | |
case class OrderRecord(...) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
object RunReports { | |
def main(args:Array[String]):Unit = { | |
// I want to do this: | |
val reports = List(new OrdersReport, new SalesReport) | |
reports.find(_.condition == something).get.runReport | |
// ... but it throws an exception (encoder cannot be found) | |
// So maybe write a wrapper like this: | |
class Wrapper[R <: Report](rpt:R)(implicit enc:Encoder[rpt.T]) { | |
def runDefer(runtimeConf:MyConf) = rpt.runReport(exchange) | |
} | |
// ... and do this: | |
val reports = List(new Wrapper(new OrdersReport), new Wrapper(new SalesReport)) | |
reports.find(_.condition == something).get.runDefer | |
//... but that doesn't work either (still 'encoder cannot be found') | |
// Maybe try the aux pattern: | |
type Aux[TE] = Report{type T = TE} | |
class Wrapper[TEA](aux:Aux[TEA])(implicit enc:Encoder[TEA]) { | |
def run(runtimeConf:MyConf) = aux.runReport(exchange) | |
} | |
val reports = List(new Wrapper(new OrdersReport), new Wrapper(new SalesReport)) | |
// ... still no luck! | |
} | |
} |
Report doesn't need to be stable, just its Instances!
How can Report instances be objects if they need to take constructor arguments? I.e. what I actually have is:
trait Report {
type T
def runReport(runtimeConf:MyConf)(encoder:spark.sql.Encoder[OrderRecord]):Unit
}
abstract class BaseReport(someArgs:SomeCaseClass) extends Report
class OrdersReport(someArgs:SomeCaseClass) extends BaseReport(someArgs)
// How do I make OrdersReport stable?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Actually my production class
Report
can't be an object because it takes input arguments! Nooooooo! Vortex of un-unifiability is sucking me in!