Skip to content

Instantly share code, notes, and snippets.

@tomaszezula
Created November 9, 2015 14:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tomaszezula/3784147bd1ff58240557 to your computer and use it in GitHub Desktop.
Save tomaszezula/3784147bd1ff58240557 to your computer and use it in GitHub Desktop.
import org.apache.spark.sql._
import org.apache.spark.streaming.dstream.DStream
trait Loader[T] {
def load(input: T)
}
class SuccessLoader(ctx: HiveContext) extends Loader[DStream[Row]] {
override def load(stream: DStream[Row]) = {
// Data aggregation and persistence
}
}
class ErrorLoader(ctx: HiveContext) extends Loader[DStream[Row]] {
// Uploads failed log entries
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment