Skip to content

Instantly share code, notes, and snippets.

@bernhardschaefer
Last active June 12, 2019 02:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save bernhardschaefer/6d8df86307b8f38f3051cc5cb188e42e to your computer and use it in GitHub Desktop.
Save bernhardschaefer/6d8df86307b8f38f3051cc5cb188e42e to your computer and use it in GitHub Desktop.
import org.apache.spark.groupon.metrics.UserMetricsSystem
import org.apache.spark.groupon.metrics.SparkTimer
import org.apache.spark.rdd.RDD
class SparkTimerExample[T] {
lazy val timer: SparkTimer = UserMetricsSystem.timer("BulkPutTimer")
def bulkPut(rdd: RDD[T]) = {
rdd.foreachPartition(partitionOfRecords => {
val timerCtx = timer.time()
// perform output operation
timerCtx.stop()
})
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment