Skip to content

Instantly share code, notes, and snippets.

View raksja's full-sized avatar

raksja raksja

  • CA
View GitHub Profile
@raksja
raksja / README.md
Created June 14, 2021 05:18 — forked from davideicardi/README.md
Write and read Avro records from bytes array

Avro serialization

There are 4 possible serialization format when using avro:

@raksja
raksja / sparkEc.scala
Created July 25, 2017 00:12 — forked from atamborrino/sparkEc.scala
Serializable Scala ExecutionContext for Spark. Allows to automatically re-create a thread-pool per Spark worker.
class ECProvider()(implicit conf: Config) extends Serializable {
@transient implicit lazy val ec: ExecutionContext = {
ExecutionContext.fromExecutorService(
Executors.newWorkStealingPool(conf.getForkJoinPoolMaxParallelism())
)
}
}
@raksja
raksja / ExecutionContextMonitor.scala
Created July 25, 2017 00:11 — forked from atamborrino/ExecutionContextMonitor.scala
Monitor Scala's ExecutionContext / Akka Dispatcher lag (number of tasks in waiting queues)
import java.util.concurrent._
import akka.dispatch.{Dispatcher, ExecutorServiceDelegate}
import config.Config
import helpers.ScalaLogger
class ExecutionContextMonitor()(implicit metricsService: MetricsClient, config: Config) {
private val log = ScalaLogger.get(this.getClass)
private val scheduler = Executors.newSingleThreadScheduledExecutor()