Skip to content

Instantly share code, notes, and snippets.


Sam Bessalah samklr

Block or report user

Report or block samklr

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
kinit -t /etc/security/keytabs/c******-*****.keytab -k ****-info**t@NX.***.somewhere
View spark_joins.scala
import spark.implicits._
case class Row(id: Int, value: String)
val r1 = Seq(Row(1, "A1"), Row(2, "A2"), Row(3, "A3"), Row(4, "A4")).toDS()
val r2 = Seq(Row(3, "A3"), Row(4, "A4"), Row(4, "A4_1"), Row(5, "A5"), Row(6, "A6")).toDS()
samklr /
Created Dec 7, 2018 — forked from noisesmith/
clean some things from zk, kafka, mongo
# vars
## EDITOR/VISUAL - what process to use to pick targets interactively
## ZK_WL - regex for zookeeper paths not to remove
## KAFKA_WL - regex for kafka topics not to remove
## MONGO_WL - regex for mongo item ids not to remove
# set -x

Keybase proof

I hereby claim:

  • I am samklr on github.
  • I am samklr_ ( on keybase.
  • I have a public key ASAJAlW3njCb2s4F77DE8jY37PhD4uZVvuKUs6x71A15PAo

To claim this, I am signing this object:

View links.txt
samklr / SparkTaskListener.scala
Created Nov 19, 2018
Listener to collect Spark execution information.
View SparkTaskListener.scala
import org.apache.spark.executor.TaskMetrics
import org.apache.spark.scheduler._
import scala.collection.mutable
class ValidationListener extends SparkListener {
private val taskInfoMetrics = mutable.Buffer[(TaskInfo, TaskMetrics)]()
private val stageMetrics = mutable.Buffer[StageInfo]()
samklr / spark-hive.scala
Created Nov 15, 2018
Integrate Spark with Hive
View spark-hive.scala
import org.apache.spark.sql.{SaveMode, SparkSession}
case class HelloWorld(message: String)
def main(args: Array[String]): Unit = {
// Creation of SparkSession
val sparkSession = SparkSession.builder()
.config("hive.metastore.warehouse.dir", params.hiveHost + "user/hive/warehouse")
samklr /
Last active Nov 12, 2018 — forked from jkpl/
Error handling pitfalls in Scala

Error handling pitfalls in Scala

There are multiple strategies for error handling in Scala.

Errors can be represented as [exceptions][], which is a common way of dealing with errors in languages such as Java. However, exceptions are invisible to the type system, which can make them challenging to deal with. It's easy to leave out the necessary error handling, which can result in unfortunate runtime errors.

samklr /
Created Nov 12, 2018 — forked from jkpl/
Enforcing invariants in Scala datatypes

Enforcing invariants in Scala datatypes

Scala provides many tools to help us build programs with less runtime errors. Instead of relying on nulls, the recommended practice is to use the Option type. Instead of throwing exceptions, Try and Either types are used for representing potential error scenarios. What’s common with these features is that they’re used for capturing runtime features in the type system, thus lifting the runtime scenario handling to the compilation phase: your program doesn’t compile until you’ve explicitly handled nulls, exceptions, and other runtime features in your code.

In his “Strategic Scala Style” blog post series,

samklr /
Created Nov 9, 2018 — forked from sahilsk/
Apache Kafka Cheat Sheet

Kafka Cheat Sheet

Display Topic Information

$ --describe --zookeeper localhost:2181 --topic beacon
Topic:beacon	PartitionCount:6	ReplicationFactor:1	Configs:
	Topic: beacon	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
	Topic: beacon	Partition: 1	Leader: 1	Replicas: 1	Isr: 1
You can’t perform that action at this time.