Skip to content

Instantly share code, notes, and snippets.

🎯
Focusing

Sam Bessalah samklr

🎯
Focusing
Block or report user

Report or block samklr

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View kinit.sh
kinit -t /etc/security/keytabs/c******-*****.keytab -k ****-info**t@NX.***.somewhere
klist
View spark_joins.scala
import spark.implicits._
case class Row(id: Int, value: String)
val r1 = Seq(Row(1, "A1"), Row(2, "A2"), Row(3, "A3"), Row(4, "A4")).toDS()
val r2 = Seq(Row(3, "A3"), Row(4, "A4"), Row(4, "A4_1"), Row(5, "A5"), Row(6, "A6")).toDS()
@samklr
samklr / clean-slate.sh
Created Dec 7, 2018 — forked from noisesmith/clean-slate.sh
clean some things from zk, kafka, mongo
View clean-slate.sh
#!/bin/sh
# vars
## EDITOR/VISUAL - what process to use to pick targets interactively
## ZK_WL - regex for zookeeper paths not to remove
## KAFKA_WL - regex for kafka topics not to remove
## MONGO_WL - regex for mongo item ids not to remove
# set -x
View keybase.md

Keybase proof

I hereby claim:

  • I am samklr on github.
  • I am samklr_ (https://keybase.io/samklr_) on keybase.
  • I have a public key ASAJAlW3njCb2s4F77DE8jY37PhD4uZVvuKUs6x71A15PAo

To claim this, I am signing this object:

View links.txt
@samklr
samklr / SparkTaskListener.scala
Created Nov 19, 2018
Listener to collect Spark execution information.
View SparkTaskListener.scala
import org.apache.spark.executor.TaskMetrics
import org.apache.spark.scheduler._
import scala.collection.mutable
class ValidationListener extends SparkListener {
private val taskInfoMetrics = mutable.Buffer[(TaskInfo, TaskMetrics)]()
private val stageMetrics = mutable.Buffer[StageInfo]()
@samklr
samklr / spark-hive.scala
Created Nov 15, 2018
Integrate Spark with Hive
View spark-hive.scala
import org.apache.spark.sql.{SaveMode, SparkSession}
case class HelloWorld(message: String)
def main(args: Array[String]): Unit = {
// Creation of SparkSession
val sparkSession = SparkSession.builder()
.appName("example-spark-scala-read-and-write-from-hive")
.config("hive.metastore.warehouse.dir", params.hiveHost + "user/hive/warehouse")
.enableHiveSupport()
@samklr
samklr / article.md
Last active Nov 12, 2018 — forked from jkpl/article.md
Error handling pitfalls in Scala
View article.md

Error handling pitfalls in Scala

There are multiple strategies for error handling in Scala.

Errors can be represented as [exceptions][], which is a common way of dealing with errors in languages such as Java. However, exceptions are invisible to the type system, which can make them challenging to deal with. It's easy to leave out the necessary error handling, which can result in unfortunate runtime errors.

@samklr
samklr / article.org
Created Nov 12, 2018 — forked from jkpl/article.org
Enforcing invariants in Scala datatypes
View article.org

Enforcing invariants in Scala datatypes

Scala provides many tools to help us build programs with less runtime errors. Instead of relying on nulls, the recommended practice is to use the Option type. Instead of throwing exceptions, Try and Either types are used for representing potential error scenarios. What’s common with these features is that they’re used for capturing runtime features in the type system, thus lifting the runtime scenario handling to the compilation phase: your program doesn’t compile until you’ve explicitly handled nulls, exceptions, and other runtime features in your code.

In his “Strategic Scala Style” blog post series,

@samklr
samklr / kafka-cheat-sheet.md
Created Nov 9, 2018 — forked from sahilsk/kafka-cheat-sheet.md
Apache Kafka Cheat Sheet
View kafka-cheat-sheet.md

Kafka Cheat Sheet

Display Topic Information

$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic beacon
Topic:beacon	PartitionCount:6	ReplicationFactor:1	Configs:
	Topic: beacon	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
	Topic: beacon	Partition: 1	Leader: 1	Replicas: 1	Isr: 1
You can’t perform that action at this time.