Skip to content

Instantly share code, notes, and snippets.

View elarib's full-sized avatar

elarib

15 followers · 30 following
View GitHub Profile
@elarib
elarib / do & grep
Created September 18, 2017 13:20
Tips for linux command
ls -l -ad dev-*
@elarib
elarib / get_original_filename_of_gzip.py
Created January 14, 2019 13:05
Get original filename of a gzip (python)
# Try to get the original filename (present in the header) of the gz file.
# Check http://www.onicos.com/staff/iz/formats/gzip.html
def get_original_fileName(path):
with open(path, 'rb') as f:
magic = f.read(2)
if magic != '\037\213':
raise IOError, 'Not a gzipped file'
@elarib
elarib / hdfs.scala
Created March 20, 2019 16:14
List recusively directories
def listSubdirectoriesRecursively(
directory: Path,
list: mutable.ListBuffer[Path] = new mutable.ListBuffer[Path]
): List[Path] = {
val iterator = fs.listLocatedStatus(directory)
while (iterator.hasNext) {
val pathStatus = iterator.next()
val path = pathStatus.getPath
@elarib
elarib / Dockerfile
Last active April 10, 2019 12:14
Docker image containing latest Ansible
FROM python:latest
RUN pip install ansible
WORKDIR "/root/project"
@elarib
elarib / Dockerfile
Created April 29, 2019 08:32
Generate Random Avro data for Kafka Topic
FROM openjdk:8-jdk
#INIT
RUN apt-get update && \
apt-get install -y jq uuid-runtime unzip
ENV CONFLUENT_VERSION 4.1.3
ENV CONFLUENT_URL http://packages.confluent.io/archive/4.1/confluent-oss-4.1.3-2.11.tar.gz
@elarib
elarib / kafka-cheat-sheet.md
Created July 19, 2019 12:17 — forked from ursuad/kafka-cheat-sheet.md
Quick command reference for Apache Kafka

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

@elarib
elarib / converter.scala
Created September 9, 2019 08:55
Json to M3U (Specific case)
import org.json4s.DefaultFormats
import org.json4s.JsonAST.JValue
import org.json4s.jackson.JsonMethods._
case class Channel(name: String, ch: String)
case class Category(id: String)
object JsonM3uTransformer extends App {
@elarib
elarib / travis-local.md
Last active November 21, 2020 05:50 — forked from fulldecent/travis-local.md
Run Travis build locally

travis-local.md

Preconditions:

  1. POSIX or Windows system
  2. Install Docker
  3. A GitHub repo that already builds on Travis

Postcondition:

@elarib
elarib / Branch.groovy
Last active August 18, 2021 10:06
Jenkins Branch From Dynamic Repo using Active Choice parameter
team_name = TOREPLACE
username = TOREPLACE
password = TOREPLACE
urlStr= "https://bitbucket.org/!api/2.0/repositories/${team_name}/${Repository}/refs/branches?sort=-target.date&fields=pagelen,next,page,size,values.name&pagelen=100"
branches = []
def getBranches(url) {
def baseUrl = new URL(url)
@elarib
elarib / EnvHacker.scala
Created August 4, 2021 12:29 — forked from vpatryshev/EnvHacker.scala
Setting environment variables in Scala JVM
import java.util.{Collections, Map => JavaMap}
import scala.collection.JavaConverters._
trait EnvHacker {
/**
* Portable method for setting env vars on both *nix and Windows.
* @see http://stackoverflow.com/a/7201825/293064
*/
def setEnv(newEnv: Map[String, String]): Unit = {
try {