Skip to content

Instantly share code, notes, and snippets.

Avatar

Fahad Siddiqui fahadsiddiqui

View GitHub Profile
@fahadsiddiqui
fahadsiddiqui / md_tree.sh
Last active Oct 20, 2020
Markdown-safe tree creation using bash command `tree`. Just go to the project root, and run it.
View md_tree.sh
tree . -aF --dirsfirst -I '.git|node_modules|.idea' | sed -e 's/│/|/g' -e 's/└/+/g' -e 's/├/+/g' -e 's/──/--/g'
@fahadsiddiqui
fahadsiddiqui / .hyper.js
Created Sep 10, 2020
My Hyper.is terminal configuration.
View .hyper.js
// Future versions of Hyper may add additional config options,
// which will not automatically be merged into this file.
// See https://hyper.is#cfg for all currently supported options.
module.exports = {
config: {
// choose either `'stable'` for receiving highly polished,
// or `'canary'` for less polished but more frequent updates
updateChannel: 'stable',
@fahadsiddiqui
fahadsiddiqui / copy.go
Created Sep 3, 2020 — forked from r0l1/copy.go
Copy a directory tree (preserving permissions) in Go.
View copy.go
/* MIT License
*
* Copyright (c) 2017 Roland Singer [roland.singer@desertbit.com]
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
View gist:54d9d61b53ec951ff897b0a153fee4a1
### Keybase proof
I hereby claim:
* I am fahadsiddiqui on github.
* I am fsdqui (https://keybase.io/fsdqui) on keybase.
* I have a public key ASA20qONU2fu3tTaU7HNsc3Oogl2LUV4XjuEYPSMXkDG1wo
To claim this, I am signing this object:
@fahadsiddiqui
fahadsiddiqui / v1.polymerapp.io.nginx
Created Jun 2, 2020
Nginx configuration for v1 server.
View v1.polymerapp.io.nginx
server {
root /var/www/v1.polymerapp.io/html;
index index.html index.htm index.nginx-debian.html;
server_name v1.polymerapp.io;
location / {
try_files $uri $uri/ /index.html $uri/ =404;
}
@fahadsiddiqui
fahadsiddiqui / zeppelin_docker_9376.md
Last active May 27, 2020
Run Apache Zeppelin docker container with port number 9376.
View zeppelin_docker_9376.md

The logs directory will be your $PWD/.zeppelin_logs and the notebook directory will be $PWD/.notebook:/notebook.

Create Folders

If you dont create folders before running the next command, the folders will be created as the root user, which will be problematic for the container to create .git/ and other folders in there. You can simply fix that too, by doing sudo chown -R $USER .zeppelin_logs .notebook.

mkdir -p $PWD/.zeppelin_logs $PWD/.notebook
@fahadsiddiqui
fahadsiddiqui / test_data_mysql.md
Created May 12, 2020
Preparing test docker-ized mysql database.
View test_data_mysql.md

First of all generate the data, here is an interesting tool to do that https://pypi.org/project/csvfaker/

Create test_data/ folder and use csvfaker to put data in there

csvfaker -r 100000 first_name last_name email ssn job country phone_number user_name zipcode invalid_ssn credit_card_number credit_card_provider credit_card_security_code bban > test_data/pii_data.csv

Then create a docker container using image mysql:5.7 and mount volume test_data/

@fahadsiddiqui
fahadsiddiqui / mutliple_dirs.sh
Last active Mar 10, 2020
Create multiple directories in one `mkdir` command.
View mutliple_dirs.sh
# create in from a list
mkdir -p temp/{api,logs,whatnot}
# create all from A to Z
mkdir -p temp/{A..Z}
@fahadsiddiqui
fahadsiddiqui / flatten_df.scala
Last active Feb 26, 2020
Flatten a nested JSON Spark DataFrame using Scala, Spark 2.2.+ — a custom solution.
View flatten_df.scala
def flattenDataFrame(df: DataFrame): DataFrame = {
val fields = df.schema.fields
val fieldNames = fields.map(x => x.name)
for (i <- fields.indices) {
val field = fields(i)
val fieldType = field.dataType
val fieldName = field.name
fieldType match {
case _: ArrayType =>
View sha_512_hash.scala
import java.math.BigInteger
import java.nio.file._
import java.security.MessageDigest
import scala.collection.JavaConverters
def sha512Hash(p: String): String = {
val path = Paths.get(p)
val text = JavaConverters
.asScalaBuffer(Files.readAllLines(path))
You can’t perform that action at this time.