n1snt

Oh my zsh.

Install ZSH.

sudo apt install zsh-autosuggestions zsh-syntax-highlighting zsh

Install Oh my ZSH.

fworks

Steps:

1. Download the latest zsh package: https://packages.msys2.org/package/zsh?repo=msys&variant=x86_64

Example:

zsh-5.7.1-1-x86_64.pkg.tar.xz

gvenzl

LOCATION=$(curl -s https://api.github.com/repos/<YOUR ORGANIZTION>/<YOUR REPO>/releases/latest \
| grep "zipball_url" \
| awk '{ print $2 }' \
| sed 's/,$//'       \
| sed 's/"//g' )     \
; curl -L -o <OUTPUT FILE NAME> $LOCATION

for example:

rspeer

kayousterhout

Generating Flame Graphs for Apache Spark

Flame graphs are a nifty debugging tool to determine where CPU time is being spent. Using the Java Flight recorder, you can do this for Java processes without adding significant runtime overhead.

When are flame graphs useful?

Shivaram Venkataraman and I have found these flame recordings to be useful for diagnosing coarse-grained performance problems. We started using them at the suggestion of Josh Rosen, who quickly made one for the Spark scheduler when we were talking to him about why the scheduler caps out at a throughput of a few thousand tasks per second. Josh generated a graph similar to the one below, which illustrates that a significant amount of time is spent in serialization (if you click in the top right hand corner and search for "serialize", you can see that 78.6% of the sampled CPU time was spent in serialization). We used this insight to spee

fdv

These are the snippets I use most of the time when administrating my ES cluster

To be updated

Settings to change before you do something

Before restarting a data node

curl -XPUT 'http://escluster:9200/_cluster/settings' -d '{

feynmanliang

import org.apache.spark.ml.feature.{CountVectorizer, RegexTokenizer, StopWordsRemover}
import org.apache.spark.mllib.clustering.{LDA, OnlineLDAOptimizer}
import org.apache.spark.mllib.linalg.Vector

import sqlContext.implicits._

val numTopics: Int = 100
val maxIterations: Int = 100
val vocabSize: Int = 10000

gccpacman

#Protecting the Docker daemon Socket with HTTPS

HOST-IP:172.17.42.1 VM-IP:172.17.0.2

openssl genrsa -aes256 -out ca-key.pem 2048
openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem ```Common Name:  172.17.42.1```
openssl genrsa -out server-key.pem 2048
openssl req -subj "/CN=172.17.42.1" -new -key server-key.pem -out server.csr

echo subjectAltName = IP:172.17.42.1, IP:172.17.0.2, IP:127.0.0.1 > extfile.cnf

angeloh

/**
      The Play (2.3) json combinator library is arguably the best in the scala world. However it doesnt
      work with case classes with greater than 22 fields.
      The following gist leverages the shapeless 'Automatic Typeclass Derivation' facility to work around this
      limitation. Simply stick it in a common location in your code base, and use like so:
      Note: ** Requires Play 2.3 and shapeless 2.1.0  
      
  
        import SWrites._
        import SReads._

solarce

# kafka_prod1.yaml
#
# The production kafka nodes for prod1

graphite_host: <%= node[:jmxtrans][:graphite][:host] %>
graphite_port: <%= node[:jmxtrans][:graphite][:port] %>

# Define sthe port that the hosts in this config listen for JMX on
# ** THIS PORT HAS TO BE THE SAME FOR ALL HOSTS **
query_port: 9999