Skip to content

Instantly share code, notes, and snippets.

Avatar

Josh Reuben jreuben11

View GitHub Profile
@jreuben11
jreuben11 / EIP.md
Last active Mar 9, 2016
Enterprise Integration Patterns - Refresher
View EIP.md

#System Architecture patterns

  • N-Tier
  • Event-Driven - Mediator / Broker
  • Microkernel
  • MicroServices
  • Blackboard
  • client - MVC / MVP / MVVM
  • server - RPC / Remoting / WS / SOA / REST
  • Space-Based
@jreuben11
jreuben11 / SparkML-QuickRef.md
Last active Mar 24, 2016
Spark.ml Pipelines QuickRef
View SparkML-QuickRef.md

in a nutshell: fit trainingData (train a model), transform testData (predict with model)

  • Transformer: DataFrame => DataFrame
  • Estimator: DataFrame => Transformer

#Transformers

  • Tokenizer: sentence => words
  • RegexTokenizer: sentence => words - setPattern
  • HashingTF: terms => feature vectors based on frequency - setNumFeatures
View JVM_Tuning.md

#JIT Compiler Tuning

##Hotspot Compilation Mechanism Selection

  • -client - client compiler (C1) - begins compiling earlier -> optimize startup time. default for 32 bit OS, choose if heap < 3GB
  • -server - server compiler (C2)- better optimize perf for long-running apps. 2 subtypes: -server 32-bit: 5-20% faster, but total process size must be < 4GB -d64 64-bit: (DEFAULT on modern machines)
  • -XX:+TieredCompilation - tiered compilation (combines both client and server) - the best perf, but requires more native memory for extra compiled code. DEFAULT.

##Advanced JIT Tuning:

View Redis-Commands.md

#Redis Commands

##Generic Commands

  • DEL key [key …] - Delete a key
  • DUMP key - Return a serialized version of value stored at specified key
  • EXISTS key - Determine if a key exists
  • EXPIRE key seconds - Set a key's time to live in seconds
  • EXPIREAT key timestamp - Set expiration for a key as a UNIX timestamp
  • KEYS pattern - Find all keys matching given pattern
  • MIGRATE host port key destination-db timeout - Atomically transfer a key from a Redis instance to another one
View pojobuilder.java
private static void jCodeModelGen(Map<String, Class<?>> mapping, String className) {
try {
int version = 10;
JCodeModel codeModel = new JCodeModel();
JPackage codeModel_Package = codeModel._package("com.example.pb");
if (className == null || className.trim().length() == 0) {
className = "Generated" + String.valueOf(version);
}
@jreuben11
jreuben11 / C++_quickref.md
Last active Apr 11, 2018
C++11 in a nutshell
View C++_quickref.md

C++11 in a nutshell

Modern C++ Coding Style

This post is based upon the standard - not supported with VC compiler ! Try these techniques using GCC / LLVM compilers and QtCreator / CLion IDEs

Avoid C programming style idioms

  • = rather than strcpy() for copying , == rather than strcmp() for comparing
  • use const , constexpr functions rather than #DEFINE, macros
  • Avoid void*, union, and raw casts (use static_cast instead)
  • use std:string instead of char*
View skleanr_quickref.md

Scikit Learn

  • supports numpy array, scipy sparse matrix, pandas dataframe.
  • Estimator - learns from data: can be a classification, regression , clustering that extracts/filters useful features from raw data - implements set_params, fit(X,y), predict(T) , score (judge the quality of fit / predict), predict_proba (confidence level)
  • Transformer - transform (reduce dimensionality)/ inverse_transform, - clean (sklearn.preprocessing), reduce dimensions (sklearn.unsupervised _reduction), expand (sklearn.kernel_approximation) or generate feature representations (sklearn.feature_extraction).

sklearn.cluster

properties: labels_, cluster_centers_. distance metrics - maximize distance between samples in different classes, and minimizes it within each class: Euclidean distance (l2), Manhattan distance (l1) - good for sparse features, cosine distance - invariant to global scalings, or any precomputed affinity matrix.

  • dbscan - deterministicly separate areas of high density from
View graph_theory_concepts.md

Graph Theory Concepts

@jreuben11
jreuben11 / gcp_guidance.md
Last active Nov 13, 2018
GCP Cloud Architecture
View gcp_guidance.md

Core

IaaS / PaaS