jreuben11

#System Architecture patterns

• N-Tier
• Event-Driven - Mediator / Broker
• Microkernel
• MicroServices
• Blackboard
• client - MVC / MVP / MVVM
• server - RPC / Remoting / WS / SOA / REST
• Space-Based

jreuben11

in a nutshell: fit trainingData (train a model), transform testData (predict with model)

• Transformer: DataFrame => DataFrame
• Estimator: DataFrame => Transformer

#Transformers

• Tokenizer: sentence => words
• RegexTokenizer: sentence => words - setPattern
• HashingTF: terms => feature vectors based on frequency - setNumFeatures

jreuben11

#JIT Compiler Tuning

##Hotspot Compilation Mechanism Selection

• -client - client compiler (C1) - begins compiling earlier -> optimize startup time. default for 32 bit OS, choose if heap < 3GB
• -server - server compiler (C2)- better optimize perf for long-running apps. 2 subtypes: -server 32-bit: 5-20% faster, but total process size must be < 4GB -d64 64-bit: (DEFAULT on modern machines)
• -XX:+TieredCompilation - tiered compilation (combines both client and server) - the best perf, but requires more native memory for extra compiled code. DEFAULT.

##Advanced JIT Tuning:

jreuben11

#Redis Commands

##Generic Commands

• DEL key [key …] - Delete a key
• DUMP key - Return a serialized version of value stored at specified key
• EXISTS key - Determine if a key exists
• EXPIRE key seconds - Set a key's time to live in seconds
• EXPIREAT key timestamp - Set expiration for a key as a UNIX timestamp
• KEYS pattern - Find all keys matching given pattern
• MIGRATE host port key destination-db timeout - Atomically transfer a key from a Redis instance to another one

jreuben11

private static void jCodeModelGen(Map<String, Class<?>> mapping, String className) {


        try {
            int version = 10;
            JCodeModel codeModel = new JCodeModel();
            JPackage codeModel_Package = codeModel._package("com.example.pb");
            if (className == null || className.trim().length() == 0) {
                className = "Generated" + String.valueOf(version);
            }

jreuben11

basic Machine Learning pre-requisites:

• python http://docs.python-guide.org/en/latest/intro/learning/
• Jupyter http://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/
• Scipy http://www.scipy-lectures.org/
• Pandas https://pandas.pydata.org/pandas-docs/stable/10min.html
• Scikit-Learn http://scikit-learn.org/stable/user_guide.html
• Seaborn charts https://seaborn.pydata.org/tutorial.html
• brief intro to Neural nets https://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks
• Spark MLLib https://spark.apache.org/mllib/
• Mathematical Foundations of Machine Learning: https://www.youtube.com/playlist?list=PLD0F06AA0D2E8FFBA

jreuben11

C++11 in a nutshell

Modern C++ Coding Style

This post is based upon the standard - not supported with VC compiler ! Try these techniques using GCC / LLVM compilers and QtCreator / CLion IDEs

Avoid C programming style idioms

• = rather than strcpy() for copying , == rather than strcmp() for comparing
• use const , constexpr functions rather than #DEFINE, macros
• Avoid void*, union, and raw casts (use static_cast instead)
• use std:string instead of char*

jreuben11

Scikit Learn

• supports numpy array, scipy sparse matrix, pandas dataframe.
• Estimator - learns from data: can be a classification, regression , clustering that extracts/filters useful features from raw data - implements set_params, fit(X,y), predict(T) , score (judge the quality of fit / predict), predict_proba (confidence level)
• Transformer - transform (reduce dimensionality)/ inverse_transform, - clean (sklearn.preprocessing), reduce dimensions (sklearn.unsupervised _reduction), expand (sklearn.kernel_approximation) or generate feature representations (sklearn.feature_extraction).

sklearn.cluster

properties: labels_, cluster_centers_. distance metrics - maximize distance between samples in different classes, and minimizes it within each class: Euclidean distance (l2), Manhattan distance (l1) - good for sparse features, cosine distance - invariant to global scalings, or any precomputed affinity matrix.

• dbscan - deterministicly separate areas of high density from

jreuben11

Graph Theory Concepts

• clique https://en.wikipedia.org/wiki/Clique_(graph_theory) - a subset of vertices of an undirected graph such that every two distinct vertices in the clique are adjacent (its induced subgraph is complete).Finding the largest clique in a graph is NP-complete problem with exponential running time see https://en.wikipedia.org/wiki/Clique_problem
• Ramsey number https://en.wikipedia.org/wiki/Ramsey%27s_theorem the minimum number of vertices, v = R(m, n), such that all undirected simple graphs of order v, contain a clique of order m, or an independent set of order n. Ramsey's theorem states that such a number exists for all m and n.
• connectivity https://en.wikipedia.org/wiki/Connectivity_(graph_theory) the minimum number of elements (nodes or edges) that need to be removed to disconnect the remaining nodes from each other. For network flow problems. The connectivity of a graph is an important measure of network resilience.
• Vertex Cover https://en.wikipedia.org/wiki/V

jreuben11

Core

• gcloud / gcp SDK - automate everything: do NOT use the console
• Resource Manager - Cloud project metadata management https://cloud.google.com/resource-manager/docs/
• Cloud Deployment Manager - Templated infrastructure deployment https://cloud.google.com/deployment-manager/docs/ infrastructure as code. if you are GCP only, you dont need Terraform
• Cloud IAM - Resource access control https://cloud.google.com/iam/docs/
• Cloud Billing - Billing and cost-management tools https://cloud.google.com/billing/docs/

IaaS / PaaS

• Compute Engine - Virtual machines, disks, and networking https://cloud.google.com/compute/docs/ the VM heavy old way ...
• App Engine - Managed app platform https://cloud.google.com/appengine/docs/standard/go111/ - I would go with Golang flavour PaaS when starting out, then switch to GKE in growth phase