jreuben11

GPU Sharing on GKE DaemonSet

NOTE: This is not a Google supported product.

Example Usage

1. Create a GKE cluster with a GPU node pool:

gcloud container clusters create gpu-sharing-demo --zone us-central1-c

jreuben11

I recently switched over to neovim (see my screenshots at the bottom). Below is my updated config file.

It's currently synchronized with my .vimrc config except for a block of neovim-specific terminal key mappings.

This is still a work in progress (everyone's own config is always a labor of love), but I'm already extremely pleased with how well this is working for me with neovim. While terminal mode isn't enough to make me stop using tmux, it is quite good and I like having it since it simplifies my documentation workflow for yanking terminal output to paste in a markdown buffer.

These days I primarily develop in Go. I'm super thrilled and grateful for fatih/vim-go,

jreuben11

Core

• gcloud / gcp SDK - automate everything: do NOT use the console
• Resource Manager - Cloud project metadata management https://cloud.google.com/resource-manager/docs/
• Cloud Deployment Manager - Templated infrastructure deployment https://cloud.google.com/deployment-manager/docs/ infrastructure as code. if you are GCP only, you dont need Terraform
• Cloud IAM - Resource access control https://cloud.google.com/iam/docs/
• Cloud Billing - Billing and cost-management tools https://cloud.google.com/billing/docs/

IaaS / PaaS

• Compute Engine - Virtual machines, disks, and networking https://cloud.google.com/compute/docs/ the VM heavy old way ...
• App Engine - Managed app platform https://cloud.google.com/appengine/docs/standard/go111/ - I would go with Golang flavour PaaS when starting out, then switch to GKE in growth phase

jreuben11

Graph Theory Concepts

• clique https://en.wikipedia.org/wiki/Clique_(graph_theory) - a subset of vertices of an undirected graph such that every two distinct vertices in the clique are adjacent (its induced subgraph is complete).Finding the largest clique in a graph is NP-complete problem with exponential running time see https://en.wikipedia.org/wiki/Clique_problem
• Ramsey number https://en.wikipedia.org/wiki/Ramsey%27s_theorem the minimum number of vertices, v = R(m, n), such that all undirected simple graphs of order v, contain a clique of order m, or an independent set of order n. Ramsey's theorem states that such a number exists for all m and n.
• connectivity https://en.wikipedia.org/wiki/Connectivity_(graph_theory) the minimum number of elements (nodes or edges) that need to be removed to disconnect the remaining nodes from each other. For network flow problems. The connectivity of a graph is an important measure of network resilience.
• Vertex Cover https://en.wikipedia.org/wiki/V

jreuben11

Scikit Learn

• supports numpy array, scipy sparse matrix, pandas dataframe.
• Estimator - learns from data: can be a classification, regression , clustering that extracts/filters useful features from raw data - implements set_params, fit(X,y), predict(T) , score (judge the quality of fit / predict), predict_proba (confidence level)
• Transformer - transform (reduce dimensionality)/ inverse_transform, - clean (sklearn.preprocessing), reduce dimensions (sklearn.unsupervised _reduction), expand (sklearn.kernel_approximation) or generate feature representations (sklearn.feature_extraction).

sklearn.cluster

properties: labels_, cluster_centers_. distance metrics - maximize distance between samples in different classes, and minimizes it within each class: Euclidean distance (l2), Manhattan distance (l1) - good for sparse features, cosine distance - invariant to global scalings, or any precomputed affinity matrix.

• dbscan - deterministicly separate areas of high density from

jreuben11

C++11 in a nutshell

Modern C++ Coding Style

This post is based upon the standard - not supported with VC compiler ! Try these techniques using GCC / LLVM compilers and QtCreator / CLion IDEs

Avoid C programming style idioms

• = rather than strcpy() for copying , == rather than strcmp() for comparing
• use const , constexpr functions rather than #DEFINE, macros
• Avoid void*, union, and raw casts (use static_cast instead)
• use std:string instead of char*

jreuben11

basic Machine Learning pre-requisites:

• python http://docs.python-guide.org/en/latest/intro/learning/
• Jupyter http://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/
• Scipy http://www.scipy-lectures.org/
• Pandas https://pandas.pydata.org/pandas-docs/stable/10min.html
• Scikit-Learn http://scikit-learn.org/stable/user_guide.html
• Seaborn charts https://seaborn.pydata.org/tutorial.html
• brief intro to Neural nets https://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks
• Spark MLLib https://spark.apache.org/mllib/
• Mathematical Foundations of Machine Learning: https://www.youtube.com/playlist?list=PLD0F06AA0D2E8FFBA

jreuben11

private static void jCodeModelGen(Map<String, Class<?>> mapping, String className) {


        try {
            int version = 10;
            JCodeModel codeModel = new JCodeModel();
            JPackage codeModel_Package = codeModel._package("com.example.pb");
            if (className == null || className.trim().length() == 0) {
                className = "Generated" + String.valueOf(version);
            }

jreuben11

#Redis Commands

##Generic Commands

• DEL key [key …] - Delete a key
• DUMP key - Return a serialized version of value stored at specified key
• EXISTS key - Determine if a key exists
• EXPIRE key seconds - Set a key's time to live in seconds
• EXPIREAT key timestamp - Set expiration for a key as a UNIX timestamp
• KEYS pattern - Find all keys matching given pattern
• MIGRATE host port key destination-db timeout - Atomically transfer a key from a Redis instance to another one

jreuben11

#JIT Compiler Tuning

##Hotspot Compilation Mechanism Selection

• -client - client compiler (C1) - begins compiling earlier -> optimize startup time. default for 32 bit OS, choose if heap < 3GB
• -server - server compiler (C2)- better optimize perf for long-running apps. 2 subtypes: -server 32-bit: 5-20% faster, but total process size must be < 4GB -d64 64-bit: (DEFAULT on modern machines)
• -XX:+TieredCompilation - tiered compilation (combines both client and server) - the best perf, but requires more native memory for extra compiled code. DEFAULT.

##Advanced JIT Tuning: