Skip to content

Instantly share code, notes, and snippets.

View karenyyng's full-sized avatar

Karen Ng karenyyng

View GitHub Profile
@karenyyng
karenyyng / debian_jesse_LTS_sources_list
Created July 2, 2019 21:55
debian_jesse_LTS_sources.list file
deb http://deb.debian.org/debian/ jessie main contrib non-free
deb-src http://deb.debian.org/debian/ jessie main contrib non-free
deb http://security.debian.org/ jessie/updates main contrib non-free
deb-src http://security.debian.org/ jessie/updates main contrib non-free
@karenyyng
karenyyng / multiple_github_account.md
Created May 13, 2019 03:55
How to set up and use multiple Github account correctly on the same machine

setting up authentication for each Github account

ref

config file

  • have one entry for each account in the ~/.ssh/config file

For the secondary Github account, choose a different Host value.

Host github.{SECONDARY_KEYWORD}.com
@karenyyng
karenyyng / bokeh_cheatsheet.md
Last active September 10, 2018 20:29
bokeh_cheatsheet

making simple plot

to show plot within Jupyter notebook

execute this following line at the top of the notebook

output_notebook()
@karenyyng
karenyyng / example_conda_setup_script.sh
Last active December 26, 2017 21:58
Example project conda setup script
# -------------
# Author: Karen Ng <mailto:karen.yyng@gmail.com>
# dependency: wget
# usage:
# source THIS_SCRIPT.sh
# -------------
# set per project settings
export CONDA_ENV=general
export INTEL_PYTHON_VERSION=2018.0.0
export PY_VERSION=35
@karenyyng
karenyyng / setup_conda.sh
Last active December 28, 2017 17:41
setup_conda.sh
# -------------
# Author: Karen Ng <mailto:karen.yyng@gmail.com>
# dependency: wget
# usage:
# source install_py${PYTHON_VERSION}_${INTEL_PYTHON_VERSION}.sh
#
# Copyright (c) 2017, Intel Corporation
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
@karenyyng
karenyyng / load_conda.sh
Last active October 30, 2017 20:45
load_conda.sh
if [[ -z $CONDA_ENV ]]; then
echo "CONDA_ENV is not defined. Quitting installation script."
exit 1
fi
if [[ $NERSC_HOST == "cori"* ]]; then
DIR=/global/common/cori/software/python/3.5-anaconda
module load python/3.5-anaconda
echo 'On Cori: echo finish loading python/3.5-anaconda'
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@karenyyng
karenyyng / parquet-benchmark-20170210.py
Created March 20, 2017 21:40 — forked from wesm/parquet-benchmark-20170210.py
Parquet multithreaded benchmarks
import gc
import os
import time
import numpy as np
import pandas as pd
from pyarrow.compat import guid
import pyarrow as pa
import pyarrow.parquet as pq
import snappy
@karenyyng
karenyyng / pydata_list.md
Last active April 7, 2016 03:28
Incomplete list of cool Python data-related projects

Personal list of cool Python related projects to play with. Projects are counted as long as they have a Python API regardless if the underlying code is written in another language.

Data analytics (pipelined) frameworks

  • PySpark
  • GraphLab Create
  • Scikit Learn
  • Tensorflow
  • Cloud Dataflow