Skip to content

Instantly share code, notes, and snippets.

View ktmud's full-sized avatar
🥌
Peace and Love~

Jesse Yang ktmud

🥌
Peace and Love~
View GitHub Profile
@ktmud
ktmud / predict.py
Created September 20, 2019 19:54
Redis Cache Decorator with mini batch ability
from redis_store import store
PRED_EXPIRE_SEC = 120
@store.cache(expire=PRED_EXPIRE_SEC)
def predict(model, photo_id):
"""Predict labels for one photo"""
# each thread uses a different learner cache
@ktmud
ktmud / .block
Last active August 29, 2019 18:23 — forked from timelyportfolio/.block
vega-lite theme
license: mit
@ktmud
ktmud / dbsync.py
Created July 20, 2019 04:50
Use SQLAlchemy to move data between servers
#!/usr/bin/env python3 -W ignore::DeprecationWarning
# -*- coding: utf-8 -*-
"""
Sync GIS data from a PostgreSQL server to another
"""
import os
import click
from io import StringIO
from sqlalchemy import create_engine
@ktmud
ktmud / future_and_purrr.R
Last active July 8, 2019 06:26
Run Parallel Job in R with Future and Purrr
library(future)
library(tidyverse)
library(datasets)
library(randomForest)
library(tictoc)
plan(multisession, workers = 10)
calc_imp <- function(idx) {
Sys.sleep(1)
@ktmud
ktmud / README.md
Last active June 23, 2019 05:28
VSCode Debug Redash Flask App

This is for launching and debugging Redash Flask app in VSCode.

We couldn't use the default Flask runner, because Redash implemented its own cli command manager.

We must use "program": "${workspaceFolder}/manage.py" instead of "module": "manage" because the relative path will cause some path issue, and result in template file not found errors.

@ktmud
ktmud / README.md
Last active March 9, 2023 01:37
Enable Okta Login for Superset

This Gist demonstrates how to enable Okta/OpenID Connect login for Superset (and with slight modifications, other Flask-Appbuilder apps).

All you need to do is to update superset_config.py as following and make sure your Cliend ID (OKTA_KEY) and Secret (OKTA_SECRET) are put into the env.

@ktmud
ktmud / datetime_between.py
Created April 3, 2019 22:16
Python iterate all days between two dates
from datetime import datetime, timedelta
ONE_DAY = timedelta(days=1)
def datetime_between(start, end, interval=timedelta(days=1),
ret_dtype='datetime'):
"""All datetime between start and end.
Default interval: 1 day"""
start = datetime.strptime(start.replace('-', ''), '%Y%m%d')
end = datetime.strptime(end.replace('-', ''), '%Y%m%d')
@ktmud
ktmud / tensorflow_1_8_sierra_gpu.md
Last active April 22, 2020 14:07 — forked from hongta/tensorflow_1_7_sierra_gpu.md
Install Tensorflow 1.8 on macOS Sierra 10.12.6 with CUDA 9.0

Tensorflow 1.7 with CUDA on macOS Sierra 10.12.6, Anaconda, and Python 3.6

Largely based on the Tensorflow 1.6 gist, this should hopefully simplify things a bit. Mixing homebrew python2/python3 with pip ends up being a mess, so here's an approach to uses the built-in python27.

Requirements

  • NVIDIA Web-Drivers 378.05.05 for 10.12.6
  • CUDA 9.0 Toolkit
  • cuDNN 7.0.5 (latest release for mac os)
  • Python 3.6

Step 1: Install Python 3.6

I use anaconda:

wget https://repo.anaconda.com/archive/Anaconda3-5.3.0-MacOSX-x86_64.sh
bash ./Anaconda3-5.3.0-MacOSX-x86_64.sh
conda create -n idp intelpython3_full python=3
source activate idp
@ktmud
ktmud / data.js
Last active October 17, 2018 20:29
MBTA Data Visualization - Brushing and Linking
var dispatch = d3.dispatch("dataLoaded", "stationHovered", "nodesUpdated");
d3.json("stations.json", function(error, stations)
{
var data = {};
data.nodes = stations.nodes;
data.links = stations.links;
for(var i = 0; i < data.nodes.length; i++)
{