Skip to content

Instantly share code, notes, and snippets.

View spark2010's full-sized avatar

Ken Park spark2010

View GitHub Profile
@spark2010
spark2010 / execute_run_models_in_remote_instances.py
Created June 2, 2018 11:01
Execute run_models.R in remote AWS instances
import boto3
import time
import subprocess
from multiprocessing import Pool
instance_types = [
't2.micro', 't2.small', 't2.medium', 't2.large', 't2.xlarge', 't2.2xlarge',
'm4.large', 'm4.xlarge', 'm4.2xlarge', 'm4.4xlarge', 'm4.10xlarge', 'm4.16xlarge',
'c4.large', 'c4.xlarge', 'c4.2xlarge', 'c4.4xlarge', 'c4.8xlarge',
@spark2010
spark2010 / run_models.py
Last active June 2, 2018 11:01
Run DRF models in one AWS instance
mem_size <- round(as.numeric(system("awk '/MemFree/ {print $2}' /proc/meminfo", intern=TRUE)) / 1024 * 0.9, digits=0)
library(h2o)
library(Hmisc)
h2o.init(min_mem_size=paste(as.character(mem_size), "M", sep=""))
test <- h2o.importFile("higgs_test_5k.csv")
y <- "response"
test[,y] <- as.factor(test[,y])
num_rows <- 500 #h2o.nrow(test)
@spark2010
spark2010 / build_models.R
Created June 2, 2018 10:56
Building DRF models with ntrees=50, 500, 5000
# Code reference: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/stacked-ensembles.html
library(h2o)
h2o.init()
# Import a sample binary outcome train set into H2O
train <- h2o.importFile("https://s3.amazonaws.com/erin-data/higgs/higgs_train_10k.csv")
# Identify predictors and response
y <- "response"
x <- setdiff(names(train), y)
@spark2010
spark2010 / run_runnable.rb
Created October 20, 2017 08:39
Demonstration of calling an API hosted in Knowru using Ruby
require 'uri'
require 'net/http'
uri = URI('https://www.knowru.com/api/runnable/your-is-prime-api-name/run/')
http = Net::HTTP.new(uri.host, uri.port)
req = Net::HTTP::Post.new(uri, 'Authorization' => 'Token your-token', 'Content-Type' => 'application/json', 'Accept' => 'application/json')
req.body = {input: {number: 3}}.to_json
res = http.request(req)
@spark2010
spark2010 / run_runnable.sh
Last active October 20, 2017 08:29
Demonstration of calling an API hosted in Knowru using curl
curl -X POST \
-d '{"input": {"number": 3}}' \
-H "Authorization: Token your-token" \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
https://www.knowru.com/api/runnable/your-is-prime-api-name/run/
@spark2010
spark2010 / run_runnable.R
Last active October 20, 2017 08:25
Demonstration of calling an API hosted in Knowru using R
install.packages("httr")
library("httr")
url <- "https://www.knowru.com/api/runnable/your-is-prime-api-name/run/"
body <- list(input=list(number=3))
response <- POST(url, add_headers(Authorization="Token your-token"), accept_json(), content_type_json(), body=body, encode="json")
content(response)$output
# TRUE
@spark2010
spark2010 / run_runnable.py
Last active October 20, 2017 08:26
Demonstration of calling an API hosted in Knowru using Python
# after running this command: pip install knowru_client
import knowru_client
kc = KnowruClient("your-token")
kc.run_runnable("my-is-prime-api-name", {"input": {"number": 3}})
# True
@spark2010
spark2010 / knowledge.R
Last active October 20, 2017 07:36
R knowledge file for an API deciding whether a number is prime or not
is.prime <- function(number) {
if (number %in% c(2, 3)) { return (TRUE) }
for (i in 2:(as.integer(sqrt(number)))) { if (number %% i == 0) return (FALSE) }
return (TRUE)
}
run <- function(data) {
return (is.prime(data$number))
}
@spark2010
spark2010 / knowledge.py
Last active October 20, 2017 07:35
Python knowledge file for an API deciding whether a number is prime or not
import math
def is_prime(number):
for i in range(2, int(math.sqrt(number))+1):
if number % i == 0:
return False
return True
def run(data):
return is_prime(data['number'])