Skip to content

Instantly share code, notes, and snippets.

View javierluraschi's full-sized avatar
👨‍💻
Coding @hal9ai

Javier Luraschi javierluraschi

👨‍💻
Coding @hal9ai
441 followers · 11 following
View GitHub Profile
@javierluraschi
javierluraschi / spark-keras-image-classify.md
Last active January 20, 2020 17:01
Image classification with Spark and Keras

This script makes use of an Amazon EMR cluster with one master and three nodes:

install.packages("sparklyr")
install.packages("keras")

sc <- spark_connect(
  master = "yarn",
  spark_home = "/usr/lib/spark/",
  config = list("sparklyr.apply.env.WORKON_HOME"  = "/tmp/.virtualenvs"))
@javierluraschi
javierluraschi / spark-tensorflow-barrier-emr-setup.md
Last active January 10, 2020 22:54
Initial setup to use Spark and TensorFlow Distributed using EMR

This script requires an Amazon EMR cluster with one master and three nodes:

library(sparklyr)
sc <- spark_connect(master = "yarn", spark_home = "/usr/lib/spark/", config = list(
    spark.dynamicAllocation.enabled = FALSE,
    `sparklyr.shell.executor-cores` = 8,
    `sparklyr.shell.num-executors` = 3,
    sparklyr.apply.env.WORKON_HOME = "/tmp/.virtualenvs"))
@javierluraschi
javierluraschi / multiverse-youtube-mlflow-intro.R
Created January 4, 2020 08:03
Multiverse YouTube MLflow Intro
library(glmnet)
wine_file <- pins::pin("https://raw.githubusercontent.com/rstudio/mlflow-example/master/wine-quality.csv")
train <- read.csv(wine_file)
train_x <- as.matrix(train[, !(names(train) == "quality")])
train_y <- train[, "quality"]
alpha <- mlflow_log_param("alpha", 0.7, "numeric")
@javierluraschi
javierluraschi / karpathys-tweets-rstats.R
Created December 11, 2019 01:57
Karpathy's Tweets per Month
library(tidyverse)
rtweet::get_timeline(rtweet::as_userid("karpathy"), n = 1000) %>%
mutate(year = as.character(lubridate::year(as.Date(created_at))),
month = lubridate::month(as.Date(created_at))) %>%
group_by(year, month) %>%
count(year, month) %>%
ggplot(aes(month, n, color = year)) + geom_point() +
geom_smooth(method = "lm", alpha = .15, aes(fill = year)) +
ggtitle("Karpathy's Tweets per Month")
@javierluraschi
javierluraschi / multiverse-youtube-slide.Rmd
Last active December 2, 2019 23:22
Multiverse YouTube Slide
---
title: "Using GPUs with TensorFlow and Docker"
subtitle: Multiverse Video Series
author: Javier Luraschi
output:
revealjs::revealjs_presentation:
df_print: paged
self_contained: true
theme: moon
---
@javierluraschi
javierluraschi / multiverse.md
Last active June 14, 2019 05:11
Multiverse Hex 
library(tidyverse)
library(rayrender)
set.seed(2019)

spheres <- expand.grid(y = (1:50) / 5 - 5, z = (1:50) / 5 - 5) %>%
  tibble::as_tibble()

noise <- ambient::noise_perlin(c(100, 100, 1))
@javierluraschi
javierluraschi / sparklyr-k8s-google-clouid.md
Last active February 2, 2020 17:18
Using sparklyr with Kubernetes and Google Cloud

Adapted for sparklyr from cloud.google.com/solutions/spark-on-kubernetes-engine:

First, configure the cluster using Google Cloud client and kubectl:

gcloud config set compute/zone us-central1-f
gcloud container clusters create spark-on-gke --machine-type n1-standard-2

Next we need to bind the cluster admin to your email (you can retrieve your account email with gcloud config get-value account if needed):

@javierluraschi
javierluraschi / simple-javascript-turorial.md
Created April 23, 2019 18:30
Simple JavaScript Intro 
<html>
    <head>
        <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.0/jquery.min.js"></script>
        <script type="text/javascript">
            function message(msg) {
                // innerHTML
                // document.getElementById("mymessage").innerText = document.getElementById("mymessage").innerText + " <b>" + msg + "</b>";
                $("#mymessage").text($("#mymessage").text() + " " + msg);
 }
@javierluraschi
javierluraschi / rstats-rayrender-spark.md
Last active July 10, 2019 21:30
Distributing Ray Tracing in Spark and R using rayrender

This script uses an Amazon EMR Spark cluster with 10 m3.xlarge instances.

Installing

install.packages(devtools)
install.packages(sparklyr)
devtools::install_github("tylermorganwall/rayrender")

system2("hadoop", args = c("fs", "-mkdir", "/rendering"))
@javierluraschi
javierluraschi / mesos-initialize-osx.md
Created March 12, 2019 02:24
Install Mesos in OS X 
brew update
brew install mesos

/usr/local/Cellar/mesos/1.6.1/sbin/mesos-master --registry=in_memory --ip=127.0.0.1

mkdir temp
cd temp