Skip to content

Instantly share code, notes, and snippets.

View cosmincatalin's full-sized avatar
👻

Cosmin Catalin SANDA cosmincatalin

👻
View GitHub Profile
@cosmincatalin
cosmincatalin / install-jupyter.sh
Last active April 17, 2023 14:23
AWS EMR bootstraps to install Jupyter (R, SparkR, Python 2, Python 3, PySpark)
#!/bin/bash
MINICONDA_VERSION="4.3.21"
PANDAS_VERSION="0.20.3"
SCIKIT_VERSION="0.19.0"
while [[ $# > 1 ]]; do
key="$1"
case $key in
@cosmincatalin
cosmincatalin / readme.md
Last active October 27, 2022 11:07
AWS EMR bootstrap to install R packages from CRAN

AWS EMR bootstrap to install R packages from CRAN

This bootstrap is useful if you want to deploy SparkR applications that run arbitrary code on the EMR cluster's workers. The R code will need to have its dependencies already installed on each of the workers, and will fail otherwise. This is the case if you use functions such as gapply or dapply.

How to use the bootstrap

  1. You will first have to download the gist to a file and then upload it to S3 in a bucket of your choice.
  2. Using the AWS EMR Console create a cluster and choose advanced options.
  3. In Step 3 you can configure your bootstraps. Choose to Configure and add a Custom action
@cosmincatalin
cosmincatalin / install-rstudio-server.sh
Last active October 27, 2022 11:07
AWS EMR bootstrap to install RStudio Server along with sparklyr
#!/bin/bash
# These variables can be overwritten using the arguments below
VERSION="1.1.463"
# drwho is listed as user in YARN's Resource Manager UI.
USER="drwho"
# Depending on where the EMR cluster lives, you might have to change this to avoid security issues.
# To change the default password (and user), use the arguments bellow.
# If the cluster is not visible on the Internet, you can just leave the defaults for convenience.
PASS="tardis"
@cosmincatalin
cosmincatalin / Shiny server with OAuth2.md
Last active October 27, 2022 10:59
Shiny community server with OAuth on Amazon EC2

Shiny community server with OAuth on Amazon EC2

Description

This is detailed description of how to set-up a Shiny server (community edition) on an AWS EC2 instance, behind an OAuth2 security layer (Google OAuth2 in this case, but it can be something else).

Prerequisites

  • A Google Developer account
  • An Amazon Web Services account
@cosmincatalin
cosmincatalin / notebook.ipynb
Last active August 18, 2020 21:42
The SageMaker attached notebook that allows building an MXNet model that counts shapes in an image
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@cosmincatalin
cosmincatalin / voice-recognition-sagemaker-script.py
Last active February 9, 2020 19:37
Voice Recognition Sagemaker Script (Part1)
import base64
import glob
import json
import logging
import subprocess
import sys
import tarfile
import traceback
import uuid
import wave
@cosmincatalin
cosmincatalin / mxnet-onnx-sagemaker-script.py
Created December 6, 2018 12:15
Training script used by SageMaker
import argparse
import logging
import os
from pickle import load
import mxnet as mx
import numpy as np
from mxnet import autograd, nd, gluon
from mxnet.contrib import onnx as onnx_mxnet
from mxnet.gluon.loss import L2Loss
@cosmincatalin
cosmincatalin / MXNet to ONNX.ipynb
Created December 6, 2018 12:11
Modeling of an MXNet regression problem
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@cosmincatalin
cosmincatalin / voice-recognition-sagmaker-notebook.ipynb
Created May 31, 2018 11:21
Voice Recognition SageMaker Notebook
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@cosmincatalin
cosmincatalin / voice-recognition-sagemaker-script.py
Created May 31, 2018 09:20
Voice Recognition Sagemaker Script (Full)
import base64
import glob
import json
import logging
import subprocess
import sys
import tarfile
import traceback
import uuid
import wave