Skip to content

Instantly share code, notes, and snippets.

View ottobricks's full-sized avatar

Otto von Sperling ottobricks

View GitHub Profile
@ottobricks
ottobricks / sagemaker_lifecycle_pyspark3_python39_oncreate.sh
Created February 10, 2021 00:00
SageMaker Lifecycle for Python3.9 + PySpark3.0.1
#!/bin/bash
# This script will create a SageMaker instance with a custom Python and Jupyter kernel,
# in addition to a PySpark3.0.1
set -e
sudo -u ec2-user -i <<'EOF'
mkdir -p /home/ec2-user/SageMaker/kernels
conda update -n base conda --yes
@ottobricks
ottobricks / download_large_files_from_gdrive.py
Last active November 30, 2020 16:51
Download large files from Google Drive using Python
import io
import os
import pickle
import sys, argparse
from googleapiclient.discovery import build
from google.auth.transport.requests import Request
from googleapiclient.http import MediaIoBaseDownload
from google_auth_oauthlib.flow import InstalledAppFlow
# If modifying these scopes, delete the file token.pickle.