Skip to content

Instantly share code, notes, and snippets.

@hsm207
hsm207 / setup_storage.sh
Created April 28, 2020 18:33
Setup dynamic FSx lustre on EKS
#!/bin/bash
# from https://aws.amazon.com/blogs/opensource/using-fsx-lustre-csi-driver-amazon-eks/
# and
# from https://github.com/kubernetes-sigs/aws-fsx-csi-driver
pushd /tmp
# create an IAM policy to allow FSx use
cat >policy.json <<EOF
{
@hsm207
hsm207 / modify_deploy.yaml
Created April 28, 2020 19:00
Modify a deployment from a template
kubectl create deployment --image=jupyter/tensorflow-notebook:latest jupyter --dry-run -o yaml > /tmp/deploy.yaml
code-insiders /tmp/deploy.yaml
kubectl apply -f /tmp/deploy.yaml
@hsm207
hsm207 / jupyter_with_fsx.yaml
Created April 28, 2020 19:07
Deploy jupyter container with fsx mounted
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: jupyter
name: jupyter
spec:
replicas: 1
selector:
@hsm207
hsm207 / fsx_pvclaim_kf.yaml
Created April 28, 2020 20:45
FSx on Kubeflow
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: fsx-claim-kf
namespace: anonymous
spec:
accessModes:
- ReadWriteMany
storageClassName: fsx-sc
resources:
@hsm207
hsm207 / cleanup.sh
Created April 28, 2020 21:39
Cleanup after playing with EKS and kubeflow
#!/bin/bash
# delete the jupyter deployment
kubectl delete -f /tmp/deploy.yaml
# delete the FSx filesystems
kubectl delete -f /tmp/claim.yaml
kubectl delete -f /tmp/claim-kf.yaml
kubectl delete -f /tmp/storage-class.yaml
kubectl delete -k github.com/kubernetes-sigs/aws-fsx-csi-driver/deploy/kubernetes/overlays/stable/?ref=master
@hsm207
hsm207 / multicondition.ipynb
Created May 7, 2020 16:16
multi condition column in pandas
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@hsm207
hsm207 / 01_create_bundle.ipynb
Created November 14, 2020 08:08
Zipline futures bug
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@hsm207
hsm207 / puzzle.ipynb
Last active October 4, 2021 22:23
Explain how pandas.core.groupby.GroupBy.apply decides the return type
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@hsm207
hsm207 / txt2vec-openai-nearText.ipynb
Created January 20, 2023 13:02
davinci-003 vs ada-002
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.