Skip to content

Instantly share code, notes, and snippets.

View vincentclaes's full-sized avatar
:octocat:

Vincent Claes vincentclaes

:octocat:
59 followers · 63 following
View GitHub Profile
@vincentclaes
vincentclaes / gist:7a5eb53acc05b316c86769fed25d4e1a
Created August 9, 2023 13:20
s3-eventbridge-event-object
{
"version": "0",
"id": "c9888434-815d-c6f4-9fc2-4dd7bdeccaaf",
"detail-type": "Object Created",
"source": "aws.s3",
"account": "528068194338",
"time": "2023-08-09T13:09:50Z",
"region": "eu-west-1",
"resources": [
"arn:aws:s3:::mxcd-magic-star-691-in"
@vincentclaes
vincentclaes / lifecycleconfig.sh
Last active January 28, 2023 12:20
lifecycle config for awsglue interactive sessions kernel for sagemaker notebook jupyter lab version 3
# #!/bin/bash
set -ex
sudo -u ec2-user -i <<'EOF'
conda create --name glue_pyspark python=3.9 ipykernel jupyter nb_conda -y
source activate glue_pyspark
python3 -m pip install --upgrade --user aws-glue-sessions jupyter_client
rm -r /home/ec2-user/anaconda3/envs/glue_pyspark/share/jupyter/kernels/python3
echo "Copying Glue PySpark Kernel"
@vincentclaes
vincentclaes / git-lfs.sh
Created October 13, 2022 20:15
install git lfs on amazon linux machines
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
sudo yum install git-lfs
@vincentclaes
vincentclaes / zero-shot-emoji-predictor.py
Last active September 27, 2022 10:59
Describe a snippet that predicts an emoji using pre-trained CLIP model.
# install these dependencies
# pip install torch transformers pillow
# import the dependencies
from PIL import Image
from transformers import CLIPProcessor, CLIPModel
import torch
# emoji images: https://public-assets-vincent-claes.s3.eu-west-1.amazonaws.com/emoji-precitor/emojis.zip
path_to_emoji_folder = "<path to the folder with images of emojis>"
@vincentclaes
vincentclaes / repeat-pandas-df.py
Created August 31, 2022 08:25
Repeat a pandas dataframe n times
n = 1000
df = pd.DataFrame(
np.repeat(df.to_numpy(), n, axis=0), columns=df.columns
)
@vincentclaes
vincentclaes / iris.csv
Created July 4, 2021 12:32 — forked from netj/iris.csv
sepal.length sepal.width petal.length petal.width variety
5.1 3.5 1.4 .2 Setosa
4.9 3 1.4 .2 Setosa
4.7 3.2 1.3 .2 Setosa
4.6 3.1 1.5 .2 Setosa
5 3.6 1.4 .2 Setosa
5.4 3.9 1.7 .4 Setosa
4.6 3.4 1.4 .3 Setosa
5 3.4 1.5 .2 Setosa
4.4 2.9 1.4 .2 Setosa
@vincentclaes
vincentclaes / bulk-delete-aws-resources.md
Last active February 17, 2021 07:33
bulk delete aws resources (stepfunctions / lambda / ....)

https://stackoverflow.com/a/65754583/1771155

stepfunctions

export PROFILE=some-profile
aws stepfunctions list-state-machines --region eu-central-1 --profile $PROFILE | jq -r '.stateMachines | .[] | .stateMachineArn' |
while read uname1; do
echo "Deleting $uname1";
aws stepfunctions delete-state-machine --region eu-central-1 --state-machine-arn $uname1 --profile $PROFILE;
@vincentclaes
vincentclaes / api-gateway-event-example.json
Created February 3, 2021 08:09
example of an AWS api gateway event that a lambda receives
{
"resource": "/",
"path": "/",
"httpMethod": "GET",
"headers": {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en-US,en;q=0.9",
"cookie": "s_fid=7AAB6XMPLAFD9BBF-0643XMPL09956DE2; regStatus=pre-register",
"Host": "70ixmpl4fl.execute-api.us-east-2.amazonaws.com",
@vincentclaes
vincentclaes / login-to-stackoverflow.py
Last active April 22, 2023 11:40
Schedule an AWS lambda with python 3.8 to run each day and logs in to stackoverflow. Put your email ("EMAIL") and password ("PASS") in the environment variables. lf you visit https://stackoverflow.com/ for 30 consecutive days, you can earn Enthusiast badge. And 100 days, Fanatic badge.
import os
import re
import datetime
import json
import urllib3
from urllib.parse import urlencode
def lambda_handler(event, context):
stack_overflow = StackOverflow()
@vincentclaes
vincentclaes / serverless-permissions-for-prod.json
Created April 7, 2020 11:02
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"cloudformation:CreateUploadBucket",
"cloudformation:CancelUpdateStack",
"cloudformation:CreateStack",
"cloudformation:DeleteStack",