Skip to content

Instantly share code, notes, and snippets.

@khalido
khalido / movies.py
Last active October 11, 2023 02:00
how to use ThreadPoolExecutor with map, tqdm and an iterator doing stuff as the futures are returned
# gets movie info from OpenaI
def get_movie_info(movie: str = None, debug=False):
"""returns movie info from openai as a dict"""
msg = f"""You love movies and are helping complete a movie database.
Give me a short plot summary, main actors and concise review of the movie '{movie}'. Return the results in Json format with the fields:
["summary", "review", "actors"]."""
completion = client.chat.completions.create(
model="gpt-3.5-turbo",
@khalido
khalido / ublock.md
Created June 15, 2020 01:07
[browsing the internet]

Install ublock origin from here - be wary of fake clones of it like ublock.org.

To block annoying accept these cookie popups:

ublock origin go to settings > filters list > annoyances, turn on easylist-cookies

@khalido
khalido / dealing_with_files_in_python.py
Last active June 3, 2020 10:14
[Dealing with files] #python
"""
For most purposes, the pathlib lib replaces all the with open() file stuff.
"""
from pathlib import Path
# to get the contents of a file
txt = Path(txt_file).read_text() # txt_file can be str or Path obj
bin_data = Path(binary_file).read_bytes() # binary_file can be str or Path obj
@khalido
khalido / cli.md
Last active June 11, 2020 05:49
[working on a remote server] #cli

Get size of directories in a folder sorted by size:

du -h --max-depth=1 | sort -hr

List files with humun readable file sizes:

ls -lh
@khalido
khalido / s3.py
Created May 17, 2020 23:30
[AWS boto3] Accessing misc AWS resources like buckets etc using boto3 in python #boto3 #aws
# add code here
# below code from
# https://alex.miller.im/posts/linear-model-custom-loss-function-regularization-python/
def mean_absolute_percentage_error(y_pred, y_true, sample_weights=None):
"""Mean absolute percentage error regression loss"""
y_true = np.array(y_true)
y_pred = np.array(y_pred)
assert len(y_true) == len(y_pred)
if np.any(y_true==0):
@khalido
khalido / 2019_day_10.py
Created December 12, 2019 00:09
[adventofcode] misc snippets
from collections import defaultdict
import numpy as np
inp1 = """.#..#
.....
#####
....#
...##"""
inp2 = """......#.#.
@khalido
khalido / plot_keras.py
Last active December 10, 2019 05:34
[keras] misc keras things #keras
# see https://github.com/stared/livelossplot for live plots, or tensorboard
# but for simple stuff the below is good enough
def plot_history(history, log=True):
"""takes in a keras history object and plots train and val loss and accuracy"""
# dict which stores train & val accuracy and losses over epochs
hist = history.history
fig, (ax, ax2) = plt.subplots(1,2, figsize=(13,6))
@khalido
khalido / clf_and_plot_cm.py
Created December 9, 2019 05:10
[sklearn] useful sklearn stuff #sklearn
from sklearn.model_selection import train_test_split
from sklearn import metrics # for evaluation
from sklearn.ensemble import RandomForestClassifier
# initiate a classifier and train on some data
rf = RandomForestClassifier(n_jobs=-1)
rf.fit(x_train, y_train)
# predict
y_predict = rf.predict(x_train)
@khalido
khalido / colors.py
Last active December 11, 2019 05:56
[Matplotlib] stuff I use often #viz
# make a list of colours
cmap = colors.ListedColormap(['red','white','black'])
# define a range of numbers: -0.5-0.5, 0.5-1.5, 1.5-2.5
bounds=[-0.5, 0.5, 1.5, 2.5]
# Generate a colormap index based on discrete intervals.
norm = colors.BoundaryNorm(bounds, cmap.N)
# plot