Skip to content

Instantly share code, notes, and snippets.

View cast42's full-sized avatar

Lode Nachtergaele cast42

View GitHub Profile
@kylemcdonald
kylemcdonald / function-calling.ipynb
Created June 14, 2023 01:10
Example of OpenAI function calling API to extract data from LAPD newsroom articles.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@shawwn
shawwn / adam.py
Last active February 15, 2023 19:48
Reformulating Adam optimizer to gain an intuition about what it's doing.
def lerp(a, b, t):
return (b - a) * t + a
def bias(i, x, beta):
return 1 - jnp.asarray(beta, x.dtype) ** (i + 1)
@optimizer
def adam(step_size, b1=0.9, b2=0.999, eps=1e-8) -> OptimizerResult:
"""Construct optimizer triple for Adam.
@Olshansk
Olshansk / video_to_gif.sh
Last active January 12, 2023 13:35
A handy bash function to convert a video (e.g. a screen cap) to a gif using ffmpeg in your shell
function video_to_gif {
local input_video_path="$1"
local output_gif_path="$2"
local fps="${3:-10}"
local scale="${4:-1080}"
local loop="${5:-0}"
ffmpeg -i "${input_video_path}" -vf "setpts=PTS/1,fps=${fps},scale=${scale}:-2:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop $loop "${output_gif_path}"
}
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@PaoloLeonard
PaoloLeonard / expect_table_row_count_to_be_more_than_others.py
Last active October 28, 2022 11:06
Full implementation of a custom table expectation that compares the considered dataset row count to other datasets row count with the possibility of using different comparison keys.
"""
Custom table expectation which checks whether the row count is greater than the row count of other tables.
There are different ways to compare the row counts:
* With absolute values, if one row count value of the other tables is greater than the current then the validation
fails,
* With mean values, if the mean of value of the other tables row count is greater than the current row count then
the validation fails.
"""
from copy import deepcopy
@rvanbruggen
rvanbruggen / colruyt datascience homework assignment.cql
Created November 7, 2019 09:37
colruyt datascience homework assignment.cql
//colruyt datascience homework assignment in Neo4j
// from: https://github.com/MarkiesFredje/data-engineering-exercise/blob/master/data_engineer_exercise.ipynb
// json file download location: https://ecgplacesmw.colruytgroup.com/ecgplacesmw/v3/nl/places/filter/clp-places
//import into neo4j using apoc
//create indexes and constraint
create index on :Address(streetName);
create index on :City(name);
@twiecki
twiecki / dask_sparse_corr.py
Created August 17, 2018 11:26
Compute large, sparse correlation matrices in parallel using dask.
import dask
import dask.array as da
import dask.dataframe as dd
import sparse
@dask.delayed(pure=True)
def corr_on_chunked(chunk1, chunk2, corr_thresh=0.9):
return sparse.COO.from_numpy((np.dot(chunk1, chunk2.T) > corr_thresh))
def chunked_corr_sparse_dask(data, chunksize=5000, corr_thresh=0.9):

Convert PascalVOC Annotations to YOLO

This script reads PascalVOC xml files, and converts them to YOLO txt files.

Note: This script was written and tested on Ubuntu. YMMV on other OS's.

Disclaimer: This code is a modified version of Joseph Redmon's voc_label.py

Instructions:

  1. Place the convert_voc_to_yolo.py file into your data folder.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jayspeidell
jayspeidell / kaggle_download.py
Last active July 18, 2023 12:23
Sample script to download Kaggle files
# Info on how to get your api key (kaggle.json) here: https://github.com/Kaggle/kaggle-api#api-credentials
!pip install kaggle
api_token = {"username":"USERNAME","key":"API_KEY"}
import json
import zipfile
import os
with open('/content/.kaggle/kaggle.json', 'w') as file:
json.dump(api_token, file)
!chmod 600 /content/.kaggle/kaggle.json
!kaggle config path -p /content