Skip to content

Instantly share code, notes, and snippets.

View joefutrelle's full-sized avatar

Joe Futrelle joefutrelle

  • Falmouth, MA
View GitHub Profile
@joefutrelle
joefutrelle / MATLAB_datetimes.py
Created April 26, 2022 16:15
Convert MATLAB datenums to/from Python datetimes
from datetime import datetime, timedelta
# util for converting MATLAB datenum to Python datetime
# as described in http://stackoverflow.com/questions/13965740/converting-matlabs-datenum-format-to-python
def datenum2datetime(matlab_datenum):
dt = datetime.fromordinal(int(matlab_datenum)) + timedelta(days=matlab_datenum%1) - timedelta(days = 366)
return dt
# util for converting datetime to MATLAB datenum
# as described in http://stackoverflow.com/questions/8776414/python-datetime-to-matlab-datenum
@joefutrelle
joefutrelle / python_sqlite.ipynb
Created June 1, 2021 16:28
Working with SQLite in Python
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@joefutrelle
joefutrelle / binpack2.py
Last active September 15, 2021 02:25
Numba implementation of Guillotine bin packing
import numba as nb
from numba.typed import List
NO_ID = -1
DOESNT_FIT = 99999999
NO_SECTION = (0, 0, 0, 0, DOESNT_FIT)
@nb.jitclass([
('x', nb.int32),
('y', nb.int32),
@joefutrelle
joefutrelle / class_scores.py
Last active December 12, 2019 17:34
save/load IFCB class scores
import h5py as h5
import numpy as np
from scipy.io import savemat
def save_class_scores(path, bin_id, scores, roi_numbers, class_labels):
assert scores.shape[0] == len(roi_numbers), 'wrong number of ROI numbers'
assert scores.shape[1] == len(class_labels), 'wrong number of class labels'
with h5.File(path,'w') as f:
ds = f.create_dataset('scores', data=scores, compression='gzip', dtype='f4')
@joefutrelle
joefutrelle / tpj.pl
Last active September 18, 2023 16:34
Obfuscated Perl contest winner from 1997 (note: no longer works)
package S2z8N3;{
$zyp=S2z8N3;use Socket;
(S2z8N3+w1HC$zyp)&
open SZzBN3,"<$0"
;while(<SZzBN3>){/\s\((.*p\))&/
&&(@S2zBN3=unpack$age,$1)}foreach
$zyp(@S2zBN3){
while($S2z8M3++!=$zyp-
30){$_=<SZz8N3>}/^(.)/|print $1
;$S2z8M3=0}s/.*//|print}sub w1HC{$age=c17
@joefutrelle
joefutrelle / wide_to_long.py
Last active March 1, 2019 18:06
utility for wide to long conversion of Pandas dataframes
def wide_to_long(df, wide_cols_list, value_cols, long_col, long_labels):
"""converts selected columns from wide to long format. params:
- df: the input dataframe
- wide_cols_list: for each set of wide columns, a list of their names
- value_cols: for each set of wide columns, the name of the long column to hold the values
- long_col: the name of the column to indicate which set of wide columns the value comes from
- long_labels: for each set of wide columns, what to call it in the long_col values.
For example if I have the following DataFrame:
@joefutrelle
joefutrelle / install-docker.sh
Last active February 1, 2019 13:35
Install Docker in Debian
# current as of 2018-12-10
# install docker-ce: https://docs.docker.com/install/linux/docker-ce/debian/#install-docker-ce-1
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable"
sudo apt-get update
@joefutrelle
joefutrelle / parse_btl_file.ipynb
Last active February 5, 2024 11:55
Parse Seabird CTD .btl file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@joefutrelle
joefutrelle / dataset.xml
Last active April 13, 2018 14:52
Convert nutrient data from mat export fmt to NetCDF
<dataset type="EDDTableFromNcCFFiles" datasetID="lter-nutrient" active="true">
<reloadEveryNMinutes>10080</reloadEveryNMinutes>
<updateEveryNMillis>10000</updateEveryNMillis>
<fileDir>/home/vagrant/lter-poc/output/</fileDir>
<fileNameRegex>.*\.nc</fileNameRegex>
<recursive>true</recursive>
<pathRegex>.*</pathRegex>
<metadataFrom>last</metadataFrom>
<preExtractRegex></preExtractRegex>
<postExtractRegex></postExtractRegex>
@joefutrelle
joefutrelle / aaa_nut_reps2netcdf.py
Last active May 1, 2018 17:54
create CF compliant NetCDF files from MVCO nutrient data
import os
from scipy.io import loadmat
import pandas as pd
from pocean.dsg.timeseriesProfile.om import OrthogonalMultidimensionalTimeseriesProfile as OMTP
MAT_FILE = '/vagrant/nut_data_reps.mat'
OUT_DIR = './output'
mat = loadmat(MAT_FILE, squeeze_me=True)