Skip to content

Instantly share code, notes, and snippets.

@shawnchin
shawnchin / shmemctypes.py
Created September 16, 2011 15:09
shm_open() version of multiprocessing.sharedctypes.RawArray
#
# Based on multiprocessing.sharedctypes.RawArray
#
# Uses posix_ipc (http://semanchuk.com/philip/posix_ipc/) to allow shared ctypes arrays
# among unrelated processors
#
# Usage Notes:
# * The first two args (typecode_or_type and size_or_initializer) should work the same as with RawArray.
# * The shared array is accessible by any process, as long as tag matches.
# * The shared memory segment is unlinked when the origin array (that returned
@takluyver
takluyver / README.md
Created September 6, 2014 21:44
Flatten notebooks for git diff

Copy nbflatten.py to somewhere on $PATH. Then, in the root of a git repository, run these commands:

echo "*.ipynb diff=ipynb" >> .gitattributes 
git config diff.ipynb.textconv nbflatten.py

When you change a notebook and run git diff, you'll see the diff of flattened, simplified notebooks, rather than the full JSON. This does lose some information (metadata, non-text output), but it makes it easier to see simple changes in the notebook.

This doesn't help with merging conflicting changes in notebooks. For that, see nbdiff.org.

@tillahoffmann
tillahoffmann / cython_line_profiler.ipynb
Created February 3, 2015 14:48
Demonstration of line-by-line profiling for `cython` functions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jcrist
jcrist / trace.py
Last active September 27, 2022 15:59
Dask Execution Tracer
import os
from dask.callbacks import Callback
from dask.dot import dot_graph
class Track(Callback):
def __init__(self, path='dasks', save_every=1):
self.path = path
self.save_every = save_every
#! /usr/bin/env python3
""" Find conda packages which use a binary prefix. """
import argparse
import bz2
import json
import os
import tarfile
import urllib.request
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
#!/usr/bin/env python
"""
Inspect MACOSX deployment targets of conda packages
Uses `otool -l` and checks for LC_VERSION_MIN_MACOSX command to verify that dylibs target the right version of OS X
BY: Min RK
LICENSE: CC0, Public Domain
"""
@ericdill
ericdill / render_env.py
Last active October 20, 2021 20:22
Render the dependency graph for your conda environment (needs graphviz!)
import json
import glob
import os
from os.path import join, basename
# install this with "conda install -c conda-forge python-graphviz"
import graphviz as gv
# path to your conda environment
path = os.environ.get('CONDA_PREFIX')
if path is None:
@jkatagi
jkatagi / tif_tools.py
Last active April 25, 2023 03:28
Read GeoTiff and convert numpy.array to GeoTiff.
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import os.path
import re
from osgeo import gdal
from osgeo import gdal_array
from osgeo import osr
@ogrisel
ogrisel / numpy_pickle_protocol_5.py
Last active October 13, 2019 09:17
Draft use of pickle protocol 5 (PEP 574) for zero-copy numpy array pickling
from pickle import Pickler, load
try:
from pickle import PickleBuffer
except ImportError:
PickleBuffer = None
import copyreg
import os
import numpy as np
import time