Skip to content

Instantly share code, notes, and snippets.

View jkmacc-LANL's full-sized avatar

jkmacc-LANL

  • Los Alamos National Laboratory
  • X @jkmacc
View GitHub Profile
@jacobtomlinson
jacobtomlinson / Dask on Fargate from scratch.ipynb
Last active February 26, 2024 14:35
Dask on Fargate from scratch
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jakevdp
jakevdp / Parsimonious-vs-Parsley.ipynb
Last active December 8, 2016 04:58
Testing Parsley vs Parsimonious
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
def sliding_window(data, size, stepsize=1, padded=False, axis=-1, copy=True):
"""
Calculate a sliding window over a signal
Parameters
----------
data : numpy array
The array to be slided over.
size : int
The sliding window size
@krischer
krischer / embarrassingly_parallel.py
Created June 21, 2015 09:38
Embarrassingly Parallel Code with MPI In Python
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
Simple script illustrating how to perform embarrassingly parallel computations
in Python using MPI/mpi4py. I like this approach a lot as its very easy to get
right without having to deal with the complications arising from forked
processes which the multiprocessing module uses.
This script can be executed with or without `mpirun`; it will just run on one
core if not executed with it. With some more logic its also possible to make
# Oracle
export ORACLE_HOME=~/bin/oracle/instantclient_11_2
export PATH=$ORACLE_HOME:$PATH
export DYLD_LIBRARY_PATH=$ORACLE_HOME
export LD_LIBRARY_PATH=$ORACLE_HOME
@dan-blanchard
dan-blanchard / .1.miniconda.md
Last active December 11, 2019 22:38
Quicker Travis builds that rely on numpy and scipy using Miniconda

For ETS's SKLL project, we found out the hard way that Travis-CI's support for numpy and scipy is pretty abysmal. There are pre-installed versions of numpy for some versions of Python, but those are seriously out of date, and scipy is not there are at all. The two most popular approaches for working around this are to (1) build everything from scratch, or (2) use apt-get to install more recent (but still out of date) versions of numpy and scipy. Both of these approaches lead to longer build times, and with the second approach, you still don't have the most recent versions of anything. To circumvent these issues, we've switched to using Miniconda (Anaconda's lightweight cousin) to install everything.

A template for installing a simple Python package that relies on numpy and scipy using Miniconda is provided below. Since it's a common s

# This is a really old post, in the comments (and stackoverflow too) you'll find better solutions.
def find(key, dictionary):
for k, v in dictionary.iteritems():
if k == key:
yield v
elif isinstance(v, dict):
for result in find(key, v):
yield result
elif isinstance(v, list):