Skip to content

Instantly share code, notes, and snippets.

View lucasb-eyer's full-sized avatar
🤔
{D,R}L Research @ Google Brain Zürich

Lucas Beyer lucasb-eyer

🤔
{D,R}L Research @ Google Brain Zürich
View GitHub Profile
@ed-alertedh
ed-alertedh / validate_tfrecords.py
Last active April 16, 2024 18:57
Utility functions to check for corruption in tfrecord files
import tensorflow as tf
def validate_dataset(filenames, reader_opts=None):
"""
Attempt to iterate over every record in the supplied iterable of TFRecord filenames
:param filenames: iterable of filenames to read
:param reader_opts: (optional) tf.python_io.TFRecordOptions to use when constructing the record iterator
"""
i = 0
# ASCII Unit separator, purpose is to separate subrecords, etc...
set __assoc_US \31
# Groks name[key] and sets varname and key variables
function assoc._parsename -S -a raw
set -l data (string match -r '(.+)\[(.+)\]' $raw)
if [ (count $data) -lt 3 ]
echo "Invalid assocatiave array specification '$raw'"
return 1
else
@yaroslavvb
yaroslavvb / smart_initialize.py
Created October 13, 2016 23:20
Better initialize_all_variables which respects variable dependencies and doesn't rerun initializers
# testing variable order init
import tensorflow as tf
def initialize_all_variables(sess=None):
"""Initializes all uninitialized variables in correct order. Initializers
are only run for uninitialized variables, so it's safe to run this multiple
times.
Args:
@rossant
rossant / benchmark.ipynb
Last active July 12, 2023 09:34
Quick HDF5 benchmark
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@kastnerkyle
kastnerkyle / preproc.py
Last active June 28, 2023 19:52
General preprocessing transforms in scikit-learn compatible format
# (C) Kyle Kastner, June 2014
# License: BSD 3 clause
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.utils import gen_batches
from scipy.linalg import eigh
from scipy.linalg import svd
import numpy as np
# From sklearn master
@gizmaa
gizmaa / Plot_Examples.md
Last active April 12, 2024 14:18
Various Julia plotting examples using PyPlot
@staticfloat
staticfloat / debugging.md
Last active February 24, 2017 03:11
Julia Debugging Procedures

Julia Debugging Procedures

So you managed to break Julia. Congratulations! Collected here are some general procedures you can undergo for common symptoms encountered when something goes awry. Including the information from these debugging steps can greatly help the maintainers when tracking down a segfault or trying to figure out why your script is running slower than expected.

If you've been directed to this page, find the symptom that best matches what you're experiencing and follow the instructions to generate the debugging information requested. Table of symptoms:

#!/bin/bash
# Script for installing tmux on systems where you don't have root access.
# tmux will be installed in $HOME/local/bin.
# It's assumed that wget and a C/C++ compiler are installed.
# exit on error
set -e
# create our directories
@spion
spion / a-warning.md
Last active March 25, 2024 03:01
C++ versus V8 versus luajit versus C benchmark - (hash) tables

Warning

This benchmark has been misleading for a while. It was originally made to demonstrate how JIT compilers can do all sorts of crazy stuff to your code - especially LuaJIT - and was meant to be a starting point of discussion about what exactly LuaJIT does and how.

As a result, its not indicative of what its performance may be on more realistic data. Differences can be expected because

  1. the text will not consist of hard-coded constants
@dwf
dwf / uses_random.py
Created January 27, 2010 09:35
Object-specific NumPy random number generator states
"""
A demonstration of how to deal with object-specific random number
generator states. One application is if you want two objects to
work with the same pseudorandom sequence but don't particularly
want to generate them in advance, or to replicate results after
serializing and de-serializing an object.
By David Warde-Farley, dwf at cs.toronto.edu, January 2010.
Released under BSD license.
"""