Skip to content

Instantly share code, notes, and snippets.

View songtaoshi's full-sized avatar

songtaoshi songtaoshi

18 followers · 403 following
  • Shanghai
View GitHub Profile
@karpathy
karpathy / gist:587454dc0146a6ae21fc
Last active July 11, 2024 10:36
An efficient, batched LSTM.
"""
This is a batched LSTM forward and backward pass
"""
import numpy as np
import code
class LSTM:
@staticmethod
def init(input_size, hidden_size, fancy_forget_bias_init = 3):
@kastnerkyle
kastnerkyle / audio_tools.py
Last active July 2, 2024 19:06
Audio tools for numpy/python. Constant work in progress.
raise ValueError("DEPRECATED/FROZEN - see https://github.com/kastnerkyle/tools for the latest")
# License: BSD 3-clause
# Authors: Kyle Kastner
# Harvest, Cheaptrick, D4C, WORLD routines based on MATLAB code from M. Morise
# http://ml.cs.yamanashi.ac.jp/world/english/
# MGC code based on r9y9 (Ryuichi Yamamoto) MelGeneralizedCepstrums.jl
# Pieces also adapted from SPTK
from __future__ import division
import numpy as np
@awni
awni / ctc_decoder.py
Last active June 1, 2024 14:21
Example CTC Decoder in Python
"""
Author: Awni Hannun
This is an example CTC decoder written in Python. The code is
intended to be a simple example and is not designed to be
especially efficient.
The algorithm is a prefix beam search for a model trained
with the CTC loss function.
@chao-ji
chao-ji / beamsearch_decoder.py
Last active March 4, 2024 07:09
Easy to understand implementation of beam search algorithm used in decoder of seq2seq models
"""NumPy implementation of Beam Search. Can be used for decoding in Seq2Seq
models or transformer.
See https://chao-ji.github.io/jekyll/update/2019/01/24/Beam_Search.html
for an in-depth disucssion.
"""
import numpy as np
NEG_INF = -1e9
@ZijiaLewisLu
ZijiaLewisLu / Tricks to Speed Up Data Loading with PyTorch.md
Last active July 11, 2024 11:00
Tricks to Speed Up Data Loading with PyTorch

In most of deep learning projects, the training scripts always start with lines to load in data, which can easily take a handful minutes. Only after data ready can start testing my buggy code. It is so frustratingly often that I wait for ten minutes just to find I made a stupid typo, then I have to restart and wait for another ten minutes hoping no other typos are made.

In order to make my life easy, I devote lots of effort to reduce the overhead of I/O loading. Here I list some useful tricks I found and hope they also save you some time.

  1. use Numpy Memmap to load array and say goodbye to HDF5.

    I used to relay on HDF5 to read/write data, especially when loading only sub-part of all data. Yet that was before I realized how fast and charming Numpy Memmapfile is. In short, Memmapfile does not load in the whole array at open, and only later "lazily" load in the parts that are required for real operations.

Sometimes I may want to copy the full array to memory at once, as it makes later operations