Skip to content

Instantly share code, notes, and snippets.

View BDHU's full-sized avatar

Edward Hu BDHU

View GitHub Profile
@abtrout
abtrout / pass.md
Created July 8, 2014 14:51
Using password-store with git repository synching

Password-store keeps your passwords (or any other sensitive information) saved in GnuPG encrypted files organized in ~/.password-store. For more information about GPG, consult the GNU Privacy Handbook.

Getting started

To get started, install pass and generate a keypair.

$ brew install pass
$ gpg --gen-key
$ gpg --list-keys
/**
* Copyright 2016 NVIDIA Corporation. All rights reserved.
*
* Please refer to the NVIDIA end user license agreement (EULA) associated
* with this source code for terms and conditions that govern your use of
* this software. Any use, reproduction, disclosure, or distribution of
* this software and related documentation outside the terms of the EULA
* is strictly prohibited.
*
*/
@yen3
yen3 / aarch64_virt_install.sh
Last active May 10, 2024 01:40
aarch64 virt-install commands
#!/bin/bash
rm -rf /home/yen3/ubuntu.qcow2
qemu-img create -f qcow2 /home/yen3/ubuntu.qcow2 10G
virsh undefine ubuntu1604arm64 --nvram
install_from_localtion() {
virt-install -n ubuntu1604arm64 --memory 1024 --arch aarch64 --vcpus 1 \
--disk /home/yen3/ubuntu.qcow2,device=disk,bus=virtio \
@ZijiaLewisLu
ZijiaLewisLu / Tricks to Speed Up Data Loading with PyTorch.md
Last active July 11, 2024 11:00
Tricks to Speed Up Data Loading with PyTorch

In most of deep learning projects, the training scripts always start with lines to load in data, which can easily take a handful minutes. Only after data ready can start testing my buggy code. It is so frustratingly often that I wait for ten minutes just to find I made a stupid typo, then I have to restart and wait for another ten minutes hoping no other typos are made.

In order to make my life easy, I devote lots of effort to reduce the overhead of I/O loading. Here I list some useful tricks I found and hope they also save you some time.

  1. use Numpy Memmap to load array and say goodbye to HDF5.

    I used to relay on HDF5 to read/write data, especially when loading only sub-part of all data. Yet that was before I realized how fast and charming Numpy Memmapfile is. In short, Memmapfile does not load in the whole array at open, and only later "lazily" load in the parts that are required for real operations.

Sometimes I may want to copy the full array to memory at once, as it makes later operations

@samskalicky
samskalicky / Makefile
Created August 6, 2018 23:43
Example cuDNN pooling operator for both forward and backward passes
all:
nvcc -arch=sm_35 -std=c++11 -O2 -Icudnn/include -L cudnn/lib64 -L /usr/local/lib test.cu -o test -lcudnn -I. -D$(FLAGS)