Skip to content

Instantly share code, notes, and snippets.

View dschaehi's full-sized avatar
🎯
Focusing

Jae Hee Lee dschaehi

🎯
Focusing
View GitHub Profile
@dschaehi
dschaehi / gradient_accumulation.py
Created September 28, 2022 15:40 — forked from thomwolf/gradient_accumulation.py
PyTorch gradient accumulation training loop
model.zero_grad() # Reset gradients tensors
for i, (inputs, labels) in enumerate(training_set):
predictions = model(inputs) # Forward pass
loss = loss_function(predictions, labels) # Compute loss function
loss = loss / accumulation_steps # Normalize our loss (if averaged)
loss.backward() # Backward pass
if (i+1) % accumulation_steps == 0: # Wait for several backward steps
optimizer.step() # Now we can do an optimizer step
model.zero_grad() # Reset gradients tensors
if (i+1) % evaluation_steps == 0: # Evaluate the model when we...
def flatten_json(json):
if type(json) == dict:
for k, v in list(json.items()):
if type(v) == dict:
flatten_json(v)
json.pop(k)
for k2, v2 in v.items():
json[k+"."+k2] = v2
@dschaehi
dschaehi / install-cuda-10-bionic.sh
Created August 10, 2019 13:11 — forked from bogdan-kulynych/install-cuda-10-bionic.sh
Install CUDA 10 on Ubuntu 18.04
#!/bin/bash
# Install CUDA Toolkit 10
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub && sudo apt update
sudo dpkg -i cuda-repo-ubuntu1804_10.0.130-1_amd64.deb
sudo apt update
sudo apt install -y cuda
@dschaehi
dschaehi / emacs-zotero-bib-fetch.el
Created June 28, 2019 13:03 — forked from andersjohansson/emacs-zotero-bib-fetch.el
Emacs-zotero-bib-fetch: fork of zotelo to use better-bibtex's pull export
;;; emacs-zotero-bib-fetch.el --- Manage Zotero collections from emacs
;;
;; Filename: emacs-zotero-bib-fetch.el
;; Author: Anders Johansson, based on zotelo by Spinu Vitalie
;; Maintainer: Anders Johansson
;; Copyright (C) 2011-2014, Anders Johansson and Spinu Vitalie
;; Created: 1 Jul 2014
;; Version: 1.2
;; URL: https://github.com/andersjohansson/emacs-zotero-bib-fetch
;; Keywords: zotero, emacs, reftex, bibtex, bibliography manager