Skip to content

Instantly share code, notes, and snippets.

@thomwolf
thomwolf / gradient_accumulation.py
Last active January 16, 2024 02:38
PyTorch gradient accumulation training loop
model.zero_grad() # Reset gradients tensors
for i, (inputs, labels) in enumerate(training_set):
predictions = model(inputs) # Forward pass
loss = loss_function(predictions, labels) # Compute loss function
loss = loss / accumulation_steps # Normalize our loss (if averaged)
loss.backward() # Backward pass
if (i+1) % accumulation_steps == 0: # Wait for several backward steps
optimizer.step() # Now we can do an optimizer step
model.zero_grad() # Reset gradients tensors
if (i+1) % evaluation_steps == 0: # Evaluate the model when we...
@sqordfish
sqordfish / organize.py
Created June 23, 2014 02:54
Python script for organizing files in windows into multiple folders by file type. I made this primarily to organize my downloads folder.
#Author: Jacob Rust
#Date: 7/8/2013
#Description:This script organizes downloaded files into separate folders depending
#on the file type. This was made for windows systems.
#Download directory should be the first command line argument
#New file types can be added to the dictionary in the main method
import os
import sys
import hashlib
@entaroadun
entaroadun / gist:1653794
Created January 21, 2012 20:10
Recommendation and Ratings Public Data Sets For Machine Learning

Movies Recommendation:

Music Recommendation: