- A simple note for how to start multi-node-training on slurm scheduler with PyTorch.
- Useful especially when scheduler is too busy that you cannot get multiple GPUs allocated, or you need more than 4 GPUs for a single job.
- Requirement: Have to use PyTorch DistributedDataParallel(DDP) for this purpose.
- Warning: might need to re-factor your own code.
- Warning: might be secretly condemned by your colleagues because using too many GPUs.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Good differentiable image resampling for PyTorch.""" | |
from functools import update_wrapper | |
import math | |
import torch | |
from torch.nn import functional as F | |
def sinc(x): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Simple example on how to log scalars and images to tensorboard without tensor ops. | |
License: BSD License 2.0 | |
""" | |
__author__ = "Michael Gygli" | |
import tensorflow as tf | |
from StringIO import StringIO | |
import matplotlib.pyplot as plt | |
import numpy as np |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@echo off | |
setlocal enableextensions | |
chcp 65001 | |
set input=%1 | |
set device=%2 | |
set loglevel=quiet | |
for %%i in ("%~f1") do set dirname=%%~dpi | |
for %%i in ("%~f1") do set extension=%%~xi | |
set tempfile=%dirname%%RANDOM%-temp.m4a |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Typical setup to include TensorFlow. | |
import tensorflow as tf | |
# Make a queue of file names including all the JPEG images files in the relative | |
# image directory. | |
filename_queue = tf.train.string_input_producer( | |
tf.train.match_filenames_once("./images/*.jpg")) | |
# Read an entire image file which is required since they're JPEGs, if the images | |
# are too large they could be split in advance to smaller files or use the Fixed |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Script for installing tmux on systems where you don't have root access. | |
# tmux will be installed in $HOME/local/bin. | |
# It's assumed that wget and a C/C++ compiler are installed. | |
# exit on error | |
set -e | |
TMUX_VERSION=1.8 |