Skip to content

Instantly share code, notes, and snippets.

View prabod's full-sized avatar

Prabod Rathnayaka prabod

36 followers · 35 following
View GitHub Profile
@TengdaHan
TengdaHan / ddp_notes.md
Last active April 22, 2024 00:19
Multi-node-training on slurm with PyTorch

Multi-node-training on slurm with PyTorch

What's this?

  • A simple note for how to start multi-node-training on slurm scheduler with PyTorch.
  • Useful especially when scheduler is too busy that you cannot get multiple GPUs allocated, or you need more than 4 GPUs for a single job.
  • Requirement: Have to use PyTorch DistributedDataParallel(DDP) for this purpose.
  • Warning: might need to re-factor your own code.
  • Warning: might be secretly condemned by your colleagues because using too many GPUs.
@cbaziotis
cbaziotis / Attention.py
Last active March 28, 2023 11:50
Keras Layer that implements an Attention mechanism for temporal data. Supports Masking. Follows the work of Raffel et al. [https://arxiv.org/abs/1512.08756]
from keras import backend as K, initializers, regularizers, constraints
from keras.engine.topology import Layer
def dot_product(x, kernel):
"""
Wrapper for dot product operation, in order to be compatible with both
Theano and Tensorflow
Args:
@guilhermemm
guilhermemm / k_shortest_paths.py
Last active May 12, 2021 02:47
Yen's K-Shortest Path Algorithm for NetworkX. Yen's algorithm computes single-source K-shortest loopless paths for a graph with non-negative edge cost. For more details, see http://en.m.wikipedia.org/wiki/Yen%27s_algorithm
# -*- coding: utf-8 -*-
"""
A NetworkX based implementation of Yen's algorithm for computing K-shortest paths.
Yen's algorithm computes single-source K-shortest loopless paths for a
graph with non-negative edge cost. For more details, see:
http://networkx.github.io
http://en.m.wikipedia.org/wiki/Yen%27s_algorithm
"""
__author__ = 'Guilherme Maia <guilhermemm@gmail.com>'