mcarilli

# This isn't supposed to run as a bash script, i named it with ".sh" for syntax highlighting.

# https://developer.nvidia.com/nsight-systems
# https://docs.nvidia.com/nsight-systems/profiling/index.html

# My preferred nsys (command line executable used to create profiles) commands
#
# In your script, write
# torch.cuda.nvtx.range_push("region name")
# ...

huchenxucs

import math
import torch
import torch.nn as nn
from torch.nn import functional as F


class RelativePositionBias(nn.Module):
    def __init__(self, bidirectional=True, num_buckets=32, max_distance=128, n_heads=2):
        super(RelativePositionBias, self).__init__()
        self.bidirectional = bidirectional

gyglim

"""Simple example on how to log scalars and images to tensorboard without tensor ops.

License: BSD License 2.0
"""
__author__ = "Michael Gygli"

import tensorflow as tf
from StringIO import StringIO
import matplotlib.pyplot as plt
import numpy as np

jagregory

So you've cloned somebody's repo from github, but now you want to fork it and contribute back. Never fear!

Technically, when you fork "origin" should be your fork and "upstream" should be the project you forked; however, if you're willing to break this convention then it's easy.

* Off the top of my head *

1. Fork their repo on Github
2. In your local, add a new remote to your fork; then fetch it, and push your changes up to it

    git remote add my-fork git@github...my-fork.git