Skip to content

Instantly share code, notes, and snippets.

Avatar

Christos Baziotis cbaziotis

View GitHub Profile
View pytorch_resume_training.py
import torch
import torch.nn as nn
import torch.optim as optim
from torch.autograd import Variable
import torch.nn.functional as F
import matplotlib.pyplot as plt
import numpy as np
@cbaziotis
cbaziotis / SelfAttention.py
Created Apr 21, 2018
SelfAttention implementation in PyTorch
View SelfAttention.py
class SelfAttention(nn.Module):
def __init__(self, attention_size, batch_first=False, non_linearity="tanh"):
super(SelfAttention, self).__init__()
self.batch_first = batch_first
self.attention_weights = Parameter(torch.FloatTensor(attention_size))
self.softmax = nn.Softmax(dim=-1)
if non_linearity == "relu":
self.non_linearity = nn.ReLU()
@cbaziotis
cbaziotis / ubuntu16_tensorflow_cuda8.sh
Created Jun 26, 2017 — forked from ksopyla/ubuntu16_tensorflow_cuda8.sh
How to set up tensorflow with CUDA 8 cuDNN 5.1 in virtualenv with Python 3.5 on Ubuntu 16.04 http://ksopyla.com/2017/02/tensorflow-gpu-virtualenv-python3/
View ubuntu16_tensorflow_cuda8.sh
# This is shorthened version of blog post
# http://ksopyla.com/2017/02/tensorflow-gpu-virtualenv-python3/
# update packages
sudo apt-get update
sudo apt-get upgrade
#Add the ppa repo for NVIDIA graphics driver
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
View Deep Learning with Python 3 on ubuntu 16.04.sh
# This is inspired by the fantastic guide https://github.com/saiprashanths/dl-setup
# I have just updated the python-related commands so that everything works in Python 3.
# Tested on Xubuntu 16.04.
# First of all let's update the repos:
sudo apt-get update
# Only if you have a CUDA-compatible Nvidia card, install CUDA.
# Check on the Nvidia website what is the latest driver version which supports your card.
# At the time of this writing it was 367.
View class_weights.py
def get_class_weights(y, smooth_factor=0):
"""
Returns the weights for each class based on the frequencies of the samples
:param smooth_factor: factor that smooths extremely uneven weights
:param y: list of true labels (the labels must be hashable)
:return: dictionary with the weight for each class
"""
counter = Counter(y)
if smooth_factor > 0:
@cbaziotis
cbaziotis / AttentionWithContext.py
Last active May 15, 2021
Keras Layer that implements an Attention mechanism, with a context/query vector, for temporal data. Supports Masking. Follows the work of Yang et al. [https://www.cs.cmu.edu/~diyiy/docs/naacl16.pdf] "Hierarchical Attention Networks for Document Classification"
View AttentionWithContext.py
def dot_product(x, kernel):
"""
Wrapper for dot product operation, in order to be compatible with both
Theano and Tensorflow
Args:
x (): input
kernel (): weights
Returns:
"""
if K.backend() == 'tensorflow':
@cbaziotis
cbaziotis / Attention.py
Last active Jun 9, 2021
Keras Layer that implements an Attention mechanism for temporal data. Supports Masking. Follows the work of Raffel et al. [https://arxiv.org/abs/1512.08756]
View Attention.py
from keras import backend as K, initializers, regularizers, constraints
from keras.engine.topology import Layer
def dot_product(x, kernel):
"""
Wrapper for dot product operation, in order to be compatible with both
Theano and Tensorflow
Args: