manasRK

This is necessitated by the open Issue: pytorch/pytorch#1591

First, the usual pack_padded_sequence and pad_packed_sequence for handling variable length sequences;

seq_len, bsz, n_dims = feats.size()
packed_input = pack_padded_sequence(feats, lengths, batch_first=False)
packed_output, self.hidden = self.lstm(packed_input, self.hidden)
# lstm_out --> seqlen X bsz X hidden_dim
lstm_out, output_lengths = pad_packed_sequence(packed_output, batch_first = False)

manasRK

def _sequence_mask(sequence_length, max_len=None):
    if max_len is None:
        max_len = sequence_length.data.max()
    batch_size = sequence_length.size(0)
    seq_range = torch.range(0, max_len - 1).long()
    seq_range_expand = seq_range.unsqueeze(0).expand(batch_size, max_len)
    seq_range_expand = Variable(seq_range_expand)
    if sequence_length.is_cuda:
        seq_range_expand = seq_range_expand.cuda()
    seq_length_expand = (sequence_length.unsqueeze(1)

manasRK

import torch
import torch.nn as nn
from torch.autograd import Variable
from torch.nn.utils.rnn import pad_packed_sequence, pack_padded_sequence

x = Variable(torch.randn(10, 20, 30)).cuda()
lens = range(10)

x = pack_padded_sequence(x, lens[::-1], batch_first=True)

manasRK

import torch
import torch.nn as nn
from torch.autograd import Variable
from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence
import torch.nn.functional as F
import numpy as np
import itertools

def flatten(l):
	return list(itertools.chain.from_iterable(l))

manasRK

Using Git

Global Settings

Related Setup: https://gist.github.com/hofmannsven/6814278

Related Pro Tips: https://ochronus.com/git-tips-from-the-trenches/

manasRK

If you deleted a file from the working tree, then commit the deletion:

$git commit -a -m "A file was deleted"

When you do a pull request on a branch, you can continue to work on another branch and make another pull request on this other branch.

Before creating a new branch, pull the changes from upstream. Your master needs to be up to date.

Create the branch on your local machine and switch in this branch :

manasRK

"""Information Retrieval metrics

Useful Resources:
http://www.cs.utexas.edu/~mooney/ir-course/slides/Evaluation.ppt
http://www.nii.ac.jp/TechReports/05-014E.pdf
http://www.stanford.edu/class/cs276/handouts/EvaluationNew-handout-6-per.pdf
http://hal.archives-ouvertes.fr/docs/00/72/67/60/PDF/07-busa-fekete.pdf
Learning to Rank for Information Retrieval (Tie-Yan Liu)
"""
import numpy as np

manasRK

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname

manasRK

import os
import math
import re
import pandas as pd
from collections import Counter
from sklearn.datasets import fetch_20newsgroups

#get a subset of the dataset

categories = [

manasRK

IASNLP-2015 Project list

LTRC, IIIT-Hyderabad

Treebanking

#####1. Shallow Parsers for different languages#####

• Description - POS Tagging and Chunking for Gujarati,Odia, Hindi, Bengali, Marathi, Telugu (individual project for each language)
• We will implement many supervised algorithms including CRF, HMM, MaxEnt, SVM, some semi-supervised classification methods, finally an unsupervised one. Will try to implement Morph Analyzer if time permits. students need to annotate data, understand the challenges, compare results given by multiple
• Mentor: Pruthwik M