shijie-wu

I follow this repo to load OPT-1.3B into metaseq. I run MASTER_ADDR=localhost MASTER_PORT=6000 python run_model.py &> output.log

shijie-wu

task1/arabic--classical-syriac/arabic-train-high
34 وَاجِهَاتِ بَرْمَجَةِ تَطْبِيقَاتٍ
38 مُدَوَّنَتَا  الْإِلِكْتْرُونِيَّتَانِ
39 الِاشْتِرَاكِيَّةَ الدِّيمُقْرَاطِيَّةَ
40 غُدَدُ  الْبَصَلِيَّةُ الْإِحْلِيلِيَّةُ
41 غُدَّةَ  الْبَصَلِيَّةَ الْإِحْلِيلِيَّةَ

shijie-wu

FROM digitalgenius/ubuntu-pytorch

MAINTAINER Shijie Wu <wushijie5@gmail.com>

WORKDIR /root

ENV PYINSTALL "pip install --no-cache-dir"

RUN pip install pip --upgrade \
    && pip install setuptools --upgrade \

shijie-wu

2017-05-05 17:22:41.617885: F tensorflow/stream_executor/cuda/cuda_driver.cc:311] Check failed: CUDA_SUCCESS == cuCtxSetCurrent(cuda_context->context()) (0 vs. 3)
2017-05-05 17:22:41.668105: E tensorflow/stream_executor/cuda/cuda_driver.cc:1185] failed to enqueue async memcpy from host to device: CUDA_ERROR_NOT_INITIALIZED; GPU dst: 0x1041ae00500; host src: 0x1041b000100; size: 4=0x4
2017-05-05 17:22:41.668148: E tensorflow/stream_executor/event.cc:40] could not create CUDA event: CUDA_ERROR_NOT_INITIALIZED
2017-05-05 17:22:41.668161: E tensorflow/stream_executor/stream.cc:289] Error recording event in stream: error recording CUDA event on stream 0x67ec520: CUDA_ERROR_INVALID_HANDLE; not marking stream as bad, as the Event object may be at fault. Monitor for further errors.
2017-05-05 17:22:41.668172: E tensorflow/stream_executor/cuda/cuda_event.cc:49] Error polling for event status: failed to query event: CUDA_ERROR_INVALID_HANDLE
2017-05-05 17:22:41.668178: F tensorflow/core/common_runtime/gpu/gpu_event_mgr

shijie-wu

Simple test to figure out the format of hiddenOutput of cudnn LSTM, see the file below.

hiddenOutput of normal LSTM

(1,.,.) =
  0.3042  0.3042  0.3042
[torch.CudaTensor of size 1x1x3]

hiddenOutput of bidirection LSTM

shijie-wu

--
-- Author: Joost van Doorn <joost.vandoorn@student.uva.nl>
--
require 'rnn'
require 'cudnn'

function toCudnnLSTM(seqLSTM)
   local rnn = cudnn.LSTM(seqLSTM.inputsize, seqLSTM.outputsize, 1)
   local H, R, D = seqLSTM.outputsize, seqLSTM.outputsize, seqLSTM.inputsize
   local biases = rnn:biases()

shijie-wu

--[[

Example of "coupled" separate encoder and decoder networks using cudnn, e.g. for sequence-to-sequence networks.

]]--

require 'rnn'
require 'cunn'
require 'cudnn'

shijie-wu

package main

import (
	"fmt"
)

type Fetcher interface {
	// Fetch returns the body of URL and
	// a slice of URLs found on that page.
	Fetch(url string) (body string, urls []string, err error)

shijie-wu

5
1 R 3 5
2 W 4 5
3 R 5 2
4 R 6 5
5 W 7 3

shijie-wu

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <pthread.h>
#include <semaphore.h>

sem_t mutex, wrt;
int readcount;
typedef struct para
{