filmo

  0%|          | 0/1000 [00:00<?, ?it/s]You're using a PreTrainedTokenizerFast tokenizer. Please note that with a fast tokenizer, using the `__call__` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.
Traceback (most recent call last):
  File "/home/philglau/PycharmProjects/tokenizersLLM/medium_article_falcon7b.py", line 87, in <module>
    trainer.train()
  File "/home/philglau/anaconda3/envs/pytorch_hug_llm_203/lib/python3.11/site-packages/transformers/trainer.py", line 1645, in train
    return inner_training_loop(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/philglau/anaconda3/envs/pytorch_hug_llm_203/lib/python3.11/site-packages/transformers/trainer.py", line 1938, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

filmo

On line 35 shuffle is imported from sklearn

from sklearn.utils import shuffle

and then later called inside the data generator with:

shuffle(keys)

However, shuffle from sklearn does ~not~ shuffle in-place. It returns a shuffled list, leaving the input parameter untouched.

filmo

import numpy as np
np.set_printoptions(precision=4,suppress=True,linewidth=180)

def clamp(n, min, max):
    if n < min:
        return min
    if n > max:
        return max
    return n

filmo

Used a 2-layer Fully Connected network with H1=100, H2=60 and ReLU

He Initialization of weights
Adam Optimizer. Initial learning rate = 0.001 Learning rate reduced using gamma of 0.50 every 350 episodes

Gamma = 0.99
Eps   = 1.00
Eps Decay = 0.98
Eps Decay every new episode. (not each step)

filmo

q_learning walkthrough. GATech OMSCS. (School policy prohibits our publishing code for honor policy)

filmo

Used Value Iteration as shown in class walkthrough. (School policy prohibits us showing code for honor policy.)
GATech OMSCS

filmo

philglau (master *) pr1 $ python submit.py gfserver
Tests that the client properly sends an ERROR response.:                  failed
Tests that the server properly sends an OK response and medium messag:    failed
Tests that the server properly sends an OK response and a long messag:    failed
Tests that the server properly sends an OK response and a short messa:    failed
Tests that the server compiles:                                           passed
Tests that the server properly handles a gfs_abort call.:                 passed
Tests that the server properly sends a FILE_NOT_FOUND response.:          passed
Tests that the server properly handles a malformed request.:              passed
Tests that the server properly handles a malformed request.:              passed

filmo

philglau (master) pr1 $ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'ubuntu/xenial64' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Loading metadata for box 'ubuntu/xenial64'
    default: URL: https://atlas.hashicorp.com/ubuntu/xenial64
==> default: Adding box 'ubuntu/xenial64' (v20160826.0.0) for provider: virtualbox
    default: Downloading: https://atlas.hashicorp.com/ubuntu/boxes/xenial64/versions/20160826.0.0/providers/virtualbox.box
==> default: Successfully added box 'ubuntu/xenial64' (v20160826.0.0) for 'virtualbox'!

filmo

make

g++ -DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -D_FORCE_INLINES -I./include -I/usr/local/cuda-7.5/include -c -o src/nnet/conv_bc01_matmul.o src/nnet/conv_bc01_matmul.cpp
nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,code=compute_35 -O3 --compiler-options '-DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -D_FORCE_INLINES' --ftz=true --prec-div=false -prec-sqrt=false --fmad=true -I./include -I/usr/local/cuda-7.5/include -c -o src/nnet/pool_b01.o src/nnet/pool_b01.cu
g++ -DCUDNN_ENABLED -O3 -fPIC -Wall -Wfatal-errors -D_FORCE_INLINES -I./include -I/usr/local/cuda-7.5/include -c -o src/nnet/cudnn.o src/nnet/cudnn.cpp
nvcc -gencode arch=compute_20,code=sm_20 -gencode arch=compute_20,code=compute_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_30,code=compute_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_35,c

filmo

https://www.youtube.com/watch?v=T_yhf2HrPfQ
https://www.youtube.com/watch?v=x9_vmhE7yiI
https://www.youtube.com/watch?v=AC7RUSqoJ3k
https://www.youtube.com/watch?v=2va57vCvvYs
https://www.youtube.com/watch?v=vwyNzW2GTtM
https://www.youtube.com/watch?v=Ry7lVkgJ1UY
https://www.youtube.com/watch?v=agLkd2ka3Uk
https://www.youtube.com/watch?v=0M-qni5JPlg
https://www.youtube.com/watch?v=l5HRWFGa9Ps
https://www.youtube.com/watch?v=jwYHU5v2YnM