keunwoochoi

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

• Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.

keunwoochoi

import tensorflow as tf
import tensorflow.keras.metrics as tfkm
from tensorflow.keras import backend as K


MASKED_VALUE = -1


def get_1d_mask(y_true, masked_value):
    """Get 1D mask by comparing y_true with masked_value.

keunwoochoi

"""This is a conversion of https://gist.github.com/yang-song/07392ed7d57a92a87968e774aef96762 
to Tensorflow 2 using GradientTape
"""
import tensorflow as tf


@tf.function
def gradients(f, x, tape, grad_ys=None):
    '''
    An easier way of computing gradients in tensorflow. The difference from tf.gradients is

keunwoochoi

import torch
import pdb


def complex_mul(t1, t2):
    """t1, t2: complex representations of torch tensor
    """
    assert (t1.dim() == t2.dim()), "dim mismatch in complex_mul, {} and {}".format(t1.dim(), t2.dim())
    if t1.dim() == 2:
        r1, i1 = t1[:, 0], t1[:, 1]

keunwoochoi

.git
.ipynb_checkpoints
.tox
*.ipynb
notebooks
*.egg-info
*.pyc
OLD-tf2-conversion
temp_data
a_really_large_folder_that_is_not_needed_to_be_in_the_docker_image

keunwoochoi

import tensorflow as tf


class LogMelgramLayer(tf.keras.layers.Layer):
    def __init__(
        self, num_fft, hop_length, num_mels, sample_rate, f_min, f_max, eps, **kwargs
    ):
        super(LogMelgramLayer, self).__init__(**kwargs)
        self.num_fft = num_fft
        self.hop_length = hop_length

keunwoochoi

from scipy.signal import firls
import numpy as np
import torch
import torch.nn.functional as F

elc = np.array([[31.5, -29.9],  # freq, response
                [63, -23.9],
                [100, -19.8],
                [200, -13.8],

keunwoochoi

# assuming num_fft = 512
NUM_FFT = 512
NUM_FREQS = 257
# some tentative constants
NUM_MEL = 60
SAMPLE_RATE = 44100
F_MIN = 0
F_MAX = 12000

keunwoochoi

# This is a simplified script of https://gist.github.com/keunwoochoi/f0ea2c49355fc21e93dad88d210efcdd
# Usage: 1. Modify the font path in the line with [ss].
#           In my case, I copied and pasted CircularSpAraTTBlack.ttf to the same folder.
#        2. Put the wav file in the same folder, say, audio_file.wav
#        3. run $./spectrogram_scrolling_video.sh audio_file.wav
#        4. You'll see audio_file.mkv in the same folder! 
# 
# Based on example here https://trac.ffmpeg.org/wiki/Encode/YouTube
text=$(basename $1 .wav)
ffmpeg -i $1 -filter_complex \

keunwoochoi

# Based on example here https://trac.ffmpeg.org/wiki/Encode/YouTube
text=$(basename $1 .wav)
ffmpeg -i $1 -filter_complex \
"[0:a]avectorscope=s=640x518,pad=1280:720[vs]; \
[0:a]showspectrum=mode=separate:color=intensity:scale=cbrt:s=640x518[ss]; \
[0:a]showwaves=s=1280x202:mode=line[sw]; \
[vs][ss]overlay=w[bg]; \
[bg][sw]overlay=0:H-h,drawtext=fontfile=/usr/share/fonts/truetype/fonts-japanese-gothic.ttf:fontcolor=white:x=10:y=10:text=$text[out]" \
-map "[out]" -map 0:a -c:v libx264 -preset fast -crf 18 -c:a copy $text.mkv