veekaybee

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

alexpaul

DSA Whiteboarding Rubric

 
// Summary of steps
/*
0. understand the prompt (question)
1. clarifying questions (ask at least 2 clarification questions)
2. pseudo code 
3. test your pseudo code 
4. ask the interviewer if you can start coding after testing your pseudocode

yunjey

def contextual_loss(x, y, h=0.5):
    """Computes contextual loss between x and y.
    
    Args:
      x: features of shape (N, C, H, W).
      y: features of shape (N, C, H, W).
      
    Returns:
      cx_loss = contextual loss between x and y (Eq (1) in the paper)
    """

rdinse

# Simple Google Drive backup script with automatic authentication
# for Google Colaboratory (Python 3)

# Instructions:
# 1. Run this cell and authenticate via the link and text box.
# 2. Copy the JSON output below this cell into the `mycreds_file_contents`
#    variable.  Authentication will occur automatically from now on.
# 3. Create a new folder in Google Drive and copy the ID of this folder
#    from the URL bar to the `folder_id` variable.
# 4. Specify the directory to be backed up in `dir_to_backup`.

AruniRC

{0: u'__background__',
 1: u'person',
 2: u'bicycle',
 3: u'car',
 4: u'motorcycle',
 5: u'airplane',
 6: u'bus',
 7: u'train',
 8: u'truck',
 9: u'boat',

jeanr84

package com.jeanr84.sparkjob;

import org.apache.spark.api.java.JavaPairRDD;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.sql.SparkSession;
import scala.Tuple2;

import java.util.Arrays;
import java.util.regex.Pattern;

zhanwenchen

Updated 4/11/2018

Here's my experience of installing the NVIDIA CUDA kit 9.0 on a fresh install of Ubuntu Desktop 16.04.4 LTS.

Table of Contents

• Install NVIDIA Graphics Driver via apt-get
• Install CUDA
• Install cuDNN

Table of contents generated with markdown-toc

wassname

import numpy as np
import h5py

class HDF5Store(object):
    """
    Simple class to append value to a hdf5 file on disc (usefull for building keras datasets)
    
    Params:
        datapath: filepath of h5 file
        dataset: dataset name within the file

Nikasa1889

# [filter size, stride, padding]
#Assume the two dimensions are the same
#Each kernel requires the following parameters:
# - k_i: kernel size
# - s_i: stride
# - p_i: padding (if padding is uneven, right padding will higher than left padding; "SAME" option in tensorflow)
# 
#Each layer i requires the following parameters to be fully represented: 
# - n_i: number of feature (data layer has n_1 = imagesize )
# - j_i: distance (projected to image pixel distance) between center of two adjacent features

panovr

import argparse
import os
import shutil
import time

import torch
import torch.nn as nn
import torch.nn.parallel
import torch.backends.cudnn as cudnn
import torch.optim