rain-1

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

JoaoLages

Maybe you've heard about this technique but you haven't completely understood it, especially the PPO part. This explanation might help.

We will focus on text-to-text language models 📝, such as GPT-3, BLOOM, and T5. Models like BERT, which are encoder-only, are not addressed.

Reinforcement Learning from Human Feedback (RLHF) has been successfully applied in ChatGPT, hence its major increase in popularity. 📈

RLHF is especially useful in two scenarios 🌟:

• You can’t create a good loss function
  • Example: how do you calculate a metric to measure if the model’s output was funny?
• You want to train with production data, but you can’t easily label your production data

ClementNerma

ZSH CheatSheet

This is a cheat sheet for how to perform various actions to ZSH, which can be tricky to find on the web as the syntax is not intuitive and it is generally not very well-documented.

Strings

Description	Syntax
Get the length of a string	${#VARNAME}
Get a single character	${VARNAME[index]}

tjvananne

#' A word vector is a giant matrix of words, and each word contains a numeric array that represents the semantic
#' meaning of that word. This is useful so we can discover relationships and analogies between words programmatically.
#' The classic example is "king" minus "man" plus "woman" is most similar to "queen"


# function definition --------------------------------------------------------------------------

    # input .txt file, exports list of list of values and character vector of names (words)
    proc_pretrained_vec <- function(p_vec) {

squarism

Tabs and Windows

Function	Shortcut
New Tab	⌘ + T
Close Tab or Window	⌘ + W (same as many mac apps)
Go to Tab	⌘ + Number Key (ie: ⌘2 is 2nd tab)
Go to Split Pane by Direction	⌘ + Option + Arrow Key
Cycle iTerm Windows	⌘ + backtick (true of all mac apps and works with desktops/mission control)

kiwanami

;;; calfw-git.el --- calendar view for git-log

;; Copyright (C) 2014  SAKURAI Masashi

;; Author: SAKURAI Masashi <m.sakurai at kiwanami.net>
;; Keywords: calendar

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or

MohamedAlaa

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname