Discover gists

## rl-for-llms.md

      
              1 file
            
          
              23 forks
            
          
              11 comments
            
          
              534 stars
            
          
                yoavg
                / rl-for-llms.md
            
            
              Last active
              June 3, 2024 02:19
            
          
    Reinforcement Learning for Language Models

Yoav Goldberg, April 2023.
Why RL?

With the release of the ChatGPT model and followup large language models (LLMs), there was a lot of discussion of the importance of "RLHF training", that is, "reinforcement learning from human feedback".
I was puzzled for a while as to why RL (Reinforcement Learning) is better than learning from demonstrations (a.k.a supervised learning) for training language models. Shouldn't learning from demonstrations (or, in language model terminology "instruction fine tuning", learning to immitate human written answers) be sufficient? I came up with a theoretical argument that was somewhat convincing. But I came to realize there is an additional argumment which not only supports the case of RL training, but also requires it, in particular for models like ChatGPT. This additional argument is spelled out in (the first half of) a talk by John Schulman from OpenAI. This post pretty much

  
## KZG10.py
from typing import List, NamedTuple, Tuple, Union
from math import ceil, log2
from random import randint
from functools import reduce
import operator
from py_ecc import bn128 as curve

"""
Implementation of PolyCommit_{DL} from:

## nbstripout_as_precommit_hook.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              1 star
            
          
                m4nh
                / nbstripout_as_precommit_hook.md
            
            
              Last active
              June 3, 2024 02:17
            
              
                #nbstripout #pre-commit #hook
              
          
    Install nbstripout

pip install --upgrade nbstripout

in target repository:
nbstripout --install


## decode_hex.zig
// compile with: -mcpu=x86_64+sse2 to ensure sse2 support
pub fn decodeHex(input: []const u8, output: []u8) !void {
    const block_size = 16; // change block size to use avx2
    const ByteBlock = @Vector(block_size, u8);
    const IntBlock = @Vector(block_size / 4, u32);

    if (input.len & 1 != 0) {
        return error.OddLenghtInput;
    }

## job_system.zig
const std = @import("std");
const Allocator = std.mem.Allocator;
const ArenaAllocator = std.heap.ArenaAllocator;
const Atomic = std.atomic.Value;

fn Queue(comptime T: type, comptime size: usize) type {
    return struct {
        front: Atomic(usize) = .{ .raw = 0 },
        back: Atomic(usize) = .{ .raw = 0 },
        items: [size]?*T = [1]?*T{null} ** size,

## installing_nvidia_driver_cuda_cudnn_linux.md

      
              1 file
            
          
              49 forks
            
          
              20 comments
            
          
              188 stars
            
          
                kmhofmann
                / installing_nvidia_driver_cuda_cudnn_linux.md
            
            
              Last active
              June 3, 2024 02:12
            
              
                Installing the NVIDIA driver, CUDA and cuDNN on Linux
              
          
    Installing the NVIDIA driver, CUDA and cuDNN on Linux (Ubuntu 20.04)

This is a companion piece to my instructions on building TensorFlow from source.
In particular, the aim is to install the following pieces of software

NVIDIA graphics card driver (v450.57)
CUDA (v11.0.2)
cuDNN (v8.0.2.39)

on an Ubuntu Linux system, in particular Ubuntu 20.04.

  
## versioning-your-saves.md

      
              1 file
            
          
              2 forks
            
          
              15 comments
            
          
              37 stars
            
          
                in5ikt
                / versioning-your-saves.md
            
            
              Created
              February 8, 2011 22:44
            
              
                Minecraft - Versioning your saves
              
          
    Minecraft - Versioning your saves

This is a tutorial show how to version your saves using  Git.
Q: Why would anyone go through this process?

A: Because if you do, you can undo an otherwise un-undoable action, such as swimming in a pit of lava and accidentally dropping your Notch armpit forged special edition diamond pickaxe and your 35 stacks of cobblestone.
Step by step tutorial

Download and install the appropriate version of Git


## README.md

      
              2 files
            
          
              0 forks
            
          
              3 comments
            
          
              7 stars
            
          
                magnetikonline
                / README.md
            
            
              Last active
              June 3, 2024 02:09
            
              
                Enable GitHub Dependabot for Golang based repositories.
              
          
    Dependabot for Golang based repositories

Minimal YAML configuration for enabling Dependabot within a Golang based repository.
Place dependabot.yaml at .github/dependabot.yaml within your repository and you should be good to go.
Reference


Configuration options for the dependabot.yml file


## ambiente-dev-ubuntu.sh
#!/bin/bash

# Executar comandos a seguir para atualizar os pacotes
sudo apt update -y
sudo apt upgrade -y

# Só o Python
sudo apt install python3.10-full python3.10-dev -y

# Instalar pacotes a seguir

## index.html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Change a map's style</title>
<meta name="viewport" content="initial-scale=1,maximum-scale=1,user-scalable=no">
<link href="https://api.mapbox.com/mapbox-gl-js/v2.2.0/mapbox-gl.css" rel="stylesheet">
<script src="https://api.mapbox.com/mapbox-gl-js/v2.2.0/mapbox-gl.js"></script>
<style>
body { margin: 0; padding: 0; }
	from typing import List, NamedTuple, Tuple, Union
	from math import ceil, log2
	from random import randint
	from functools import reduce
	import operator
	from py_ecc import bn128 as curve

	"""
	Implementation of PolyCommit_{DL} from:
	// compile with: -mcpu=x86_64+sse2 to ensure sse2 support
	pub fn decodeHex(input: []const u8, output: []u8) !void {
	const block_size = 16; // change block size to use avx2
	const ByteBlock = @Vector(block_size, u8);
	const IntBlock = @Vector(block_size / 4, u32);

	if (input.len & 1 != 0) {
	return error.OddLenghtInput;
	}
	const std = @import("std");
	const Allocator = std.mem.Allocator;
	const ArenaAllocator = std.heap.ArenaAllocator;
	const Atomic = std.atomic.Value;

	fn Queue(comptime T: type, comptime size: usize) type {
	return struct {
	front: Atomic(usize) = .{ .raw = 0 },
	back: Atomic(usize) = .{ .raw = 0 },
	items: [size]?T = [1]?T{null} ** size,
	#!/bin/bash

	# Executar comandos a seguir para atualizar os pacotes
	sudo apt update -y
	sudo apt upgrade -y

	# Só o Python
	sudo apt install python3.10-full python3.10-dev -y

	# Instalar pacotes a seguir
	<!DOCTYPE html>
	<html>
	<head>
	<meta charset="utf-8">
	<title>Change a map's style</title>
	<meta name="viewport" content="initial-scale=1,maximum-scale=1,user-scalable=no">
	<link href="https://api.mapbox.com/mapbox-gl-js/v2.2.0/mapbox-gl.css" rel="stylesheet">
	<script src="https://api.mapbox.com/mapbox-gl-js/v2.2.0/mapbox-gl.js"></script>
	<style>
	body { margin: 0; padding: 0; }