-
Find the Discord channel in which you would like to send commits and other updates
-
In the settings for that channel, find the Webhooks option and create a new webhook. Note: Do NOT give this URL out to the public. Anyone or service can post messages to this channel, without even needing to be in the server. Keep it safe!
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from transformers import AutoModelForCausalLM, AutoTokenizer, StaticCache | |
import torch | |
from typing import Optional | |
device = "cuda" | |
# Copied from the gpt-fast repo | |
def multinomial_sample_one_no_sync(probs_sort): # Does multinomial sampling without a cuda synchronization | |
q = torch.empty_like(probs_sort).exponential_(1) | |
return torch.argmax(probs_sort / q, dim=-1, keepdim=True).to(dtype=torch.int) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# A very useful script for giving credit where it is due: by replacing (most) variable names by the authors who wrote them. | |
# Requirements: pip install gitpython | |
# Usage: python3 assign-credits.py <input_file> <output_file> | |
# Example: python3 assign-credits.py ./src/main.py ./src/main-credited.py | |
# NOTE that this is an awful idea with an awful implementation. The "generated" code likely does not work | |
# (e.g. typing stuff is skipped, class attribute names are replaced). The "author" is decided by the current HEAD of git repo, and whoever | |
# defines the variable first will get the credits (hahaa dunno if even this is right). | |
# It is getting late and I am tired of typing so I will let Github Copilot write something for me. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# ==== Non-communicating processes | |
# 4x 1 chip per process: | |
os.environ["TPU_CHIPS_PER_PROCESS_BOUNDS"] = "1,1,1" | |
os.environ["TPU_PROCESS_BOUNDS"] = "1,1,1" | |
# Different per process: | |
os.environ["TPU_VISIBLE_DEVICES"] = "0" # "1", "2", "3" | |
# Pick a unique port per process | |
os.environ["TPU_MESH_CONTROLLER_ADDRESS"] = "localhost:8476" | |
os.environ["TPU_MESH_CONTROLLER_PORT"] = "8476" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
model.zero_grad() # Reset gradients tensors | |
for i, (inputs, labels) in enumerate(training_set): | |
predictions = model(inputs) # Forward pass | |
loss = loss_function(predictions, labels) # Compute loss function | |
loss = loss / accumulation_steps # Normalize our loss (if averaged) | |
loss.backward() # Backward pass | |
if (i+1) % accumulation_steps == 0: # Wait for several backward steps | |
optimizer.step() # Now we can do an optimizer step | |
model.zero_grad() # Reset gradients tensors | |
if (i+1) % evaluation_steps == 0: # Evaluate the model when we... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package main | |
import ( | |
"encoding/json" | |
"fmt" | |
"io/ioutil" | |
"log" | |
"net/http" | |
"net/url" | |
"os" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
var http = require("http"), | |
url = require("url"), | |
path = require("path"), | |
fs = require("fs") | |
port = process.argv[2] || 8888, | |
mimeTypes = { | |
"html": "text/html", | |
"jpeg": "image/jpeg", | |
"jpg": "image/jpeg", | |
"png": "image/png", |