Skip to content

Instantly share code, notes, and snippets.

@NaxAlpha
NaxAlpha / long_gpt.py
Last active October 15, 2023 11:21
Training script for LongGPT; Fine-tunes GPT-2 (335M) on The Pile Dataset with a context size of 8k tokens. (requires > 16GB RAM)
import time
from contextlib import suppress
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torch.backends.cuda as cuda
from torch.utils.data import DataLoader, IterableDataset
@welch
welch / installing-opencv-2.4.5-macosx-10.8.4-anaconda-1.6.0
Created September 6, 2013 19:17
How to to build OpenCV 2.4.5 from the distribution tarball using cmake, on Mac OSX 10.8.4, linked to an anaconda installation.
It is a rite of passage to post one's successful build instructions for OpenCV on a Mac
after you've tried all the other conflicting instructions out there and still failed.
brew failed for me (was this because I could never get a happy brew doctor situation?
I'll never know). macports? nope. build-from-source recipes? I didn't find one that
worked for me.
Here's what did work to build OpenCV 2.4.5 from the distribution tarball using cmake,
on Mac OSX 10.8.4, linked to an anaconda installation rather than the system python.
It is a mashup of various bits of advice out there. If you're already comfortable with
build/install from source, all you need to read is the cmake invocation in step 3 and