Skip to content

Instantly share code, notes, and snippets.

View odp's full-sized avatar
:octocat:

Omkar Pangarkar odp

:octocat:
  • Petuum, Inc.
  • Pittsburgh, PA
View GitHub Profile
@willccbb
willccbb / grpo_demo.py
Last active October 14, 2025 12:30
GRPO Llama-1B
# train_grpo.py
#
# See https://github.com/willccbb/verifiers for ongoing developments
#
"""
citation:
@misc{brown2025grpodemo,
title={Granular Format Rewards for Eliciting Mathematical Reasoning Capabilities in Small Language Models},
author={Brown, William},
import os
import torch
import torch.distributed as dist
import lovely_tensors as lt; lt.monkey_patch()
def split_tensor(data: torch.Tensor, dim: int) -> torch.Tensor:
rank = dist.get_rank()
world_size = dist.get_world_size()
@3outeille
3outeille / pipeline-model-parallel-visualization.ipynb
Created June 14, 2024 19:58 — forked from sighingnow/pipeline-model-parallel-visualization.ipynb
Visualizing various different pipeline model parallel scheduling algorithms: GPipe, Pipedream(1F1B), Pipedream-2BW(async, no-flushes), and eager-1F1B
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@NaxAlpha
NaxAlpha / long_gpt.py
Last active July 23, 2024 13:07
Training script for LongGPT; Fine-tunes GPT-2 (335M) on The Pile Dataset with a context size of 8k tokens. (requires > 16GB RAM)
import time
from contextlib import suppress
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torch.backends.cuda as cuda
from torch.utils.data import DataLoader, IterableDataset
@mitchellh
mitchellh / RadixTrees.tla
Last active September 8, 2025 00:01
Radix tree and validation in TLA+
This module contains operations for working with radix trees. A radix tree
is a data structure for efficient storage and lookup of values that often
share prefixes, typically used with strings.
A common question when I show this to people is: how do I add to the tree?
delete? update? For these, grab the Range of the tree, use set logic to
add/remove any elements, and construct a new tree with RadixTree.
For educational purposes, I've heavily commented all the operations. I
recommend using the constant expression evaluator to try the building blocks
@jacobtomlinson
jacobtomlinson / Dask on Fargate from scratch.ipynb
Last active September 18, 2025 21:05
Dask on Fargate from scratch
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Teggy
Teggy / four-solutions-to-a-trivial-problem.hs
Last active December 21, 2024 10:21
A Haskell "transcript" of Guy Steele's talk "Four Solutions to a Trivial Problem" (https://www.youtube.com/watch?v=ftcIcn8AmSY)
{-# LANGUAGE TypeSynonymInstances #-}
import Data.Monoid
import Data.Maybe
-- How much water does a "histogram" hold?
--
-- Inspired by Guy Steele's talk "Four Solutions to a Trivial Problem"
-- https://www.youtube.com/watch?v=ftcIcn8AmSY
@briantjacobs
briantjacobs / storytelling_from_space.md
Last active August 28, 2024 07:14
Storytelling from Space

Storytelling from Space: Tools/Resources

This list of resources is all about acquring and processing aerial imagery. It's generally broken up in three ways: how to go about this in Photoshop/GIMP, using command-line tools, or in GIS software, depending what's most comfortable to you. Often these tools can be used in conjunction with each other.

Acquiring Landsat & MODIS

Web Interface

  • Landsat archive
@hadley
hadley / .gitignore
Last active February 25, 2024 02:10
Benchmark different ways of reading a file
.Rproj.user
.Rhistory
.RData
*.Rproj
*.html