Skip to content

Instantly share code, notes, and snippets.


Brendan Dolan-Gavitt moyix

View GitHub Profile
moyix / 00_output.txt
Created Aug 30, 2022
Demo of extending a rotary position embedding model to a longer context than it was trained on
View 00_output.txt
(sfcodegen) moyix@isabella:~$ python
vocab_file vocab.json
merges_file merges.txt
tokenizer_file tokenizer.json
added_tokens_file added_tokens.json
special_tokens_map_file special_tokens_map.json
tokenizer_config_file tokenizer_config.json
Partial prompt from /usr/include/stdlib.h:
[...] restrict __nptr,
#!/usr/bin/env python
import torch
from transformers import CodeGenConfig, CodeGenForCausalLM, CodeGenTokenizer
from transformers.utils.hub import cached_file
NEW_SIZE = 4096
cg_config = CodeGenConfig.from_pretrained('Salesforce/codegen-350M-mono')
cg_config.n_ctx = NEW_SIZE
moyix /
Created Jul 29, 2022
Proxy between Copilot and a Triton inference server running FasterTransformer
#!/usr/bin/env python
import time
import random
import string
from flask import Flask, request
import numpy as np
import tritonclient.grpc as client_util
import json
from tritonclient.utils import np_to_triton_dtype
moyix /
Created Jul 25, 2022
Convert HuggingFace GPT-J model to FasterTransformers
# Copyright (c) 2021-2022, NVIDIA CORPORATION. All rights reserved.
# Modified by Brendan Dolan-Gavitt, 2022
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# Unless required by applicable law or agreed to in writing, software
moyix /
Last active Nov 17, 2022
How to convert the SalesForce CodeGen models to GPT-J

Using Linear Algebra to Convert a Large Code Model


The SalesForce CodeGen models are a family of large language models trained on a large amount of natural language data and then fine-tuned on specialized datasets of code. Models of size 350M, 2B, 6B, and 16B parameters are provided in three flavors:

  • nl, the base model trained on The Pile, a large natural language dataset compiled by EleutherAI
  • multi, which is fine-tuned from the nl model on a dataset of code in multiple languages, scraped from GitHub, and
  • mono, which is fine-tuned from the multi model on Python code only.
moyix /
Created Jul 22, 2022
Convert a SalesForce CodeGen model's weights to plain GPT-J
#!/usr/bin/env python
import argparse
import torch
from transformers import GPTJForCausalLM, GPTJConfig
# Note: these need the git version of Transformers as of 7/22/2022
from transformers import CodeGenTokenizer, CodeGenForCausalLM
parser = argparse.ArgumentParser('Convert SalesForce CodeGen model to GPT-J')
moyix / shadowstack.c
Last active Mar 23, 2022
Example of using a constructor to set up the shadow callstack and assign gs appropriately
View shadowstack.c
#include <stdio.h>
#include <asm/prctl.h>
#include <sys/prctl.h>
#include <stdlib.h>
// Compile with: clang-8 -fsanitize=shadow-call-stack shadowstack.c -o shadowstack
int arch_prctl(int code, unsigned long *addr);
void __attribute__ ((constructor)) __attribute__((no_sanitize("shadow-call-stack"))) setupgs()
moyix / tiffcrop.c
Created Mar 22, 2022
Buggy code from libtiff
View tiffcrop.c
static int readSeparateTilesIntoBuffer (TIFF* in, uint8 *obuf,
uint32 imagelength, uint32 imagewidth,
uint32 tw, uint32 tl,
uint16 spp, uint16 bps)
int i, status = 1, sample;
int shift_width, bytes_per_pixel;
uint16 bytes_per_sample;
uint32 row, col; /* Current row and col of image */
uint32 nrow, ncol; /* Number of rows and cols in current tile */
moyix / codex_commands.txt
Created Mar 11, 2022
Commands executed by Codex across 100 five-minute runs in a Linux VM
View codex_commands.txt
91 {
42 y
40 ")
40 """
34 #!/usr/bin/env python
33 import numpy as np
33 # -*- coding: utf-8 -*-
31 ubuntu
28 Authentication failure
moyix /
Last active Jul 20, 2022
Retrieve reflist for a repository on Github using httpx and asyncio
#!/usr/bin/env python
import os
import itertools
import asyncio
import sys
import httpx
import time
import random
from binascii import crc32