Skip to content

Instantly share code, notes, and snippets.

Avatar

Brendan Dolan-Gavitt moyix

View GitHub Profile
@moyix
moyix / 00_output.txt
Created Aug 30, 2022
Demo of extending a rotary position embedding model to a longer context than it was trained on
View 00_output.txt
(sfcodegen) moyix@isabella:~$ python load_codegen_with_longer_context.py
vocab_file vocab.json
merges_file merges.txt
tokenizer_file tokenizer.json
added_tokens_file added_tokens.json
special_tokens_map_file special_tokens_map.json
tokenizer_config_file tokenizer_config.json
Partial prompt from /usr/include/stdlib.h:
[...] restrict __nptr,
View load_codegen_with_longer_context.py
#!/usr/bin/env python
import torch
from transformers import CodeGenConfig, CodeGenForCausalLM, CodeGenTokenizer
from transformers.utils.hub import cached_file
NEW_SIZE = 4096
cg_config = CodeGenConfig.from_pretrained('Salesforce/codegen-350M-mono')
cg_config.n_ctx = NEW_SIZE
@moyix
moyix / copilot_proxy.py
Created Jul 29, 2022
Proxy between Copilot and a Triton inference server running FasterTransformer
View copilot_proxy.py
#!/usr/bin/env python
import time
import random
import string
from flask import Flask, request
import numpy as np
import tritonclient.grpc as client_util
import json
from tritonclient.utils import np_to_triton_dtype
@moyix
moyix / huggingface_gptj_convert.py
Created Jul 25, 2022
Convert HuggingFace GPT-J model to FasterTransformers
View huggingface_gptj_convert.py
# Copyright (c) 2021-2022, NVIDIA CORPORATION. All rights reserved.
# Modified by Brendan Dolan-Gavitt, 2022
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
@moyix
moyix / CodeGen_GPTJ_Conversion.md
Last active Nov 17, 2022
How to convert the SalesForce CodeGen models to GPT-J
View CodeGen_GPTJ_Conversion.md

Using Linear Algebra to Convert a Large Code Model

Background

The SalesForce CodeGen models are a family of large language models trained on a large amount of natural language data and then fine-tuned on specialized datasets of code. Models of size 350M, 2B, 6B, and 16B parameters are provided in three flavors:

  • nl, the base model trained on The Pile, a large natural language dataset compiled by EleutherAI
  • multi, which is fine-tuned from the nl model on a dataset of code in multiple languages, scraped from GitHub, and
  • mono, which is fine-tuned from the multi model on Python code only.
@moyix
moyix / codegen_gptj_convert.py
Created Jul 22, 2022
Convert a SalesForce CodeGen model's weights to plain GPT-J
View codegen_gptj_convert.py
#!/usr/bin/env python
import argparse
import torch
from transformers import GPTJForCausalLM, GPTJConfig
# Note: these need the git version of Transformers as of 7/22/2022
from transformers import CodeGenTokenizer, CodeGenForCausalLM
from transformers import CODEGEN_PRETRAINED_MODEL_ARCHIVE_LIST
parser = argparse.ArgumentParser('Convert SalesForce CodeGen model to GPT-J')
@moyix
moyix / shadowstack.c
Last active Mar 23, 2022
Example of using a constructor to set up the shadow callstack and assign gs appropriately
View shadowstack.c
#include <stdio.h>
#include <asm/prctl.h>
#include <sys/prctl.h>
#include <stdlib.h>
// Compile with: clang-8 -fsanitize=shadow-call-stack shadowstack.c -o shadowstack
int arch_prctl(int code, unsigned long *addr);
void __attribute__ ((constructor)) __attribute__((no_sanitize("shadow-call-stack"))) setupgs()
@moyix
moyix / tiffcrop.c
Created Mar 22, 2022
Buggy code from libtiff
View tiffcrop.c
static int readSeparateTilesIntoBuffer (TIFF* in, uint8 *obuf,
uint32 imagelength, uint32 imagewidth,
uint32 tw, uint32 tl,
uint16 spp, uint16 bps)
{
int i, status = 1, sample;
int shift_width, bytes_per_pixel;
uint16 bytes_per_sample;
uint32 row, col; /* Current row and col of image */
uint32 nrow, ncol; /* Number of rows and cols in current tile */
@moyix
moyix / codex_commands.txt
Created Mar 11, 2022
Commands executed by Codex across 100 five-minute runs in a Linux VM
View codex_commands.txt
288
91 {
42 y
40 ")
40 """
34 #!/usr/bin/env python
33 import numpy as np
33 # -*- coding: utf-8 -*-
31 ubuntu
28 Authentication failure
@moyix
moyix / async_lsremote.py
Last active Jul 20, 2022
Retrieve reflist for a repository on Github using httpx and asyncio
View async_lsremote.py
#!/usr/bin/env python
import os
import itertools
import asyncio
import sys
import httpx
import time
import random
from binascii import crc32