moyix

Using Linear Algebra to Convert a Large Code Model

Background

The SalesForce CodeGen models are a family of large language models trained on a large amount of natural language data and then fine-tuned on specialized datasets of code. Models of size 350M, 2B, 6B, and 16B parameters are provided in three flavors:

• nl, the base model trained on The Pile, a large natural language dataset compiled by EleutherAI
• multi, which is fine-tuned from the nl model on a dataset of code in multiple languages, scraped from GitHub, and
• mono, which is fine-tuned from the multi model on Python code only.

d-mart

#!/usr/bin/env bash
set -e

## Prereqs:
## brew install heroku
## brew install kafkacat
## heroku login -i

## Usage exmaple
## $ APP=roadie-admin-qa ./kafka-listener.sh gig.events

vjeux

I'm taking down this post. I just posted this as a side comment to explain a sentence on my latest blog post. This wasn't meant to be #1 on HN to start a huge war on functional programming... The thoughts are not well formed enough to have a huge audience. Sorry for all the people reading this. And please, don't dig through the history...

mlavin

Running the Example

Start the worker:

celery -A tasks worker --loglevel=info -c 2 --pidfile=celery.pid

In another terminal send 6 tasks:

python script.py

jakevdp

"""
Bayesian Generative Classifier
------------------------------
"""
# Author: Jake Vanderplas <jakevdp@cs.washington.edu>

import numpy as np
from sklearn.neighbors.kde import KernelDensity
from sklearn.mixture import GMM
from sklearn.base import BaseEstimator, clone