yoavg

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of chatGPT, maybe played with it a little, and was imressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.

Intro

Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labour costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We

ncammarata

import scipy.ndimage as nd
from scipy.ndimage import gaussian_filter
import lucid.optvis.transform as transform

def mean_alpha():
    def inner(T):
        input_t = T("input")
        return 1.0 - tf.reduce_mean(input_t[..., 3:])

    return objectives.Objective(inner)

leferrad

#!/usr/bin/env python
# -*- coding: utf-8 -*-

""""""

import matplotlib
matplotlib.use("TkAgg")

from datetime import datetime as dt
import numpy as np

rspeer

domenic

Redirecting GitHub Pages after a repository move

The problem

You have a repository, call it alice/repo. You would like to transfer it to the user bob, so it will become bob/repo.

However, you make heavy use of the GitHub Pages feature, so that people are often accessing https://alice.github.io/repo/. GitHub will helpfully redirect all of your repository stuff hosted on github.com after the move, but will not redirect the GitHub Pages hosted on github.io.

The solution

ghaiklor

iTerm2

NOTE: this gist is outdated, refer to this repository instead - https://github.com/ghaiklor/iterm-fish-fisher-osx

Install iTerm 2

• Download and install iTerm2 (it has better color fidelity than the built in Terminal).

Install Color Scheme

chebee7i

from __future__ import division

import numpy as np

def multinomial_ci(counts, alpha):
    """
    Calculate a simultaneous (1-alpha) * 100 percent confidence interval.

    Parameters
    ----------

matthewmueller

# OSX for Hackers (Mavericks/Yosemite)
#
# Source: https://gist.github.com/brandonb927/3195465

#!/bin/sh

# Some things taken from here
# https://github.com/mathiasbynens/dotfiles/blob/master/.osx

# Ask for the administrator password upfront

pocotan001

• 🎨 when improving the format/structure of the code
• 🚀 when improving performance
• ✏️ when writing docs
• 💡 new idea
• 🚧 work in progress
• ➕ when adding feature
• ➖ when removing feature
• 🔈 when adding logging
• 🔇 when reducing logging
• 🐛 when fixing a bug

dearaujoj

git tag -d TagName && git push origin :refs/tags/TagName