Skip to content

Instantly share code, notes, and snippets.

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of chatGPT, maybe played with it a little, and was imressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.


Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labour costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We

Dammmien /
Last active June 12, 2024 13:29
wget cheat sheet
# POST a JSON file and redirect output to stdout
wget -q -O - --header="Content-Type:application/json" --post-file=foo.json
# Download a complete website
wget -m -r -linf -k -p -q -E -e robots=off
# But it may be sufficient
wget -mpk
# Download all images of a website
bjtitus /
Last active May 18, 2022 09:14 — forked from rodw/
A shell script to backup your Github organization's repos.
# A simple script to backup an organization's GitHub repositories.
GHBU_BACKUP_DIR=${GHBU_BACKUP_DIR-"github-backups"} # where to place the backup files
GHBU_ORG=${GHBU_ORG-"<CHANGE-ME>"} # the GitHub organization whose repos will be backed up
GHBU_API=${GHBU_API-""} # base URI for the GitHub API
GHBU_GITHOST=${GHBU_GITHOST-"<CHANGE-ME>"} # the GitHub hostname (see comments)
# I recommend using an API token so it is easily trackable and removable.
# Note that you MUST have SSH keys for a user with the access to all repos set up
brandonkelly /
Last active February 7, 2024 15:20
Templating in EE vs. Craft
kenzie / craft.conf
Last active May 17, 2018 17:48
Nginx virtual host configuration for Craft CMS, PHP5-FPM, NGINX 1.2.1 and craft/config/general.php for friendly URLs.
server {
listen 80;
root /var/www/;
index index.php index.html index.htm;
location / {
try_files $uri $uri/ @rewrites;
ndarville /
Last active January 13, 2024 17:27
Business models based on the compiled list at I find the link very hard to browse, so I made a simple version in Markdown instead.

Business Models


Models Examples
Display ads Yahoo!
Search ads Google
tdreyno / airports.json
Created December 13, 2012 18:50
JSON data for airports and their locations
This file has been truncated, but you can view the full file.
"code": "AAA",
"lat": "-17.3595",
"lon": "-145.494",
"name": "Anaa Airport",
"city": "Anaa",
"state": "Tuamotu-Gambier",
"country": "French Polynesia",
"woeid": "12512819",
require 'bcrypt'
module EasyAuth
# To generate a crypted password (in irb):
# require 'easy_auth'
# EasyAuth.encrypt_password('my_password') # Put returned array in AUTHORIZED_USERS