SwathiMystery

Some remarks on Large Language Models

Yoav Goldberg, January 2023

Audience: I assume you heard of chatGPT, maybe played with it a little, and was imressed by it (or tried very hard not to be). And that you also heard that it is "a large language model". And maybe that it "solved natural language understanding". Here is a short personal perspective of my thoughts of this (and similar) models, and where we stand with respect to language understanding.

Intro

Around 2014-2017, right within the rise of neural-network based methods for NLP, I was giving a semi-academic-semi-popsci lecture, revolving around the story that achieving perfect language modeling is equivalent to being as intelligent as a human. Somewhere around the same time I was also asked in an academic panel "what would you do if you were given infinite compute and no need to worry about labour costs" to which I cockily responded "I would train a really huge language model, just to show that it doesn't solve everything!". We

SwathiMystery

@Override public void launchTask(final ExecutorDriver executorDriver, final Protos.TaskInfo taskInfo) { ... }

SwathiMystery

MesosSchedulerDriver schedulerDriver = new MesosSchedulerDriver(scheduler, frameworkBuilder.build(), args[0]);
int status = schedulerDriver.run() == Protos.Status.DRIVER_STOPPED ? 0 : 1;
schedulerDriver.stop();
System.exit(status);

SwathiMystery

Scheduler scheduler = args.length == 1 ?				new PinScheduler(userProfileExecutorInfo, userBoardExecutorInfo) :				new PinScheduler(userProfileExecutorInfo, userBoardExecutorInfo, Integer.parseInt(args[1]), args[2]);

SwathiMystery

Protos.FrameworkInfo.Builder frameworkBuilder = Protos.FrameworkInfo.newBuilder().setFailoverTimeout(120000)
.setUser("").setName("Pinspider Framework");

SwathiMystery

Protos.ExecutorInfo userProfileExecutorInfo = Protos.ExecutorInfo.newBuilder().setExecutorId(Protos.ExecutorID.newBuilder().setValue("PinUserProfileExecutor")).setCommand(commandInfoUserProfile).setName("PinUserProfileExecutor Java").setSource("java").build();

SwathiMystery

# init_mongo.sh script 
......................
sudo yum update
df -h
cat /proc/partitions 
sudo fdisk -l
sudo su
yum install lvm2
fdisk -l
umount /mnt

SwathiMystery

versions-maven plugin:

Set
mvn versions:set -DnewVersion=0.3.0

Commit 
mvn versions:commit

Revert
mvn versions:revert

SwathiMystery

#!/bin/sh
#
# init.d script with LSB support.
#
# Copyright (c) 2007 Javier Fernandez-Sanguino <jfs@debian.org>
#
# This is free software; you may redistribute it and/or modify
# it under the terms of the GNU General Public License as
# published by the Free Software Foundation; either version 2,
# or (at your option) any later version.

SwathiMystery

sudo apt-get install oozie oozie-client
sudo apt-get install mysql-server-5.1

mysql -u root -p password
create database oozie;
grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';
grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';
exit

sudo vim /etc/oozie/conf/oozie-site.xml