vihari

This document contains rough calculations of the power consumption and CO2e due to AI. If you're a ML/AI researcher or a heavy user of AI-related applications, please read on.

Common emitters (for comparison)

	CO2e (in Kg)	Ref.
Lifetime car emissions	57,152	[1]
London->NY flight (per person)	179	[3]
London->NY flight (overall)	59,600	[3]
UK Collective (per-annum)	CO2e (in Million ton)	[4]

vihari

The following is a short summary of AI alignment that you may find handy.

Imagine a maid robot with which we are interacting.

• Outer alignment problem, aka Reward hacking, task misspecification, specification gaming.

  You ask for a coffee. It understood the assignment, but grabbed it from your father and gave it to you. You got the coffee, but that is not how you want it.
  Problem: Your values and preferences are not encoded.
  Challenging part: How to specify innumerably many preferences and ensure they are adhered?
  Methods: Tune it to be honest, harmless and helpful: RLHF. Feedback at scale for super-intelligence: Scalable oversight, weak-to-strong generalisation, super-alignment. Explain the process instead of simply specifying the outcome: process-based feedback.

vihari

def csd(embeds, label_placeholder, domain_placeholder, num_classes, num_domains, K=1, is_training=False, scope=""):  
  """CSD layer to be used as a replacement for your final classification layer
   
  Args:
    embeds (tensor): final layer representations of dim 2
    label_placeholder (tensor): tf tensor with label index of dim 1
    domain_placeholder (tensor): tf tensor with domain index of dim 1 -- set to all zeros when testing
    num_classes (int): Number of label classes: scalar
    num_domains (int): Number of domains: scalar
    K (int): Number of domain specific components to use. should be >=1 and <=num_domains-1

vihari

def csd(self, embeds, labels, domains, num_classes, num_domains, K=1, is_training=False, scope=""):  
  """CSD layer to be used as a replacement for your final classification layer
   
  Args:
    embeds (tensor): final layer representations of dim 2
    labels (tensor): tf tensor with label index of dim 1
    domains (tensor): tf tensor with domain index of dim 1 -- set to all zeros when testing
    num_classes (int): Number of label classes: scalar
    num_domains (int): Number of domains: scalar
    K (int): Number of domain specific components to use. should be >=1 and <=num_domains-1

vihari

def rankL(np_rank):
    r = int(np_rank[-1])
    _l = 0
    for k in range(1, r+1):
        _l += 1./k
    return np.float32(_l)


"""
labels are assumed to be 1 hot encoded

vihari

"""
Exports data into tfrecords to the save_dir
train_data, validation_data and test_data are list of tuples containing: (image_data, label, domain id, file_path (if available))
"""
def export_tfrecord(save_dir, train_data, validation_data, test_data):
    import math
    import itertools

    random.shuffle(train_data)
    

vihari

#!/usr/bin/python
"""
Is Convergence rate of perceptron update dependent on the input dimensionality?
"""
import numpy as np

N = 100
lr = 1
for sz in [5, 10, 100, 500, 1000, 5000, 10000]:
    dat = np.random.normal(scale=10, size=[N, sz])

vihari

#!/usr/bin/python
"""
I have written this script to see if the optimal policy solution obtained by Linear Programming is same as the one that would be obtained by a Gradient based method
The Gradient based approach is implemented with Tensorflow and LP with a scipy library. 
I found that n of the nk constraints (greater than or equal to) for the solution become equalities for the solution obtained by LinProg and that is not so in the case of Gradient-based approach. This shows that the constraints do not sufficiently specify the solution, but works in case of LP because of the way it finds the solution.

TF - Tensorflow's Gradient Descent
Run: python <script> [LP|TF]
"""
import tensorflow as tf

vihari

#!/usr/bin/perl -w
# set the query
$query = "www.google.com\\\/maps\\\/embed";
# path to CommonCrawl dataset
$S3_URL = "s3://commoncrawl/crawl-data/CC-MAIN-2017-26/segments/";
$all = `aws s3 ls $S3_URL|perl -ane 'print "\$F[1]\n"'`;
print "Launching search for: $query...\n";
@segs = split(/[\n\s]+/, $all);
$nf=0;
for ($i=0;$i<=$#segs;$i++){

vihari

"""
The default tf.Print op goes to STDERR
Use the function below to direct the output to stdout instead
Usage: 
> x=tf.ones([1, 2])
> y=tf.zeros([1, 3])
> p = x*x
> p = tf_print(p, [x, y], "hello")
> p.eval()
hello [[ 0.  0.]]