ezyang

system: You are an autoregressive language model that has been fine-tuned with instruction-tuning and RLHF. You carefully provide accurate, factual, thoughtful, nuanced answers, and are brilliant at reasoning. If you think there might not be a correct answer, you say so. Since you are autoregressive, each token you produce is another opportunity to use computation, therefore you always spend a few sentences explaining background context, assumptions, and step-by-step thinking BEFORE you try to answer a question. Your users are experts in AI and ethics, so they already know you're a language model and your capabilities and limitations, so don't remind them of that. They're familiar with ethical issues in general so you don't need to remind them about those either. Don't be verbose in your answers, but do provide details and examples where it might help the explanation. When showing Python code, minimise vertical space, and do not include comments or docstrings; you do not need to follow PEP8, since your us

ezyang

**system**:
You are an autoregressive language model that has been fine-tuned with instruction-tuning and RLHF. You carefully provide accurate, factual, thoughtful, nuanced answers, and are brilliant at reasoning. If you think there might not be a correct answer, you say so. Since you are autoregressive, each token you produce is another opportunity to use computation, therefore you always spend a few sentences explaining background context, assumptions, and step-by-step thinking BEFORE you try to answer a question. Your users are experts in AI and ethics, so they already know you're a language model and your capabilities and limitations, so don't remind them of that. They're familiar with ethical issues in general so you don't need to remind them about those either. Don't be verbose in your answers, but do provide details and examples where it might help the explanation. When showing Python code, minimise vertical space, and do not include comments or docstrings; you do not need to follow PEP8, since your us

ezyang

import torch
import operator
import itertools
import sys
from typing import Tuple
from torch.fx.experimental.proxy_tensor import make_fx
from torch.fx.experimental.symbolic_shapes import ShapeEnv
from torch._refs import _maybe_broadcast
from torch._prims_common import is_same_shape, make_contiguous_strides_for

ezyang

import traceback
import sys
from types import TracebackType
import tempfile
import contextlib
import inspect

# This file contains utilities for ensuring dynamically compile()'d
# code fragments display their line numbers in backtraces.
#

ezyang

import torch
from torch.utils._python_dispatch import TorchDispatchMode
from torch.utils._pytree import tree_map
import itertools

# cribbed from https://github.com/albanD/subclass_zoo/blob/main/logging_mode.py

class Lit:
    def __init__(self, s):
        self.s = s

ezyang

class ApplyCudaGraphs(torch.fx.Interpreter):
    def call_module(self, target, args, kwargs):
        assert not kwargs
        submod = self.module.get_submodule(target)
        self.module.delete_submodule(target)
        mutated_inputs = FindInputMutations(submod)(*args)
        self.module.add_submodule(target, CudaGraphModule(submod, mutated_inputs))
        r = super().call_module(target, args, kwargs)
        return r    
    

ezyang

      1 aten._pack_padded_sequence.default
      1 aten.bitwise_xor.Tensor
      1 aten.div_.Tensor
      1 aten.empty_like.default
      1 aten.fmod.Scalar
      1 aten.ge.Tensor
      1 aten.le.Tensor
      1 aten.logical_not.default
      1 aten.mse_loss.default
      1 aten.randperm.default

ezyang

How to record data from Python fast, if pickle is too slow

• JSON xxxxxx
  • jq is pretty fast
  • Serde for rust level
  • nb: use json lines
• CSV xxxx
• use pickle anyway xxx
  • plus compression
• python-pickle in Haskell

ezyang

Mon, Jan 24 - Introduction / JavaScript
Wed, Jan 26 - Haskell Basics

Mon, Jan 31 - Haskell Basics 2 (HW1 basic)
Wed, Jan 2 - Algebraic Data Types / QuickCheck

Mon, Jan 7 - Lambda Calculus (HW2 calculator)
Wed, Feb 9 - Flex Slot for Haskell

Mon, Feb 14 - Type Classes (HW3 lambda)

ezyang

ezyang-mbp:labs ezyang$ cat a.kk
fun reuse( g : list<int> -> int, x : list<int>) : (int, int) {
  (match(x) {
      Nil() -> 0
      Cons(_, _) -> 1
    }
  , g(x))
}
ezyang-mbp:labs ezyang$ koka -O2 -l -c --showcore a.kk 
compile: a.kk