Ben Thompson tbenthompson

## attn_mask_bug.py
"""
An explanation for discrepancies between three different ways of generating tokens with Llama-2-7b-chat-hf:
1. Huggingface's `model.generate` defaults to using a mask with a zero in the first position and ones elsewhere.*
2. Huggingface `model.forward` defaults to using a mask with all ones.
3. VLLM defaults to using a mask with all ones, matching Huggingface `model.forward` but not `model.generate`.

* Why? I think maybe HF generate is excluding the BOS <s> token. Is this correct? I don't know!

I ran with:
- transformers 4.34.0

## config.py
"""
Usage:

A dataclass/YAML/CLI config system:
- write a @dataclass with your config options
- make sure every option has a default value
- include a `config: str = ""` option in the dataclass.
- write a main function that takes a single argument of the dataclass type
- decorate your main function with @dataclass_cli
- make sure your main function has a docstring.

## block_slider.ipynb

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                tbenthompson
                / block_slider.ipynb
            
            
              Created
              March 10, 2018 22:55
            
          
        Loading

      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## function_serialize.cpp
#include <iostream>
#include <sstream>
#include <fstream>

/* Serialize a function by writing out a pointer to its location in memory.
 * This will only work between two processes running identical binaries.
 *
 * One difficulty is ASLR:
 * Address space layout randomization (ASLR) puts functions in a different
 * place in memory everytime a program is loaded. Within a given binary
	"""
	An explanation for discrepancies between three different ways of generating tokens with Llama-2-7b-chat-hf:
	1. Huggingface's `model.generate` defaults to using a mask with a zero in the first position and ones elsewhere.*
	2. Huggingface `model.forward` defaults to using a mask with all ones.
	3. VLLM defaults to using a mask with all ones, matching Huggingface `model.forward` but not `model.generate`.

	* Why? I think maybe HF generate is excluding the BOS <s> token. Is this correct? I don't know!

	I ran with:
	- transformers 4.34.0
	"""
	Usage:

	A dataclass/YAML/CLI config system:
	- write a @dataclass with your config options
	- make sure every option has a default value
	- include a `config: str = ""` option in the dataclass.
	- write a main function that takes a single argument of the dataclass type
	- decorate your main function with @dataclass_cli
	- make sure your main function has a docstring.
	#include <iostream>
	#include <sstream>
	#include <fstream>

	/* Serialize a function by writing out a pointer to its location in memory.
	* This will only work between two processes running identical binaries.
	*
	* One difficulty is ASLR:
	* Address space layout randomization (ASLR) puts functions in a different
	* place in memory everytime a program is loaded. Within a given binary