sirus20x6

## Pretraining-LLM.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              9 stars
            
          
                ritwikraha
                / Pretraining-LLM.md
            
            
              Last active
              May 24, 2024 01:06
            
              
                Pretraining of Large Language Models
              
          
    Pretraining


A Map for Studying Pre-training in LLMs

Data Collection

General Text Data
Specialized Data


Data Preprocessing

Quality Filtering


Deduplication


## i2s_stream.c
//Almost entirelly lifted directly from https://github.com/igrr/esp32-cam-demo
//Just clocked a little differently and has chained buffers.
//This totes works with the I2S bus on the ESP32 for READING 16 wires simultaneously.
//Can be clocked off of I2S's internal controller or an external clock.

#define I2S_D0 4
#define I2S_D1 5
#define I2S_D2 18
#define I2S_D3 19
#define I2S_D4 36
	//Almost entirelly lifted directly from https://github.com/igrr/esp32-cam-demo
	//Just clocked a little differently and has chained buffers.
	//This totes works with the I2S bus on the ESP32 for READING 16 wires simultaneously.
	//Can be clocked off of I2S's internal controller or an external clock.

	#define I2S_D0 4
	#define I2S_D1 5
	#define I2S_D2 18
	#define I2S_D3 19
	#define I2S_D4 36