Radi Radev radiradev

## normcore-llm.md

      
              1 file
            
          
              0 forks
            
          
              0 comments
            
          
              0 stars
            
          
                radiradev
                / normcore-llm.md
            
            
              Created
              August 31, 2023 12:54
                — forked from veekaybee/normcore-llm.md
            
              
                Normcore LLM Reads
              
          
    Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts and experience preferred (super rare at this point).
My own notes from a few months back.
Background


Survey of LLMS
Self-attention and transformer networks


## Dockerfile.masked_autoencoder
FROM rradev/minkowski-engine:latest

RUN pip install timm tensorboardx einops pytorch-lightning

CMD ["bash"]

# the minkowski-engine:latest uses the dockerfile defined here https://gist.github.com/radiradev/7af53500133773948c813d3e133e62c8

## Dockerfile.ME.DepthWise
FROM deeplearnphysics/larcv2:ub20.04-cuda11.6-pytorch1.13-larndsim

# Install ninja
RUN pip install ninja


ENV TORCH_CUDA_ARCH_LIST="6.0 7.0 7.5 8.0"

# Install MinkowskiEngine with DepthwiseConvolution from GitHub
RUN pip install -U --target=/usr/local/lib/python3.8/dist-packages \
	FROM rradev/minkowski-engine:latest

	RUN pip install timm tensorboardx einops pytorch-lightning

	CMD ["bash"]

	# the minkowski-engine:latest uses the dockerfile defined here https://gist.github.com/radiradev/7af53500133773948c813d3e133e62c8
	FROM deeplearnphysics/larcv2:ub20.04-cuda11.6-pytorch1.13-larndsim

	# Install ninja
	RUN pip install ninja


	ENV TORCH_CUDA_ARCH_LIST="6.0 7.0 7.5 8.0"

	# Install MinkowskiEngine with DepthwiseConvolution from GitHub
	RUN pip install -U --target=/usr/local/lib/python3.8/dist-packages \