Julian David Rath julian-r

## llama-7b-m1.md

      
              1 file
            
          
              21 forks
            
          
              16 comments
            
          
              125 stars
            
          
                cedrickchee
                / llama-7b-m1.md
            
            
              Last active
              July 13, 2024 04:59
            
              
                4 Steps in Running LLaMA-7B on a M1 MacBook with `llama.cpp`
              
          
    4 Steps in Running LLaMA-7B on a M1 MacBook

The large language models usability

The problem with large language models is that you can’t run these locally on your laptop. Thanks to Georgi Gerganov and his llama.cpp project, it is now possible to run Meta’s LLaMA on a single computer without a dedicated GPU.
Running LLaMA

There are multiple steps involved in running LLaMA locally on a M1 Mac after downloading the model weights.

  
## Dockerfile
FROM nvidia/cuda:11.1-cudnn8-devel-ubuntu20.04

ENV NVIDIA_VISIBLE_DEVICES ${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics

# add new sudo user
ENV USERNAME autoware
ENV HOME /home/$USERNAME
RUN useradd -m $USERNAME && \
        echo "$USERNAME:$USERNAME" | chpasswd && \
	FROM nvidia/cuda:11.1-cudnn8-devel-ubuntu20.04

	ENV NVIDIA_VISIBLE_DEVICES ${NVIDIA_VISIBLE_DEVICES:-all}
	ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics

	# add new sudo user
	ENV USERNAME autoware
	ENV HOME /home/$USERNAME
	RUN useradd -m $USERNAME && \
	echo "$USERNAME:$USERNAME" \| chpasswd && \