sueszli/allennlp.md

## allennlp.md

      
    Raw
  

              allennlp.md
            
          
    how to install


conda:
works great. no issues.


docker:
works well - unless you want to use the latest stable vscode version as of may 2024.
see: microsoft/vscode#174632


google colab:
not an option.
see: sebastian-hofstaetter/teaching#129 (ok these guys might have made it, but i don't know how)
see: googlecolab/colabtools#4212 (comment)
see: https://www.reddit.com/r/GoogleColab/comments/13605i3/comment/jinqaz3/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button


docker

docker-compose.yml
services:
    main:
        container_name: noodle-retrieval
        build: .

        # mount volume
        volumes:
            - type: bind
              source: .
              target: /workspace
        working_dir: /workspace

        # config for pytorch IPC
        # for gpu access see: https://docs.docker.com/compose/gpu-support/
        stdin_open: true
        tty: true
        ipc: host

        # map ports to host
        ports:
            - '8888:8888'
Dockerfile
FROM --platform=linux/amd64 python:3.6.15-slim-bullseye

RUN apt-get update && apt-get install -y make build-essential
RUN pip install jsonnet --no-build-isolation  
RUN pip install --upgrade pip
RUN pip install allennlp==1.2.2

RUN pip install blingfire==0.1.7
RUN pip install PyYAML==5.4
RUN pip install transformers==3.5.1
RUN pip install --find-links https://download.pytorch.org/whl/torch_stable.html torch==1.6.0

RUN pip install overrides

# convenience
RUN apt-get install -y git
RUN pip install numpy pandas matplotlib seaborn

# jupyter server
# attaching vscode can be buggy: https://github.com/microsoft/vscode-remote-release/issues/8169#issuecomment-1543987445
RUN pip install jupyter jupyterlab jupyter_contrib_nbextensions
ENV JUPYTER_ENABLE_LAB=yes
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--port=8888", "--allow-root", "--no-browser", "--ServerApp.token=''", "--ServerApp.password=''", "--ServerApp.allow_origin='*'", "--ServerApp.disable_check_xsrf=True", "--ServerApp.allow_root=True", "--ServerApp.open_browser=False", "--ServerApp.disable_check_xsrf=True", "--ServerApp.disable_check_xsrf=True"]
EXPOSE 8888
run.sh
# ------------------------------------------- start
docker-compose up

docker ps --all
docker exec -it noodle-retrieval /bin/bash
open http://localhost:8888/lab

# ------------------------------------------- stop
docker-compose down

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
docker rmi $(docker images -q)

yes | docker container prune
yes | docker image prune
yes | docker volume prune
yes | docker network prune
yes | docker system prune

docker ps --all
docker images
docker system df
docker volume ls
docker network ls


conda

conda is a lot more GPU programming friendly.
# ----------------------------------------------------------------------------- install conda
brew install --cask miniconda
conda update conda

conda init zsh
conda init bash
exit # restart shell

conda config --set auto_activate_base false # disable auto-activation
conda config --env --set subdir osx-64 # emulate x86_64 (reset for other projects)

# ----------------------------------------------------------------------------- start
conda activate base

# see: https://blog.balasundar.com/install-older-versions-of-python-using-miniconda-on-mac-m1
conda create --yes --name noodle-retrieval python=3.6.12 anaconda
conda activate noodle-retrieval

# setuptools requirements (order matters)
pip install 'pyqtwebengine<5.13'
pip install 'pyqt5<5.13'
pip install pyls-black
pip install --no-deps ruamel.yaml # kind of a hack

# allennlp requirements (order matters)
pip install --upgrade pip
pip install --upgrade setuptools
pip install --upgrade wheel
pip install --upgrade Cython
pip install think --no-build-isolation
pip install spacy --no-build-isolation
pip install jsonnet --no-build-isolation

# project requirements
pip install allennlp==1.2.2 --no-build-isolation
pip install blingfire==0.1.7
pip install PyYAML==5.4
pip install transformers==3.5.1
pip install --find-links https://download.pytorch.org/whl/torch_stable.html torch==1.6.0
pip install overrides

# convenience
pip install black isort flake8 mypy
pip install numpy pandas matplotlib seaborn

# ----------------------------------------------------------------------------- stop
conda deactivate
conda remove --yes --name noodle-retrieval --all
conda env list
^ note: the amd64 / osx-64 flags are only necessary if you're using an arm64 chip (apple silicon)