Skip to content

Instantly share code, notes, and snippets.

Last active August 4, 2023 20:20
Show Gist options
  • Save jedi4ever/be65552811a3966d438be46d6e053898 to your computer and use it in GitHub Desktop.
Save jedi4ever/be65552811a3966d438be46d6e053898 to your computer and use it in GitHub Desktop.
improved devcontainer for langchain
# This is a Dockerfile for running unit tests
# Use the Python base image
FROM python:3.11.2-bullseye
# install bash completion - to make make work
RUN apt-get update && apt-get install -y bash-completion
# Install nodejs & yarn (for docs)
# inspired by
echo "deb bullseye main" > /etc/apt/sources.list.d/nodesource.list && \
wget -qO- | apt-key add - && \
echo "deb stable main" > /etc/apt/sources.list.d/yarn.list && \
wget -qO- | apt-key add - && \
apt-get update && \
apt-get upgrade -yqq && \
apt-get install -yqq nodejs yarn
# Add user python - this also creates the homedir
RUN useradd -ms /bin/bash python
# Create some dirs
# used for poetry env
RUN mkdir -p /opt/poetry
# used for poetry virtual envs
RUN mkdir -p /opt/virtual-envs
# set the correct ownership
RUN chown python /opt/virtual-envs
RUN chown python /opt/poetry
# Switch to python user
USER python
# Create a Python virtual environment for Poetry and install it
# Define the version of Poetry to install (default is 1.4.2)
ARG POETRY_HOME=/opt/poetry
RUN python3 -m venv ${POETRY_HOME} && \
$POETRY_HOME/bin/pip install --upgrade pip && \
$POETRY_HOME/bin/pip install poetry==${POETRY_VERSION}
# Test if Poetry is installed in the expected path
RUN echo "Poetry version:" && $POETRY_HOME/bin/poetry --version
# add poetry to paths
RUN echo $PATH
RUN echo "export PATH=$PATH:/opt/poetry/bin" >> ~/.bashrc
RUN cat ~/.bashrc
RUN echo $PATH
# Set the working directory for the app
# This is the path where the code gets mounted
# This is crucial as poetry will calculate the name of the poetry env based on the path
WORKDIR /workspaces/
# force the settings to not store things in code-dir/.venv
# Copy only the dependency files for installation
# We put them on the right place so poetry can find them again
COPY libs/langchain/pyproject.toml libs/langchain/poetry.lock libs/langchain/poetry.toml libs/langchain/
# Install the Poetry dependencies (this layer will be cached as long as the dependencies don't change)
RUN cd libs/langchain/ && $POETRY_HOME/bin/poetry install --sync --no-interaction --no-ansi --no-root --with dev,test
# Unable to find installation candidates for nvidia-cudnn-cu11 (
#RUN $POETRY_HOME/bin/poetry install --no-interaction --no-ansi --with dev,test,test_integration
# adding extended-testing makes it fail
# CRITICAL:root:A GDAL API version must be specified. Provide a path to gdal-config using a GDAL_CONFIG environment variable or use a GDAL_VERSION environment variable.
#RUN $POETRY_HOME/bin/poetry install --no-interaction --no-ansi --with dev,test -E extended-testing
# This is more to calculate a hash for bumping
# We copy this (though it's overridden by the mount)
# This allows us to detect if something changed in libs/langchain
# and if needed re-install local
COPY libs/langchain libs/langchain
# If there is a need to rebuild , the previous poetry packages will be cached
RUN cd libs/langchain/ && $POETRY_HOME/bin/poetry install --sync --with dev,test
# note this also depends on langchain
# todo: figure out how to also cache this based in the files , so we don't need to reinstall
COPY pyproject.toml poetry.lock poetry.toml ./
RUN $POETRY_HOME/bin/poetry install --sync --no-interaction --no-ansi --with docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment