Installing numpy, scipy, pandas and matplotlib in Alpine (Docker)

Dockerfile

# Below are the dependencies required for installing the common combination of numpy, scipy, pandas and matplotlib 
# in an Alpine based Docker image.
FROM alpine:3.4
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install numpy scipy pandas matplotlib

Better use following:
RUN pip install --no-cache-dir numpy scipy pandas matplotlib

This was working until about yesterday. Could use some help with a fix. I think something got updated, I'm not sure what.

numpy updated and is not compatible with alpine. use numpy==1.14.3 or an older version and it should work

It still fails on older versions of Numpy where it was working before. I think the update was in one of the C headers it needs, During the compile, it complains about some typing issues. I've gone from version 1.14.2 - 1.13.0. Even the last known version that was working 1.14.2

I had the same problem as you and found that pandas is still trying to install most recent version of numpy even if you already satisfied its requirement of numpy > 1.9 so I resolved it by:

• install numpy version < 1.14.4 so 1.14.3 is fine
• install python-dateutil (which is a pandas dependency)
• install pandas without dependencies by running pip install --no-deps pandas==0.23.0
• install other packages

Also pytz>=2011k is a Pandas dependency, let's include it together with python-dateuti.

Thanks a lot for the tip to use numpy==1.14.3! 👍

Is this still working? I'm getting:

ERROR: unsatisfiable constraints:
  so:libgfortran.so.5 (missing):
    required by: openblas-ilp64-0.3.3-r2[so:libgfortran.so.5] openblas-ilp64-0.3.3-r2[so:libgfortran.so.5] openblas-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]

I'm getting

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
fetch http://dl-8.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  so:libgfortran.so.5 (missing):
    required by:
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
The command '/bin/sh -c apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev' returned a non-zero code: 4

I'm getting

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
fetch http://dl-8.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  so:libgfortran.so.5 (missing):
    required by:
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
The command '/bin/sh -c apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev' returned a non-zero code: 4

Had the same issue... changed the repos from dl-8 to CDN and it worked!

Here's the new Dockerfile

FROM alpine:3.4
RUN echo "http://dl-cdn.alpinelinux.org/alpine/latest-stable/main" > /etc/apk/repositories
RUN echo "http://dl-cdn.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install numpy scipy pandas matplotlib

gcc is likely not needed as build-base depends on it.

This works for me:

FROM: python:alpine

RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories

# Add HDF5 support
RUN apk add --no-cache --allow-untrusted --repository http://dl-3.alpinelinux.org/alpine/edge/testing hdf5 hdf5-dev
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h

RUN pip install numpy

The rest could be installed via the dependencies automatically by pip.

@MartinThoma what's the HDF5 library for?

This isn't working..
ERROR: Command "/usr/local/bin/python -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-install-2u_c2pqi/numpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-gfsn5nsl/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-2u_c2pqi/numpy/
This ois i am getting..

First add py-numpy, since directly installing with pip doesn't work.
RUN apk add py-numpy
In the next step install it with pip.
RUN python -m pip install numpy

First add py-numpy, since directly installing with pip doesn't work.
RUN apk add py-numpy
In the next step install it with pip.
RUN python -m pip install numpy

pip install numpy fails ... tries to install it normally. And yet without this step Python gives module not found error. So how do you tell Python that the package was already installed via linux package manager?

I figured out how to make numpy package from Alpine repository to work. Actually py-numpy does not really install bumpy but py3-numpy does.

But even though, py3-numpy installs libraries into /usr/lib/python3.7/site-packagesdirectory but default Python module path does not use it :

$ docker run -it python:3-alpine sh
/ # apk add --update --no-cache py3-numpy
...
/ # python
>>> import numpy
>>> ... module not found ...
>>> import sys
>>> sys.path
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/site-packages']

I fixed the problem by settings $PYTHONPATH environment variable to /usr/liblocated site packages :

FROM python:3-alpine

RUN apk add --update --no-cache py3-numpy
ENV PYTHONPATH=/usr/lib/python3.7/site-packages

EDIT: you might also need to install py-numpy-dev to get Numpy C headers required by other Python compiled libraries.

I had various problems on the way starting FROM alpine, FROM python:alpine, but with the following I had a smooth docker build experience:

FROM python:slim
pip install numpy scipy

I assume you can add matplotlib and pandas as extra packages without problems.
See https://pythonspeed.com/articles/alpine-docker-python/ for my choice to start from python:slim.

@sanderzz91 Thanks!

I tried using both alpine:latest image and python:slim image, both throws an error:
1)alpine:latest
Dockerfile:
FROM alpine:latest

ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ENV PROJ_DIR=/usr
ENV PYTHONPATH=/usr/lib/python3.7/site-packages

RUN addgroup -S appuser ; adduser -S appuser -G appuser -h /app ;
chown appuser:appuser /app

WORKDIR /app
COPY --chown=appuser:appuser / /app

RUN apk add --update --no-cache python3 libpq uwsgi-python3;
apk add --no-cache python3-dev py3-pip alpine-sdk postgresql-dev postgresql;
apk add --update --no-cache py3-numpy;
apk add libgfortran;
apk add lapack-dev;
apk add proj proj-dev;
apk add proj-util;
pip3 install --no-cache-dir --upgrade pip ;
pip3 install --upgrade setuptools;
pip3 install --no-cache-dir -r requirements.txt

EXPOSE 5432

CMD ["python3", "/app/subscriber_db.py", "/app/resolver.py"]

Requirements:
numpy==1.14.3
pandas==1.0.5
sqlalchemy==1.3.16
paho-mqtt==1.5.0
psycopg2==2.8.5
six==1.14.0
requests==2.24.0
cython
pyproj
scipy==1.3.1
statistics==1.0.3.5
matplotlib

Console Output:
PS C:\Users\thangaraj\PycharmProjects> docker build -t grs-docker .
Collecting scipy==1.3.1
Downloading scipy-1.3.1.tar.gz (23.6 MB)
Installing build dependencies: started
Installing build dependencies: still running...
Installing build dependencies: still running...
Installing build dependencies: still running...
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
.
.
.
.
numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.8/site-packages/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp9m_79xyv Check the logs for full command output.
The command '/bin/sh -c apk add --update --no-cache python3 libpq uwsgi-python3; apk add --no-cache python3-dev py3-pip alpine-sdk postgresql-dev postgresql; apk add --update --no-cache py3-numpy; apk add libgfortran; apk add
lapack-dev; apk add proj proj-dev; apk add proj-util; pip3 install --no-cache-dir --upgrade pip ; pip3 install --upgrade setuptools; pip3 install --no-cache-dir -r requirements.txt' returned a non-zero code: 1

2) python:slim (I think its not suited for hosting postgresql DB)

Console logs:
PS C:\Users\thangaraj\PycharmProjects> docker build -t grs-docker .
Sending build context to Docker daemon 1.394MB
Step 1/10 : FROM python:slim
slim: Pulling from library/python
8559a31e96f4: Pull complete
62e60f3ef11e: Pull complete
93c8ae153782: Pull complete
ea222f757df7: Pull complete
e97d3933bbbe: Pull complete
Digest: sha256:938fd520a888e9dbac3de374b8ba495cc50fe96440030264a40f733052001895
Status: Downloaded newer image for python:slim

.
.
.

chown: invalid user: ‘appuser:appuser’
The command '/bin/sh -c addgroup -S appuser ; adduser -S appuser -G appuser -h /app ; chown appuser:appuser /app' returned a non-zero code: 1

Could anyone help/ advice me with these errors? I would really appreciate it. Thanks in advance.

Thanks @orenitamar.

@mkdthanga: The following worked for me - I simply used python3-alpine build, and removed python, python-dev and py-pip from the apk command (line 5, OP):

FROM python:3-alpine
WORKDIR /usr/source/app
COPY requirements.txt ./

RUN apk update
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install --no-cache-dir -r requirements.txt

I'm getting

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
fetch http://dl-8.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  so:libgfortran.so.5 (missing):
    required by:
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
The command '/bin/sh -c apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev' returned a non-zero code: 4

Had the same issue... changed the repos from dl-8 to CDN and it worked!

Here's the new Dockerfile

FROM alpine:3.4
RUN echo "http://dl-cdn.alpinelinux.org/alpine/latest-stable/main" > /etc/apk/repositories
RUN echo "http://dl-cdn.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install numpy scipy pandas matplotlib

Thanks man your answers solved my problem!

I had various problems on the way starting FROM alpine, FROM python:alpine, but with the following I had a smooth docker build experience:

FROM python:slim
pip install numpy scipy

I assume you can add matplotlib and pandas as extra packages without problems.
See https://pythonspeed.com/articles/alpine-docker-python/ for my choice to start from python:slim.

Thanks - this is a great option for my use case

In case you cant move to slim-buster, the natural answer to this is
pip install --extra-index-url https://alpine-wheels.github.io/index ...

Hi, can anyone help. I tried to execute the line in the Docker file but it takes very long to run .
Can anyone can help and what should i do ?
i didn't include panda version in my requirements - i tried but it run error . should i include and what version ? thanks

Pull a pre-built alpine docker image with nginx and python3 installed

FROM tiangolo/uwsgi-nginx-flask:python3.6-alpine3.7
ENV LISTEN_PORT=8000
EXPOSE 8000
COPY /app /app
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install pandas

Thanks @orenitamar.

@mkdthanga: The following worked for me - I simply used python3-alpine build, and removed python, python-dev and py-pip from the apk command (line 5, OP):

FROM python:3-alpine
WORKDIR /usr/source/app
COPY requirements.txt ./

RUN apk update
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install --no-cache-dir -r requirements.txt

This leads to ERROR: Failed building wheel for pyarrow for me

@Frikster If you are using python:3-alpine which means you prefer the latest python3.10 version, you could consider using package from alpine instead of from pip.

This way need python package from alpine!!!!

FROM alpine:3.16
RUN apk --no-cache --update-cache add  python3 py3-pip py3-arrow  py3-pandas # and py3-anything package need to be compiled
RUN pip install --no-cache-dir -r requirements.txt

=> ERROR [6/8] RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev 5.3s

[6/8] RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev:
nity/x86_64/APKINDEX.tar.gz
#11 5.137 WARNING: Ignoring http://dl-8.alpinelinux.org/alpine/edge/community: DNS lookup error
#11 5.215 ERROR: unable to select packages:
#11 5.252 python (no such package):
#11 5.252 required by: world[python]
#11 5.252 python-dev (no such package):
#11 5.252 required by: world[python-dev]

---

executor failed running [/bin/sh -c apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev]: exit code: 2

I am getting error like this pls can anyone help me out