Skip to content

Instantly share code, notes, and snippets.

@orenitamar
Last active March 22, 2024 05:13
Show Gist options
  • Save orenitamar/f29fb15db3b0d13178c1c4dd611adce2 to your computer and use it in GitHub Desktop.
Save orenitamar/f29fb15db3b0d13178c1c4dd611adce2 to your computer and use it in GitHub Desktop.
Installing numpy, scipy, pandas and matplotlib in Alpine (Docker)
# Below are the dependencies required for installing the common combination of numpy, scipy, pandas and matplotlib
# in an Alpine based Docker image.
FROM alpine:3.4
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install numpy scipy pandas matplotlib
@aviatorBeijing
Copy link

Better use following:
RUN pip install --no-cache-dir numpy scipy pandas matplotlib

@windlessuser
Copy link

This was working until about yesterday. Could use some help with a fix. I think something got updated, I'm not sure what.

@stetsonwood
Copy link

numpy updated and is not compatible with alpine. use numpy==1.14.3 or an older version and it should work

@windlessuser
Copy link

It still fails on older versions of Numpy where it was working before. I think the update was in one of the C headers it needs, During the compile, it complains about some typing issues. I've gone from version 1.14.2 - 1.13.0. Even the last known version that was working 1.14.2

@LKozlowski
Copy link

I had the same problem as you and found that pandas is still trying to install most recent version of numpy even if you already satisfied its requirement of numpy > 1.9 so I resolved it by:

  • install numpy version < 1.14.4 so 1.14.3 is fine
  • install python-dateutil (which is a pandas dependency)
  • install pandas without dependencies by running pip install --no-deps pandas==0.23.0
  • install other packages

@gbroccolo
Copy link

gbroccolo commented Jun 12, 2018

Also pytz>=2011k is a Pandas dependency, let's include it together with python-dateuti.

@byaminov
Copy link

Thanks a lot for the tip to use numpy==1.14.3! 👍

@tgsmith61591
Copy link

Is this still working? I'm getting:

ERROR: unsatisfiable constraints:
  so:libgfortran.so.5 (missing):
    required by: openblas-ilp64-0.3.3-r2[so:libgfortran.so.5] openblas-ilp64-0.3.3-r2[so:libgfortran.so.5] openblas-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]

@flyrev
Copy link

flyrev commented Nov 12, 2018

I'm getting

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
fetch http://dl-8.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  so:libgfortran.so.5 (missing):
    required by:
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
The command '/bin/sh -c apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev' returned a non-zero code: 4

@mdiazcl
Copy link

mdiazcl commented Nov 28, 2018

I'm getting

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
fetch http://dl-8.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  so:libgfortran.so.5 (missing):
    required by:
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
The command '/bin/sh -c apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev' returned a non-zero code: 4

Had the same issue... changed the repos from dl-8 to CDN and it worked!

Here's the new Dockerfile

FROM alpine:3.4
RUN echo "http://dl-cdn.alpinelinux.org/alpine/latest-stable/main" > /etc/apk/repositories
RUN echo "http://dl-cdn.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install numpy scipy pandas matplotlib

@JensRantil
Copy link

gcc is likely not needed as build-base depends on it.

@MartinThoma
Copy link

MartinThoma commented May 6, 2019

This works for me:

FROM: python:alpine

RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories

# Add HDF5 support
RUN apk add --no-cache --allow-untrusted --repository http://dl-3.alpinelinux.org/alpine/edge/testing hdf5 hdf5-dev
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h

RUN pip install numpy

The rest could be installed via the dependencies automatically by pip.

@orhun
Copy link

orhun commented May 7, 2019

@MartinThoma what's the HDF5 library for?

@promehul
Copy link

This isn't working..
ERROR: Command "/usr/local/bin/python -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-install-2u_c2pqi/numpy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-gfsn5nsl/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-2u_c2pqi/numpy/
This ois i am getting..

@ozichert
Copy link

First add py-numpy, since directly installing with pip doesn't work.
RUN apk add py-numpy
In the next step install it with pip.
RUN python -m pip install numpy

@felixraboacct
Copy link

First add py-numpy, since directly installing with pip doesn't work.
RUN apk add py-numpy
In the next step install it with pip.
RUN python -m pip install numpy

pip install numpy fails ... tries to install it normally. And yet without this step Python gives module not found error. So how do you tell Python that the package was already installed via linux package manager?

@Faylixe
Copy link

Faylixe commented Aug 13, 2019

I figured out how to make numpy package from Alpine repository to work. Actually py-numpy does not really install bumpy but py3-numpy does.

But even though, py3-numpy installs libraries into /usr/lib/python3.7/site-packagesdirectory but default Python module path does not use it :

$ docker run -it python:3-alpine sh
/ # apk add --update --no-cache py3-numpy
...
/ # python
>>> import numpy
>>> ... module not found ...
>>> import sys
>>> sys.path
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7', '/usr/local/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/site-packages']

I fixed the problem by settings $PYTHONPATH environment variable to /usr/liblocated site packages :

FROM python:3-alpine

RUN apk add --update --no-cache py3-numpy
ENV PYTHONPATH=/usr/lib/python3.7/site-packages

EDIT: you might also need to install py-numpy-dev to get Numpy C headers required by other Python compiled libraries.

@sandervh14
Copy link

sandervh14 commented Apr 6, 2020

I had various problems on the way starting FROM alpine, FROM python:alpine, but with the following I had a smooth docker build experience:

FROM python:slim
pip install numpy scipy

I assume you can add matplotlib and pandas as extra packages without problems.
See https://pythonspeed.com/articles/alpine-docker-python/ for my choice to start from python:slim.

@photostu
Copy link

photostu commented Apr 7, 2020

@sanderzz91 Thanks!

@mkdthanga
Copy link

mkdthanga commented Jun 29, 2020

I tried using both alpine:latest image and python:slim image, both throws an error:
1)alpine:latest
Dockerfile:
FROM alpine:latest

ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ENV PROJ_DIR=/usr
ENV PYTHONPATH=/usr/lib/python3.7/site-packages

RUN addgroup -S appuser ; adduser -S appuser -G appuser -h /app ;
chown appuser:appuser /app

WORKDIR /app
COPY --chown=appuser:appuser / /app

RUN apk add --update --no-cache python3 libpq uwsgi-python3;
apk add --no-cache python3-dev py3-pip alpine-sdk postgresql-dev postgresql;
apk add --update --no-cache py3-numpy;
apk add libgfortran;
apk add lapack-dev;
apk add proj proj-dev;
apk add proj-util;
pip3 install --no-cache-dir --upgrade pip ;
pip3 install --upgrade setuptools;
pip3 install --no-cache-dir -r requirements.txt

EXPOSE 5432

CMD ["python3", "/app/subscriber_db.py", "/app/resolver.py"]

Requirements:
numpy==1.14.3
pandas==1.0.5
sqlalchemy==1.3.16
paho-mqtt==1.5.0
psycopg2==2.8.5
six==1.14.0
requests==2.24.0
cython
pyproj
scipy==1.3.1
statistics==1.0.3.5
matplotlib

Console Output:
PS C:\Users\thangaraj\PycharmProjects> docker build -t grs-docker .
Collecting scipy==1.3.1
Downloading scipy-1.3.1.tar.gz (23.6 MB)
Installing build dependencies: started
Installing build dependencies: still running...
Installing build dependencies: still running...
Installing build dependencies: still running...
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
.
.
.
.
numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.8/site-packages/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp9m_79xyv Check the logs for full command output.
The command '/bin/sh -c apk add --update --no-cache python3 libpq uwsgi-python3; apk add --no-cache python3-dev py3-pip alpine-sdk postgresql-dev postgresql; apk add --update --no-cache py3-numpy; apk add libgfortran; apk add
lapack-dev; apk add proj proj-dev; apk add proj-util; pip3 install --no-cache-dir --upgrade pip ; pip3 install --upgrade setuptools; pip3 install --no-cache-dir -r requirements.txt' returned a non-zero code: 1

2) python:slim (I think its not suited for hosting postgresql DB)

Console logs:
PS C:\Users\thangaraj\PycharmProjects> docker build -t grs-docker .
Sending build context to Docker daemon 1.394MB
Step 1/10 : FROM python:slim
slim: Pulling from library/python
8559a31e96f4: Pull complete
62e60f3ef11e: Pull complete
93c8ae153782: Pull complete
ea222f757df7: Pull complete
e97d3933bbbe: Pull complete
Digest: sha256:938fd520a888e9dbac3de374b8ba495cc50fe96440030264a40f733052001895
Status: Downloaded newer image for python:slim

.
.
.

chown: invalid user: ‘appuser:appuser’
The command '/bin/sh -c addgroup -S appuser ; adduser -S appuser -G appuser -h /app ; chown appuser:appuser /app' returned a non-zero code: 1

Could anyone help/ advice me with these errors? I would really appreciate it. Thanks in advance.

@rendorHaevyn
Copy link

Thanks @orenitamar.

@mkdthanga: The following worked for me - I simply used python3-alpine build, and removed python, python-dev and py-pip from the apk command (line 5, OP):

FROM python:3-alpine
WORKDIR /usr/source/app
COPY requirements.txt ./

RUN apk update
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install --no-cache-dir -r requirements.txt

@PJCAfonso
Copy link

I'm getting

fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
fetch http://dl-8.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  so:libgfortran.so.5 (missing):
    required by:
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-ilp64-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
                 openblas-0.3.3-r2[so:libgfortran.so.5]
The command '/bin/sh -c apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev' returned a non-zero code: 4

Had the same issue... changed the repos from dl-8 to CDN and it worked!

Here's the new Dockerfile

FROM alpine:3.4
RUN echo "http://dl-cdn.alpinelinux.org/alpine/latest-stable/main" > /etc/apk/repositories
RUN echo "http://dl-cdn.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install numpy scipy pandas matplotlib

Thanks man your answers solved my problem!

@aboutmedicine
Copy link

I had various problems on the way starting FROM alpine, FROM python:alpine, but with the following I had a smooth docker build experience:

FROM python:slim
pip install numpy scipy

I assume you can add matplotlib and pandas as extra packages without problems.
See https://pythonspeed.com/articles/alpine-docker-python/ for my choice to start from python:slim.

Thanks - this is a great option for my use case

@pPanda-beta
Copy link

In case you cant move to slim-buster, the natural answer to this is
pip install --extra-index-url https://alpine-wheels.github.io/index ...

@samanthahw21
Copy link

Hi, can anyone help. I tried to execute the line in the Docker file but it takes very long to run .
Can anyone can help and what should i do ?
i didn't include panda version in my requirements - i tried but it run error . should i include and what version ? thanks

Pull a pre-built alpine docker image with nginx and python3 installed

FROM tiangolo/uwsgi-nginx-flask:python3.6-alpine3.7
ENV LISTEN_PORT=8000
EXPOSE 8000
COPY /app /app
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install pandas

@Frikster
Copy link

Thanks @orenitamar.

@mkdthanga: The following worked for me - I simply used python3-alpine build, and removed python, python-dev and py-pip from the apk command (line 5, OP):

FROM python:3-alpine
WORKDIR /usr/source/app
COPY requirements.txt ./

RUN apk update
RUN echo "http://dl-8.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk --no-cache --update-cache add gcc gfortran build-base wget freetype-dev libpng-dev openblas-dev
RUN ln -s /usr/include/locale.h /usr/include/xlocale.h
RUN pip install --no-cache-dir -r requirements.txt

This leads to ERROR: Failed building wheel for pyarrow for me

@PaleNeutron
Copy link

PaleNeutron commented Jul 19, 2022

@Frikster If you are using python:3-alpine which means you prefer the latest python3.10 version, you could consider using package from alpine instead of from pip.

This way need python package from alpine!!!!

FROM alpine:3.16
RUN apk --no-cache --update-cache add  python3 py3-pip py3-arrow  py3-pandas # and py3-anything package need to be compiled
RUN pip install --no-cache-dir -r requirements.txt

@Tejas9019
Copy link

=> ERROR [6/8] RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev 5.3s

[6/8] RUN apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev:
nity/x86_64/APKINDEX.tar.gz
#11 5.137 WARNING: Ignoring http://dl-8.alpinelinux.org/alpine/edge/community: DNS lookup error
#11 5.215 ERROR: unable to select packages:
#11 5.252 python (no such package):
#11 5.252 required by: world[python]
#11 5.252 python-dev (no such package):
#11 5.252 required by: world[python-dev]


executor failed running [/bin/sh -c apk --no-cache --update-cache add gcc gfortran python python-dev py-pip build-base wget freetype-dev libpng-dev openblas-dev]: exit code: 2

I am getting error like this pls can anyone help me out

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment