matplotlib font question on Stack Overflow

README.md

matplotlib mathtext font warning: Font 'default' does not have a glyph

When trying to run a script (given below in the minimal working example) that sets non-default values for matplotlib.rcParams I am getting the maptplotlib warnings:

$ python example.py
Font 'default' does not have a glyph for '-' [U+2212], substituting with a dummy symbol.
Font 'default' does not have a glyph for '-' [U+2212], substituting with a dummy symbol.
Font 'default' does not have a glyph for '-' [U+2212], substituting with a dummy symbol.
Font 'default' does not have a glyph for '-' [U+2212], substituting with a dummy symbol.
Font 'default' does not have a glyph for '-' [U+2212], substituting with a dummy symbol.
Font 'default' does not have a glyph for '-' [U+2212], substituting with a dummy symbol.
Font 'default' does not have a glyph for '-' [U+2212], substituting with a dummy symbol.
Font 'default' does not have a glyph for '-' [U+2212], substituting with a dummy symbol.

This is on Ubuntu 20.04 with Tex Live 2020 installed from source that is findable to matplotlib and Python 3.8.6 built from source using pyenv

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
$ which latex
/usr/local/texlive/2020/bin/x86_64-linux/latex
$ $(pyenv which python) --version --version
Python 3.8.6 (default, Jan  5 2021, 00:14:15)
[GCC 9.3.0]

However, when I try to build the following Docker image for a minimal failing example I am unable to replicate the error.

Minimal Working/Failing Example

For the following Dockerfile

FROM ubuntu:20.04

RUN apt-get update -y && \
    apt-get install -y \
        fontconfig \
        fonts-dejavu \
        fonts-freefont-ttf \
        python3 \
        python3-dev \
        python3-pip \
        python3-venv \
        vim && \
        apt-get -y autoclean && \
        apt-get -y autoremove && \
        rm -rf /var/lib/apt/lists/*

RUN python3 -m pip install --upgrade --no-cache-dir pip setuptools wheel && \
    python3 -m pip install --no-cache-dir "mplhep~=0.2.9" && \
    python3 -m pip list

WORKDIR /code

COPY example.py example.py

with example.py of

import numpy as np
import matplotlib.pyplot as plt
import mplhep


def make_plot(title=None):
    fig, ax = plt.subplots()
    x = np.linspace(0, 10, 101)
    y = np.square(x)
    ax.plot(x, y)
    ax.semilogy()

    ax.set_xlabel("$x$")
    ax.set_ylabel("$x^2$")
    if title is not None:
        ax.set_title(title)

    return fig, ax


def main():
    image_types = ["pdf", "png"]
    fig, ax = make_plot("Default matplotlib settings")

    for type in image_types:
        fig.savefig(f"default.{type}")

    mplhep.style.set_style("ATLAS")
    # above is equivalent to: plt.style.use(mplhep.style.ATLAS)
    fig, ax = make_plot("mplhep ATLAS style")
    for type in image_types:
        fig.savefig(f"ATLAS_style.{type}")


if __name__ == "__main__":
    main()

if built with

docker build . \
--pull \
-f Dockerfile \
-t matplotlib-font-question:debug-local

giving

$ docker run --rm -ti matplotlib-font-question:debug-local /bin/bash -c "pip list"
Package         Version
--------------- ---------
certifi         2020.12.5
chardet         4.0.0
cycler          0.10.0
idna            2.10
kiwisolver      1.3.1
matplotlib      3.3.3
mplhep          0.2.9
numpy           1.19.5
packaging       20.8
Pillow          8.1.0
pip             20.3.3
pyparsing       2.4.7
python-dateutil 2.8.1
requests        2.25.1
scipy           1.6.0
setuptools      51.1.1
six             1.15.0
urllib3         1.26.2
wheel           0.36.2

and then run with

docker run --rm --user 1000:1000 -v $PWD:$PWD -w $PWD matplotlib-font-question:debug-local /bin/bash -c "python3 /code/example.py"

the resulting figures will be produced with no warnings or errors, unlike locally.

This is confusing given that if I check the font libraries installed on the Docker image (which does not have LaTeX)

$ docker run --rm matplotlib-font-question:debug-local /bin/bash -c "apt list --installed | grep font"

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

fontconfig-config/now 2.13.1-2ubuntu3 all [installed,local]
fontconfig/now 2.13.1-2ubuntu3 amd64 [installed,local]
fonts-dejavu-core/now 2.37-1 all [installed,local]
fonts-dejavu-extra/now 2.37-1 all [installed,local]
fonts-dejavu/now 2.37-1 all [installed,local]
fonts-freefont-ttf/now 20120503-10 all [installed,local]
libfontconfig1/now 2.13.1-2ubuntu3 amd64 [installed,local]

all of the found font libraries are also found on my local machine

$ apt list --installed | grep font | grep "fontconfig\|dejavu"

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

fontconfig-config/focal,focal,now 2.13.1-2ubuntu3 all [installed,automatic]
fontconfig/focal,now 2.13.1-2ubuntu3 amd64 [installed,automatic]
fonts-dejavu-core/focal,focal,now 2.37-1 all [installed,automatic]
fonts-dejavu-extra/focal,focal,now 2.37-1 all [installed]
fonts-dejavu/focal,focal,now 2.37-1 all [installed]
libfontconfig1-dev/focal,now 2.13.1-2ubuntu3 amd64 [installed,automatic]
libfontconfig1/focal,now 2.13.1-2ubuntu3 amd64 [installed]

So it is unclear to me why I am getting warnings locally when I have all the same fonts installed and more and when they additionally give the same information about matplotlib.rcParams:

Docker

$ docker run --rm -ti matplotlib-font-question:debug-local
root@2e6cff635604:/code# python3
Python 3.8.5 (default, Jul 28 2020, 12:59:40)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> import mplhep
>>> mplhep.style.set_style("ATLAS")
>>> for key, value in matplotlib.rcParams.items():
...     if "mathtext" in key:
...             print(key, value)
...
axes.formatter.use_mathtext False
mathtext.bf sans:bold
mathtext.cal cursive
mathtext.default rm
mathtext.fallback cm
mathtext.fallback_to_cm None
mathtext.fontset stixsans
mathtext.it sans:italic
mathtext.rm sans
mathtext.sf sans
mathtext.tt monospace
>>> for key, value in matplotlib.rcParams.items():
...     if "default" in key:
...             print(key, value)
...
mathtext.default rm

local machine

$ rm ~/.cache/matplotlib/fontlist-v330.json # not a cache issue
$ python
Python 3.8.6 (default, Jan  5 2021, 00:14:15)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import matplotlib
>>> import mplhep
>>> mplhep.style.set_style("ATLAS")
>>> for key, value in matplotlib.rcParams.items():
...     if "mathtext" in key:
...             print(key, value)
...
axes.formatter.use_mathtext False
mathtext.bf sans:bold
mathtext.cal cursive
mathtext.default rm
mathtext.fallback cm
mathtext.fallback_to_cm None
mathtext.fontset stixsans
mathtext.it sans:italic
mathtext.rm sans
mathtext.sf sans
mathtext.tt monospace
>>> for key, value in matplotlib.rcParams.items():
...     if "default" in key:
...             print(key, value)
...
mathtext.default rm

Questions

Why do the results differ between my local machine and the Docker container? How would I go about attempting to debug this in general, as this is about as far as I know how to get when debugging font issues?

Relevant GitHub Issues

I ran into this issue before on Ubuntu 19.10 (described in the Issues linked below), but things just magically started working then and I do not know how or why as the approaches I had there aren't working here.

• mplhep Issue 178: ATLAS style susceptible to matplotlib math font bugs
• matplotlib Issue 14829: Font/Mathfont and fallbacks buggy
• GitHub Gist that contains all of this code for this Stack Overflow question

apt.txt

fontconfig
fonts-dejavu
fonts-freefont-ttf
python3
python3-dev
python3-pip
python3-venv
vim

docker_ATLAS_style.png

docker_default.png

Dockerfile

FROM ubuntu:20.04

RUN apt-get update -y && \
    apt-get install -y \
        fontconfig \
        fonts-dejavu \
        fonts-freefont-ttf \
        python3 \
        python3-dev \
        python3-pip \
        python3-venv \
        vim && \
        apt-get -y autoclean && \
        apt-get -y autoremove && \
        rm -rf /var/lib/apt/lists/*

RUN python3 -m pip install --upgrade --no-cache-dir pip setuptools wheel && \
    python3 -m pip install --no-cache-dir "mplhep~=0.2.9" && \
    python3 -m pip list

WORKDIR /code

COPY example.py example.py

example.py

import numpy as np
import matplotlib.pyplot as plt
import mplhep


def make_plot(title=None):
    fig, ax = plt.subplots()
    x = np.linspace(0, 10, 101)
    y = np.square(x)
    ax.plot(x, y)
    ax.semilogy()

    ax.set_xlabel("$x$")
    ax.set_ylabel("$x^2$")
    if title is not None:
        ax.set_title(title)

    return fig, ax


def main():
    image_types = ["pdf", "png"]
    fig, ax = make_plot("Default matplotlib settings")

    for type in image_types:
        fig.savefig(f"default.{type}")

    mplhep.style.set_style("ATLAS")
    # above is equivalent to: plt.style.use(mplhep.style.ATLAS)
    fig, ax = make_plot("mplhep ATLAS style")
    for type in image_types:
        fig.savefig(f"ATLAS_style.{type}")


if __name__ == "__main__":
    main()

local_ATLAS_style.png

local_default.png

Makefile

default: debug

debug:
	docker build . \
	--pull \
	-f Dockerfile \
	-t matplotlib-font-question:debug-local

The fix ended up being more subtle than just the font list cache. I had more than just fontlist-v330.json in the matplotlib cache

$ ls -htra ~/.cache/matplotlib/
..  tex.cache  fontlist-v330.json  .

so rebuilding the font list cache with

import matplotlib.font_manager
matplotlib.font_manager._rebuild()

or

rm ~/.cache/matplotlib/fontlist-v330.json

wasn't sufficient. Instead the entire matplotlib cache needs to be updated, which is most easily done with

rm ~/.cache/matplotlib/*

Why do the results differ between my local machine and the Docker container?

As the Docker image has no previous history it never had a cache in the first place, so there was nothing to mess it up.

How would I go about attempting to debug this in general, as this is about as far as I know how to get when debugging font issues?

I'm now going to make sure that I totally clean and remove my caches before trying to do anything else.