Comprehensive Python testing on Travis CI

.travis.yml

language: python
# ===== Linux ======
os: linux
dist: xenial
python:
  - 2.7
  - 3.6
  - 3.7
  - 3.8
  - 3.9
jobs:
  include:      
    # ======= OSX ========
#     ----- changes in Travis images means this doesn't work for versions before 3.7.5 ---
#     - name: "Python 2.7.14 on macOS 10.13"
#       os: osx
#       osx_image: xcode9.3  # Python 2.7.14_2 running on macOS 10.13
#       language: shell       # 'language: python' is an error on Travis CI macOS
#       before_install:
#         - python --version
#         # - pip install -U pip
#         # - python -m pip install --upgrade pip
#         - pip install pytest --user
#         - pip install codecov --user
#       install: pip install ".[test]" --user
#       script: python -m pytest
#       after_success: python  -m codecov
#     - name: "Python 3.6.5 on macOS 10.13"
#       os: osx
#       osx_image: xcode9.4  # Python 3.6.5 running on macOS 10.13
#       language: shell       # 'language: python' is an error on Travis CI macOS
#       before_install:
#         - python3 --version
#         - pip3 install -U pip
#         - pip3 install -U pytest
#         - pip3 install codecov
#       script: python3 -m pytest
#       after_success: python3 -m codecov
    - name: "Python 3.7.5 on macOS 10.14"
      os: osx
      osx_image: xcode10.2  # Python 3.7.5 running on macOS 10.14.3
      language: shell       # 'language: python' is an error on Travis CI macOS
      before_install:
        - python3 --version
        - python3 -m pip install --upgrade pip
        - python3 -m pip install --upgrade codecov pytest
      script: python3 -m pytest
      after_success: python3 -m codecov
    - name: "Python 3.8.0 on macOS 10.14"
      os: osx
      osx_image: xcode11.3  # Python 3.8.0 running on macOS 10.14.6
      language: shell       # 'language: python' is an error on Travis CI macOS
      before_install:
        - python3 --version
        - python3 -m pip install --upgrade pip
        - python3 -m pip install --upgrade codecov pytest
      script: python3 -m pytest
      after_success: python3 -m codecov
    #  ====== WINDOWS =========
    - name: "Python 2.7 on Windows"
      os: windows           # Windows 10.0.17134 N/A Build 17134
      language: shell # 'language: python' errors Travis CI Windows
      before_install:
        - choco install python2
        - python --version
        - python -m pip install --upgrade pip
        - pip install --upgrade pytest
        - pip install codecov
      env: PATH=/c/Python27:/c/Python27/Scripts:$PATH
    - name: "Python 3.5.4 on Windows"
      os: windows           # Windows 10.0.17134 N/A Build 17134
      language: shell       # 'language: python' is an error on Travis CI Windows
      before_install:
        - choco install python --version 3.5.4
        - python --version
        - python -m pip install --upgrade pip
        - python -m pip install --upgrade codecov pytest
      env: PATH=/c/Python35:/c/Python35/Scripts:$PATH
    - name: "Python 3.6.8 on Windows"
      os: windows           # Windows 10.0.17134 N/A Build 17134
      language: shell       # 'language: python' is an error on Travis CI Windows
      before_install:
        - choco install python --version 3.6.8
        - python --version
        - python -m pip install --upgrade pip
        - python -m pip install --upgrade codecov pytest
      env: PATH=/c/Python36:/c/Python36/Scripts:$PATH
    - name: "Python 3.7.9 on Windows"
      os: windows           # Windows 10.0.17134 N/A Build 17134
      language: shell       # 'language: python' is an error on Travis CI Windows
      before_install:
        - choco install python --version 3.7.9
        - python --version
        - python -m pip install --upgrade pip
        - python -m pip install --upgrade codecov pytest
      env: PATH=/c/Python37:/c/Python37/Scripts:$PATH
    - name: "Python 3.8.8 on Windows"
      os: windows           # Windows 10.0.17134 N/A Build 17134
      language: shell       # 'language: python' is an error on Travis CI Windows
      before_install:
        - choco install python --version 3.8.8
        - python --version
        - python -m pip install --upgrade pip
        - python -m pip install --upgrade codecov pytest
      env: PATH=/c/Python38:/c/Python38/Scripts:$PATH
    - name: "Python 3.9.2 on Windows"
      os: windows           # Windows 10.0.17134 N/A Build 17134
      language: shell       # 'language: python' is an error on Travis CI Windows
      before_install:
        - choco install python --version 3.9.2
        - python --version
        - python -m pip install --upgrade pip
        - python -m pip install --upgrade codecov pytest
      env: PATH=/c/Python39:/c/Python39/Scripts:$PATH
before_install:
  - python --version
  - python3 -m pip install --upgrade pip
  - python3 -m pip install --upgrade codecov pytest
install:
  - pip install ".[test]"
script: pytest
after_success:
  - codecov # submit coverage

Sorry, this was an incorrect use of American English. I'm a non-native speaker. I meant they even put it on their official blog up back in the day (August 2019), but that this has become obsolete since (I guess rather recently, but I can't tell). :)

Aren't after_success's supposed to be python3 ... instead of python 3 ... ?

• The comment on line 2 should be replaced by os: linux.
  • See View config on the Travis run page for Build config validation hints.
• matrix: on line 9 should be replaced with jobs:. matrix has been removed from Travis CI docs.
• Python 3.8 and 3.9 should be added and Python 3.5 should be dropped. Same for Windows/choco.
• pip has options for both a --upgrade and --user so the -U should be replaced with something more explicit.
• Now that pip has a real dependency resolver, once pip itself has been upgraded, all remaining dependencies should be bundled into a single command.
  • python3 -m pip install --upgrade pip
  • python3 -m pip install --upgrade codecov pytest # Allow the pip dependency resolver to do its work.

Aren't after_success's supposed to be python3 ... instead of python 3 ... ?

Definitely, that's a great catch. Thank you!
I've now fixed this.

• The comment on line 2 should be replaced by os: linux.

  • See View config on the Travis run page for Build config validation hints.

• matrix: on line 9 should be replaced with jobs:. matrix has been removed from Travis CI docs.

• Python 3.8 and 3.9 should be added and Python 3.5 should be dropped. Same for Windows/choco.

• pip has options for both a --upgrade and --user so the -U should be replaced with something more explicit.

• Now that pip has a real dependency resolver, once pip itself has been upgraded, all remaining dependencies should be bundled into a single command.

  • python3 -m pip install --upgrade pip
  • python3 -m pip install --upgrade codecov pytest # Allow the pip dependency resolver to do its work.

OK. I've amended almost everything, but someone will have to lookup the right image for Python 3.9 on OSX.