Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Comprehensive Python testing on Travis CI
language: python
# ===== Linux ======
dist: xenial
python:
- 2.7
- 3.5
- 3.6
- 3.7
matrix:
include:
# ======= OSX ========
# ----- changes in Travis images means this doesn't work for versions before 3.7.5 ---
# - name: "Python 2.7.14 on macOS 10.13"
# os: osx
# osx_image: xcode9.3 # Python 2.7.14_2 running on macOS 10.13
# language: shell # 'language: python' is an error on Travis CI macOS
# before_install:
# - python --version
# # - pip install -U pip
# # - python -m pip install --upgrade pip
# - pip install pytest --user
# - pip install codecov --user
# install: pip install ".[test]" --user
# script: python -m pytest
# after_success: python -m codecov
# - name: "Python 3.6.5 on macOS 10.13"
# os: osx
# osx_image: xcode9.4 # Python 3.6.5 running on macOS 10.13
# language: shell # 'language: python' is an error on Travis CI macOS
# before_install:
# - python3 --version
# - pip3 install -U pip
# - pip3 install -U pytest
# - pip3 install codecov
# script: python3 -m pytest
# after_success: python 3 -m codecov
- name: "Python 3.7.5 on macOS 10.14"
os: osx
osx_image: xcode10.2 # Python 3.7.5 running on macOS 10.14.3
language: shell # 'language: python' is an error on Travis CI macOS
before_install:
- python3 --version
- pip3 install -U pip
- pip3 install -U pytest
- pip3 install codecov
script: python3 -m pytest
after_success: python 3 -m codecov
- name: "Python 3.8.0 on macOS 10.14"
os: osx
osx_image: xcode11.3 # Python 3.8.0 running on macOS 10.14.6
language: shell # 'language: python' is an error on Travis CI macOS
before_install:
- python3 --version
- pip3 install -U pip
- pip3 install -U pytest
- pip3 install codecov
script: python3 -m pytest
after_success: python 3 -m codecov
# ====== WINDOWS =========
- name: "Python 2.7 on Windows"
os: windows # Windows 10.0.17134 N/A Build 17134
language: shell # 'language: python' errors Travis CI Windows
before_install:
- choco install python2
- python --version
- python -m pip install --upgrade pip
- pip install --upgrade pytest
- pip install codecov
env: PATH=/c/Python27:/c/Python27/Scripts:$PATH
- name: "Python 3.5.4 on Windows"
os: windows # Windows 10.0.17134 N/A Build 17134
language: shell # 'language: python' is an error on Travis CI Windows
before_install:
- choco install python --version 3.5.4
- python --version
- python -m pip install --upgrade pip
- pip3 install --upgrade pytest
- pip3 install codecov
env: PATH=/c/Python35:/c/Python35/Scripts:$PATH
- name: "Python 3.6.8 on Windows"
os: windows # Windows 10.0.17134 N/A Build 17134
language: shell # 'language: python' is an error on Travis CI Windows
before_install:
- choco install python --version 3.6.8
- python --version
- python -m pip install --upgrade pip
- pip3 install --upgrade pytest
- pip3 install codecov
env: PATH=/c/Python36:/c/Python36/Scripts:$PATH
- name: "Python 3.7.4 on Windows"
os: windows # Windows 10.0.17134 N/A Build 17134
language: shell # 'language: python' is an error on Travis CI Windows
before_install:
- choco install python --version 3.7.4
- python --version
- python -m pip install --upgrade pip
- pip3 install --upgrade pytest
- pip3 install codecov
env: PATH=/c/Python37:/c/Python37/Scripts:$PATH
before_install:
- python --version
- pip install -U pip
- pip install -U pytest
- pip install codecov
install:
- pip install ".[test]"
script: pytest
after_success:
- codecov # submit coverage
@cclauss

This comment has been minimized.

Copy link

@cclauss cclauss commented Apr 9, 2020

Would it be possible to update for Python 3.8 and osx_image: xcode11.3 (and drop legacy Python)?

Also pip will be getting a real dependency resolver soon so python3 -m pip install --upgrade codecov pytest might be better.

@shaypal5

This comment has been minimized.

Copy link
Owner Author

@shaypal5 shaypal5 commented Apr 26, 2020

Well, I've added the 3.8.0 job for macOS, but it doesn't make since to remove jobs for legacy Python, since this is a template, meant for people to take and remove what they don't need. Some projects still test on legacy Python, even Python 2 after its deprecation.

About the dependency resolver - when will this happen? And why add this before this happens?

@shaypal5

This comment has been minimized.

Copy link
Owner Author

@shaypal5 shaypal5 commented Apr 26, 2020

Also, thank you for the input! :)

@cclauss

This comment has been minimized.

Copy link

@cclauss cclauss commented Apr 26, 2020

What about Python 3.8 on Linux just after line 8?

Why prepare now for pip dependency resolution? Because it is now in beta and preparing has no negative effects but not preparing will mean that anyone who grabs this template before you change modify it will not benefit from pip having true dependency resolution.

@pmeier

This comment has been minimized.

Copy link

@pmeier pmeier commented Apr 27, 2020

Thanks for this awesome template 💯


@shaypal5 Could you have a look if the macOs python versions are still correct? I've tried your trick with the differnt osx_images but they resolve as follows:

osx_image python3 --version
xcode9.4 Python 3.7.5
xcode10.2 Python 3.7.5
xcode11.3 Python 3.7.7

Here is a build of a sample project of mine.

@shaypal5

This comment has been minimized.

Copy link
Owner Author

@shaypal5 shaypal5 commented Apr 27, 2020

Hey @pmeier,

Unfortunately, changes in Travis images means this doesn't work for versions before 3.7.5 anymore.

his used to work - Travis even put my blog post on this up on their own official blog - but I guess they keep rebuilding the images, and so now even old xcode images come with Python 3.7.5.

This is the image that gets pulled by the job, not something we can control. At the time, when I was figuring this out, this was the only way I found to easily test different Python versions on macOS. I think building it from source each time is too expensive an overhead, so we can bare to not test this on earlier Pythons specifically on macOS (they ARE tested on linux, after all).

I've encountered this for one of my projects as well:
shaypal5/cachier#23

I had to move to only testing Python 3.7.5 and Python 3.8.0 on macOS. :\

@pmeier

This comment has been minimized.

Copy link

@pmeier pmeier commented Apr 27, 2020

Hm, that is unfortunate. Heads up for you: even with xcode11.3 you are only testing against 3.7. See for example one of your latest builds.


Travis even pulled my blog post on this to their own official blog

Are you sure? I came to this gist by this

https://blog.travis-ci.com/2019-08-07-extensive-python-testing-on-travis-ci

@shaypal5

This comment has been minimized.

Copy link
Owner Author

@shaypal5 shaypal5 commented Apr 27, 2020

Sorry, this was an incorrect use of American English. I'm a non-native speaker. I meant they even put it on their official blog up back in the day (August 2019), but that this has become obsolete since (I guess rather recently, but I can't tell). :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment