Skip to content

Instantly share code, notes, and snippets.

@scop
Last active October 24, 2020 10:19
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save scop/5910ee85a1352988554e8b8d5d675e65 to your computer and use it in GitHub Desktop.
Save scop/5910ee85a1352988554e8b8d5d675e65 to your computer and use it in GitHub Desktop.
Why not install tests?

Why not install tests?

For the majority of users, test code is just waste of space and bandwidth. For example, for Home Assistant, in the full dependency set du -hc **/tests **/test reports 82M here (2020-10-24). Granted, most users don't install the full dependency set, so they are affected to a lesser extent, and that's a smallish fraction of the total dep set size, but still.

But isn't it good to install test code so it's around as a usage example? Test code is often not good code to be used as a general use example. It is often entangled with mockery, accesses things that should not be accessed in normal use to begin with, or accesses things in a way that is not supposed to be done outside of tests, etc, or all of that.

Isn't it good to be able to run tests suites on installed setups e.g. when diagnosing problems? Yes, that would be nice. But a bunch of things makes this less practical as it would seem on the surface.

  • Test code can have dependencies that are not the package's runtime dependencies. When these dependencies are not installed along with the package and its real runtime dependencies (hopefully at least these are always excluded!), users must take measures to install them before they are able to run the installed tests. Info what those dependencies are is likely not included in installs.
  • Test code may require write access to places where it's not ok in runtime setups, or leave trash behind. Granted, properly written test code does not have this problem, whether it's installed or not. But not that well written test code may "hurt" more when run if installed.
  • There is no standard way to run installed tests. One needs to figure it out on package by package basis. And docs for that are likely not included in installs.
  • Test code tends to be more often excluded from installs than installed (which is great). So one cannot generally assume to find tests installed.

Considering the above points, it's better just to go get the source distribution or repository and run the tests from there. Doing so won't clutter one's runtime setup that easily, the docs or examples how to run the tests are almost always there in form of direct docs or CI configs or build system configs or the like, as well as is the info on the test dependencies. One needs to prepare to do this anyway in general due to the last bullet point above.

Please do include test code in source distributions so anyone building from source is able to run the tests. But please do exclude them from runtime installs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment