Skip to content

Instantly share code, notes, and snippets.

@gregelin
Last active April 29, 2019 11:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gregelin/ac0b983ca767fbb2776ef77da06f4e0e to your computer and use it in GitHub Desktop.
Save gregelin/ac0b983ca767fbb2776ef77da06f4e0e to your computer and use it in GitHub Desktop.
Testing GovReady-Q

This gist contains information on code scanning soon to be added to https://govready-q.readthedocs.io/en/latest/Test.html

Testing

Running Tests

GovReady-Q's unit tests and integration tests are currently combined. Our integration tests uses Selenium to simulate user interactions with the interface.

To run the integration tests, you'll also need to install chromedriver:

sudo apt-get install chromium-chromedriver   (on Ubuntu)
brew install chromedriver                    (on Mac)

Navigate within your terminal to GovReady-Q top level directory.

Then run the test suite with:

./manage.py test

NOTE: Depending on your Python3 configuration, you may need to run:

python3 manage.py test

To selectively run tests from individual modules:

# test rendering of guided modules
./manage.py test guidedmodules

# test general siteapp logic
./manage.py test siteapp

# test discussion functionality
./manage.py test discussion

Or to selectively run tests from individual classes or methods:

# run tests from individual test class
./manage.py test siteapp.tests.GeneralTests

# run tests from individual test method
./manage.py test siteapp.tests.GeneralTests.test_login

Test Coverage Report

To produce a code coverage report, run the tests with coverage:

coverage run --source='.' --branch manage.py test
coverage report

Code Scanning and Analysis

GovReady-Q is a Python web application written on top of the Django framework and uses a variety of industry standard Javascript libraries. See Software Requirements for high level view and the requirement*.txt files for detailed view.

GovReady-Q's Python application code is found in the *.py files in the following directories and their subdirectories:

  • discussion/
  • guidedmodules/
  • siteapp/

The small manage.py script in the root directory is part of the Django framework. We use bash utilities scripts (*.sh) to automate installation and maintenance tasks of the code base. Python scripts in .circleci directory are used within our Continuous Implementation pipeline.

Simple Static Code Analysis

To run a static code analysis with our typical settings:

bandit -s B101,B110,B603 -r discussion/ guidedmodules/ siteapp/

We use -s on the command-line and nosec in limited places in the source code to disable some checks that are determined after review to be false positives.

Detailed Static and Dynamic Code Analysis

We periodically scan GovReady-Q's code base with more traditional/powerful tools and remediate critical and high vulnerabilities.

To scan GovReady-Q's codebase, you will need to configure your tools to scan Python code. You are looking for the *.py files across the code base.

To scan or do other penetration tests on the code base, we recommend deploying GovReady-Q with Docker.

Dependency Management and Vulnerability Testing

Our requirements.txt file is designed to work with pip install --require-hashes, which ensures that every installed dependency matches a hash stored in this repository. The option requires that every dependency (including dependencies of dependencies) be listed, pinned to a version number, and paired with a hash. We therefore don't manually edit requirements.txt. Instead, we place our immediate dependencies in requirements.in and run requirements_txt_updater.sh (which calls pip-tools's pip-compile command) to update the requirements.txt file for production.

Continuous integration is set up with CircleCI at https://circleci.com/gh/GovReady/govready-q and performs unit tests, integration tests, and security checks on our dependencies.

  1. CI runs requirements_txt_checker.sh which ensures requirements.txt is in sync with requirements.in. This script is set up to run against any similar files as well, such as MySQL-specific requirements_mysql.* files.
  2. CI checks that there are no known vulnerabilities in the dependencies using pyup.io.
  3. CI checks that all packages are up to date with upstream sources (unless the package and its latest upstream version are listed in requirements_txt_checker_ignoreupdates.txt).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment