Heroku CI compatible DiscoverRunner class for Django 1.10+ that does not require createdb or dropdb permissions.

heroku_test_runner.py

import os

from django.test.runner import DiscoverRunner

"""
WARNING:  WHEN USED INCORRECTLY THIS TEST RUNNER WILL DROP ALL TABLES IN YOUR PRODUCTION 
DATABASE!!!

Heroku does not give users createdb/dropdb permissions, therefore Heroku CI cannot run tests for django.
In order to fix this, use this test runner instead which attempts to minimally override the 
default test runner by a)  forcing keepdb=True to stop database create/drop, and b) by dropping all
tables after a test run and resetting the database to its initial blank state.

Usage:

1. In your django test settings file add the following two lines to ensure that the test 
database name is the same as the Heroku provided database name.

DATABASES['default'] = env.db('DATABASE_URL')  # or whatever you use to load the Heroku database settings
DATABASES['default']['TEST'] = {'NAME': DATABASES['default']['NAME']}

2. Set the testrunner to this file
TEST_RUNNER = 'your_modules.HerokuDiscoverRunner'

3. Set an environment variable on heroku CI of IS_HEROKU_TEST=1 to enable this runner, otherwise
the runner will exit as a safety measure.
"""

class HerokuDiscoverRunner(DiscoverRunner):
    def setup_databases(self, **kwargs):
        if not os.environ.get('IS_HEROKU_TEST'):
            raise ValueError(
                "The IS_HEROKU_TEST env variable must be set to enable this.  WARNING:  "
                "This test runner will wipe all tables in the database it targets!")
        self.keepdb = True
        return super(HerokuDiscoverRunner, self).setup_databases(**kwargs)

    def _wipe_tables(self, connection):
        with connection.cursor() as cursor:
            cursor.execute(
                """
                    DROP SCHEMA public CASCADE;
                    CREATE SCHEMA public;
                    GRANT ALL ON SCHEMA public TO postgres;
                    GRANT ALL ON SCHEMA public TO public;
                    COMMENT ON SCHEMA public IS 'standard public schema';
                """
            )

    def teardown_databases(self, old_config, **kwargs):
        self.keepdb = True
        for connection, old_name, destroy in old_config:
            if destroy:
                self._wipe_tables(connection)
        super(HerokuDiscoverRunner, self).teardown_databases(old_config, **kwargs)

For anyone who finds this, Heroku now allows in-dyno databases that allow creation of databases.

I'm on Django 2.0, Python 3.6, trying to get my tests to pass in Heroku CI. I tried in-dyno databases, I tried the HerokuTestRunner, I tried --keepdb. For the life of me, I just could not get around

Got an error creating the test database: permission denied to create database

Here's my hack. First, heroku_ci_test_settings.py

# DANGER DANGER DANGER: If you use these settings and run tests, they will
# install vacuous testing data on your database.

# Note that the default ./manage.py test behavior creates a fresh database
# to test on. But the whole point of Heroku is to magically create all the
# stuff you need. It's totally appropriate that THEY make the testing database
# and we just use it. Simple. Easy. But the assumption that your web framework
# will administer your testing databases is pretty well baked into Django,
# hence this kind of hack.

# See https://gist.github.com/cordery/d52d9ba44541fabaf4b012f4e62d675b
# for more discussion.

# The environments.test section in app.json is really important here.

from django.test.runner import DiscoverRunner
from <my app>.settings import *

TEST_RUNNER = '<my app>.heroku_ci_test_settings.ExistingDBDiscoverRunner'

class ExistingDBDiscoverRunner(DiscoverRunner):
  def setup_databases(self, **kwargs):
    pass

  def teardown_databases(self, old_config, **kwargs):
    pass

Then this is in app.json

"environments": {
  "test": {
    "scripts": {
      "test-setup": "python manage.py migrate",
      "test": "python manage.py test --settings=<my app>.heroku_ci_test_settings"
    }
  }
}

For anyone who finds this, Heroku now allows in-dyno databases that allow creation of databases.

Yes! This fixed it for me. I didn't need to change anything other than have this in my app.json to use the in-dyno db:

"environments": {
    "test": {
      "addons": ["heroku-postgresql:in-dyno"],
      "scripts": {
        "test": "python manage.py test"
      }
    }
  }