Skip to content

Instantly share code, notes, and snippets.

@acdha
Created October 3, 2016 21:26
Show Gist options
  • Save acdha/ee4e4efee0f47e6953c05b2f060eb4ad to your computer and use it in GitHub Desktop.
Save acdha/ee4e4efee0f47e6953c05b2f060eb4ad to your computer and use it in GitHub Desktop.
Traceback from a crash introduced by a recent Homebrew update when Python ctypes tries to load gdal
Traceback (most recent call last):
File "/Users/cadams/.virtualenvs/wdl/bin/django-admin", line 11, in <module>
sys.exit(execute_from_command_line())
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
utility.execute()
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/core/management/__init__.py", line 328, in execute
django.setup()
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/apps/config.py", line 86, in create
module = import_module(entry)
File "/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/reversion/__init__.py", line 11, in <module>
from reversion.revisions import default_revision_manager, revision_context_manager, VersionAdapter
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/reversion/revisions.py", line 28, in <module>
from reversion.models import Revision, Version, has_int_pk, pre_revision_commit, post_revision_commit
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/reversion/models.py", line 48, in <module>
class Revision(models.Model):
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/models/base.py", line 139, in __new__
new_class.add_to_class('_meta', Options(meta, **kwargs))
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/models/base.py", line 324, in add_to_class
value.contribute_to_class(cls, name)
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/models/options.py", line 250, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/__init__.py", line 36, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/utils.py", line 241, in __getitem__
backend = load_backend(db['ENGINE'])
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/db/utils.py", line 112, in load_backend
return import_module('%s.base' % backend_name)
File "/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/db/backends/mysql/base.py", line 5, in <module>
from .features import DatabaseFeatures
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/db/backends/mysql/features.py", line 1, in <module>
from django.contrib.gis.db.backends.base.features import BaseSpatialFeatures
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/db/backends/base/features.py", line 3, in <module>
from django.contrib.gis.db.models import aggregates
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/db/models/__init__.py", line 7, in <module>
from django.contrib.gis.geos import HAS_GEOS
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/geos/__init__.py", line 15, in <module>
from .geometry import GEOSGeometry, wkt_regex, hex_regex
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/geos/geometry.py", line 10, in <module>
from django.contrib.gis.gdal.error import SRSException
File "/Users/cadams/.virtualenvs/wdl/lib/python2.7/site-packages/django/contrib/gis/gdal/__init__.py", line 37, in <module>
ctypes.CDLL('/usr/local/lib/libgdal.dylib')
File "/usr/local/Cellar/python/2.7.12_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 362, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/lib/libgdal.dylib, 6): Symbol not found: _GEOSArea
Referenced from: /usr/local/opt/libspatialite/lib/libspatialite.7.dylib
Expected in: flat namespace
in /usr/local/opt/libspatialite/lib/libspatialite.7.dylib
@acdha
Copy link
Author

acdha commented Oct 3, 2016

There's some sort of load-time dependency here as this can be “fixed” by putting something like this early in the load process – such as Django's settings module:

# MAGIC HACK: without this, a recent Homebrew update broke GDAL!
GDAL_LIBRARY_PATH = "/usr/local/lib/libgdal.dylib"
import ctypes
ctypes.CDLL(GDAL_LIBRARY_PATH)

Done later, such as by the call in the GeoDjango initialization, import ctypes; ctypes.CDLL('/usr/local/lib/libgdal.dylib') will fail with the same error as above.

@PaoloC68
Copy link

@acdha Where should I put that code above?

@Azd325
Copy link

Azd325 commented Oct 18, 2016

@PaoloC68 You can add to the top of you settings.py

@schinckel
Copy link

Is there a better solution than this? I'm not sure my production environment will have libgdal in that path...

@acdha
Copy link
Author

acdha commented Oct 28, 2016

See Homebrew/homebrew-core#5161 for a tracking ticket. It's not clear exactly where the problem is.

Copy link

ghost commented Feb 25, 2017

@acdha Fix worked .Thanks

@frankwiles
Copy link

Just an FYI I ran into this same issue and removing an old template_postgis database from an old dump/restore from older GeoDjangos apparently fixed the issue for me.

@cewing
Copy link

cewing commented Jul 18, 2017

Ran into this same issue today and did some digging. For me, the issue was in an older Django app (1.8.12). It appears that the problem arises because of the way that HAS_GEOS is checked in that version of Django. There, in __init__.py in django.contrib.gis.models has the following code to check if GEOS is installed:

from django.contrib.gis.geos import HAS_GEOS

if not HAS_GEOS:
    raise ImproperlyConfigured(
        "GEOS is required and has not been detected. Are you sure it is installed? "
        "See also https://docs.djangoproject.com/en/%s/ref/contrib/gis/install/geolibs/" % get_docs_version())

This import causes ctypes to load the geos_c dynamic library. A short while later, django.contrib.gis.geos.geometry attempts to import something from django.contrib.gis.gdal. That causes ctypes to load the libgdal dynamic library. Unfortunately, it appears that if geos is loaded before gdal is loaded, then gdal fails to load with the given error.

Starting in In Django 1.9 this conditional check was moved and imports were reshuffled in such a way as to fix this problem. I have been unable to reproduce it in a version of Django after 1.9a1.

If you (like me) are forced to keep an older Django, then ensuring that gdal is loaded by ctypes at any point before geos is loaded is sufficient to avoid this error. Adding the block above in settings.py ensures that it is executed before any database setup occurs, which then ensures that gdal is loaded before geos and solves the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment