AWX's worker stalls when trying to authenticate against AD

README

Using a blank LDAP User DN Template as suggested in https://github.com/ansible/awx/issues/496 and 
using the LDAP User Search configuration from that same issue will cause this issue.

awx_web.log

2018-02-28 15:08:53,888 INFO     Starting server at tcp:port=8051:interface=127.0.0.1, channel layer awx.asgi:channel_layer.                                                                                                          
2018-02-28 15:08:53,889 INFO     HTTP/2 support not enabled (install the http2 and tls Twisted extras)                                                                                                                                
2018-02-28 15:08:53,890 INFO     Using busy-loop synchronous mode on channel layer                                                                                                                                    
2018-02-28 15:08:53,890 INFO     Listening on endpoint tcp:port=8051:interface=127.0.0.1                                                                                                                              
2018-02-28 15:09:13,752 ERROR    awx.sso.backends authenticating REDACTED
2018-02-28 15:09:13,752 ERROR    django_auth_ldap before _authenticate_user_dn<django_auth_ldap.backend._LDAPUser object at 0x7b86bd0>
2018-02-28 15:09:13,752 ERROR    django_auth_ldap inside _authenticate_user_dn 
2018/02/28 15:11:13 [warn] 26#0: *1 upstream server temporarily disabled while reading response header from upstream, client: 172.17.0.1, server: _, request: "POST /api/v2/authtoken/ HTTP/1.1", upstream: "uwsgi://127.0.0.1:8050",
host: "127.0.0.1:8080", referrer: "https://awx.pink.zhdk.ch/"
2018/02/28 15:11:13 [error] 26#0: *1 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 172.17.0.1, server: _, request: "POST /api/v2/authtoken/ HTTP/1.1", upstream: "uwsgi://127.0$0.1:8050", host: "127.0.0.1:8080", referrer: "https://awx.pink.zhdk.ch/"
172.17.0.1 - - [28/Feb/2018:15:11:13 +0000] "POST /api/v2/authtoken/ HTTP/1.1" 504 183 "https://awx.pink.zhdk.ch/" "Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0" "10.128.36.210"                             
Wed Feb 28 15:11:14 2018 - *** HARAKIRI ON WORKER 4 (pid: 30, try: 1) ***
Wed Feb 28 15:11:14 2018 - HARAKIRI !!! worker 4 status !!! 
Wed Feb 28 15:11:14 2018 - HARAKIRI [core 0] 172.17.0.1 - POST /api/v2/authtoken/ since 1519830553 
Wed Feb 28 15:11:14 2018 - HARAKIRI !!! end of worker 4 status !!!
DAMN ! worker 4 (pid: 30) died, killed by signal 9 :( trying respawn ...

backend.py

#/var/lib/awx/venv/awx/lib/python2.7/site-packages/django_auth_ldap/backend.py

    # L#508
    def _search_for_user_dn(self):
        """
        Searches the directory for a user matching AUTH_LDAP_USER_SEARCH.
        Populates self._user_dn and self._user_attrs.
        """
        search = self.settings.USER_SEARCH
        if search is None:
            raise ImproperlyConfigured('AUTH_LDAP_USER_SEARCH must be an LDAPSearch instance.')

        logger.error('trying to get results from search.execute')
        results = search.execute(self.connection, {'user': self._username})
        if results is not None and len(results) == 1:
            logger.error('iterating through results now')
            (self._user_dn, self._user_attrs) = next(iter(results))


    def _authenticate_user_dn(self, password):
        logger.error('inside _authenticate_user_dn')
        """
        Binds to the LDAP server with the user's DN and password. Raises
        AuthenticationFailed on failure.
        """
        logger.error('self.dn is ' + self.dn)
        if self.dn is None:  # This is where it stalls, self.dn appears to never return
            raise self.AuthenticationFailed("failed to map the username to a DN.")

        try:
            logger.error('self.dn was *not* none')
            sticky = self.settings.BIND_AS_AUTHENTICATING_USER
            logger.error('settings applied')

            logger.error('the user has DN' + str(self.dn))

            self._bind_as(self.dn, password, sticky=sticky)
        except ldap.INVALID_CREDENTIALS:
            raise self.AuthenticationFailed("user DN/password rejected by LDAP server.")

backends.py

#/usr/lib/python2.7/site-packages/awx/sso/backends.py

    def authenticate(self, username, password):
        if self.settings.START_TLS and ldap.OPT_X_TLS_REQUIRE_CERT in self.settings.CONNECTION_OPTIONS:
            # with python-ldap, if you want to set connection-specific TLS
            # parameters, you must also specify OPT_X_TLS_NEWCTX = 0
            # see: https://stackoverflow.com/a/29722445
            # see: https://stackoverflow.com/a/38136255
            self.settings.CONNECTION_OPTIONS[ldap.OPT_X_TLS_NEWCTX] = 0

        if not self.settings.SERVER_URI:
            return None
        if not feature_enabled('ldap'):
            logger.error("Unable to authenticate, license does not support LDAP authentication")
            return None
        try:
            user = User.objects.get(username=username)
            if user and (not user.profile or not user.profile.ldap_dn):
                return None
        except User.DoesNotExist:
            pass
        try:
            logger.error("authenticating " + username)
            return super(LDAPBackend, self).authenticate(username, password)
        except Exception:
            logger.exception("Encountered an error authenticating to LDAP")
            return None