rebrow Redis password token with JWT

rebrow.redis.password.jwt.token.gist.py

# This gist is a basic example of adding Redis password to rebrow in a modified implementation of
# @elky84 https://github.com/marians/rebrow/pull/20 but using PyJWT to encode the password in the POST
# to a JWT token and a client token as a replacement for the plaintext password URL parameter.
# This example includes logging which is not in rebrow and this example rebrow stores the JWT encode string in Redis.
# With normal rebrow this would not be possible and Flask seesion or some internal Python method would need to be
# used.  The version of rebrow that is used here is a version that embedded in another application that does have
# access to Redis, hence here rebrow stores the data in Redis, rebrow here requires auth and it is also run behind
# a SSL terminated endpoint and therefore the POST data is encrypted.  If rebrow was just run as is, then the POST
# data would not be encrypted and the password would still be sent plaintext.

# Please also see my first commit after this code relating to how this could still be succesfully attacked under the
# correct conditions.

# new requirements
import jwt
import hashlib
from sys import version_info
from ast import literal_eval

# new defs
def get_redis(host, port, db, password):
    if password == "":
        return redis.StrictRedis(host=host, port=port, db=db)
    else:
        return redis.StrictRedis(host=host, port=port, db=db, password=password)

# Added token, client_id and salt to replace password parameter and determining
# client protocol
def get_client_details():
    """
    Gets the first X-Forwarded-For address and sets as the IP address.
    Gets the client_id by simply using a md5 hash of the client IP address
    and user agent.
    Determines whether the request was proxied.
    Determines the client protocol.

    :return: client_id, protocol, proxied
    :rtype: str, str, boolean, str

    """
    proxied = False
    if request.headers.getlist('X-Forwarded-For'):
        client_ip = str(request.headers.getlist('X-Forwarded-For')[0])
        logger.info('rebrow access :: client ip set from X-Forwarded-For[0] to %s' % (str(client_ip)))
        proxied = True
    else:
        client_ip = str(request.remote_addr)
        logger.info('rebrow access :: client ip set from remote_addr to %s, no X-Forwarded-For header was found' % (str(client_ip)))
    client_user_agent = request.headers.get('User-Agent')
    logger.info('rebrow access :: %s client_user_agent set to %s' % (str(client_ip), str(client_user_agent)))
    client_id = '%s_%s' % (client_ip, client_user_agent)
    if python_version == 2:
        client_id = hashlib.md5(client_id).hexdigest()
    else:
        client_id = hashlib.md5(client_id.encode('utf-8')).hexdigest()
    logger.info('rebrow access :: %s has client_id %s' % (str(client_ip), str(client_id)))

    if request.headers.getlist('X-Forwarded-Proto'):
        protocol_list = request.headers.getlist('X-Forwarded-Proto')
        protocol = str(protocol_list[0])
        logger.info('rebrow access :: protocol for %s was set from X-Forwarded-Proto to %s' % (client_ip, str(protocol)))
    else:
        protocol = 'unknown'
        logger.info('rebrow access :: protocol for %s was not set from X-Forwarded-Proto to %s' % (client_ip, str(protocol)))

    if not proxied:
        logger.info('rebrow access :: Skyline is not set up correctly, the expected X-Forwarded-For header was not found')

    return client_id, protocol, proxied


def decode_token(client_id):
    """
    Use the app.secret, client_id and salt to decode the token JWT encoded
    payload and determine the Redis password.

    :param client_id: the client_id string
    :type client_id: str
    return token, decoded_redis_password, fail_msg, trace
    :return: token, decoded_redis_password, fail_msg, trace
    :rtype: str, str, str, str

    """
    fail_msg = False
    trace = False
    token = False
    logger.info('decode_token for client_id - %s' % str(client_id))

    if not request.args.getlist('token'):
        fail_msg = 'No token url parameter was passed, please log into Redis again through rebrow'
    else:
        token = request.args.get('token', type=str)
        logger.info('token found in request.args - %s' % str(token))

    if not token:
        client_id, protocol, proxied = get_client_details()
        fail_msg = 'No token url parameter was passed, please log into Redis again through rebrow'
        trace = 'False'

    client_token_data = False
    if token:
        try:
            if settings.REDIS_PASSWORD:
                redis_conn = redis.StrictRedis(password=settings.REDIS_PASSWORD, unix_socket_path=settings.REDIS_SOCKET_PATH)
            else:
                redis_conn = redis.StrictRedis(unix_socket_path=settings.REDIS_SOCKET_PATH)
            key = 'rebrow.token.%s' % token
            client_token_data = redis_conn.get(key)
        except:
            trace = traceback.format_exc()
            fail_msg = 'Failed to get client_token_data from Redis key - %s' % key
            client_token_data = False
            token = False

    client_id_match = False
    if client_token_data is not None:
        logger.info('client_token_data retrieved from Redis - %s' % str(client_token_data))
        try:
            client_data = literal_eval(client_token_data)
            logger.info('client_token_data - %s' % str(client_token_data))
            client_data_client_id = str(client_data[0])
            logger.info('client_data_client_id - %s' % str(client_data_client_id))
        except:
            trace = traceback.format_exc()
            logger.error('%s' % trace)
            err_msg = 'error :: failed to get client data from Redis key'
            logger.error('%s' % err_msg)
            fail_msg = 'Invalid token. Please log into Redis through rebrow again.'
            client_data_client_id = False

        if client_data_client_id != client_id:
            logger.error(
                'rebrow access :: error :: the client_id does not match the client_id of the token - %s - %s' %
                (str(client_data_client_id), str(client_id)))
            try:
                if settings.REDIS_PASSWORD:
                    redis_conn = redis.StrictRedis(password=settings.REDIS_PASSWORD, unix_socket_path=settings.REDIS_SOCKET_PATH)
                else:
                    redis_conn = redis.StrictRedis(unix_socket_path=settings.REDIS_SOCKET_PATH)
                key = 'rebrow.token.%s' % token
                redis_conn.delete(key)
                logger.info('due to possible attempt at unauthorised use of the token, deleted the Redis key - %s' % str(key))
            except:
                pass
            fail_msg = 'The request data did not match the token data, due to possible attempt at unauthorised use of the token it has been deleted.'
            trace = 'this was a dodgy request'
            token = False
        else:
            client_id_match = True
    else:
        fail_msg = 'Invalid token, there was no data found associated with the token, it has probably expired.  Please log into Redis again through rebrow'
        trace = client_token_data
        token = False

    client_data_salt = False
    client_data_jwt_payload = False
    if client_id_match:
        client_data_salt = str(client_data[1])
        client_data_jwt_payload = str(client_data[2])

    decoded_redis_password = False
    if client_data_salt and client_data_jwt_payload:
        try:
            jwt_secret = '%s.%s.%s' % (app.secret_key, client_id, client_data_salt)
            jwt_decoded_dict = jwt.decode(client_data_jwt_payload, jwt_secret, algorithms=['HS256'])
            jwt_decoded_redis_password = str(jwt_decoded_dict['auth'])
            decoded_redis_password = jwt_decoded_redis_password
        except:
            trace = traceback.format_exc()
            logger.error('%s' % trace)
            err_msg = 'error :: failed to decode the JWT token with the salt and client_id'
            logger.error('%s' % err_msg)
            fail_msg = 'failed to decode the JWT token with the salt and client_id. Please log into rebrow again.'
            token = False

    return token, decoded_redis_password, fail_msg, trace


@app.route('/rebrow', methods=['GET', 'POST'])
@requires_auth
def login():
    """
    Start page
    """
    if request.method == 'POST':
        # TODO: test connection, handle failures
        host = str(request.form['host'])
        port = int(request.form['port'])
        db = int(request.form['db'])
        password = str(request.form['password'])
        token_valid_for = int(request.form['token_valid_for'])
        if token_valid_for > 3600:
            token_valid_for = 3600
        if token_valid_for < 30:
            token_valid_for = 30

        # Added auth to rebrow as per https://github.com/marians/rebrow/pull/20 by
        # elky84 and add encryption to the password URL parameter trying to use
        # pycrypto/pycryptodome to encode it, but no, used PyJWT instead
        # padded_password = password.rjust(32)
        # secret_key = '1234567890123456' # create new & store somewhere safe
        # cipher = AES.new(app.secret_key,AES.MODE_ECB) # never use ECB in strong systems obviously
        # encoded = base64.b64encode(cipher.encrypt(padded_password))

        # Added client_id, token and salt
        salt = salt = str(uuid.uuid4())
        client_id, protocol, proxied = get_client_details()

        # Use pyjwt - JSON Web Token implementation to encode the password and
        # pass a token in the URL password parameter, the password in the POST
        # data should be encrypted via the reverse proxy SSL endpoint
        # encoded = jwt.encode({'some': 'payload'}, 'secret', algorithm='HS256')
        # jwt.decode(encoded, 'secret', algorithms=['HS256'])
        # {'some': 'payload'}
        try:
            jwt_secret = '%s.%s.%s' % (app.secret_key, client_id, salt)
            jwt_encoded_payload = jwt.encode({'auth': str(password)}, jwt_secret, algorithm='HS256')
        except:
            message = 'Failed to create set jwt_encoded_payload for %s' % client_id
            trace = traceback.format_exc()
            return internal_error(message, trace)

        # HERE WE WANT TO PUT THIS INTO REDIS with a TTL key and give the key
        # a salt and have the client use that as their token
        client_token = str(uuid.uuid4())
        logger.info('rebrow access :: generated client_token %s for client_id %s' % (client_token, client_id))
        try:
            if settings.REDIS_PASSWORD:
                redis_conn = redis.StrictRedis(password=settings.REDIS_PASSWORD, unix_socket_path=settings.REDIS_SOCKET_PATH)
            else:
                redis_conn = redis.StrictRedis(unix_socket_path=settings.REDIS_SOCKET_PATH)
            key = 'rebrow.token.%s' % client_token
            value = '[\'%s\',\'%s\',\'%s\']' % (client_id, salt, jwt_encoded_payload)
            redis_conn.setex(key, token_valid_for, value)
            logger.info('rebrow access :: set Redis key - %s' % (key))
        except:
            message = 'Failed to set Redis key - %s' % key
            trace = traceback.format_exc()
            return internal_error(message, trace)

        # Change password parameter to token parameter
        # url = url_for("rebrow_server_db", host=host, port=port, db=db, password=password)
        url = url_for(
            "rebrow_server_db", host=host, port=port, db=db, token=client_token)
        return redirect(url)
    else:
        start = time.time()

        client_id, protocol, proxied = get_client_details()

        # Added client message to give relevant messages on the login page
        client_message = False

        return render_template(
            'rebrow_login.html',
            # Change password parameter to token parameter and added protocol,
            # proxied
            # redis_password=redis_password,
            protocol=protocol, proxied=proxied, client_message=client_message,
            version=skyline_version,
            duration=(time.time() - start))

#########
#
# In all subsequent @app.route requests before redis is called, get_client_details and decode_token need to
# called and the the token parameter needs to be passed to all render_template calls

    client_id, protocol, proxied = get_client_details()
    token, redis_password, fail_msg, trace = decode_token(client_id)
    if not token:
        abort(401)

    try:
        r = get_redis(host, port, db, redis_password)
    except:
        logger.error(traceback.format_exc())
        logger.error('error :: rebrow access :: failed to login to Redis with token')

This is not unhackable, if an attacker knew originating IP, user agent, had the token parameter and had access to your rebrow instance, then they could access rebrow with your token, by spoofing the X-Forwarded-For header and their user agent as that is used to generate the client token. Flask and rebrow do not have the X-Forwarded-For header by default anyway, in this example rebrow is fronted by an Apache reverse proxy which handles SSL termination and authenticate.

That all said they could still not read the Redis password even if they had the token and spoofed. They could only access Redis and delete keys :) This is because the Redis password is encoded with the rebrow app.secret, the client_id (IP, user agent -> md5) and the salt, which is stored in a Redis key (here it is stored in Rdis, but in Flask session, I do not know). The Redis would then be accessible to the attacker. These 3 "keys" are used to encode the JWT token, which is all so stored in the Redis, but without the rebrow app.secret, the attacker cannot decode the encoded JWT string in the Redis, so the Redis password cannot be read by the the attacker OR the user valid themselves via rebrow.

In this modified version of rebrow the app.secret is not user declared in runserver.py as a user sting really but using uuid

# For secret_key
import uuid

# key for cookie safety. Shal be overridden using ENV var SECRET_KEY
#app.secret_key = os.getenv("SECRET_KEY", "lasfuoi3ro8w7gfow3bwiubdwoeg7p23r8g23rg")
secret_key = str(uuid.uuid5(uuid.NAMESPACE_DNS, settings.GRAPHITE_HOST))
app.secret_key = secret_key

But I do not think there is a reason why that could not even be:

app.secret_key = str(uuid.uuid4())

That way not even the rebrow admin knows the app.secret and it is regenerated on every rebrow start. I do not think there is any need to the rebrow user to have to know the app.secret_key as far as I can tell in rebrow itself.

Anyway just sharing thoughts, And do not get me started on adding stunnel for rebrow to remote Redis instances access :)