Skip to content

Instantly share code, notes, and snippets.

@swdevbali
Last active June 1, 2022 19:24
Show Gist options
  • Save swdevbali/f4adcacd402216256dd8 to your computer and use it in GitHub Desktop.
Save swdevbali/f4adcacd402216256dd8 to your computer and use it in GitHub Desktop.
Redis store data as string, getting it back and trying to dump it as Json will still create json with string data type. We need to convert it back to correct data type using our own dict of types
__author__ = 'ekowibowo'
import os
import redis
from flask import jsonify, Flask
import json
REDIS_HOST = os.getenv('SS_ABTEST_REDIS_HOST', '192.168.59.103')
hkey = 'redisobject:whyweneedit'
r = redis.StrictRedis(host=REDIS_HOST)
r.delete(hkey)
dict_to_store = {
"id": 2,
"name": "desktopCatalogueFeedColumning",
"device": "desktop",
"active": True,
"buckets": [
{
"name": "TWO-COLUMN"
},
{
"name": "THREE-COLUMN"
}
]
}
r.hmset(hkey, dict_to_store)
dict_from_redis = r.hgetall(hkey)
print 'Python dict_to_store', dict_to_store
print 'Python dict_from_redis', dict_from_redis
print 'Converted with json.dumps() ', json.dumps(dict_from_redis)
DEFAULT_VALUE = {str(int): 0, str(float): 0.0, str(list): [], str(dict): {}, str(bool): False, str(str): ''}
def decode(key_types, redis_dict):
result = {}
# construct initial Python dict result with default value
for k in key_types:
type = key_types[k]
result[k] = DEFAULT_VALUE[str(type)]
for k in redis_dict.keys():
v = redis_dict[k]
if k in key_types:
type = key_types[k]
if type == list or type == bool:
result[k] = eval(v)
else:
if v == '' and (type == int or type == float):
v = '0'
result[k] = type(v)
else:
result[k] = v
return result
types = {}
types['experiment'] = {'id': int, 'active': bool, 'buckets': list, 'device': str}
print 'Converting it the right way...'
redis_dict_in_python = decode(types['experiment'], dict_from_redis)
print 'redis_dict_in_python', redis_dict_in_python
redis_dict_in_python_to_json = json.dumps(redis_dict_in_python)
print 'redis_dict_in_python_to_json', redis_dict_in_python_to_json
app = Flask(__name__)
@app.route('/')
def json():
return jsonify(data=dict_from_redis, success=True, error={})
app.run(host='0.0.0.0', port=9999, debug=True)
@swdevbali
Copy link
Author

Python dict_to_store {'device': 'desktop', 'active': True, 'buckets': [{'name': 'TWO-COLUMN'}, {'name': 'THREE-COLUMN'}], 'id': 2, 'name': 'desktopCatalogueFeedColumning'}

Python dict_from_redis {'device': 'desktop', 'active': 'True', 'buckets': "[{'name': 'TWO-COLUMN'}, {'name': 'THREE-COLUMN'}]", 'id': '2', 'name': 'desktopCatalogueFeedColumning'}

Converted with json.dumps()  {"device": "desktop", "active": "True", "buckets": "[{'name': 'TWO-COLUMN'}, {'name': 'THREE-COLUMN'}]", "id": "2", "name": "desktopCatalogueFeedColumning"}

Converting it the right way...
redis_dict_in_python {'active': True, 'device': 'desktop', 'buckets': [{'name': 'TWO-COLUMN'}, {'name': 'THREE-COLUMN'}], 'id': 2, 'name': 'desktopCatalogueFeedColumning'}

redis_dict_in_python_to_json {"active": true, "device": "desktop", "buckets": [{"name": "TWO-COLUMN"}, {"name": "THREE-COLUMN"}], "id": 2, "name": "desktopCatalogueFeedColumning"}`

@swdevbali
Copy link
Author

Notice how in Converted with json.dumps() in "active": "True" we lost our boolean type. If we return this to www, it'll create error as it expected true, not "True"

While in redis_dict_in_python_to_json we already have proper boolean value for active

@swdevbali
Copy link
Author

Update: I have already activate the use of Flask in the above gist, and you can browse to http://0.0.0.0:9999 and see the following output:

{
  "data": {
    "active": "True",
    "buckets": "[{'name': 'TWO-COLUMN'}, {'name': 'THREE-COLUMN'}]",
    "device": "desktop",
    "id": "2",
    "name": "desktopCatalogueFeedColumning"
  },
  "error": {},
  "success": true
}

You can see that "active": "True", which is coming from jsonify(dict_from_redis). Also buckets is in type of string, not list. Which also create error in www.

In summary, we really need to first properly decode() the dict coming from redis to correct Python data type, before using jsonify or json.dumps(). If not, all dict fields will retained its string type as per redis behavior

@garindra
Copy link

garindra commented Nov 6, 2015

As I said in the original PR, all of these conversion nightmare is because you're trying to use Redis hash (hmset) to store the experiment information. The better way would be to rely on JSON encoding, and just store the JSON string in a Redis string.

@swdevbali
Copy link
Author

Yep, got it. My bad.
Wil test it first

@abhimanyu3-zz
Copy link

I want to store a python dict in redis but as i am getting data every 15 minutes so i also need to update the redis every 15 minutes. Can you help me how I can achieve that? I mean, How i can dynamically assign a key every time i write data in redis as when i am using the same key.. it is overwriting the old data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment