Skip to content

Instantly share code, notes, and snippets.

@LouisAmon
Forked from alexland/serialize-numpy-array.py
Last active November 16, 2017 10:03
Show Gist options
  • Save LouisAmon/380301d6173e4a5d70789ae4ffafe02a to your computer and use it in GitHub Desktop.
Save LouisAmon/380301d6173e4a5d70789ae4ffafe02a to your computer and use it in GitHub Desktop.
serialize, persist, retrieve, and de-serialize a NumPy array as a binary string (any dimension, any dtype); exemplary use case: a web app calculates some result--eg, from a Machine Learning algorithm, using NumPy and the result is a NumPy array; it is efficient to just return that result to rather than persist the array then retrieve it via query
import time
import numpy
import redis
# a 2D array to serialize
A = 10 * numpy.random.randn(10000).reshape(1000, 10)
# flatten the 2D NumPy array and save it as a binary string
array_dtype = str(A.dtype)
l, w = A.shape
A = A.ravel().tostring()
# create a key as a UNIX timestamp w/ array shape appended to end of key delimited by '|'
db = redis.StrictRedis(db=0)
key = '{0}|{1}#{2}#{3}'.format(int(time.time()), array_dtype, l, w)
# store the binary string in redis
db.set(key, A)
# retrieve the proto-array from redis
A1 = db.get(key)
# deserialize it
array_dtype, l, w = key.split('|')[1].split('#')
A = numpy.fromstring(A1, dtype=array_dtype).reshape(int(l), int(w))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment