Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@Ademord
Forked from alexland/serialize-numpy-array.py
Created August 7, 2017 11:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Ademord/78862e3f09e56edc9416f49779c9138c to your computer and use it in GitHub Desktop.
Save Ademord/78862e3f09e56edc9416f49779c9138c to your computer and use it in GitHub Desktop.
serialize, persist, retrieve, and de-serialize a NumPy array as a binary string (any dimension, any dtype); exemplary use case: a web app calculates some result--eg, from a Machine Learning algorithm, using NumPy and the result is a NumPy array; it is efficient to just return that result to rather than persist the array then retrieve it via query
import time
import numpy as NP
from redis import StrictRedis as redis
# a 2D array to serialize
A = 10 * NP.random.randn(10000).reshape(1000, 10)
# flatten the 2D NumPy array and save it as a binary string
array_dtype = str(A.dtype)
l, w = A.shape
A = A.ravel().tostring()
# create a key as a UNIX timestamp w/ array shape appended to end of key delimited by '|'
db = redis(db=0)
key = '{0}|{1}#{2}#{3}'.format(int(time.time()), array_dtype, l, w)
# store the binary string in redis
db.set(key, A)
# retrieve the proto-array from redis
A1 = db.get(key)
# deserialize it
array_dtype, l, w = key.split('|')[1].split('#')
A = NP.fromstring(A1, dtype=array_dtype).reshape(int(l), int(w))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment