Skip to content

Instantly share code, notes, and snippets.

@chezou
Last active October 26, 2019 13:05
Show Gist options
  • Save chezou/27a975565fb83afb78c33cb1af2ff6a0 to your computer and use it in GitHub Desktop.
Save chezou/27a975565fb83afb78c33cb1af2ff6a0 to your computer and use it in GitHub Desktop.
Convert R data frame to msgpack stream
devtools::install_github("crowding/msgpack-r")
library(msgpack)
x <- data.frame(a=1:3, b=4:6)
conn <- gzfile("test.msgpack.gz", open="w+b")
apply(x, 1, function(x) {writeMsg(c(x, time=as.integer(Sys.time())), conn)})
flush(conn)
close(conn)
import gzip
import msgpack
with gzip.GzipFile("test.msgpack.gz", "rb") as f:
unpacker = msgpack.Unpacker(f, raw=False)
for u in unpacker:
print(u)
# {'a': 1, 'b': 4, 'time': 1572094906}
# {'a': 2, 'b': 5, 'time': 1572094906}
# {'a': 3, 'b': 6, 'time': 1572094906}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment