Skip to content

Instantly share code, notes, and snippets.

@Atala
Created January 19, 2015 14:56
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Atala/71e662b9de13d4e32eb9 to your computer and use it in GitHub Desktop.
Save Atala/71e662b9de13d4e32eb9 to your computer and use it in GitHub Desktop.
Note on messagepack unpacking

As I understand, this won't work:

import msgpack
fd = sock.makefile("rb")
unpacker = msgpack.Unpacker(fd, encoding="utf-8")
packet = unpacker.next()

From the msgepack python implementation, the unpacker will emit this:

fd.read(min(1024**2, max_buffer_size))
```
(ref: https://github.com/msgpack/msgpack-python/blob/master/msgpack/_unpacker.pyx#L323-L326)

This will block forever, as the read will return either at EOF or when the length is reached.

I came up with this solution:
```python
unpacker = msgpack.Unpacker(encoding="utf-8")
# do not worry about timeouts
while True:
    buf = sock.recv(1024**2)
    unpacker.feed(buf)
    try:
        packet = unpacker.next()
        break
    except StopIteration:
        continue
```

A real stream process implementation is discussed here:
https://github.com/msgpack/msgpack-python/issues/48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment