Skip to content

Instantly share code, notes, and snippets.

View takluyver's full-sized avatar

Thomas Kluyver takluyver

View GitHub Profile
@takluyver
takluyver / h5py_strings.py
Created October 15, 2020 08:59
Read HDF5 strings as str with h5py 2.x and 3.0
"""Read HDF5 strings to Python str
h5py 3.0 changed the APIs for storing and reading strings.
This shows how to read strings as Python 3 str if you need
to support both old and new h5py.
"""
import h5py
import numpy as np
f = h5py.File('foo.h5', 'r')
@takluyver
takluyver / cache_format_options.py
Created August 27, 2019 11:26
Karabo data cache format experimentation
import json
import h5py
import numpy as np
def read_json(path):
with open(path) as f:
loaded_data = json.load(f)
res = {}
for info in loaded_data:
@takluyver
takluyver / numpy_array_sum0.npy
Last active June 7, 2019 16:47
Bizarre numpy sum behaviour
@takluyver
takluyver / VDS issue.ipynb
Created April 9, 2019 14:21
HDF5 virtual dataset access issue
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@takluyver
takluyver / atomicity_issue_pull.py
Created December 7, 2018 15:42
Pyzmq multipart atomicity issue
import os
from time import monotonic
import zmq
# Set up a PULL socket
ctx = zmq.Context()
sock = ctx.socket(zmq.PULL)
sock.set_hwm(1)
sock.connect("tcp://127.0.0.1:8541")
@takluyver
takluyver / pull.py
Created December 7, 2018 10:29
ZMQ push/pull HWM experiment
import zmq
# Set up the socket
ctx = zmq.Context()
sock = ctx.socket(zmq.PULL)
sock.set_hwm(1)
sock.setsockopt(zmq.RCVBUF, 16 * 1024)
sock.connect("tcp://127.0.0.1:8541")
print("RCVHWM", sock.getsockopt(zmq.RCVHWM))
@takluyver
takluyver / README.md
Created October 19, 2018 13:02
Investigating h5py chunked read performance
@takluyver
takluyver / Twitter archive.ipynb
Created August 4, 2018 11:38
Inspecting archived tweets
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.