Skip to content

Instantly share code, notes, and snippets.

View maartenbreddels's full-sized avatar

Maarten Breddels maartenbreddels

View GitHub Profile
@maartenbreddels
maartenbreddels / h5map.ipynb
Created October 25, 2017 12:25
memory mapping hdf5 continuous data
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maartenbreddels
maartenbreddels / reacton-in-panel.py
Created December 13, 2022 19:58
Reacton running in Panel
# $ panel serve reacton-in-panel.py
import panel as pn
import reacton
import reacton.ipywidgets as w
@reacton.component
def ButtonClick(label="Hi"):
clicks, set_clicks = reacton.use_state(0)
def increment():
@maartenbreddels
maartenbreddels / gist:82a3778c9a79b7ef048e
Last active June 19, 2022 22:26
simple example on how to combine python and numpy with some fast c function using a c++ template function
#include <Python.h>
#include <math.h>
#include <stdexcept>
#include <cstdio>
#include <numpy/arrayobject.h>
template<typename T>
void object_to_numpy1d_nocopy(T* &ptr, PyObject* obj, long long &count, int& stride=stride_default, int type=NPY_DOUBLE) {
if(obj == NULL)
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maartenbreddels
maartenbreddels / Makefile
Created April 28, 2018 18:10
Makefile for converting GaiaDR2 cvs files to a single hdf5 file
# Makefile for converting the CSV files from http://cdn.gea.esac.esa.int/Gaia/gdr2/gaia_source/csv/
# to a single (vaex) hdf5 file
# * https://docs.vaex.io
# * https://github.com/maartenbreddels/vaex/
# It is multistage to work around opening 60 000 files at once.
# Strategy is
# * stage1: convert all cvs.gz to csv to hdf5
# * do this via xargs and calling make again, since gmake has trouble matching 60 000 rules
# * stage2: Create part-<NUMBER>.txt files containing max FILES_PER_PART per file
# * stage3: convert the list of hdf5 files to single hdf5 files (part-<NUMBER>.hdf5)
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@maartenbreddels
maartenbreddels / ipython_thread.py
Last active July 28, 2020 14:07
IPython snippet for doing work in a thread, and updating a progressbar
import threading
from IPython.display import display
import ipywidgets as widgets
import time
def get_ioloop():
import IPython, zmq
ipython = IPython.get_ipython()
if ipython and hasattr(ipython, 'kernel'):
return zmq.eventloop.ioloop.IOLoop.instance()
ioloop = get_ioloop()
@maartenbreddels
maartenbreddels / qt_and_tornado.py
Created October 1, 2015 10:57
Combining Qt and tornado, both which want to have their own event loop.
__author__ = 'breddels'
"""
Demonstrates combining Qt and tornado, both which want to have their own event loop.
The solution is to run tornado in a thread, the issue is that callbacks will then also be executed in this thread, and Qt doesn't like that.
To fix this, I show how to use execute the callback in the main thread, using a Qt signal/event in combination with Promises.
The output of the program is:
fetch page, we are in thread <_MainThread(MainThread, started 47200787479520)>
response is 191548 bytes, we are in thread <Thread(Thread-1, started daemon 47201018689280)>
the other thread should fulfil the result to this promise, we are in thread <Thread(Thread-1, started daemon 47201018689280)>
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.