Skip to content

Instantly share code, notes, and snippets.

View danielballan's full-sized avatar

Dan Allan danielballan

View GitHub Profile
danielballan /
Created May 10, 2024 20:18
Build Adapter from DataSource
import collections
from tiled.mimetypes import DEFAULT_ADAPTERS_BY_MIMETYPE
def build_adapter(data_source, adapters_by_mimetype=None):
# We use a dictionary mapping a mimetype to a callable that returns an Adapter instance.
# This might be a class, classmethod constructor, factory function...
# it does not matter here; it is just a callable.
danielballan /
Last active April 1, 2024 16:39
Flyscanning data setup

Flyscanning data setup


  1. Documents, including StreamResource and StreamDatum, are emitted by RunEngine.
  2. New TiledWriter callback extracts information from documents and makes HTTP calls to Tiled, storing the locations and relative alignment of the files.
  3. Export code uses Tiled as a "directory service" (borrowing Callum's term) to locate the files for a given scan. It packs up "NeXus file", in accordance with beamline requirements, by making h5py.ExternalLinks into the raw files.

At this time we are using Tiled only as a directory service, but of course it could also be used access sliced and transcoded data over HTTP or stream the whole raw files over HTTP.

danielballan /
Last active March 6, 2024 19:12
Demo Stream documents into Tiled
import json
from pathlib import Path
from bluesky.callbacks.tiled_writer import TiledWriter
from tiled.client import from_uri
documents = json.loads(Path("documents.json").read_text())
client = from_uri("http://localhost:8000", api_key="secret")
tw = TiledWriter(client)
danielballan /
Created January 13, 2024 13:29
dask h5py lifecycle issue
import dask
import dask.array as da
import distributed
from distributed.protocol.serialize import dask_deserialize, dask_serialize
from dask.distributed import Client
import h5py
import numpy as np
import logging
logger = logging.Logger(__name__)
danielballan /
Last active May 18, 2023 20:01
Device with custom set
from ophyd.sim import Signal
from ophyd import Component, Device
class Filters(Device):
a = Component(Signal, value=0)
b = Component(Signal, value=1)
def set(self, a, b):
return (

Example of how to extend Tiled with a custom route

  1. Clone this gist and enter the directory.

  2. Install Tiled and and, optionally, a HTTP client to test with.

    pip install tiled[all] httpie
danielballan /
Created March 10, 2023 15:21
Middle exploration


  1. Run python
  2. Using HTTPie, request data: http :8000/data. This should return the list of numbers from the server.
  3. Add a simple middleware that just uses print to print a message in the server logs before and after a request is seen.
  4. Add a second middleware that also prints.

Question: Do the middlewares get run in the order that they are added in or in the reverse order?

danielballan /
Created November 21, 2022 14:45
Writing data into Tiled experimental sandbox
conda activate sandbox
pip install --pre tiled[all] databroker[all] ipython

Note for Mac: Mac uses zsh by default and it requires single quotes like 'databroker[all]'. On the other hand, Windows requires that you not have quotes.

mkdir sandbox
cd sandbox/

Owners - Tech Steering Committee



  • Dan Allan
  • Garrett Bischof
  • Maksim Rakitin
  • Jakob Wlodek