Skip to content

Instantly share code, notes, and snippets.

@perrygeo
Last active October 6, 2022 15:04
Show Gist options
  • Save perrygeo/f3e5afcfe8ed9d4064598d6a041f85b9 to your computer and use it in GitHub Desktop.
Save perrygeo/f3e5afcfe8ed9d4064598d6a041f85b9 to your computer and use it in GitHub Desktop.
GDALOpenShared does not properly handle open options
"""
GDALOpenShared with open options.
This script demonstrates a race condition whereby
the first dataset opened in shared mode will take precedent
over subsequent datasets that are with different open options.
$ python test-open-order.py
2015 mean blue value: 77.47204
2017 mean blue value: 77.47204
2019 mean blue value: 77.47204
$ python test-open-order.py
2019 mean blue value: 82.556031
2017 mean blue value: 82.556031
2015 mean blue value: 82.556031
"""
import os
import random
from osgeo import gdal
GDAL_OF_SHARED = int("0x20", 16)
OPEN_FLAGS = GDAL_OF_SHARED | gdal.GA_ReadOnly
# Read a 1000x1000 chunk of a tileDB array with data at multiple timestamps
path = os.path.abspath("../naip/naip-combined")
x = y = 1000
cols = rows = 1000
years = [2019, 2017, 2015]
random.shuffle(years)
for year in years:
src = gdal.OpenEx(path, OPEN_FLAGS, open_options=[f"TILEDB_TIMESTAMP={year}"])
band = src.GetRasterBand(3)
arr = band.ReadAsArray(x, y, cols, rows)
print(f"{year} mean blue value: {arr.mean()}")
@perrygeo
Copy link
Author

perrygeo commented Oct 6, 2022

In a distributed/multi-process scenario serving web map tiles, this means your server nodes could be initialized with a different timestamp depending on the order and how the requests are routed. This makes for a fun, non-deterministic mixture of timestamps

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment