Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
Fetch and untile tiled Himawari-8 images from the PNG endpoint
import requests as req
import sys
from dateutil.parser import parse
from PIL import Image
from StringIO import StringIO
# <date> <zoom level> <output>
# E.g.: 2016-01-13T22:10:00 8 2016-01-13T221000-z8.png
# Fetch Himawari-8 full disks at a given zoom level.
# Valid zoom levels seem to be powers of 2, 1..16, and 20.
# To do:
# - Better errors (e.g., catch the "No Image" image).
# - Don't ignore seconds, and/or:
# - option to snap to nearest valid time.
# - Librarify.
# Tile size for this dataset:
width = 550
height = 550
time = parse(sys.argv[1])
scale = int(sys.argv[2])
out = sys.argv[3]
base = '' % (scale)
tiles = [[None] * scale] * scale
def pathfor(t, x, y):
return "%s/%s/%02d/%02d/%02d%02d00_%s_%s.png" \
% (base, t.year, t.month,, t.hour, t.minute, x, y)
def fetch(session, path, retries=1, verbose=False, failure='error'):
# FIXME: this is kinda messy:
if retries < 0:
if failure == 'warning':
raise Warning('Could not download %s (filling with black)')
return np.zeros((width, height, 3))
elif failure == 'silent':
return np.zeros((width, height, 3))
else: # presumed failure == 'error':
raise IOError('Could not download %s')
tiledata = session.get(path).content
tile =
return tile
if verbose:
print('Retrying %s (%s retries left)' % (path, retries))
return fetch(
retries=(retries - 1),
sess = req.Session() # so requests will reuse the connection
png ='RGB', (width*scale, height*scale))
for x in range(scale):
for y in range(scale):
path = pathfor(time, x, y)
tile = fetch(sess, path, retries=4, verbose=True, failure='error')
png.paste(tile, (width*x, height*y, width*(x+1), height*(y+1))), 'PNG')
Copy link

joe32097 commented Feb 1, 2016

If I am using py 3.5, how can I get StringIO. Are there any up-to-date-versions?

Copy link

wizpig64 commented Feb 4, 2016

Copy link

willwhitney commented Feb 4, 2016

Any idea how to avoid or catch the "No Image" images? I've got this running every ten minutes and setting my desktop background.

Copy link

mnstrspeed commented Feb 5, 2016

@willwhitney tiles in the "No Image" frames all have the same tiledata, so as long as you're using a scale >= 2 (i.e., at least two frames) you can just check whether any two tiles have the same tiledata. Roughly:

prev_tiledata = ""
for x in range(scale):
  for y in range(scale):
    # ...
    if tiledata == prev_tiledata:
    prev_tiledata = tiledata
    # ...

I think I also read somewhere that the "No Image" frames happen at specific times, but this works for me.

Copy link

willwhitney commented Feb 5, 2016

Nice, that's a solid answer.

Copy link

celoyd commented Jun 14, 2016

I also read somewhere that the "No Image" frames happen at specific times

They happen at 02:40 and at 14:40 virtually every day, but they can also happen at other times because of maintenance or random errors, so it’s not enough to hard-code those times.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment