Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Takes a large image as the input, outputs map tiles at the appropriate size and file structure for use in frameworks like leaflet.js, MapBox, etc.
# -*- coding: utf-8 -*-
import argparse, magic, re, os, math, glob, shutil
'''
GENERATE SLIPPY MAP TILES
Jeff Thompson | 2016 | jeffreythompson.org
Takes a large image as the input, outputs map tiles
at the appropriate size and file structure for use
in frameworks like leaflet.js, MapBox, etc.
ARGS:
input_file large image file to split (JPG, PNG, or TIFF)
zoom_level zoom level(s) to generate (0 to 18); either
integer or range (ex: 2-6)
output_folder folder name to write tiles to (will be created
if does not exist)
OPTIONAL:
-h, --help show this help message and exit
-w --resize_width dimension in pixels for outputted tiles (default 256px)
-q, --quiet suppress all output from program (useful for
integrating into larger projects)
DETAILS:
Resulting tiles are 256px square, regardless of the
size of the source image. The number of tiles wide/
high is determined by the "zoom level", which is
2^zoom. In other words, a zoom level of 3 = 8 tiles,
each resized to 256 pixels square.
Way more info here:
http://wiki.openstreetmap.org/wiki/Slippy_map_
tilenames#Resolution_and_Scale
REQUIRES:
ImageMagick and Python bindings for splitting
images, resizing tiles, etc
http://www.imagemagick.org
https://github.com/ahupp/python-magic
FILE STRUCTURE
Slippy maps require tiles to be stored in a specific
file structure:
output_folder/zoom_level/x/y.png
This is the standard arrangement (some frameworks let
you specify others), and should be noted in your Javascript.
For example, if using leaflet.js, you would use:
tiles/{z}/{x}/{y}.png
ADDING MORE ZOOM LEVELS
Want to add more levels? Just run this script again; it
will append the new zoom level to the same location.
CREATING A SOURCE IMAGE
If combining many smaller images, the easiest method
is to use ImageMagick's 'montage' command.
Your images should be the same size, or at least the
same height. You can do this using ImageMagick as well:
mogrify -geometry x400 *.jpg
Arguments:
x400 height to set images to
*.jpg gets all jpg images from a folder
Then combine into a single image:
montage *.jpg -gravity center -tile NxN -geometry +0+0 output.jpg
Arguments:
*.jpg gets all jpg images from a folder
-gravity centers rows/columns
-tile how many images per row/column in final image
-geometry no extra space between images (or +N+N for padding)
-background none or "rgb(255,255,255)"
output.jpg output filename and format
VERY LARGE IMAGES:
When working with extra big images, ImageMagick makes
some suggestions where RAM may run out:
http://www.imagemagick.org/Usage/files/#massive
'''
# ==============
def power_of(num, base):
''' checks if a number is a power another '''
while(num % base == 0):
num = num / base
return num == 1
def generate(input_file, output_folder, zoom_level, resize_width, quiet):
''' generates slippy map tiles from large image '''
# how many tiles will that be?
num_tiles = pow(2, zoom_level)
if not quiet: print 'Zoom level ' + str(zoom_level) + ' = ' + str(num_tiles) + ' tiles'
# get image dims (without loading into memory)
# via: http://stackoverflow.com/a/19035508/1167783
if not quiet: print 'Getting source image dimensions...'
t = magic.from_file(input_file)
try:
if input_file.endswith('.jpg') or input_file.endswith('.jpeg'):
dims = re.search(', (\d+)x(\d+)', t)
width = int(dims.group(1))
height = int(dims.group(2))
elif input_file.endswith('.tif') or input_file.endswith('.tiff'):
width = int(re.search('width=(\d+)', t).group(1))
height = int(re.search('height=(\d+),', t).group(1))
elif input_file.endswith('.png'):
dims = re.search(', (\d+) x (\d+)', t)
width = int(dims.group(1))
height = int(dims.group(2))
else:
if not quiet: print 'ERROR: Unknown source image type; JPG, TIFF, or PNG only! Quitting...'
exit(0)
except:
if not quiet: print 'ERROR: Could not parse source image dims! Quitting...'
exit(0)
if not quiet: print '- ' + str(width) + ' x ' + str(height) + ' pixels'
# errors and warnings
if not power_of(int(width), 2):
if not quiet: print 'WARNING: Source image dims should be power of 2! Continuing anyway...'
if width != height:
if not quiet: print 'ERROR: Source image should be square! Quitting...'
exit(0)
# get details for ImageMagick
if not quiet: print 'Splitting to...'
tile_width = int(math.ceil(width / num_tiles))
if not quiet: print '- ' + str(tile_width) + ' x ' + str(tile_width) + ' px tiles'
pad = len(str(num_tiles * num_tiles))
# split using ImageMagic, then resize to the expected tile size
# use filename padding so glob gets them in the right order
if not os.path.exists(output_folder):
os.mkdir(output_folder)
cmd = 'convert ' + input_file + ' -quiet -crop ' + str(tile_width) + 'x' + str(tile_width) + ' -resize ' + str(resize_width) + 'x' + str(resize_width) + ' ' + output_folder + '/%0' + str(pad) + 'd.png'
os.popen(cmd)
if not quiet: print '- done!'
# rename/move images into tile server format
if not quiet: print 'Moving files into column folders...'
# 1. make cols
for x in range(0, num_tiles):
folder = output_folder + '/' + str(zoom_level) + '/' + str(x)
if not os.path.exists(folder):
os.makedirs(folder)
# 2. move tiles into their column folders
tiles = glob.glob(output_folder + '/*.png')
for i, tile in enumerate(tiles):
col = i % num_tiles
f = i / num_tiles
dst = output_folder + '/' + str(zoom_level) + '/' + str(col) + '/' + str(f) + '.png'
shutil.move(tile, dst)
if not quiet: print '- done!'
# ==============
if __name__ == '__main__':
p = argparse.ArgumentParser(description='Takes a large image as the input, outputs map tiles at the appropriate size and file structure for use in frameworks like leaflet.js, MapBox, etc. Much more info in the source code.', usage='python GenerateSlippyMapTiles.py input_file zoom_level output_folder [options]')
p.add_argument('input_file', help='large image file to split (JPG, PNG, or TIFF)')
p.add_argument('zoom_level', help='zoom level(s) to generate (0 to 18); either integer or range (ex: 2-6)')
p.add_argument('output_folder', help='folder name to write tiles to (will be created if does not exist)')
p.add_argument('-w', '--resize_width', help='dimension in pixels for outputted tiles (default 256px)', metavar='', type=int, default=256)
p.add_argument('-q', '--quiet', help='suppress all output from program (useful for integrating into larger projects)', action='store_true')
args = p.parse_args()
input_file = args.input_file
zoom_level = args.zoom_level
output_folder = args.output_folder
resize_width = args.resize_width
quiet = args.quiet
if not quiet: print 'GENERATING SLIPPY-MAP TILES'
if not quiet: print ('- ' * 14)
# if multiple zoom levels, run them all
# otherwise, run just once
if '-' in zoom_level:
try:
match = re.search(r'([0-9]+)-([0-9]+)', zoom_level)
zoom_min = int(match.group(1))
zoom_max = int(match.group(2))
except:
if not quiet: print "ERROR: Couldn't parse zoom levels; should be int or 'min-max'! Quitting..."
exit(0)
for z in range(zoom_min, zoom_max+1):
generate(input_file, output_folder, z, resize_width, quiet)
if not quiet: print ('- ' * 14)
else:
generate(input_file, output_folder, int(zoom_level), resize_width, quiet)
if not quiet: print ('- ' * 14)
# that's it!
if not quiet: print 'FINISHED!'
@jeffThompson

This comment has been minimized.

Copy link
Owner Author

@jeffThompson jeffThompson commented May 17, 2016

Official (ie more likely to be updated) version in this repo: https://github.com/jeffThompson/EmptyApartments/tree/master/GenerateSlippyMapTiles

@calsurferpunk

This comment has been minimized.

Copy link

@calsurferpunk calsurferpunk commented Jul 25, 2019

Hi. Thanks for sharing this. I don't know if it's the same for you, but I had to change line 158 from

tiles = glob.glob(output_folder + '/.png')
to
tiles = sorted(glob.glob(output_folder + '/
.png'))

in order to have the tiles in the correct order. I'm currently running Python 2.7.16 on Ubuntu 19.04.

@jeffThompson

This comment has been minimized.

Copy link
Owner Author

@jeffThompson jeffThompson commented Jul 26, 2019

@calsurferpunk – hmm, it shouldn't need that, I don't think? But it probably can't hurt either.

@Qwerty-Space

This comment has been minimized.

Copy link

@Qwerty-Space Qwerty-Space commented Aug 29, 2019

This doesn't seem to be creating any files, just directories.

@jeffThompson

This comment has been minimized.

Copy link
Owner Author

@jeffThompson jeffThompson commented Sep 10, 2019

@Qwerty-Space – so weird! Do you get a full set of nested directories (ie in the format of tiles/{z}/{x}/{y}.png)? Are you getting any errors?

@Qwerty-Space

This comment has been minimized.

Copy link

@Qwerty-Space Qwerty-Space commented Sep 11, 2019

No errors, it just completes with the directories in place

@calsurferpunk

This comment has been minimized.

Copy link

@calsurferpunk calsurferpunk commented Sep 11, 2019

I was getting all of the files in each set of folders, but they weren't correct for the needed coordinates. I'm pretty sure I had empty folders when I tried to use an incorrect image size (i.e. not perfectly square).

@Pysis868

This comment has been minimized.

Copy link

@Pysis868 Pysis868 commented Sep 20, 2019

> pip2 install python-magic
> python2 GenerateSlippyMapTiles.py --help
usage: python GenerateSlippyMapTiles.py input_file zoom_level output_folder [options]

Takes a large image as the input, outputs map tiles at the appropriate size
and file structure for use in frameworks like leaflet.js, MapBox, etc. Much
more info in the source code.

positional arguments:
  input_file            large image file to split (JPG, PNG, or TIFF)
  zoom_level            zoom level(s) to generate (0 to 18); either integer or
                        range (ex: 2-6)
  output_folder         folder name to write tiles to (will be created if does
                        not exist)

optional arguments:
  -h, --help            show this help message and exit
  -w , --resize_width   dimension in pixels for outputted tiles (default
                        256px)
  -q, --quiet           suppress all output from program (useful for
                        integrating into larger projects)
@Pysis868

This comment has been minimized.

Copy link

@Pysis868 Pysis868 commented Sep 20, 2019

Added magic file specification parameter: https://gist.github.com/Pysis868/99e4b6f6592cbc0e59c5849316d0f86c
On my Mac with Homebrew it was at /usr/local/Cellar/libmagic/5.37/share/misc/magic.
It uses the executable from file-formula and requires the dependency libmagic that provides this.

@jeffThompson

This comment has been minimized.

Copy link
Owner Author

@jeffThompson jeffThompson commented Sep 21, 2019

Sorry all that I haven't had time to look into this.

@Pysis868 – did this solve the issue and you now get tiles correctly?

@prasanjitdash

This comment has been minimized.

Copy link

@prasanjitdash prasanjitdash commented Apr 19, 2020

Hi Jeff,
I have a question that is related but not exactly about your tiler.

I have looked through many os resources where codes are available to create tiles, based on the assumption that the source image is available. But I would love to understand more about the creation of the source image first. Do you have any tip or publicly available resource on creating a source image from a data file using python3?

E.g.,

  1. How to make a source image from a netCDF file available from:
    ftp://ftp.nodc.noaa.gov/pub/data.nodc/ghrsst/L4/GLOB/JPL/MUR/2020/
    These are 1 KM spatial resolution data, meaning effectively 36000 x 18000 pixels.

  2. Is the so-called source image a large raster image with 36000 x 18000 pixels (pre-generated)?

  3. Then how to generate multiple CRS datasets for WGS84 projection? Namely, EPSG 4326, 3413, 3031. I am not very much interested in the Google Mercator EPSG 3857. Can the same source map be reused for each of these CRSs or does it have to be generated separately for each CRS.

I understand this is unrelated to your tiler (which needs a source map you generated using imageMagick), but some pointers would be useful to reuse the tiler subsequently.

Hope you are keeping safe in these turbulent time in your location.
Thanks

@jeffThompson

This comment has been minimized.

Copy link
Owner Author

@jeffThompson jeffThompson commented Apr 20, 2020

@prasanjitdash – sorry, not sure I can be much help here! I don't use GIS or mapping tools much. I think you'd probably look for existing satellite data, then combine them into a single image (maybe using ImageMagick or a similar tool) and then split it into tiles? I know NASA and ESA have some data, as does NOAA too I believe? Or is your question what to do once you have the imagery downloaded already?

@prasanjitdash

This comment has been minimized.

Copy link

@prasanjitdash prasanjitdash commented Apr 20, 2020

Thanks @jeffThompson for your response. Satellite data availability is no issue - there is plenty (as in the example link in my question). The question is rather amateurish and related to the procedure - what is a 'Source Map' and how to make it. Tiling is the second step.

1) Source Map is a full resolution big image? or you make small high-res images and stitch?

2) Making a big image of the earth, let's say with 100 m resolution will be a very huge file, approx 360000 x 180000 size and bytes

3) Do you have to make Source Maps with various CRS, depending on your target tiles or one Big Source Map can be reused for all?

I will do some more digging and will get back here for mentioning if I find a procedural answer. Thas again Jeff for your response and stay safe.

@jeffThompson

This comment has been minimized.

Copy link
Owner Author

@jeffThompson jeffThompson commented Apr 20, 2020

@prasanjitdash - aha, got it! Thanks for clarifying. If I remember correctly, yes: you make one really big image(mine was 150,000 pixels square) which then gets sliced into the tiles. The files will be huge but you don't need more than one source image – basically this code just slices it into different sized squares, one for each zoom level. Hope that helps!

@prasanjitdash

This comment has been minimized.

Copy link

@prasanjitdash prasanjitdash commented Apr 20, 2020

@jeffThompson Hi Jeff, Thanks, again. I will move in this direction then of generating huge source images and then do the slicing. If some other ideas dawn upon me on some of the uneventful evenings, I will get back here to report as a source for those who may be pondering over the same question. Something like a code to generate a source image, sample input, sample output, and sliced tiles with a few different CRS would be useful I guess. Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.