MGLMapView+MBTiles

readme.md

MGLMapView+MBTiles.swift

What it does

It enables an MGLMapView (from the Mapbox Maps iOS SDK) to display MBTiles.

How it works

It adds an extension to MGLMapView for adding MBTiles sources. When an MBTiles source is added to an MGLMapView, it starts a web server within your app and points the map's style to localhost.

Installation

1. Add MGLMapView+MBTiles.swift (from this Gist) to your app.
2. Add the GCDWebServer library to your app.
3. Add the SQLite.swift library to your app.
4. Add the following to your app's Info.plist:

    <key>NSAppTransportSecurity</key>
    <dict>
        <key>NSExceptionDomains</key>
        <dict>
            <key>localhost</key>
            <dict>
                <key>NSExceptionAllowsInsecureHTTPLoads</key>
                <true/>
                <key>NSIncludesSubdomains</key>
                <false/>
            </dict>
        </dict>
    </dict> 

Usage

Add a source to mapView

let path = "/path/to/source.mbtiles"
if let source = try? MBTilesSource(filePath: path),
   let index = mapView.style?.layers.count {
    try? mapView.insertMBTilesSource(source, at: index)
}

Remove a source from mapView

mapView.remove(MBTilesSource: source)

Known Limitations

• Only MBTiles that contain raster data sources (jpg, png) are currently supported. Vector sources (i.e. pbf) are not (yet) supported.

Why?

I created this for my app, Forest Maps.

MGLMapView+MBTiles.swift

import Foundation
import Mapbox
import SQLite

// MARK: MbtilesSource

enum MBTilesSourceError: Error {
    case CouldNotReadFileError
    case UnknownFormatError
    case UnsupportedFormatError
}

class MBTilesSource: NSObject {
    
    // MBTiles spec, including details about vector tiles:
    // https://github.com/mapbox/mbtiles-spec/
    
    // MARK: Properties
    
    var filePath: String
    fileprivate var isVector = false
    fileprivate var tileSize: Int?
    fileprivate var layersJson: String?
    fileprivate var attribution: String?
    var minZoom: Float?
    var maxZoom: Float?
    var bounds: MGLCoordinateBounds?
    
    fileprivate var mglTileSource: MGLTileSource?
    fileprivate var mglStyleLayer: MGLForegroundStyleLayer?
    var isVisible: Bool {
        get {
            return mglStyleLayer == nil ? false : mglStyleLayer!.isVisible
        }
        set {
            mglStyleLayer?.isVisible = newValue
        }
    }
    
    private var database: Connection?
    private let validRasterFormats = ["jpg", "png"]
    private let validVectorFormats = ["pbf", "mvt"]
    
    // MARK: Functions
    
    init(filePath: String) throws {
        self.filePath = filePath
        super.init()
        
        do {
            self.database = try Connection(filePath, readonly: true)
            
            guard let anyTile = try database?.scalar("SELECT tile_data FROM tiles LIMIT 1") as? Blob else {
                throw MBTilesSourceError.CouldNotReadFileError
            }
            let tileData = Data(bytes: anyTile.bytes)
            
            // https://stackoverflow.com/a/42104538
            var format: String?
            let headerData = [UInt8](tileData)[0]
            if headerData == 0x89 {
                format = "png"
            } else if headerData == 0xFF {
                format = "jpg"
            } else {
                format = getMetadata(fieldName: "format")
            }
            
            if format == nil {
                throw MBTilesSourceError.UnknownFormatError
            } else if validRasterFormats.contains(format!) {
                isVector = false
            } else if validVectorFormats.contains(format!) {
                isVector = true
            } else {
                throw MBTilesSourceError.UnsupportedFormatError
            }
            
            if let tileImage = UIImage(data: tileData) {
                let screenScale = UIScreen.main.scale > 1 ? 2 : 1
                self.tileSize = Int(tileImage.size.height / CGFloat(screenScale))
            }
            
            if let minString = getMetadata(fieldName: "minzoom") {
                minZoom = Float(minString)
            }
            if let maxString = getMetadata(fieldName: "maxzoom") {
                maxZoom = Float(maxString)
            }
            if let boundsString = getMetadata(fieldName: "bounds") {
                let coordinates = boundsString.split(separator: ",")
                if let west  = CLLocationDegrees(coordinates[0]),
                    let south = CLLocationDegrees(coordinates[1]),
                    let east = CLLocationDegrees(coordinates[2]),
                    let north  = CLLocationDegrees(coordinates[3]) {
                    bounds = MGLCoordinateBoundsMake(CLLocationCoordinate2DMake(south, west),
                                                     CLLocationCoordinate2DMake(north, east))
                }
            }
            attribution = getMetadata(fieldName: "attribution")
            layersJson = getMetadata(fieldName: "json")
        } catch {
            throw MBTilesSourceError.CouldNotReadFileError
        }
    }
    
    deinit {
        database = nil
    }
    
    private func getMetadata(fieldName: String) -> String? {
        let query = "SELECT value FROM metadata WHERE name=\"\(fieldName)\""
        if let binding = try? database?.scalar(query) {
            return binding as? String
        }
        return nil
    }
    
    fileprivate func getTile(x: Int, y: Int, z: Int) -> Data? {
        let query = "SELECT tile_data FROM tiles WHERE tile_column=\(x) AND tile_row=\(y) AND zoom_level=\(z)"
        if let binding = try? database?.scalar(query),
            let blob = binding as? Blob {
            return Data(bytes: blob.bytes)
        }
        return nil
    }
}

// MARK: - MGLMapView Extension

extension MGLMapView {
    func insertMBTilesSource(_ source: MBTilesSource, at index: UInt, tileSize: Int? = nil) throws {
    
        remove(MBTilesSource: source)
        
        let port: UInt = 54321
        
        MbtilesServer.shared.start(port: port)
        MbtilesServer.shared.sources[source.filePath] = source
        
        let filePath = source.filePath as NSString
        
        guard let escapedPath = filePath.addingPercentEncoding(withAllowedCharacters: NSCharacterSet.urlPathAllowed) else {
            return
        }
        
        let id = filePath.lastPathComponent
        let urlTemplate = "http://localhost:\(port)\(escapedPath)?x={x}&y={y}&z={z}"
        var options = [MGLTileSourceOption : Any]()
        options[MGLTileSourceOption.tileCoordinateSystem] = MGLTileCoordinateSystem.TMS.rawValue
        options[MGLTileSourceOption.tileSize] = tileSize ?? source.tileSize
        options[MGLTileSourceOption.minimumZoomLevel] = source.minZoom
        options[MGLTileSourceOption.maximumZoomLevel] = source.maxZoom
        if let bounds = source.bounds {
            options[MGLTileSourceOption.coordinateBounds] = NSValue(mglCoordinateBounds: bounds)
        }
        if let attributionString = source.attribution {
            options[MGLTileSourceOption.attributionInfos] = [MGLAttributionInfo(title: NSAttributedString(string: attributionString) , url: nil)]
        }
        
        if source.isVector {
            // Only raster data (jpg, png) is currently supported.
            // Raster data (i.e. pbf) is not (yet) supported.
            throw MBTilesSourceError.UnsupportedFormatError
        } else {
            source.mglTileSource = MGLRasterTileSource(identifier: id, tileURLTemplates: [urlTemplate], options: options)
            source.mglStyleLayer = MGLRasterStyleLayer(identifier: id, source: source.mglTileSource!)
        }
        
        if source.minZoom != nil {
            source.mglStyleLayer!.minimumZoomLevel = source.minZoom!
        }

        if source.maxZoom != nil {
            source.mglStyleLayer!.maximumZoomLevel = source.maxZoom!
        }
        
        DispatchQueue.main.async { [weak self] in
            self?.style?.addSource(source.mglTileSource!)
            self?.style?.insertLayer(source.mglStyleLayer!, at: index)
        }
    }
    
    func remove(MBTilesSource source: MBTilesSource) {
        let filePath = source.filePath
        let id = (filePath as NSString).lastPathComponent
        DispatchQueue.main.async { [weak self] in
            if let layer = self?.style?.layer(withIdentifier: id) {
                self?.style?.removeLayer(layer)
            }
            if let source = self?.style?.source(withIdentifier: id) {
                self?.style?.removeSource(source)
            }
        }
        MbtilesServer.shared.sources[filePath] = nil
    }
}

// MARK: - Server

fileprivate class MbtilesServer: NSObject {
    
    // MARK: Properties
    
    static let shared = MbtilesServer()
    var sources = [String : MBTilesSource]()
    
    private var webServer: GCDWebServer? = nil
    
    // MARK: Functions
    
    func start(port: UInt) {
        guard Thread.isMainThread else {
            DispatchQueue.main.sync { [weak self] in
                self?.start(port: port)
            }
            return
        }
        
        if webServer == nil {
            webServer = GCDWebServer()
        }
        guard !webServer!.isRunning else {
            return
        }
        GCDWebServer.setLogLevel(3)
        webServer?.addDefaultHandler(forMethod: "GET", request: GCDWebServerRequest.self, processBlock: { [weak self] (request: GCDWebServerRequest) -> GCDWebServerResponse? in
            return self?.handleGetRequest(request)
        })
        webServer?.start(withPort: port, bonjourName: nil)
    }
    
    func stop() {
        guard Thread.isMainThread else {
            DispatchQueue.main.sync { [weak self] in
                self?.stop()
            }
            return
        }
        
        webServer?.stop()
        webServer?.removeAllHandlers()
    }
    
    private func handleGetRequest(_ request: GCDWebServerRequest) -> GCDWebServerResponse? {
        if (request.path as NSString).pathExtension == "mbtiles",
            let source = sources[request.path],
            let query = request.query,
            let xString = query["x"] as? String,
            let yString = query["y"] as? String,
            let zString = query["z"] as? String,
            let x = Int(xString),
            let y = Int(yString),
            let z = Int(zString),
            let tileData = source.getTile(x: x, y: y, z: z) {
            
            return GCDWebServerDataResponse(data: tileData, contentType: "")
        } else {
            return GCDWebServerResponse(statusCode: 404)
        }
    }
}

Thanks, @typebrook, for your suggestion over in mapbox/mapbox-gl-native#6862 (comment). I think you might be right that creating and adding an MGLVectorSource would work for vector tiles. You might try replacing the following line:

throw MBTilesSourceError.UnsupportedFormatError

with:

source.mglTileSource = MGLVectorSource(identifier: id, tileURLTemplates: [urlTemplate], options: options)

and seeing what happens.

The part I'm not sure about is the style layer. For vector tiles, would we need to use the vector equivalent to MGLRasterStyleLayer? There are several different MGLVectorStyleLayer subclasses, and I'm not sure which one (if any) to use.

https://www.mapbox.com/ios-sdk/api/3.7.2/Classes/MGLVectorStyleLayer.html

Thoughts?

@namannik
There is no certain way to add layers for vector source, unlike raster source, user can define many things (filter/zoom/type)with same vector data. For example, a single default Mapbox street style use 20 different layers for a single vector source.

I think we can directly put local host url into existing json style file. And define layers using this vector source in json style file. Later, just activate local host service, and tiles should be rendered.

As I mentioned, unlike raster source, add subclass of MGLVectorStyleLayer in code spend more efforts, I prefer using json style file.(Of course we can dynamically add all layers we want in code, but put them into style file should be the best choice)

Android Kotlin solution that supports vector tiles:

https://gist.github.com/typebrook/7d25be326f0e9afd58e0bbc333d2a175

Firstly, thanks for sharing this gist. It's proved to be a great starting point for our use case, which is to serve raster/vector mbtiles for offline mapping.

There were a couple of gotchas I encountered whilst putting together a prototype for vector tiles and I thought I'd mention them here in case anyone encounters similar issues:

Gzip

I had to manually un-gzip vector tiles before returning them from the GCDWebServer. Without doing so, I encountered an error as follows:

Unknown pbf field type exception

I used the library GzipSwift for the unzipping:

if tileData.isGzipped {
    let data = try! tileData.gunzipped()
    return GCDWebServerDataResponse(data: data, contentType: "")
} else {
    return GCDWebServerDataResponse(data: tileData, contentType: "")
}

Note that I tried configuring the GCDWebServerDataResponse with "Content-Encoding": "gzip" and this had no effect. I also tried setting .isGZipContentEncodingEnabled = true and this too had no effect. Not sure if I'm missing something obvious here?

Working Offline

If you're looking to serve tiles without an internet connection, then the following steps may be required:

I had to modify my .plist in the following way. Note that our apps target iOS 11+ (and thus 10+, where I think this change was introduced). To my knowledge the following is functionally equivalent, but preferred, to Step 4 (detailed in the opening post).

<dict>
    <key>NSAllowsLocalNetworking</key>
    <true/>
</dict>

Secondly, I had to start the GCDWebServer as follows in order to access via 'http://localhost':

try! webServer?.start(options: ["Port": port, "BindToLocalhost": true])

---

With those changes in place the (very early prototype) seems to function well enough that we're now once again considering Mapbox as a solution. Once again, a big thanks for the initial concept.

First thing first, thanks for this gist!
I'm loading a simple mbtiles, which is correctly shown via Mapkit. With this gist, though, I notice an blocking issue:
libc++abi.dylib: terminating with uncaught exception of type std::out_of_range: vector

The code is super simple:
let tilesURL = Bundle.main.url(forResource: "mytiles", withExtension: "mbtiles") if let source = try? MBTilesSource(filePath: tilesURL?.path ?? ""), let index = mapView.style?.layers.count { try? mapView.insertMBTilesSource(source, at: UInt(index)) }

I made myself sure that the try statement gets triggered.
Any hints?

Yes, I also got the same error using oc mixture. How are you doing now? @StefanoBiasu

I tried a method to load the offline mbtiles.
In didFinishLoadingStyle add code, I found the self. The mapView. The style.css. The layers of two layer for mapbox default, so I will index is set to 0, a perfect load.
Of course, I do not understand what is the reason, I hope there is a god passing by to explain, thank you!

By the way, the error should be because addlayer is overstepping the bounds, it's reading the count of the array, if it's a subscript, it should be -1, thank you!

As far as I know, @songyuyang0918, the only way to fix it is change mapbox gl native code, explained here:
mapbox/mapbox-gl-native#11538 (comment)
However, it takes a while to recompile mapbox gl native..

Great piece of sample code, it works for me with vector tiles. But the only problem is that when I start to pan and zoom, the tiles disappear for some zoom ranges. Does this mean that the .mbtiles file misses tiles for this range?

@StefanoBiasu @namannik Who can share a class file that supports swift4.2?
Swift is my pain point,
My ios version no longer supports swift3.0

Are there any specific errors preventing it from compiling?

Has been resolved if compiled using swift4.2
Init is preceded by @objc public
Add @objc before func insertMBTilesSource
Thank you very much!

I am trying this solution to show mbtiles on my iPad app.
GCDWebServer started the local server correctly but iPad app doesn't trigger this url
"http://localhost:\(port)\(escapedPath)?x={x}&y={y}&z={z}"
Which code line must trigger this url?
Actually self?.style?.insertLayer(source.mglStyleLayer!, at: index) make the app crashes so I changed this line as follows
self?.style?.insertLayer(source.mglStyleLayer!, at: 0)
Then app doesn't crashed but mbtiles are not showing up. Is this why the localhost url doesn't triggered?
I am trying to show mbtiles for several days but still no luck.
I would be appreciate any help. Thanks in advance,

I just replaced the mbtiles with another one and I see the local url are triggering successfully.
Some mbtiles are not working and some mbtiles are working properly. I don't know why.
I am sure all mbtiles are correct and if I open the mbtiles with a tool like mbtiles viewer, it shows the mbtiles successfully.
Is there any criteria on mbtiles that can work on iOS? I am testing on iOS13.1.2.

Hi, @danielsgit. Make sure you're calling insertMBTilesSource from either mapView(_ mapView: MGLMapView, didFinishLoading style: MGLStyle) or mapViewDidFinishLoadingMap(_ mapView: MGLMapView) inside your MGLMapViewDelegate. For example:

func mapView(_ mapView: MGLMapView, didFinishLoading style: MGLStyle) {
    let path = "/path/to/source.mbtiles"
    if let source = try? MBTilesSource(filePath: path),
        let index = mapView.style?.layers.count {
        try? mapView.insertMBTilesSource(source, at: index)
    }
}

hi,@namannik
If I have a lot of mbtiles files, using the current method, which can only be stacked layer by layer, would you consider updating a method that can read more than one mbtiles?

This error occurred recently while swiping the map:
[ERROR] ERROR while writing to socket 65: Broken pipe (32)
I saw this log on the real machine

@songyuyang0918, using the MGLMapView+MBTiles extension, you can add as many MBTiles layers as you'd like. Just be sure to use the appropriate index when calling insertMBTilesSource to ensure the tiles are shown in the order you'd like.

I'm not sure what would be the cause of the broken pipe error you're seeing.

hi @namannik
About Mapbox initialization. If I'm using offline, I also have to have a custom white space source and then add it to styleURL?
I wonder how you handled it?

@songyuyang0918 My app uses a hybrid of an online tile sours and multiple offline tile source. I'm not sure what you mean about "white space source".

@ namannik
Forgive me for not describing it clearly, I just want to show the source of the block multiple offline tile source.
But MapBox requires that I have the online tile sours to use the offline tile source.
Do you have any good ideas?

My app works with a styleURL that references a style.json file contained locally within the app and tiles hosted online. The user can download tiles for offline use if they want. I'm not aware that having an online tile source is a requirement of MapBox. As far as I know, you can set the styleURL to get tiles from anywhere, including locally on the device.

StyleUrl can indeed be accessed from a.JSON file or an online hosting tile.
Because I'm using the Mapbox-based extended offline function MBTilesSource that you developed.
If I only show the offline map, then only show MBTilesSource without showing the styleUrl?
In fact, if styleUrl = nil, Then MBTilesSource will not work.

hey, any suggestions on how to implement this approach with Mapbox V10.10.0?

@hjhimanshu01 @namannik

Do you know whether this is possible with https://docs.mapbox.com/mapbox-gl-js/api/ for hybrid web wrapper native apps?

If not:
I am thinking of developing a plugin. How hard would that be, to get that to work? What is standing in the way currently?