typebrook/MBTilesServer.kt

## README.md

      
    Raw
  

              README.md
            
          
    MBTilesSource.kt + MbtilesServer.kt

What it does?

It enables a MapboxMap (from Mapbox Android SDK) to use MBTiles as Source.
Inspired from similar Swift approach in iOS SDK, made by @namannik.
How it works?

In you APP, when a MBTilesSource instance activates, it starts a localhost MBTilesServer and works like a common Mapbox Raster/Vector Source.
MBTilesServer is a Singleton. It can hosts multiple MBTilesSource. The URL for each MBTilesSource is http://localhost:8888/[Source ID]/{z}/{x}/{y}.[jpg|png|pbf|mvt]
Since MBTilesSource only works as common Mapbox Raster/Vector Source, you always need to add layer or set style for MapboxMap by yourself. Mapbox Style is a bit more complex. If your MBTiles contains vector tile (.mvt), make sure your specify correct value for source-layer. See spec here.
Also, you may check an example by MapTiler, an OSM vector tiles provider.
Installation


Add MBTilesSource.kt and MBTilesServer.kt (from this gist) into your Android project.

Usage

Prepare network configuration

To make sure MBTilesServer works without network connection above Android 9 (API level 28), you have to allow HTTP traffic to localhost.
For more information, see Android Developers.

Add network configuration into AndroidManifest.xml
android:networkSecurityConfig="@xml/network_security_config" >

A resource file named in AndroidManifest.xml
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <domain-config cleartextTrafficPermitted="true">
        <domain includeSubdomains="false">localhost</domain>
    </domain-config>
</network-security-config>


Create a MBTilesSource to read your MBTiles file

// Connect to localhost even when device is not connected to internet
Mapbox.setConnected(true)

// Create MBTilesSource
val path = "/path/to/source.mbtiles" // file path
val sourceId = "source1" // used as Mapbox Source ID
val mbSource = try {
    MBTilesSource(path, sourceId).apply { activate() }
} catch (e: MBTilesSourceException.CouldNotReadFileException){
    // Deal with error if fail to read MBTiles
}
Now, a XYZ Tile Service is working on http://localhost:8888/source1/{z}/{x}/{y}.[mvt|pbf|jpg|png].
The file extension of each tile is judged by the format in MBTiles. (Defined in table metadata, only mvt, pbf, jpg, png are allowed)
Also, if your MBTiles is at asset folder, you may use companion object in MBTilesSource to copy it into internal storage, and get the path directly.
val path = MBTilesSource.readAsset(context, "FILENAME.mbtiles")
Remove a source from MBTilesServer

 mbSource.deactivate()
Work with Mapbox Style

// In callback of Style.OnStyleLoaded
style.addSource(mbSource.instance)

// If mbSource contains raster tiles
val rasterLayer = RasterLayer("raster_layer_id", mbSource.id)
style.addLayer(rasterLayer)

// If mbSource contains vector tiles
val vectorLayer = LineLayer("vector_layer_id", mbSource.id)
style.addLayer(vectorLayer)
How it looks like?

I created this for my app, Five More Minutes.
This demo use vector MBTiles and style from OpenMapTiles.
Alternatives


If you want to create an APP with map which really reads MBTiles directly, consider use Tangram SDK instead of Mapbox SDK. It natively support use a MBTiles as source offline, see Documentation here.
Do not challenge why Mapbox doesn't support a format they created :D


Or, use Maplibre-gl-native, a mapbox-gl-native fork which embraces open source!


If it won't bother you to store tiles with folders, you can use asset:// and file:// protocol with source. See related thread in StackOverflow here.


CHANGELOG


2021-01-26

Add Alternatives section


2020-11-25

Add network configuration for newer Android version, thanks @Cacaonut !
Remove Anko from dependencies
Code refactor
Exmaple refactor for latest Mapbox API


2020-05-26

Fix connection issue about localhost, thanks @grzesiek2010 !


## MBTilesServer.kt
package com.example.sample.offline

import android.util.Log
import java.io.BufferedReader
import java.io.ByteArrayOutputStream
import java.io.FileNotFoundException
import java.io.PrintStream
import java.net.ServerSocket
import java.net.Socket
import kotlin.math.pow

/*
 * Localhost tile server with MBTilesSource
 */

object MBTilesServer : Runnable {

    const val port = 8888
    private val serverSocket: ServerSocket = ServerSocket(port)
    var isRunning = false
    val sources: MutableMap<String, MBTilesSource> = mutableMapOf()

    fun start() {
        isRunning = true
        Thread(this).start()
    }

    fun stop() {
        isRunning = false
        serverSocket.close()
    }

    override fun run() {
        try {
            while (isRunning) {
                serverSocket.accept().use { socket ->
                    Log.d(javaClass.simpleName, "Handling request")
                    handle(socket)
                    Log.d(javaClass.simpleName, "Request handled")
                }
            }
        } catch (e: Exception) {
            Log.d(
                javaClass.simpleName,
                e.localizedMessage ?: "Exception while running MBTilesServer"
            )
        } finally {
            Log.d(javaClass.simpleName, "request handled")
        }
    }

    @Throws
    private fun handle(socket: Socket) {

        val reader: BufferedReader = socket.getInputStream().reader().buffered()
        // Output stream that we send the response to
        val output = PrintStream(socket.getOutputStream())

        try {
            var route: String? = null

            // Read HTTP headers and parse out the route.
            do {
                val line = reader.readLine() ?: ""
                if (line.startsWith("GET")) {
                    // the format for route should be {source}/{z}/{x}/{y}
                    route = line.substringAfter("GET /").substringBefore(".")
                    break
                }
            } while (line.isNotEmpty())

            // the source which this request target to
            val source = sources[route?.substringBefore("/")] ?: return

            // Prepare the content to send.
            if (null == route) {
                writeServerError(output)
                return
            }

            val bytes = loadContent(source, route) ?: run {
                writeServerError(output)
                return
            }

            // Send out the content.
            with(output) {
                println("HTTP/1.0 200 OK")
                println("Content-Type: " + detectMimeType(source.format))
                println("Content-Length: " + bytes.size)
                if (source.isVector) println("Content-Encoding: gzip")
                println()
                write(bytes)
                flush()
            }
        } finally {
            reader.close()
            output.close()
        }
    }

    @Throws
    private fun loadContent(source: MBTilesSource, route: String): ByteArray? = try {
        val (z, x, y) = route.split("/").subList(1, 4).map { it.toInt() }
        source.getTile(z, x, (2.0.pow(z)).toInt() - 1 - y)
    } catch (e: FileNotFoundException) {
        e.printStackTrace()
        null
    }

    private fun writeServerError(output: PrintStream) {
        output.println("HTTP/1.0 500 Internal Server Error")
        output.flush()
    }

    private fun detectMimeType(format: String): String? = when (format) {
        "jpg" -> "image/jpeg"
        "png" -> "image/png"
        "mvt" -> "application/x-protobuf"
        "pbf" -> "application/x-protobuf"
        else -> "application/octet-stream"
    }
}

## MBTilesSource.kt
package com.example.sample.offline

import android.content.Context
import android.database.sqlite.SQLiteDatabase
import com.mapbox.mapboxsdk.style.sources.RasterSource
import com.mapbox.mapboxsdk.style.sources.Source
import com.mapbox.mapboxsdk.style.sources.TileSet
import com.mapbox.mapboxsdk.style.sources.VectorSource
import java.io.File
import java.io.FileOutputStream
import kotlin.properties.Delegates

/*
 *  Mapbox Source backend by localhost tile server
 */

sealed class MBTilesSourceException : Exception() {
    class CouldNotReadFileException : MBTilesSourceException()
    class UnsupportedFormatException : MBTilesSourceException()
}

class MBTilesSource(filePath: String, sourceId: String? = null) {

    val id = sourceId ?: filePath.substringAfterLast("/").substringBefore(".")
    val url get() = "http://localhost:${MBTilesServer.port}/$id/{z}/{x}/{y}.$format"
    private val db: SQLiteDatabase = try {
        SQLiteDatabase.openOrCreateDatabase(filePath, null)
    } catch (e: RuntimeException) {
        throw MBTilesSourceException.CouldNotReadFileException()
    }
    val instance: Source by lazy {
        if (isVector) VectorSource(id, TileSet(null, url))
        else RasterSource(id, TileSet(null, url))
    }

    var isVector by Delegates.notNull<Boolean>()
    lateinit var format: String
//    var tileSize: Int? = null
//    var layersJson: String? = ""
//    var attributions: String? = ""
//    var minZoom: Float? = null
//    var maxZoom: Float? = null
//    var bounds: LatLngBounds? = null

    init {
        try {
            format = db.query(
                "metadata", null, "name = ?",
                arrayOf("format"), null, null, null
            ).use { cursor ->
                cursor.moveToFirst()
                val index = cursor.getColumnIndex("value")
                cursor.getString(index)
            }

            isVector = when (format) {
                in validVectorFormats -> true
                in validRasterFormats -> false
                else -> throw MBTilesSourceException.UnsupportedFormatException()
            }

        } catch (error: MBTilesSourceException) {
            print(error.localizedMessage)
        }
    }

    fun getTile(z: Int, x: Int, y: Int): ByteArray? {
        return db.query(
            "tiles", null, "zoom_level = ? AND tile_column = ? AND tile_row = ?",
            arrayOf("$z", "$x", "$y"), null, null, null
        ).use { cursor ->
            if (cursor.count == 0) return null

            cursor.moveToFirst()
            val index = cursor.getColumnIndex("tile_data")
            cursor.getBlob(index)
        }
    }

    fun activate() = with(MBTilesServer) {
        sources[id] = this@MBTilesSource
        if (!isRunning) start()
    }

    fun deactivate() = with(MBTilesServer) {
        sources.remove(id)
        if (isRunning && sources.isEmpty()) stop()
    }

    companion object {
        val validRasterFormats = listOf("jpg", "png")
        val validVectorFormats = listOf("pbf", "mvt")

        fun readAsset(context: Context, asset: String): String =
            context.assets.open(asset).use { inputStream ->
                val path = context.getDatabasePath(asset).path
                val outputFile = File(path)
                FileOutputStream(outputFile).use { outputStream ->
                    inputStream.copyTo(outputStream)
                    outputStream.flush()
                }
                return path
            }
    }
}
	package com.example.sample.offline

	import android.util.Log
	import java.io.BufferedReader
	import java.io.ByteArrayOutputStream
	import java.io.FileNotFoundException
	import java.io.PrintStream
	import java.net.ServerSocket
	import java.net.Socket
	import kotlin.math.pow

	/*
	* Localhost tile server with MBTilesSource
	*/

	object MBTilesServer : Runnable {

	const val port = 8888
	private val serverSocket: ServerSocket = ServerSocket(port)
	var isRunning = false
	val sources: MutableMap<String, MBTilesSource> = mutableMapOf()

	fun start() {
	isRunning = true
	Thread(this).start()
	}

	fun stop() {
	isRunning = false
	serverSocket.close()
	}

	override fun run() {
	try {
	while (isRunning) {
	serverSocket.accept().use { socket ->
	Log.d(javaClass.simpleName, "Handling request")
	handle(socket)
	Log.d(javaClass.simpleName, "Request handled")
	}
	}
	} catch (e: Exception) {
	Log.d(
	javaClass.simpleName,
	e.localizedMessage ?: "Exception while running MBTilesServer"
	)
	} finally {
	Log.d(javaClass.simpleName, "request handled")
	}
	}

	@Throws
	private fun handle(socket: Socket) {

	val reader: BufferedReader = socket.getInputStream().reader().buffered()
	// Output stream that we send the response to
	val output = PrintStream(socket.getOutputStream())

	try {
	var route: String? = null

	// Read HTTP headers and parse out the route.
	do {
	val line = reader.readLine() ?: ""
	if (line.startsWith("GET")) {
	// the format for route should be {source}/{z}/{x}/{y}
	route = line.substringAfter("GET /").substringBefore(".")
	break
	}
	} while (line.isNotEmpty())

	// the source which this request target to
	val source = sources[route?.substringBefore("/")] ?: return

	// Prepare the content to send.
	if (null == route) {
	writeServerError(output)
	return
	}

	val bytes = loadContent(source, route) ?: run {
	writeServerError(output)
	return
	}

	// Send out the content.
	with(output) {
	println("HTTP/1.0 200 OK")
	println("Content-Type: " + detectMimeType(source.format))
	println("Content-Length: " + bytes.size)
	if (source.isVector) println("Content-Encoding: gzip")
	println()
	write(bytes)
	flush()
	}
	} finally {
	reader.close()
	output.close()
	}
	}

	@Throws
	private fun loadContent(source: MBTilesSource, route: String): ByteArray? = try {
	val (z, x, y) = route.split("/").subList(1, 4).map { it.toInt() }
	source.getTile(z, x, (2.0.pow(z)).toInt() - 1 - y)
	} catch (e: FileNotFoundException) {
	e.printStackTrace()
	null
	}

	private fun writeServerError(output: PrintStream) {
	output.println("HTTP/1.0 500 Internal Server Error")
	output.flush()
	}

	private fun detectMimeType(format: String): String? = when (format) {
	"jpg" -> "image/jpeg"
	"png" -> "image/png"
	"mvt" -> "application/x-protobuf"
	"pbf" -> "application/x-protobuf"
	else -> "application/octet-stream"
	}
	}
	package com.example.sample.offline

	import android.content.Context
	import android.database.sqlite.SQLiteDatabase
	import com.mapbox.mapboxsdk.style.sources.RasterSource
	import com.mapbox.mapboxsdk.style.sources.Source
	import com.mapbox.mapboxsdk.style.sources.TileSet
	import com.mapbox.mapboxsdk.style.sources.VectorSource
	import java.io.File
	import java.io.FileOutputStream
	import kotlin.properties.Delegates

	/*
	* Mapbox Source backend by localhost tile server
	*/

	sealed class MBTilesSourceException : Exception() {
	class CouldNotReadFileException : MBTilesSourceException()
	class UnsupportedFormatException : MBTilesSourceException()
	}

	class MBTilesSource(filePath: String, sourceId: String? = null) {

	val id = sourceId ?: filePath.substringAfterLast("/").substringBefore(".")
	val url get() = "http://localhost:${MBTilesServer.port}/$id/{z}/{x}/{y}.$format"
	private val db: SQLiteDatabase = try {
	SQLiteDatabase.openOrCreateDatabase(filePath, null)
	} catch (e: RuntimeException) {
	throw MBTilesSourceException.CouldNotReadFileException()
	}
	val instance: Source by lazy {
	if (isVector) VectorSource(id, TileSet(null, url))
	else RasterSource(id, TileSet(null, url))
	}

	var isVector by Delegates.notNull<Boolean>()
	lateinit var format: String
	// var tileSize: Int? = null
	// var layersJson: String? = ""
	// var attributions: String? = ""
	// var minZoom: Float? = null
	// var maxZoom: Float? = null
	// var bounds: LatLngBounds? = null

	init {
	try {
	format = db.query(
	"metadata", null, "name = ?",
	arrayOf("format"), null, null, null
	).use { cursor ->
	cursor.moveToFirst()
	val index = cursor.getColumnIndex("value")
	cursor.getString(index)
	}

	isVector = when (format) {
	in validVectorFormats -> true
	in validRasterFormats -> false
	else -> throw MBTilesSourceException.UnsupportedFormatException()
	}

	} catch (error: MBTilesSourceException) {
	print(error.localizedMessage)
	}
	}

	fun getTile(z: Int, x: Int, y: Int): ByteArray? {
	return db.query(
	"tiles", null, "zoom_level = ? AND tile_column = ? AND tile_row = ?",
	arrayOf("$z", "$x", "$y"), null, null, null
	).use { cursor ->
	if (cursor.count == 0) return null

	cursor.moveToFirst()
	val index = cursor.getColumnIndex("tile_data")
	cursor.getBlob(index)
	}
	}

	fun activate() = with(MBTilesServer) {
	sources[id] = this@MBTilesSource
	if (!isRunning) start()
	}

	fun deactivate() = with(MBTilesServer) {
	sources.remove(id)
	if (isRunning && sources.isEmpty()) stop()
	}

	companion object {
	val validRasterFormats = listOf("jpg", "png")
	val validVectorFormats = listOf("pbf", "mvt")

	fun readAsset(context: Context, asset: String): String =
	context.assets.open(asset).use { inputStream ->
	val path = context.getDatabasePath(asset).path
	val outputFile = File(path)
	FileOutputStream(outputFile).use { outputStream ->
	inputStream.copyTo(outputStream)
	outputStream.flush()
	}
	return path
	}
	}
	}