Skip to content

Instantly share code, notes, and snippets.

@dgaubert
Last active September 11, 2018 14:03
Show Gist options
  • Save dgaubert/3645cfad0656d00829cf4d854b85094e to your computer and use it in GitHub Desktop.
Save dgaubert/3645cfad0656d00829cf4d854b85094e to your computer and use it in GitHub Desktop.

Maps API: getting started

Basic concepts:

  • What's a map?: User's data (sql/analyses) + basemap + pop-ups + dataviews

Components:

  • Browser (carto[JS|VL]), cdn, nginx, varnish, maps-api, batch-api, redis-metadata, user's database

Map Instatiation Workflow: Map Config (request) & Layergroup (response):

Fetching tiles

  • cartodb/mapnik: png(32)/mvt/grid.json/
  • torque: png/torque.json
  • http: png
  • plain: png

Static views

{
"layergroupid": "d3f5254e023bc3805576822e5b0908bb:1536591621591",
"metadata": {
"layers": [ ... ],
"analyses": [
{
"nodes": {
"a1": {
"status": "ready",
"url": {
"http": "http://2.gusc.cartocdn.com/dgaubert/api/v1/map/d3f5254e023bc3805576822e5b0908bb:1536591621591/analysis/node/a0901a62fa55adf044985c5a610ec3b3820663ce",
"https": "https://cartocdn-gusc-c.global.ssl.fastly.net/dgaubert/api/v1/map/d3f5254e023bc3805576822e5b0908bb:1536591621591/analysis/node/a0901a62fa55adf044985c5a610ec3b3820663ce"
},
"query": "select * from analysis_7530d60ffc_a0901a62fa55adf044985c5a610ec3b3820663ce"
},
"a0": {
"status": "ready",
"url": {
"http": "http://2.gusc.cartocdn.com/dgaubert/api/v1/map/d3f5254e023bc3805576822e5b0908bb:1536591621591/analysis/node/2f4e2bf43fe25d97b2d8cdfdbd06ad1c26c3cdfe",
"https": "https://cartocdn-gusc-c.global.ssl.fastly.net/dgaubert/api/v1/map/d3f5254e023bc3805576822e5b0908bb:1536591621591/analysis/node/2f4e2bf43fe25d97b2d8cdfdbd06ad1c26c3cdfe"
},
"query": "SELECT * FROM dgaubert.ne_10m_populated_places_simple"
}
}
}
],
"tilejson": { ... },
"url": { ... }
},
"cdn_url": { ... },
"last_updated": "2018-09-10T15:00:21.591Z"
}
{
"layers": [
{
"type": "mapnik",
"options": {
"cartocss": "#layer {\n marker-width: 7;\n marker-fill: #EE4D5A;\n marker-fill-opacity: 0.9;\n marker-allow-overlap: true;\n marker-line-width: 1;\n marker-line-color: #FFFFFF;\n marker-line-opacity: 1;\n}",
"cartocss_version": "2.1.1",
"source": {
"id": "a1"
}
}
}
],
"analyses": [
{
"id": "a1",
"type": "sampling",
"params": {
"source": {
"id": "a0",
"type": "source",
"params": {
"query": "SELECT * FROM dgaubert.ne_10m_populated_places_simple"
}
},
"sampling": 0.5
}
}
]
}
{
"layergroupid": "271f0059a3c354b0b88557e2b24fe930:1536590939651",
"metadata": {
"layers": [
{
"type": "mapnik",
"id": "2f40bf64-317c-4c56-b914-099cf3c647cb",
"meta": {
"cartocss": "#layer {\n marker-width: 7;\n marker-fill: #EE4D5A;\n marker-fill-opacity: 0.9;\n marker-line-color: #FFFFFF;\n marker-line-width: 1;\n marker-line-opacity: 1;\n marker-placement: point;\n marker-type: ellipse;\n marker-allow-overlap: true;\n}",
"stats": {
"estimatedFeatureCount": 7322
},
"cartocss_meta": {
"rules": []
}
},
"tilejson": {
"vector": {
"tilejson": "2.2.0",
"tiles": [
"https://cartocdn-gusc-a.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.mvt",
"https://cartocdn-gusc-b.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.mvt",
"https://cartocdn-gusc-c.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.mvt",
"https://cartocdn-gusc-d.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.mvt"
]
},
"raster": {
"tilejson": "2.2.0",
"tiles": [
"https://cartocdn-gusc-a.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.png",
"https://cartocdn-gusc-b.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.png",
"https://cartocdn-gusc-c.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.png",
"https://cartocdn-gusc-d.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.png"
],
"grids": [
"https://cartocdn-gusc-a.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.grid.json",
"https://cartocdn-gusc-b.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.grid.json",
"https://cartocdn-gusc-c.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.grid.json",
"https://cartocdn-gusc-d.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/2f40bf64-317c-4c56-b914-099cf3c647cb/{z}/{x}/{y}.grid.json"
]
}
}
}
],
"tilejson": {
"vector": {
"tilejson": "2.2.0",
"tiles": [
"https://cartocdn-gusc-a.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.mvt",
"https://cartocdn-gusc-b.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.mvt",
"https://cartocdn-gusc-c.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.mvt",
"https://cartocdn-gusc-d.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.mvt"
]
},
"raster": {
"tilejson": "2.2.0",
"tiles": [
"https://cartocdn-gusc-a.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.png",
"https://cartocdn-gusc-b.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.png",
"https://cartocdn-gusc-c.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.png",
"https://cartocdn-gusc-d.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.png"
]
}
},
"url": {
"vector": {
"urlTemplate": "https://cartocdn-gusc-{s}.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.mvt",
"subdomains": [
"a",
"b",
"c",
"d"
]
},
"raster": {
"urlTemplate": "https://cartocdn-gusc-{s}.global.ssl.fastly.net/dgaubert/api/v1/map/271f0059a3c354b0b88557e2b24fe930:1536590939651/{z}/{x}/{y}.png",
"subdomains": [
"a",
"b",
"c",
"d"
]
}
}
},
"cdn_url": {
"templates": {
"http": {
"subdomains": [
"0",
"1",
"2",
"3"
],
"url": "http://{s}.gusc.cartocdn.com"
},
"https": {
"subdomains": [
"a",
"b",
"c",
"d"
],
"url": "https://cartocdn-gusc-{s}.global.ssl.fastly.net"
}
},
"http": "gusc.cartocdn.com",
"https": "cartocdn-gusc.global.ssl.fastly.net"
},
"last_updated": "2018-09-10T14:48:59.651Z"
}
{
"layers": [
{
"type": "mapnik",
"options": {
"cartocss": "#layer {\n marker-width: 23.5;\n marker-fill: #E4F1E1;\n marker-fill-opacity: 1;\n marker-allow-overlap: true;\n marker-line-width: 2;\n marker-line-color: #FFFFFF;\n marker-line-opacity: 1;\n}",
"cartocss_version": "2.1.1",
"sql": "SELECT * FROM dgaubert.ne_10m_populated_places_simple"
}
}
]
}
{
"layergroupid": "45ca43924c4b5d70bcc023bc30c92c60:1536590939651",
"metadata": {
"layers": [ ... ],
"dataviews": {
"106eddc8-1418-4f2f-b8a8-08de45b53866": {
"url": {
"http": "http://0.gusc.cartocdn.com/dgaubert/api/v1/map/45ca43924c4b5d70bcc023bc30c92c60:1536590939651/dataview/106eddc8-1418-4f2f-b8a8-08de45b53866",
"https": "https://cartocdn-gusc-a.global.ssl.fastly.net/dgaubert/api/v1/map/45ca43924c4b5d70bcc023bc30c92c60:1536590939651/dataview/106eddc8-1418-4f2f-b8a8-08de45b53866"
}
}
},
"tilejson": { ... },
"url": { ... }
},
"cdn_url": { ... },
"last_updated": "2018-09-10T14:48:59.651Z"
}
{
"layers": [
{
"type": "mapnik",
"options": {
"cartocss": "#layer {\n marker-width: 7;\n marker-fill: #EE4D5A;\n marker-fill-opacity: 0.9;\n marker-line-color: #FFFFFF;\n marker-line-width: 1;\n marker-line-opacity: 1;\n marker-placement: point;\n marker-type: ellipse;\n marker-allow-overlap: true;\n}",
"cartocss_version": "2.1.1",
"source": {
"id": "a0"
}
}
}
],
"dataviews": {
"6484261a-7210-4b3f-bfb9-13ec290c58ce": {
"type": "histogram",
"source": {
"id": "a0"
},
"options": {
"column": "latitude",
"bins": 10
}
}
},
"analyses": [
{
"id": "a0",
"type": "source",
"params": {
"query": "SELECT * FROM dgaubert.ne_10m_populated_places_simple"
}
}
]
}
{
"layergroupid": "d48005ccd77434e48530f3d0102e5fd8:1536590939651",
"metadata": {
"layers": [
{
"type": "mapnik",
"id": "layer0",
"meta": {
"cartocss": "\n /* cache buster 1536601885048 */\n #layer {\n marker-width: 7;\n marker-fill: #EE4D5A;\n marker-fill-opacity: 1;\n marker-allow-overlap: true;\n marker-line-width: 1;\n marker-line-color: #FFFFFF;\n marker-line-opacity: 1;\n }",
"stats": {
"estimatedFeatureCount": 7322
},
"cartocss_meta": {
"rules": []
},
"aggregation": {
"png": true,
"mvt": true
}
},
"tilejson": { ... }
}
],
"dataviews": {},
"analyses": [],
"tilejson": { ... }
},
"url": { ... }
},
"cdn_url": { ... },
"last_updated": "2018-09-10T14:48:59.651Z"
}
{
"layers": [
{
"type": "mapnik",
"options": {
"sql": "SELECT * FROM dgaubert.ne_10m_populated_places_simple",
"aggregation": {
"placement": "point-sample",
"columns": {
"pop_max_sum": {
"aggregate_function": "sum",
"aggregated_column": "pop_max"
}
}
}
}
}
]
}
{
"layergroupid": "45bbe2e586afdd6a87df3ddb0f35a161:1536590939651",
"metadata": {
"layers": [
{
"type": "mapnik",
"id": "c1522705-9b7c-46a3-97a6-e6f0fb48a741",
"meta": {
"cartocss": "#layer {\n polygon-fill: #245668;\n [ agg_value > 2 ] {\n polygon-fill: #04817E;\n }\n [ agg_value > 4 ] {\n polygon-fill: #39AB7E;\n }\n [ agg_value > 15 ] {\n polygon-fill: #8BD16D;\n }\n [ agg_value > 42 ] {\n polygon-fill: #EDEF5D;\n }\n}\n#layer::outline {\n line-width: 1;\n line-color: #FFFFFF;\n line-opacity: 1;\n}",
"stats": {
"estimatedFeatureCount": 7322
},
"cartocss_meta": {
"rules": [
{
"selector": "#layer",
"prop": "polygon-fill",
"column": "agg_value",
"mapping": ">",
"buckets": [
{
"filter": {
"type": "range",
"start": 1,
"end": 2
},
"value": "#245668"
},
{
"filter": {
"type": "range",
"start": 2,
"end": 4
},
"value": "#04817E"
},
{
"filter": {
"type": "range",
"start": 4,
"end": 15
},
"value": "#39AB7E"
},
{
"filter": {
"type": "range",
"start": 15,
"end": 42
},
"value": "#8BD16D"
},
{
"filter": {
"type": "range",
"start": 42,
"end": 141
},
"value": "#EDEF5D"
}
],
"stats": {
"filter_avg": 21.85373134328358
}
}
]
}
},
"tilejson": { ... }
}
],
"tilejson": { ... },
"url": { ... }
},
"cdn_url": { ... },
"last_updated": "2018-09-10T14:48:59.651Z"
}
{
"layers": [
{
"type": "mapnik",
"options": {
"cartocss": "#layer {\n polygon-fill: ramp([agg_value], (#245668, #04817E, #39AB7E, #8BD16D, #EDEF5D), quantiles);\n}\n#layer::outline {\n line-width: 1;\n line-color: #FFFFFF;\n line-opacity: 1;\n}",
"cartocss_version": "2.1.0",
"sql": "SELECT * FROM dgaubert.ne_10m_populated_places_simple",
"sql_wrap": "WITH hgrid AS (SELECT CDB_HexagonGrid(ST_Expand(!bbox!, CDB_XYZ_Resolution(2) * 10), CDB_XYZ_Resolution(2) * 10) as cell) SELECT hgrid.cell as the_geom_webmercator, count(1) as agg_value, count(1)/power( 10 * CDB_XYZ_Resolution(2), 2 ) as agg_value_density, row_number() over () as cartodb_id FROM hgrid, (<%= sql %>) i where ST_Intersects(i.the_geom_webmercator, hgrid.cell) GROUP BY hgrid.cell"
}
}
]
}
{
"layers": [
{
"type": "mapnik",
"options": {
"cartocss": "#layer {\n marker-width: 23.5;\n marker-fill: #E4F1E1;\n marker-fill-opacity: 1;\n marker-allow-overlap: true;\n marker-line-width: 2;\n marker-line-color: #FFFFFF;\n marker-line-opacity: 1;\n}",
"cartocss_version": "2.1.1",
"sql": "SELECT * FROM dgaubert.ne_10m_populated_places_simple",
"interactivity": [
"cartodb_id",
"name",
"pop_max"
],
"attributes": {
"id": "cartodb_id",
"columns": [
"name",
"pop_max"
]
}
}
}
]
}
{
"layergroupid": "1f21870ffb786eedb9f1dfae439ab913:1536590939651",
"metadata": {
"layers": [
{
"type": "http",
"id": "251a08bd-6b8f-4d46-a89f-b5feeb7b3e4b",
"meta": {
"stats": {}
},
"tilejson": {
"raster": {
"tilejson": "2.2.0",
"tiles": [
"https://cartocdn-gusc-a.global.ssl.fastly.net/dgaubert/api/v1/map/1f21870ffb786eedb9f1dfae439ab913:1536590939651/251a08bd-6b8f-4d46-a89f-b5feeb7b3e4b/{z}/{x}/{y}.png",
"https://cartocdn-gusc-b.global.ssl.fastly.net/dgaubert/api/v1/map/1f21870ffb786eedb9f1dfae439ab913:1536590939651/251a08bd-6b8f-4d46-a89f-b5feeb7b3e4b/{z}/{x}/{y}.png",
"https://cartocdn-gusc-c.global.ssl.fastly.net/dgaubert/api/v1/map/1f21870ffb786eedb9f1dfae439ab913:1536590939651/251a08bd-6b8f-4d46-a89f-b5feeb7b3e4b/{z}/{x}/{y}.png",
"https://cartocdn-gusc-d.global.ssl.fastly.net/dgaubert/api/v1/map/1f21870ffb786eedb9f1dfae439ab913:1536590939651/251a08bd-6b8f-4d46-a89f-b5feeb7b3e4b/{z}/{x}/{y}.png"
]
}
}
},
{
"type": "mapnik",
"id": "c1522705-9b7c-46a3-97a6-e6f0fb48a741",
"meta": { ... }
},
"tilejson": { ... }
},
{
"type": "http",
"id": "ae51be24-29d9-4451-b7f2-af768a6c75ed",
"meta": {
"stats": {}
},
"tilejson": { ... }
}
],
"dataviews": {},
"analyses": [],
"tilejson": { ... },
"url": {
"vector": { ... },
"raster": { ... }
}
},
"cdn_url": { ... },
"last_updated": "2018-09-10T14:48:59.651Z"
}
{
"layers": [
{
"type": "http",
"options": {
"urlTemplate": "https://{s}.basemaps.cartocdn.com/rastertiles/voyager_nolabels/{z}/{x}/{y}.png",
"subdomains": "abcd"
}
},
{
"type": "mapnik",
"options": {
"cartocss": "#layer {\n marker-width: 23.5;\n marker-fill: #E4F1E1;\n marker-fill-opacity: 1;\n marker-allow-overlap: true;\n marker-line-width: 2;\n marker-line-color: #FFFFFF;\n marker-line-opacity: 1;\n}",
"cartocss_version": "2.1.1",
"sql": "SELECT * FROM dgaubert.ne_10m_populated_places_simple",
}
},
{
"type": "http",
"options": {
"urlTemplate": "https://{s}.basemaps.cartocdn.com/rastertiles/voyager_only_labels/{z}/{x}/{y}.png",
"subdomains": "abcd"
}
}
]
}
{
"layergroupid": "d480ba89581d7f658b2c11d2b67d5b5e:1536590939651",
"metadata": {
"layers": [
{
"type": "torque",
"id": "2f40bf64-317c-4c56-b914-099cf3c647cb",
"meta": {
"start": 1000,
"end": 7322000,
"steps": 256,
"data_steps": 7322,
"column_type": "number",
"cartocss": "Map {\n -torque-frame-count: 256;\n -torque-animation-duration: 30;\n -torque-time-attribute: \"cartodb_id\";\n -torque-aggregation-function: \"count(1)\";\n -torque-resolution: 4;\n -torque-data-aggregation: linear;\n}\n#layer {\n marker-width: 7;\n marker-fill: #EE4D5A;\n marker-fill-opacity: 0.9;\n marker-line-width: 1;\n marker-line-color: #FFFFFF;\n marker-line-opacity: 1;\n comp-op: lighter;\n}\n#layer[frame-offset=1] {\n marker-width: 9;\n marker-fill-opacity: 0.45;\n}\n#layer[frame-offset=2] {\n marker-width: 11;\n marker-fill-opacity: 0.225;\n}",
"stats": {},
"cartocss_meta": {
"rules": []
}
},
"tilejson": {
"raster": { ... }
}
}
],
"torque": {
"1": {
"start": 1000,
"end": 7322000,
"steps": 256,
"data_steps": 7322,
"column_type": "number",
"cartocss": "Map {\n -torque-frame-count: 256;\n -torque-animation-duration: 30;\n -torque-time-attribute: \"cartodb_id\";\n -torque-aggregation-function: \"count(1)\";\n -torque-resolution: 4;\n -torque-data-aggregation: linear;\n}\n#layer {\n marker-width: 7;\n marker-fill: #EE4D5A;\n marker-fill-opacity: 0.9;\n marker-line-width: 1;\n marker-line-color: #FFFFFF;\n marker-line-opacity: 1;\n comp-op: lighter;\n}\n#layer[frame-offset=1] {\n marker-width: 9;\n marker-fill-opacity: 0.45;\n}\n#layer[frame-offset=2] {\n marker-width: 11;\n marker-fill-opacity: 0.225;\n}",
"stats": {},
"cartocss_meta": {
"rules": []
}
}
}
},
"cdn_url": { ... },
"last_updated": "2018-09-10T14:48:59.651Z"
}
{
"layers": [
{
"type": "torque",
"options": {
"cartocss": "Map {\n -torque-frame-count: 256;\n -torque-animation-duration: 30;\n -torque-time-attribute: \"cartodb_id\";\n -torque-aggregation-function: \"count(1)\";\n -torque-resolution: 4;\n -torque-data-aggregation: linear;\n}\n#layer {\n marker-width: 7;\n marker-fill: #EE4D5A;\n marker-fill-opacity: 0.9;\n marker-line-width: 1;\n marker-line-color: #FFFFFF;\n marker-line-opacity: 1;\n comp-op: lighter;\n}\n#layer[frame-offset=1] {\n marker-width: 9;\n marker-fill-opacity: 0.45;\n}\n#layer[frame-offset=2] {\n marker-width: 11;\n marker-fill-opacity: 0.225;\n}",
"cartocss_version": "2.1.0",
"sql": "SELECT * FROM dgaubert.ne_10m_populated_places_simple"
}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment