View both_geoms.vrt
<?xml version="1.0"?>
<OGRVRTDataSource>
<OGRVRTWarpedLayer>
<OGRVRTLayer name="cartogeom">
<SrcDataSource>cartogeom.csv</SrcDataSource>
<GeometryField name="the_geom" encoding="PointFromColumns" x="longitude" y="latitude">
<GeometryType>wkbPoint</GeometryType>
<SRS>EPSG:4326</SRS>
</GeometryField>
<GeometryField name="the_geom_webmercator" encoding="PointFromColumns" x="longitude" y="latitude">
View copy-test.py
import sys
import os
import gzip
import random
import md5
from carto.auth import APIKeyAuthClient
from carto.sql import SQLClient
import requests
import hashlib
View index.md

So, it looks like COPY support will have to be divided into two parts, /copyfrom and /copyto, which is fine. They are both interesting, in that since the idea is to support scaling, the implementations absolutely must stream, rather than taking files or holding data in memory.

The parts needed are

The parts do in fact all fit together and work, as can been seen in this small proof-of-concept:

View rect_node_distance.sql
-- Large primary geometry
select st_distance(e.geom, v.geom)
from ed_2017 e, va_ply_17 v
where e.ed_abbrev = 'KLA' and v.ed_abbrev != 'KLA';
-- 25s
select _ST_DistanceRectTree(e.geom, v.geom)
from ed_2017 e, va_ply_17 v
where e.ed_abbrev = 'KLA' and v.ed_abbrev != 'KLA';
View postgis-gis.md

PostGIS && GIS

cdb-manager

A simple browser-based terminal for running SQL against Carto using the SQL API

  • http://github.com/cartodb/cdb-manager
  • git clone git@github.com:CartoDB/cdb-manager.git
  • When you're done cloning, enter the directory and run ./httpserv.py
  • Point your browser at http://locahost:8000
View small-it-bibliography.md
  • Google Presentation Link [goo.gl]
  • PDF Presentation Link [s3.cleverelephant.ca]
  • "Ottawa turns to U.S. tech giants too often: internal memo" [cbc.ca]
  • "Government as a Platform: the next phase of digital transformation" [gds.blog.gov.uk]
  • DevOps Real Talk [www.theregister.co.uk] "incremental change, tight feedback loops, shared knowledge, and mutual respect" "If you're a developer releasing large changesets, you're part of the problem."
  • The Government IT Self-Harm Playbook [medium.com]
  • Better For Less (UK IT) [[drive.google.com](https:/
View performance.md

Carto Patched PostGIS/PostgreSQL Performance

The REL_10_CARTO PostgreSQL branch and svn-2.4-cartodb branch of PostGIS carry patches to improve performance around our core use cases (generating resolution-appropriate data for rendering in Mapnik and creating vector tiles). They also include patches that make the PostgreSQL planner more likely to pick parallel plans, in particular when using PostGIS functions.

  • The first round of improvements went into PostGIS and were focussed on the functions used commonly in feeding Mapnik and MVT.
  • The second round went into both PostGIS and PostgreSQL and were focussed on improving parallel query for our use cases.
2.4 Aug 2.4cdb Oct 2.4cdb Dec
# features ms μs ms μs
View index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Geonames Heatmap</title>
<!-- Include Leaflet 1.2.0 Library -->
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.2.0/dist/leaflet.css" />
<script src="https://unpkg.com/leaflet@1.2.0/dist/leaflet.js"></script>
View parallel_paths_include_tlist_cost_v5.patch
diff --git a/src/backend/optimizer/geqo/geqo_eval.c b/src/backend/optimizer/geqo/geqo_eval.c
index b5cab0c..faa5bb7 100644
--- a/src/backend/optimizer/geqo/geqo_eval.c
+++ b/src/backend/optimizer/geqo/geqo_eval.c
@@ -40,7 +40,7 @@ typedef struct
} Clump;
static List *merge_clump(PlannerInfo *root, List *clumps, Clump *new_clump,
- bool force);
+ int num_gene, bool force);
View random-test-data.sql
--
-- Polygons in contained in a 10000x10000
-- square, with just enough size/density to mostly
-- cover the whole area.
--
DROP TABLE IF EXISTS polygon_table_10000;
CREATE TABLE polygon_table_10000 AS
SELECT ST_Buffer(
ST_SetSRID(
ST_MakePoint(random() * 10000, random() * 10000),