Skip to content

Instantly share code, notes, and snippets.

@marklit
Last active July 16, 2024 08:47
Show Gist options
  • Save marklit/39b3ea4f1c3177f5f48ef2a995bafc3e to your computer and use it in GitHub Desktop.
Save marklit/39b3ea4f1c3177f5f48ef2a995bafc3e to your computer and use it in GitHub Desktop.
COPY (
SELECT * EXCLUDE (geometry,
bbox,
names,
sources),
ST_GEOMFROMWKB(geometry) geom,
JSON(bbox) AS bbox,
JSON(names) AS names,
JSON(sources) AS sources
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-06-13-beta.0/theme=buildings/type=building/*00208*',
filename=true,
hive_partitioning=1)
WHERE bbox.xmin BETWEEN 116.28749 AND 116.52790 AND
bbox.ymin BETWEEN 39.83297 AND 40.01533
)
TO 'beijing.overture.gpkg'
WITH (FORMAT GDAL,
DRIVER 'GPKG',
LAYER_CREATION_OPTIONS 'WRITE_BBOX=YES');
@marklit
Copy link
Author

marklit commented Jul 11, 2024

beijing overture 2024-06-zoomed-in

@marklit
Copy link
Author

marklit commented Jul 11, 2024

beijing overture 2024-06

@marklit
Copy link
Author

marklit commented Jul 13, 2024

COPY (
    SELECT * EXCLUDE (geometry,
                      bbox,
                      names,
                      sources),
           ST_GEOMFROMWKB(geometry) geom,
           JSON(bbox)    AS bbox,
           JSON(names)   AS names,
           JSON(sources) AS sources
    FROM read_parquet('s3://overturemaps-us-west-2/release/2024-06-13-beta.0/theme=buildings/type=building/*00129*',
                      filename=true,
                      hive_partitioning=1)
    WHERE ST_CONTAINS('POLYGON ((54.519999999999996 24.28, 54.51423558412097 24.221472903395163, 54.497163859753385 24.165194970290475, 54.469440883690766 24.11332893009412, 54.432132034355966 24.067867965644037, 54.38667106990588 24.030559116309238, 54.33480502970953 24.002836140246615, 54.27852709660484 23.985764415879032, 54.22 23.98, 54.16147290339516 23.985764415879032, 54.10519497029047 24.002836140246615, 54.05332893009412 24.030559116309238, 54.00786796564403 24.067867965644037, 53.97055911630923 24.11332893009412, 53.94283614024661 24.165194970290475, 53.92576441587903 24.221472903395163, 53.92 24.28, 53.92576441587903 24.33852709660484, 53.94283614024661 24.394805029709527, 53.97055911630923 24.446671069905882, 54.00786796564403 24.492132034355965, 54.05332893009412 24.529440883690764, 54.10519497029047 24.557163859753388, 54.16147290339516 24.57423558412097, 54.22 24.580000000000002, 54.27852709660484 24.57423558412097, 54.33480502970953 24.557163859753388, 54.38667106990588 24.529440883690764, 54.432132034355966 24.492132034355965, 54.469440883690766 24.446671069905882, 54.497163859753385 24.394805029709527, 54.51423558412097 24.33852709660484, 54.519999999999996 24.28))'::GEOMETRY,
                      ST_POINT(bbox.xmin, bbox.ymin))
)
TO 'AE.gpkg'
    WITH (FORMAT GDAL,
          DRIVER 'GPKG',
          LAYER_CREATION_OPTIONS 'WRITE_BBOX=YES');

qgis-bin_2FrjQlmVEW

@marklit
Copy link
Author

marklit commented Jul 16, 2024

for RELEASE in 2024-06-13-beta.1 \
               2024-06-13-beta.0 \
               2024-05-16-beta.0 \
               2024-04-16-beta.0; do
    echo $RELEASE
    WORKING_DB="working.`uuidgen`.duckdb"
    echo "COPY (
              SELECT * EXCLUDE (geometry,
                                bbox,
                                names,
                                sources),
                     ST_GEOMFROMWKB(geometry) geom,
                     JSON(bbox)    AS bbox,
                     JSON(names)   AS names,
                     JSON(sources) AS sources
              FROM read_parquet('s3://overturemaps-us-west-2/release/$RELEASE/theme=buildings/type=building/*',
                                filename=true,
                                hive_partitioning=1)
              WHERE bbox.xmin BETWEEN 54.31391 AND 54.71493
              AND   bbox.ymin BETWEEN 24.33754 AND 24.53911
          )
          TO 'abu.dhabi.$RELEASE.gpkg'
              WITH (FORMAT GDAL,
                    DRIVER 'GPKG',
                    LAYER_CREATION_OPTIONS 'WRITE_BBOX=YES');" \
            | ~/duckdb -unsigned $WORKING_DB
    rm $WORKING_DB $WORKING_DB.wal || true
done

qgis-bin_OkN96SmTAj

@marklit
Copy link
Author

marklit commented Jul 16, 2024

The footprints largely haven't changed since at least April. It looks like orthofication has taken place. The tall buildings were photographed at an angle so it's only with linestring outlines that it is clear the footprints are probably very accurate. This data looks fine and doesn't seem to suffer the offset issue the Beijing data is having.

explorer_7gfNUQUkhC

@marklit
Copy link
Author

marklit commented Jul 16, 2024

Microsoft's ML buildings dataset data for Abu Dhabi leaves a lot to be desired.

qgis-bin_HUDHlcKk9k

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment