Skip to content

Instantly share code, notes, and snippets.

@goergen95
Last active June 29, 2023 09:14
Show Gist options
  • Save goergen95/6b13c39af41d831cce315e06b6fde54e to your computer and use it in GitHub Desktop.
Save goergen95/6b13c39af41d831cce315e06b6fde54e to your computer and use it in GitHub Desktop.
Using GDAL to read/write geospatial data from minio S3
services:
minio:
image: quay.io/minio/minio
container_name: minio
environment:
MINIO_ROOT_USER: miniouser
MINIO_ROOT_PASSWORD: miniopass
ports:
- 9000:9000
- 9090:9090
volumes:
- ./minio/data:/data
command: server --console-address ":9090" /data
rstudio:
image: rocker/geospatial
container_name: r-studio
environment:
- USER=rstudio
- PASSWORD=supersecret
- ROOT=TRUE
volumes:
- .:/home/rstudio/r-s3
ports:
- 8787:8787
links:
- minio
Sys.setenv(AWS_ACCESS_KEY_ID = "miniouser",
AWS_SECRET_ACCESS_KEY = "miniopass",
AWS_DEFAULT_REGION = "",
AWS_SESSION_TOKEN = "",
AWS_S3_ENDPOINT = "minio:9000",
AWS_VIRTUAL_HOSTING = "FALSE", # for GDAL to resolve endpoint correctly
AWS_HTTPS = "FALSE", # for GDAL to not use https
CPL_VSIL_USE_TEMP_FILE_FOR_RANDOM_WRITE = "YES") # for GDAL to allow writing GTiff
if(!requireNamespace("aws.s3")) install.packages("aws.s3")
library(aws.s3)
install.packages("terra")
library(terra)
library(stars)
library(sf)
# create a bucket
put_bucket(bucket = "my-bucket", use_https = FALSE, region = "")
# showcase behavior for raster
file <- system.file("ex/elev.tif", package="terra")
raster_terra <- rast(file)
raster_stars <- read_stars(file)
write_stars(raster_stars, dsn = "/vsis3/my-bucket/raster_stars.tif")
(raster_stars <- read_stars("/vsis3/my-bucket/raster_stars.tif"))
writeRaster(raster_terra, filename = "/vsis3/my-bucket/raster_terra.tif")
(raster_terra <- rast("/vsis3/my-bucket/raster_terra.tif"))
# showcase behavior for vector
file <- system.file("gpkg/nc.gpkg", package="sf")
vector_sf <- read_sf(file)
vector_terra <- vect(file)
st_write(vector_sf, dsn = "/vsis3/my-bucket/vector_sf.gpkg", driver = "GPKG")
writeVector(vector_terra, filename = "/vsis3/my-bucket/vector_terra.gpkg")
(vector_sf <- read_sf("/vsis3/my-bucket/vector_sf.gpkg"))
(vector_terra - vect("/vsis3/my-bucket/vector_terra.gpkg"))
get_bucket("my-bucket", use_https = FALSE, region = "")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment