Skip to content

Instantly share code, notes, and snippets.

View Rotzke's full-sized avatar
🇩🇪

Nils Bergmüller Rotzke

🇩🇪
  • Berlin, Germany
  • 07:36 (UTC +02:00)
View GitHub Profile
@Rotzke
Rotzke / load_directory_json.gz.sh
Created February 21, 2022 21:00 — forked from rjurney/load_directory_json.gz.sh
How to bulk load gzip'd JSON in Elastic
# Bulk load the Foo data we prepared via PySpark in etl/transform_foo.spark.py
for path in data/foo/elastic/part*
do
file=$(basename ${path})
echo "Submitting ${path} to Elastic index foo ..."
curl ${USER_STRING} \
-X POST \
-H "Content-encoding: gzip" \
-H "Content-Type: application/x-ndjson" \