Skip to content

Instantly share code, notes, and snippets.

@kmatt
Last active September 18, 2023 16:23
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kmatt/56f2e188dc675402c30ddf8666177204 to your computer and use it in GitHub Desktop.
Save kmatt/56f2e188dc675402c30ddf8666177204 to your computer and use it in GitHub Desktop.
Spark Delta Jars
# Thrift is not finding Delta jars in Ivy2 cache, even when specified in spark-defaults.conf (spark.sql.catalog.spark_catalog) ?
# Probably a bad solution...
wget https://repo1.maven.org/maven2/io/delta/delta-core_2.12/2.4.0/delta-core_2.12-2.4.0.jar && \
mv delta-core_2.12-2.4.0.jar jars/
wget https://repo1.maven.org/maven2/io/delta/delta-storage/2.4.0/delta-storage-2.4.0.jar && \
mv delta-storage-2.4.0.jar jars/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment