nadine farah nfarah86
nfarah86
/ cliStep1.sh
Created
February 28, 2023 05:47
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| hudi-> connect -–path TABLE_PATH |
nfarah86
/ localStep5.sh
Created
February 28, 2023 05:46
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| export SPARK_HOME=<spark-home-folder> | |
| export CLI_BUNDLE_JAR=<cli-bundle-jar-to-use> | |
| export SPARK_BUNDLE_JAR=<spark-bundle-jar-to-use> | |
| ./hudi-cli-with-bundle.sh |
nfarah86
/ localStep4.sh
Created
February 28, 2023 05:45
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cp $HUDI_REPO_ROOT/packaging/hudi-cli-bundle/hudi-cli-with-bundle.sh cli-staging/ | |
| mkdir cli-staging/conf | |
| cp $HUDI_REPO_ROOT/packaging/hudi-cli-bundle/conf/* cli-staging/conf/ |
nfarah86
/ localStep3.sh
Created
February 28, 2023 05:42
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cp $HUDI_REPO_ROOT/packaging/hudi-spark-bundle/target/hudi-spark3.2-bundle_2.12-0.13.0.jar cli-staging/ | |
| cp $HUDI_REPO_ROOT/packaging/hudi-cli-bundle/target/hudi-cli-bundle_2.12-0.13.0.jar cli-staging/ | |
| cp $HUDI_REPO_ROOT/packaging/hudi-cli-bundle/hudi-cli-with-bundle.sh cli-staging/ |
nfarah86
/ localStep2.sh
Created
February 28, 2023 05:41
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mkdir cli-staging/ |
nfarah86
/ localStep1.sh
Created
February 28, 2023 05:40
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mvn -T 2C clean package -DskipTests -Dspark[YourVersion] -Dscala-[YourVersion] |
nfarah86
/ mvnStep4.sh
Created
February 28, 2023 04:59
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cd ../ | |
| export SPARK_HOME=<spark-home-folder> | |
| export CLI_BUNDLE_JAR=<cli-bundle-jar-to-use> | |
| export SPARK_BUNDLE_JAR=<spark-bundle-jar-to-use> | |
| ./hudi-cli-with-bundle.sh |
nfarah86
/ mvnStep3.sh
Created
February 28, 2023 04:58
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mkdir conf/ | |
| cd conf/ | |
| $ curl https://raw.githubusercontent.com/apache/hudi/release-0.13.0/packaging/hudi-cli-bundle/conf/hudi-env.sh -o hudi-env.sh | |
| $ curl https://raw.githubusercontent.com/apache/hudi/release-0.13.0/packaging/hudi-cli-bundle/conf/log4j2.properties -o log4j2.properties |
nfarah86
/ mvnStep2.sh
Created
February 28, 2023 04:56
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ curl https://repo1.maven.org/maven2/org/apache/hudi/hudi-cli-bundle_2.12/0.13.0/hudi-cli-bundle_2.12-0.13.0.jar -o hudi-cli-bundle_2.12-0.13.0.jar | |
| $ curl https://repo1.maven.org/maven2/org/apache/hudi/hudi-spark3.2-bundle_2.12/0.13.0/hudi-spark3.2-bundle_2.12-0.13.0.jar -o hudi-spark3.2-bundle_2.12-0.13.0.jar | |
| $ curl https://raw.githubusercontent.com/apache/hudi/release-0.13.0/packaging/hudi-cli-bundle/hudi-cli-with-bundle.sh -o hudi-cli-with-bundle.sh |
nfarah86
/ mvnStep1.sh
Last active
February 28, 2023 04:57
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| mkdir cli-staging/ | |
| cd cli-staging/ |