Skip to content

Instantly share code, notes, and snippets.

@findepi
Last active June 10, 2024 15:19
Show Gist options
  • Save findepi/da2ea85d095eaa0f3cfdcbb311ff59ed to your computer and use it in GitHub Desktop.
Save findepi/da2ea85d095eaa0f3cfdcbb311ff59ed to your computer and use it in GitHub Desktop.

quick build

./gradlew -x test -x integrationTest build

full build, like mvn install

By default only modules for one Spark/Flink/Scala versions are built. To get the full build, component versions need to be enumerated.

./gradlew \
    -DallVersions=true \
    `# TODO drop scala versions after https://github.com/apache/iceberg/pull/10478` \
    -DscalaVersions=$(sed -n 's/^systemProp.knownScalaVersions=//p' gradle.properties) \
    -x test -x integrationTest build publishToMavenLocal 

reformat code

./gradlew \
    -DallVersions=true \
    `# TODO drop scala versions after https://github.com/apache/iceberg/pull/10478` \
    -DscalaVersions=$(sed -n 's/^systemProp.knownScalaVersions=//p' gradle.properties) \
    spotlessApply

run a single Spark test class

./gradlew :iceberg-spark:iceberg-spark-extensions:test --tests TestRemoveOrphanFilesProcedure

# or with non-default Spark version
./gradlew -DsparkVersions=3.2 :iceberg-spark:iceberg-spark-extensions:test --tests TestRemoveOrphanFilesProcedure

run individual Spark test methods

# Note that * at the end of the method name is required
./gradlew -DsparkVersions=3.3 :iceberg-spark:iceberg-spark-extensions:test \
    --tests TestRemoveOrphanFilesProcedure.testRemoveOrphanFilesWithStatisticFiles\* \
    --tests TestRemoveOrphanFilesAction3.testRemoveOrphanFilesWithStatisticFiles\*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment