Skip to content

Instantly share code, notes, and snippets.

@mikeananev
Created November 20, 2022 07:09
Show Gist options
  • Save mikeananev/86c63ef167d197ad2383ecd6563a6ed5 to your computer and use it in GitHub Desktop.
Save mikeananev/86c63ef167d197ad2383ecd6563a6ed5 to your computer and use it in GitHub Desktop.
Run Clojure Geni using Java 19
(defproject geni-cookbook "0.0.1-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
:url "https://www.eclipse.org/legal/epl-2.0/"}
:dependencies [[org.clojure/clojure "1.11.1"]
[zero.one/geni "0.0.40"]
[com.fasterxml.jackson.core/jackson-annotations "2.12.3"
:exclusions [com.fasterxml.jackson.core/jackson-databind]]
[metosin/jsonista "0.3.3"
:exclusions [com.fasterxml.jackson.core/jackson-databind]]
[expound "0.8.9"]
;; Spark
[org.apache.spark/spark-core_2.12 "3.2.0"]
[org.apache.spark/spark-hive_2.12 "3.2.0"]
[org.apache.spark/spark-mllib_2.12 "3.2.0"]
[org.apache.spark/spark-sql_2.12 "3.2.0"]
[org.apache.spark/spark-streaming_2.12 "3.2.0"]
[org.apache.spark/spark-yarn_2.12 "3.2.0"]
[com.github.fommil.netlib/all "1.1.2" :extension "pom"]
; Arrow
[org.apache.arrow/arrow-memory-netty "4.0.0"]
[org.apache.arrow/arrow-memory-core "4.0.0"]
[org.apache.arrow/arrow-vector "4.0.0"
:exclusions [commons-codec com.fasterxml.jackson.core/jackson-databind]]
;; Databases
[mysql/mysql-connector-java "8.0.25"]
[org.postgresql/postgresql "42.2.20"]
[org.xerial/sqlite-jdbc "3.34.0"]]
:jvm-opts ["--add-opens=java.base/java.lang=ALL-UNNAMED"
"--add-opens=java.base/java.lang.invoke=ALL-UNNAMED"
"--add-opens=java.base/java.lang.reflect=ALL-UNNAMED"
"--add-opens=java.base/java.io=ALL-UNNAMED"
"--add-opens=java.base/java.net=ALL-UNNAMED"
"--add-opens=java.base/java.nio=ALL-UNNAMED"
"--add-opens=java.base/java.util=ALL-UNNAMED"
"--add-opens=java.base/sun.nio.ch=ALL-UNNAMED"
"--add-opens=java.base/sun.nio.cs=ALL-UNNAMED"
]
:profiles {:uberjar {:aot :all}
:dev {:plugins [[lein-ancient "0.7.0"]]}}
:main ^:skip-aot geni-cookbook.core
:target-path "target/%s")
@mikeananev
Copy link
Author

To run spark 3.3 this may be useful

java
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.lang.invoke=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.net=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
--add-opens=java.base/sun.nio.cs=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED
--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED
-jar <JAR_FILE_PATH>

Taken from here https://stackoverflow.com/questions/73465937/apache-spark-3-3-0-breaks-on-java-17-with-cannot-access-class-sun-nio-ch-direct

@mikeananev
Copy link
Author

To run uberjar from Geni
java --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED -jar ~/Downloads/geni-repl-uberjar-0.0.40.jar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment