Weird errros with Apache Spark / Beam / Scio?
For most of 2019 and early 2020, this will most likely be caused by not having the right Java version.
As of August 2019, all big data frame works mostly are written and compatible with Java 8.
Check your java version here:
$ java -version
If this outputs:
java version "1.8.xxx" Java(TM) SE Runtime Environment (build 1.8.xxx) Java HotSpot(TM) 64-Bit Server VM (build xxx, mixed mode)
You are all set, otherwise:
Download Java 8 from here and then install it.
Next export java home (on a Mac) as follows:
$ export JAVA_HOME=$(/usr/libexec/java_home -v 1.8)
You might also want to add the above line to your
.bash_profile so that your default
java version is Java 8, and you don't have to change it everytime.
$ echo 'export JAVA_HOME=$(/usr/libexec/java_home -v 1.8)' >> ~/.bash_profile $ source ~/.bash_profile
If you are an advanced user or you want to switch versions frequently,
you might want to use