Instantly share code, notes, and snippets.

Embed
What would you like to do?
Exception in thread "main" java.lang.NoSuchMethodError: java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView;

Interaction of Covariance and Java Cross-compile

Here is a Java class with a compatibility problem:

HelloCovariance.java:

import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

public class HelloCovariance {
  public static void main(String[] args) {
    ConcurrentHashMap<String, String> properties = new ConcurrentHashMap<>();
    Set<String> keySet = properties.keySet();
  }
}

Here is a session log that seems implausible:

$ /usr/lib/jvm/java-8-oracle/bin/javac -source 1.7 -target 1.7 HelloCovariance.java 
warning: [options] bootstrap class path not set in conjunction with -source 1.7
1 warning
$ /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java HelloCovariance 
Exception in thread "main" java.lang.NoSuchMethodError: java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView;
	at HelloCovariance.main(HelloCovariance.java:7)

Why does this NoSuchMethodError happen?

Compare the JavaDoc for ConcurrentHashMap#keySet() in Java 1.7 and 1.8:

Notably the Java 1.7 ConcurrentHashMap#keySet() returns a Set<K> while the 1.8 ConcurrentHashMap#keySet() returns a ConcurrentHashMap.KeySetView<K,V>.

How can this work? It turns out that a overriding method is allowed to return a sub-type of the parent methods return type. This is due to covariance. This is useful in certain cases where you know the concrete types and want to avail of them somehow.

You'll notice that there is a warning about bootstrap classpath. It turns out those bootstrap classpath warnings have a lot of merit. The safest fix is clearly to compile with a bootstrap classpath.

Failing that you can avoid the unnecessary use of a concrete type for a variable here.

Using the general Map interface in place of the concrete ConcurrentHashMap type here side-steps the coupling to the Java 8 return type and will allow this code to be compiled with Java 8 and run on Java 7.

HelloSidestep.java:

import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;

public class HelloSidestep {
  public static void main(String[] args) {
    Map<String, String> properties = new ConcurrentHashMap<>();
    Set<String> keySet = properties.keySet();
  }
}
$ /usr/lib/jvm/java-8-oracle/bin/javac -source 1.7 -target 1.7 HelloCovariance.java 
warning: [options] bootstrap class path not set in conjunction with -source 1.7
1 warning
$ /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java HelloCovariance 

Hey look! No crash. This is not really the right fix, and you will be playing whack-a-mole with similar bugs in a large system. The correct and recommended fix is to use a bootstrap classpath.

$ /usr/lib/jvm/java-8-oracle/bin/javac -source 1.7 -target 1.7 HelloCovariance.java -bootclasspath /usr/lib/jvm/java-1.7.0-openjdk-amd64/jre/lib/rt.jar
$ /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java HelloCovariance 

Now we get no warnings and no crash.

This is not a new problem, but it seems like there is still a lot of confusion around it.

Here are some useful resources to consider:

#!/usr/bin/env bash
# Tested on Ubuntu with WebUpd8 install of Oracle JDK 8
/usr/lib/jvm/java-8-oracle/bin/javac -source 1.7 -target 1.7 HelloCovariance.java
/usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java HelloCovariance
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
public class HelloCovariance {
public static void main(String[] args) {
ConcurrentHashMap<String, String> properties = new ConcurrentHashMap<>();
Set<String> keySet = properties.keySet();
}
}
@ijuma

This comment has been minimized.

Show comment
Hide comment
@ijuma

ijuma May 23, 2016

Java 9 has a good solution for this: http://openjdk.java.net/jeps/247. In the meantime, one can also simply compile with Java 7 if that is the minimum supported version. It doesn't seem like there is much of an advantage in compiling with Java 8 if one still has to pass Java 7's rt.jar in the bootclasspath.

ijuma commented May 23, 2016

Java 9 has a good solution for this: http://openjdk.java.net/jeps/247. In the meantime, one can also simply compile with Java 7 if that is the minimum supported version. It doesn't seem like there is much of an advantage in compiling with Java 8 if one still has to pass Java 7's rt.jar in the bootclasspath.

@medusar

This comment has been minimized.

Show comment
Hide comment
@medusar

medusar Jun 8, 2016

hey, thank you for your share, i wonder that if i specify the jdk version to 1.7 when i compile the source code , this problem still happens, why is that ?

medusar commented Jun 8, 2016

hey, thank you for your share, i wonder that if i specify the jdk version to 1.7 when i compile the source code , this problem still happens, why is that ?

@medusar

This comment has been minimized.

Show comment
Hide comment
@medusar

medusar Jun 8, 2016

i mean , i use a JDK8 to develop the code, and when i compile the source, i use the command :'javac -source 1.7 -target 1.7', this problem still happens, why ?

medusar commented Jun 8, 2016

i mean , i use a JDK8 to develop the code, and when i compile the source, i use the command :'javac -source 1.7 -target 1.7', this problem still happens, why ?

@rdblue

This comment has been minimized.

Show comment
Hide comment
@rdblue

rdblue Jul 26, 2016

The problem is that -source and -target are used for the language version you're compiling (-source) and the bytecode version you're creating (-target) but the version of the runtime library, rt.jar, isn't configured by either one. When you compile with Java 8, you're creating byte code that references its rt.jar classes, which differs in small and incompatible ways with Java 7's runtime.

rdblue commented Jul 26, 2016

The problem is that -source and -target are used for the language version you're compiling (-source) and the bytecode version you're creating (-target) but the version of the runtime library, rt.jar, isn't configured by either one. When you compile with Java 8, you're creating byte code that references its rt.jar classes, which differs in small and incompatible ways with Java 7's runtime.

@rdblue

This comment has been minimized.

Show comment
Hide comment
@rdblue

rdblue Aug 26, 2016

I just noticed that Spark has a configuration that fixes this when building: https://github.com/apache/spark/blob/master/pom.xml#L2595

rdblue commented Aug 26, 2016

I just noticed that Spark has a configuration that fixes this when building: https://github.com/apache/spark/blob/master/pom.xml#L2595

@erikdw

This comment has been minimized.

Show comment
Hide comment
@erikdw

erikdw Oct 6, 2016

@AlainODea: you have a thrice-repeated typo of Covariance as Co**_n**_variance, I fixed it in a fork of this gist, but seems GitHub still doesn't allow PRs for gists.

erikdw commented Oct 6, 2016

@AlainODea: you have a thrice-repeated typo of Covariance as Co**_n**_variance, I fixed it in a fork of this gist, but seems GitHub still doesn't allow PRs for gists.

@AlainODea

This comment has been minimized.

Show comment
Hide comment
@AlainODea

AlainODea Jul 19, 2017

@erikdw thank you. I'm not sure why I didn't get the notification on your mention here. I've merged your changes in.

@ijuma good to know! I'm in the process of assessing Java 9 readiness now so JEP 247 will be useful. Agreed on compiling with JDK 7 if you need to run on JRE 7.

Owner

AlainODea commented Jul 19, 2017

@erikdw thank you. I'm not sure why I didn't get the notification on your mention here. I've merged your changes in.

@ijuma good to know! I'm in the process of assessing Java 9 readiness now so JEP 247 will be useful. Agreed on compiling with JDK 7 if you need to run on JRE 7.

@stuart-marks

This comment has been minimized.

Show comment
Hide comment
@stuart-marks

stuart-marks Jan 22, 2018

Thanks for writing this up. The recommendation to use -bootclasspath (with 8 and earlier) is correct, as is the recommendation from @ijuma to use --release when compiling with JDK 9 and later.

People are running into similar issues with the 8 => 9 transition. In particular, the various Buffer subclasses now have a whole new family of covariant overrides, and there are new overloads of Math.floorDiv and floorMod that can resolve differently between 8 and 9. Both of these can lead to the same NoSuchMethodError problem if code is compiled on JDK 9 using the old (and broken) technique of specifying -source and -target but not -bootclasspath. Please publicize the proper use of --release if you get a chance.

stuart-marks commented Jan 22, 2018

Thanks for writing this up. The recommendation to use -bootclasspath (with 8 and earlier) is correct, as is the recommendation from @ijuma to use --release when compiling with JDK 9 and later.

People are running into similar issues with the 8 => 9 transition. In particular, the various Buffer subclasses now have a whole new family of covariant overrides, and there are new overloads of Math.floorDiv and floorMod that can resolve differently between 8 and 9. Both of these can lead to the same NoSuchMethodError problem if code is compiled on JDK 9 using the old (and broken) technique of specifying -source and -target but not -bootclasspath. Please publicize the proper use of --release if you get a chance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment