An example of how to use Caliper: http://code.google.com/p/caliper/.

poml.xml

<dependency>
    <groupId>com.google.caliper</groupId>
    <artifactId>caliper</artifactId>
    <version>0.5-rc1</version>
    <scope>test</scope>
</dependency>

SetBenchmark.java

import com.google.caliper.Runner;
import com.google.caliper.SimpleBenchmark;

import java.util.HashSet;
import java.util.TreeSet;

public class SetBenchmark extends SimpleBenchmark {
    // If you add a main function, you can run it from your IDE.
    public static void main(String[] args) throws Exception {
        new Runner().run(
                // These are the command line arguments for Runner. You can add
                // "--trials", "10" to run each benchmark 10 times each.
                SetBenchmark.class.getName()
        );
    }
    
    private static final int SET_SIZE = 100000;
    
    // Do the up-front allocation of things in the benchmark constructor,
    // just like a unit test.
    private final HashSet<Integer> hash = new HashSet<Integer>(100);
    private final TreeSet<Integer> tree = new TreeSet<Integer>();

    // Caliper looks for methods which start with "time"; gross, I know.
    // The reps argument here is injected by Caliper, which uses it to factor
    // out the cost of method invocation.
    public void timeHashSetAdd(int reps) {
        // Be sure to add this outer loop.
        for (int i = 0; i < reps; i++) {
            // Then, in the inner loop, go ahead and do some meaningful work.
            // Here we're seeing how long it takes to add this many integers
            // to a set.
            for (int j = 0; j < SET_SIZE; j++) {
                hash.add(j);
            }
        }
    }
    
    // And now we've got a fight on our hands!
    public void timeTreeSetAdd(int reps) {
        for (int i = 0; i < reps; i++) {
            for (int j = 0; j < 10000; j++) {
                tree.add(j);
            }
        }
    }
}

Which version of guava will require to run this example?

benchmark=HashSetAdd}Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.io.Closeables.closeQuietly(Ljava/io/Closeable;)V

Closeables.closeQuietly has been removed with Guava 16: https://google.github.io/guava/releases/16.0/api/diffs/changes/com.google.common.io.Closeables.html