Skip to content

Instantly share code, notes, and snippets.

@ivannov
Last active July 6, 2017 01:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ivannov/906096dcff2100ccd5b2ef11f97362dd to your computer and use it in GitHub Desktop.
Save ivannov/906096dcff2100ccd5b2ef11f97362dd to your computer and use it in GitHub Desktop.
The disappointments I had during setting up jCache workshop for BG JUG

Now this is not really disappointment with jCache spec per se as it was my first one. It is more in the Hazelcast - Payara integration.

Note
the current version of Payara is working with Hazelcast 3.5.2. As far as I saw, the serialization implementation, which is the one that is heavily involved here, was changed a lot since then. So maybe all these issues were fixed.

So, as part of our workshop, we tried to showcase as well chapter 9 of the spec: Entry Processors. We wanted when a comment was put in our comments cache, it’s description to contain also the comment author at the end.

So we implemented our simple entry processor by the spec:

public class CommentsAuthorEntryProcessor implements
        EntryProcessor<Long, Comment, Comment>, Serializable {

    @Override
    public Comment process(MutableEntry<Long, Comment> entry,
                           Object... arguments) throws EntryProcessorException {
        Comment comment = entry.getValue();
        comment.setContent(comment.getContent() + " ["
                + comment.getByUser().getUserName() + "]");
        entry.setValue(comment);
        return comment;
    }

}

Next, following the spec, we configured the entry processor in our cache implementation:

    private Cache<Long, Comment> cache;

    @PostConstruct
    public void getCommentsCache() {

        cache = cacheManager.getCache(COMMENTS_CACHE_NAME, Long.class, Comment.class);
        if (cache == null) {
            cache = cacheManager.createCache(
                    COMMENTS_CACHE_NAME,
                    new MutableConfiguration<Long, Comment>()
                            .setTypes(Long.class, Comment.class)
                            .addCacheEntryListenerConfiguration(new MutableCacheEntryListenerConfiguration<>(
                                    FactoryBuilder.factoryOf(EntryCreatedLogListener.class), null, true, true)));
        }
    }

And finally let’s try to submit a comment:

    public void submitComment(Comment newComment) {
        cache.put(newComent.getId(), newComent);
        cache.invoke(newComment, new CommentsAuthorEntryProcessor());
    }

This will fail:

java.lang.ClassNotFoundException: bg.jug.guestbook.entities.Comment
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at com.hazelcast.nio.ClassLoaderUtil.tryLoadClass(ClassLoaderUtil.java:125)
	at com.hazelcast.nio.ClassLoaderUtil.loadClass(ClassLoaderUtil.java:114)
	at com.hazelcast.nio.IOUtil$1.resolveClass(IOUtil.java:113)
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1613)
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
	at com.hazelcast.nio.serialization.DefaultSerializers$ObjectSerializer.read(DefaultSerializers.java:201)
	at com.hazelcast.nio.serialization.StreamSerializerAdapter.read(StreamSerializerAdapter.java:41)
	at com.hazelcast.nio.serialization.SerializationServiceImpl.toObject(SerializationServiceImpl.java:276)
	at com.hazelcast.spi.impl.NodeEngineImpl.toObject(NodeEngineImpl.java:200)
	at com.hazelcast.cache.impl.AbstractCacheService.toObject(AbstractCacheService.java:270)
	at com.hazelcast.cache.impl.CacheEntryProcessorEntry.getRecordValue(CacheEntryProcessorEntry.java:132)
	at com.hazelcast.cache.impl.CacheEntryProcessorEntry.getValue(CacheEntryProcessorEntry.java:114)

The reason for this failure is the classloader that is trying to de-serialize the Comment value that I am trying to pass to the entry processor. This one is the current thread’s context class loader. From ClassLoaderUtil::loadClass:

        if (theClassLoader == null) {
            theClassLoader = Thread.currentThread().getContextClassLoader();
        }

Which is really different than the one of the injected CacheManager.

A workaround here was to not inject the CacheManager, but to create it manually. Practice not really common for a Java EE application:

    private static CacheManager cacheManager;

    static {
        ClassLoader appClassLoader = JCacheCommentsManager.class.getClassLoader();
        Config config = new Config();
        config.setClassLoader(appClassLoader);
        HazelcastInstance instance = Hazelcast.newHazelcastInstance(config);
        Properties props = HazelcastCachingProvider.propertiesByInstanceName(instance.getName());
        CachingProvider cp = Caching.getCachingProvider();
        cacheManager = cp.getCacheManager(cp.getDefaultURI(), appClassLoader, props);
    }

Now calling as simple method as cache.iterator(); of a cache created by the above CacheManager hangs and after some time starts spitting these exceptions in the server log:

com.hazelcast.cache.CacheNotExistsException: Cache is already destroyed or not created yet, on Member [192.168.124.1]:5900 this
	at com.hazelcast.cache.impl.AbstractCacheRecordStore.<init>(AbstractCacheRecordStore.java:117)
	at com.hazelcast.cache.impl.CacheRecordStore.<init>(CacheRecordStore.java:61)
	at com.hazelcast.cache.impl.CacheService.createNewRecordStore(CacheService.java:76)
	at com.hazelcast.cache.impl.CachePartitionSegment$1.createNew(CachePartitionSegment.java:56)
	at com.hazelcast.cache.impl.CachePartitionSegment$1.createNew(CachePartitionSegment.java:53)
	at com.hazelcast.util.ConcurrencyUtil.getOrPutSynchronized(ConcurrencyUtil.java:40)
	at com.hazelcast.cache.impl.CachePartitionSegment.getOrCreateCache(CachePartitionSegment.java:76)
	at com.hazelcast.cache.impl.AbstractCacheService.getOrCreateCache(AbstractCacheService.java:128)
	at com.hazelcast.cache.impl.operation.AbstractCacheOperation.beforeRun(AbstractCacheOperation.java:68)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:131)
	at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:315)
	at com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.processPacket(OperationThread.java:142)
	at com.hazelcast.spi.impl.operationexecutor.classic.OperationThread.process(OperationThread.java:115)

This was miraculously solved by my colleage Martin Toshev by replacing all the Comment occurnces in the generic parameters of our Cache and CacheEntryProcessor implementations and instances. Something like that:

        @PostConstruct
        public void getCommentsCache() {

            cache = cacheManager.getCache(COMMENTS_CACHE_NAME, Long.class, PayaraValueHolder.class);
            if (cache == null) {
                cache = cacheManager.createCache(
                        COMMENTS_CACHE_NAME,
                        new MutableConfiguration<Long, PayaraValueHolder>()
                                .setTypes(Long.class, PayaraValueHolder.class)
                                .addCacheEntryListenerConfiguration(new MutableCacheEntryListenerConfiguration<>(
                                        FactoryBuilder.factoryOf(EntryCreatedLogListener.class), null, true, true)));
            }
        }

        @Override
        public void submitComment(Comment newComment) {
            try {
                cache.put(newComment.getId(), new PayaraValueHolder(submittedComment));
                cache.invoke(newComment.getId(), new CommentsAuthorEntryProcessor());
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

Ugly, isn’t it:

  • I create the CacheManager myself as if I am working outside of a Java EE container

  • I use a Payara-Hazelcast internal type (PayaraValueHolder) instead of my own Comment

BTW, this didn’t work either. After hanging for 12 seconds, Payara-Hazelcast threw some kind of TimeoutException.

@OndroMih
Copy link

OndroMih commented Dec 7, 2016

I believe that the solution with creation of a CacheManager in the application should work with later versions of Payara Server. I suspect that the second cachemanager was created in a separate Hazelcast cluster, which collided with the server's cluster. In later versions, Payara server specifies a custom Hazelcast group name and password to avoid collisions with other Hazelcast clusters.

@lprimak
Copy link

lprimak commented Feb 13, 2017

I am working on this issue, it's not an easy one but will get there

@lprimak
Copy link

lprimak commented Jul 4, 2017

This requires some integration on the part of Hazelcast. This will be fixed in Payara 5

@lprimak
Copy link

lprimak commented Jul 6, 2017

Working version of jcache-workshop with Payara experimental PR merged:
payara/Payara#1744 (comment)

Basically all the workarounds were removed and it runs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment