Skip to content

Instantly share code, notes, and snippets.

@lanwen
Created May 5, 2020 18:39
Show Gist options
  • Save lanwen/769a1204b26f8392c3929c7457a980e2 to your computer and use it in GitHub Desktop.
Save lanwen/769a1204b26f8392c3929c7457a980e2 to your computer and use it in GitHub Desktop.
18:31:38.972 [public/default/serde-0] INFO org.apache.pulsar.functions.instance.JavaInstanceRunnable - Load JAR: /pulsar/download/pulsar_functions/public/default/serde/0/functions-all.jar
18:31:39.127 [public/default/serde-0] WARN org.apache.pulsar.common.nar.NarClassLoader - /tmp/pulsar-nar/functions-all.jar-unpacked does not contain META-INF/bundled-dependencies!
18:31:39.128 [public/default/serde-0] INFO org.apache.pulsar.functions.instance.JavaInstanceRunnable - Initialize function class loader for function serde at function cache manager
18:31:39.704 [client-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.clients.impl.channel.StorageServerChannelManager - Added range server (hostname: "172.17.0.3"
port: 4181
) into the channel manager.
18:31:39.868 [public/default/serde-0] INFO org.apache.pulsar.functions.instance.JavaInstanceRunnable - Starting state table for function serde
18:31:39.914 [client-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.clients.impl.channel.StorageServerChannelManager - Added range server (hostname: "172.17.0.3"
port: 4181
) into the channel manager.
18:31:39.949 [ForkJoinPool.commonPool-worker-1] INFO org.apache.bookkeeper.clients.StorageClientImpl - Retrieved table properties for table public_default/serde : stream_id: 1024
storage_container_id: 1
stream_name: "serde"
stream_conf {
key_type: HASH
min_num_ranges: 4
initial_num_ranges: 4
split_policy {
fixed_range_policy {
num_ranges: 2
}
}
rolling_policy {
size_policy {
max_segment_size: 134217728
}
}
retention_policy {
time_policy {
retention_minutes: -1
}
}
storage_type: TABLE
}
18:31:40.130 [client-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.clients.impl.kv.PByteBufTableImpl - Updated the active ranges to StreamRanges(keyType=HASH, ranges={-9223372036854775808=start_hash_key: -9223372036854775808
end_hash_key: -4611686018427387905
range_id: 1024
, -4611686018427387905=start_hash_key: -4611686018427387905
end_hash_key: -2
range_id: 1025
storage_container_id: 1
, -2=start_hash_key: -2
end_hash_key: 4611686018427387901
range_id: 1026
storage_container_id: 1
, 4611686018427387901=start_hash_key: 4611686018427387901
end_hash_key: 9223372036854775807
range_id: 1027
}, maxRangeId=1027)
18:31:40.135 [client-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.clients.impl.kv.PByteBufTableImpl - Create table range client for range 1024
18:31:40.136 [client-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.clients.impl.kv.PByteBufTableImpl - Create table range client for range 1025
18:31:40.136 [client-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.clients.impl.kv.PByteBufTableImpl - Create table range client for range 1026
18:31:40.136 [client-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.clients.impl.kv.PByteBufTableImpl - Create table range client for range 1027
18:31:40.171 [public/default/serde-0] INFO org.apache.pulsar.functions.sink.PulsarSink - Opening pulsar sink with config: PulsarSinkConfig(processingGuarantees=ATLEAST_ONCE, topic=test-output-topic, serdeClassName=ru.lanwen.pulsar.functions.TreeNodeSerDe, schemaType=null, typeClassName=ru.lanwen.pulsar.com.fasterxml.jackson.databind.JsonNode)
18:31:40.613 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0xdab8663a, L:/127.0.0.1:34446 - R:localhost/127.0.0.1:6650]] Connected to server
18:31:40.987 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {
"topicName" : "test-output-topic",
"producerName" : null,
"sendTimeoutMs" : 0,
"blockIfQueueFull" : true,
"maxPendingMessages" : 1000,
"maxPendingMessagesAcrossPartitions" : 50000,
"messageRoutingMode" : "CustomPartition",
"hashingScheme" : "Murmur3_32Hash",
"cryptoFailureAction" : "FAIL",
"batchingMaxPublishDelayMicros" : 10000,
"batchingPartitionSwitchFrequencyByPublishDelay" : 10,
"batchingMaxMessages" : 1000,
"batchingMaxBytes" : 131072,
"batchingEnabled" : true,
"compressionType" : "LZ4",
"initialSequenceId" : null,
"autoUpdatePartitions" : true,
"multiSchema" : true,
"properties" : {
"application" : "pulsar-function",
"id" : "public/default/serde",
"instance_id" : "0"
}
}
18:31:40.991 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {
"serviceUrl" : "pulsar://localhost:6650",
"authPluginClassName" : null,
"authParams" : null,
"operationTimeoutMs" : 30000,
"statsIntervalSeconds" : 60,
"numIoThreads" : 1,
"numListenerThreads" : 1,
"connectionsPerBroker" : 1,
"useTcpNoDelay" : true,
"useTls" : false,
"tlsTrustCertsFilePath" : null,
"tlsAllowInsecureConnection" : true,
"tlsHostnameVerificationEnable" : false,
"concurrentLookupRequest" : 5000,
"maxLookupRequest" : 50000,
"maxNumberOfRejectedRequestPerConnection" : 50,
"keepAliveIntervalSeconds" : 30,
"connectionTimeoutMs" : 10000,
"requestTimeoutMs" : 60000,
"initialBackoffIntervalNanos" : 100000000,
"maxBackoffIntervalNanos" : 60000000000
}
18:31:41.004 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [test-output-topic] [null] Creating producer on cnx [id: 0xdab8663a, L:/127.0.0.1:34446 - R:localhost/127.0.0.1:6650]
18:31:41.027 [pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ProducerImpl - [test-output-topic] [standalone-0-3] Created producer on cnx [id: 0xdab8663a, L:/127.0.0.1:34446 - R:localhost/127.0.0.1:6650]
18:31:41.050 [public/default/serde-0] INFO org.apache.pulsar.functions.source.PulsarSource - Opening pulsar source with config: PulsarSourceConfig(processingGuarantees=ATLEAST_ONCE, subscriptionType=Failover, subscriptionName=public/default/serde, subscriptionPosition=Latest, maxMessageRetries=2, deadLetterTopic=test-dlq-topic, topicSchema={test-inputs-topic=ConsumerConfig(schemaType=null, serdeClassName=ru.lanwen.pulsar.functions.TreeNodeSerDe, isRegexPattern=false, receiverQueueSize=null)}, typeClassName=ru.lanwen.pulsar.com.fasterxml.jackson.databind.JsonNode, timeoutMs=null)
18:31:41.052 [public/default/serde-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - Source open produced uncaught exception:
java.lang.RuntimeException: User class must be in class path
at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:67) ~[org.apache.pulsar-pulsar-functions-utils-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.instance.InstanceUtils.createInstance(InstanceUtils.java:84) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.instance.InstanceUtils.initializeSerDe(InstanceUtils.java:43) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.source.TopicSchema.newSchemaInstance(TopicSchema.java:202) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.source.TopicSchema.newSchemaInstance(TopicSchema.java:210) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.source.TopicSchema.lambda$getSchema$0(TopicSchema.java:66) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[?:1.8.0_232]
at org.apache.pulsar.functions.source.TopicSchema.getSchema(TopicSchema.java:66) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.source.PulsarSource.lambda$setupConsumerConfigs$5(PulsarSource.java:163) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at java.util.TreeMap.forEach(TreeMap.java:1005) ~[?:1.8.0_232]
at org.apache.pulsar.functions.source.PulsarSource.setupConsumerConfigs(PulsarSource.java:160) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.source.PulsarSource.open(PulsarSource.java:66) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupInput(JavaInstanceRunnable.java:732) [org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupJavaInstance(JavaInstanceRunnable.java:215) [org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:244) [org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Caused by: java.lang.ClassNotFoundException: ru.lanwen.pulsar.functions.TreeNodeSerDe
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_232]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_232]
at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_232]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_232]
at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_232]
at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:65) ~[org.apache.pulsar-pulsar-functions-utils-2.5.0.jar:2.5.0]
... 15 more
18:31:41.055 [public/default/serde-0] ERROR org.apache.pulsar.functions.instance.JavaInstanceRunnable - [public/default/serde:0] Uncaught exception in Java Instance
java.lang.RuntimeException: User class must be in class path
at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:67) ~[org.apache.pulsar-pulsar-functions-utils-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.instance.InstanceUtils.createInstance(InstanceUtils.java:84) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.instance.InstanceUtils.initializeSerDe(InstanceUtils.java:43) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.source.TopicSchema.newSchemaInstance(TopicSchema.java:202) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.source.TopicSchema.newSchemaInstance(TopicSchema.java:210) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.source.TopicSchema.lambda$getSchema$0(TopicSchema.java:66) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[?:1.8.0_232]
at org.apache.pulsar.functions.source.TopicSchema.getSchema(TopicSchema.java:66) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.source.PulsarSource.lambda$setupConsumerConfigs$5(PulsarSource.java:163) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at java.util.TreeMap.forEach(TreeMap.java:1005) ~[?:1.8.0_232]
at org.apache.pulsar.functions.source.PulsarSource.setupConsumerConfigs(PulsarSource.java:160) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.source.PulsarSource.open(PulsarSource.java:66) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:2.5.0]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupInput(JavaInstanceRunnable.java:732) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupJavaInstance(JavaInstanceRunnable.java:215) ~[org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:244) [org.apache.pulsar-pulsar-functions-instance-2.5.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_232]
Caused by: java.lang.ClassNotFoundException: ru.lanwen.pulsar.functions.TreeNodeSerDe
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_232]
at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_232]
at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_232]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_232]
at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_232]
at org.apache.pulsar.functions.utils.Reflections.createInstance(Reflections.java:65) ~[org.apache.pulsar-pulsar-functions-utils-2.5.0.jar:2.5.0]
... 15 more
18:31:41.059 [public/default/serde-0] INFO org.apache.pulsar.functions.instance.JavaInstanceRunnable - Closing instance
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment