Skip to content

Instantly share code, notes, and snippets.

@realazthat

realazthat/execute.sh Secret

Created October 30, 2018 03:51
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save realazthat/d1ccb6556a3d4ce186cecc877a9702a7 to your computer and use it in GitHub Desktop.
Save realazthat/d1ccb6556a3d4ce186cecc877a9702a7 to your computer and use it in GitHub Desktop.
Download ZIP
Issue uploading segments
Raw
execute.sh
# pinot server: SNAPSHOT-2017-12-22T224303Z release
# pinot client: @da1b31a485b7cff717a20aa6223481a9665f8280 (same as SNAPSHOT-2017-12-22T224303Z release)
JAVA_HOME=/usr/lib/jvm/java-8-oracle/jre /usr/local/hadoop/bin/hadoop jar \
./pinot/pinot-hadoop/target/pinot-hadoop-0.016-shaded.jar SegmentTarPush schema/hadoop-job.properties
Raw
hadoop-job.properties
# Segment creation job config:
path.to.input=avro
path.to.output=segments
path.to.schema=schema/events.json
segment.table.name=events
# Segment tar push job config:
push.to.hosts=localhost
push.to.port=8081
Raw
output.log
2018-10-30 03:45:58,492 INFO job.SegmentTarPushJob: ******** Now uploading segments tar from dir: file:/home/ubuntu/cloud9/acstats.site.pinot/db/segments
2018-10-30 03:45:58,501 INFO job.SegmentTarPushJob: ******** Upoading file: events_0 to Host: localhost and Port: 8081 *******
2018-10-30 03:45:58,547 INFO utils.FileUploadUtils: Sending file events_0 to localhost:8081
2018-10-30 03:45:58,592 INFO utils.FileUploadUtils: URI is: localhost:8081
2018-10-30 03:45:58,628 WARN utils.FileUploadUtils: Retry 0 of Upload of File events_0 to URI localhost:8081 after error trying to send file Sleeping for 60 seconds. Caught Exception
java.lang.RuntimeException: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: localhost:8081
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:466)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFileRetry(FileUploadUtils.java:378)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendSegment(FileUploadUtils.java:370)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.pushOneTarFile(SegmentTarPushJob.java:90)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.pushDir(SegmentTarPushJob.java:72)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.run(SegmentTarPushJob.java:57)
at com.linkedin.pinot.hadoop.PinotHadoopJobLauncher.kickOffPinotHadoopJob(PinotHadoopJobLauncher.java:57)
at com.linkedin.pinot.hadoop.PinotHadoopJobLauncher.main(PinotHadoopJobLauncher.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: localhost:8081
at org.apache.http.impl.client.CloseableHttpClient.determineTarget(CloseableHttpClient.java:94)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:426)
... 13 more
2018-10-30 03:46:58,634 INFO utils.FileUploadUtils: Sending file events_0 to localhost:8081
2018-10-30 03:46:58,634 INFO utils.FileUploadUtils: URI is: localhost:8081
2018-10-30 03:46:58,635 WARN utils.FileUploadUtils: Retry 1 of Upload of File events_0 to URI localhost:8081 after error trying to send file Sleeping for 60 seconds. Caught Exception
java.lang.RuntimeException: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: localhost:8081
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:466)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFileRetry(FileUploadUtils.java:378)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendSegment(FileUploadUtils.java:370)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.pushOneTarFile(SegmentTarPushJob.java:90)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.pushDir(SegmentTarPushJob.java:72)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.run(SegmentTarPushJob.java:57)
at com.linkedin.pinot.hadoop.PinotHadoopJobLauncher.kickOffPinotHadoopJob(PinotHadoopJobLauncher.java:57)
at com.linkedin.pinot.hadoop.PinotHadoopJobLauncher.main(PinotHadoopJobLauncher.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: localhost:8081
at org.apache.http.impl.client.CloseableHttpClient.determineTarget(CloseableHttpClient.java:94)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:426)
... 13 more
2018-10-30 03:47:58,636 INFO utils.FileUploadUtils: Sending file events_0 to localhost:8081
2018-10-30 03:47:58,636 INFO utils.FileUploadUtils: URI is: localhost:8081
2018-10-30 03:47:58,637 WARN utils.FileUploadUtils: Retry 2 of Upload of File events_0 to URI localhost:8081 after error trying to send file Sleeping for 60 seconds. Caught Exception
java.lang.RuntimeException: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: localhost:8081
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:466)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFileRetry(FileUploadUtils.java:378)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendSegment(FileUploadUtils.java:370)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.pushOneTarFile(SegmentTarPushJob.java:90)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.pushDir(SegmentTarPushJob.java:72)
at com.linkedin.pinot.hadoop.job.SegmentTarPushJob.run(SegmentTarPushJob.java:57)
at com.linkedin.pinot.hadoop.PinotHadoopJobLauncher.kickOffPinotHadoopJob(PinotHadoopJobLauncher.java:57)
at com.linkedin.pinot.hadoop.PinotHadoopJobLauncher.main(PinotHadoopJobLauncher.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: org.apache.http.client.ClientProtocolException: URI does not specify a valid host name: localhost:8081
at org.apache.http.impl.client.CloseableHttpClient.determineTarget(CloseableHttpClient.java:94)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at com.linkedin.pinot.common.utils.FileUploadUtils.sendFile(FileUploadUtils.java:426)
... 13 more
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment