Skip to content

Instantly share code, notes, and snippets.

@nsphung
Created February 9, 2017 12:00
Show Gist options
  • Save nsphung/91dd5c0509f534d59345f10d6969d54f to your computer and use it in GitHub Desktop.
Save nsphung/91dd5c0509f534d59345f10d6969d54f to your computer and use it in GitHub Desktop.
dataproc
$ gcloud dataproc clusters diagnose fca-cluster-202 [12:47:56]
Waiting on operation [projects/fcms-sandbox-gce/regions/global/operations/cddd33c1-1a4a-4e23-8309-54e0fcfb4063].
Waiting for cluster diagnose operation...done.
Output from diagnostic:
-----------------------------------------------
Running Google Cloud Dataproc diagnostic
Collecting daemon and system state information
17/02/09 11:48:14 INFO client.RMProxy: Connecting to ResourceManager at fca-cluster-202-m/10.132.0.30:8032
17/02/09 11:48:16 INFO gcs.GoogleHadoopFileSystemBase: GHFS version: 1.5.5-hadoop2
ping: unknown host fca-cluster-202-sw-3por.c.fcms-sandbox-gce.internal
ping: unknown host fca-cluster-202-sw-v7fz.c.fcms-sandbox-gce.internal
ping: unknown host fca-cluster-202-w-6.c.fcms-sandbox-gce.internal
Collecting daemon logs
Collecting configs
Collecting node startup logs
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-m/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-m/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-m/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-3por/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-3por/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-3por/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-v7fz/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-v7fz/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-0/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-0/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-sw-v7fz/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-0/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-1/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-1/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-1/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-2/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-2/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-2/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-3/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-3/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-3/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-4/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-4/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-5/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-4/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-5/dataproc-initialization-script-0_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-6/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-5/dataproc-startup-script_SUCCESS...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-6/dataproc-startup-script_output...
Copying gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/fca-cluster-202-w-6/dataproc-initialization-script-0_output...
/ [30/30 files][733.9 KiB/733.9 KiB] 100% Done
Operation completed over 30 objects/733.9 KiB.
Collecting stack traces
22719: Unable to open socket file: target process not responding or HotSpot VM not loaded
The -F option can be used when the target process is not responding
Generating archive
hadoop_conf/
hadoop_conf/mapred-queues.xml.template
hadoop_conf/hadoop-policy.xml
hadoop_conf/capacity-scheduler.xml
hadoop_conf/configuration.xsl
hadoop_conf/hadoop-env.sh
hadoop_conf/mapred-site.xml
hadoop_conf/hdfs-site.xml
hadoop_conf/mapred-env.sh
hadoop_conf/yarn-env.sh
hadoop_conf/ssl-client.xml.example
hadoop_conf/log4j.properties
hadoop_conf/slaves
hadoop_conf/mapred-site.xml.template
hadoop_conf/nodes_exclude
hadoop_conf/hadoop-metrics2.properties
hadoop_conf/ssl-server.xml.example
hadoop_conf/hadoop-metrics.properties
hadoop_conf/distcp-default.xml
hadoop_conf/container-executor.cfg
hadoop_conf/yarn-site.xml
hadoop_conf/nodes_include
hadoop_conf/core-site.xml
jstack/
jstack/driver_22719.jstack
jstack/agent.jstack
logs/
logs/hive-metastore.log
logs/hadoop-hdfs-secondarynamenode-fca-cluster-202-m.log
logs/google-dataproc-agent.0.log
logs/hadoop-hdfs-namenode-fca-cluster-202-m.log
logs/yarn-yarn-resourcemanager-fca-cluster-202-m.log
logs/mapred-mapred-historyserver-fca-cluster-202-m.log
logs/hive-server2.log
logs/syslog
node_startup/
node_startup/fca-cluster-202-sw-3por/
node_startup/fca-cluster-202-sw-3por/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-sw-3por/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-sw-3por/dataproc-startup-script_output
node_startup/fca-cluster-202-w-3/
node_startup/fca-cluster-202-w-3/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-w-3/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-w-3/dataproc-startup-script_output
node_startup/fca-cluster-202-w-4/
node_startup/fca-cluster-202-w-4/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-w-4/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-w-4/dataproc-startup-script_output
node_startup/fca-cluster-202-w-0/
node_startup/fca-cluster-202-w-0/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-w-0/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-w-0/dataproc-startup-script_output
node_startup/fca-cluster-202-sw-v7fz/
node_startup/fca-cluster-202-sw-v7fz/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-sw-v7fz/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-sw-v7fz/dataproc-startup-script_output
node_startup/fca-cluster-202-w-2/
node_startup/fca-cluster-202-w-2/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-w-2/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-w-2/dataproc-startup-script_output
node_startup/fca-cluster-202-m/
node_startup/fca-cluster-202-m/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-m/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-m/dataproc-startup-script_output
node_startup/fca-cluster-202-w-6/
node_startup/fca-cluster-202-w-6/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-w-6/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-w-6/dataproc-startup-script_output
node_startup/fca-cluster-202-w-1/
node_startup/fca-cluster-202-w-1/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-w-1/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-w-1/dataproc-startup-script_output
node_startup/fca-cluster-202-w-5/
node_startup/fca-cluster-202-w-5/dataproc-startup-script_SUCCESS
node_startup/fca-cluster-202-w-5/dataproc-initialization-script-0_output
node_startup/fca-cluster-202-w-5/dataproc-startup-script_output
system/
system/cluster-ping.log
system/df.log
system/netstat.log
system/fs-file-nr.log
system/yarn-nodes.log
system/sysctl.log
system/free.log
system/hdfs-nodes.log
system/ps.log
system/service.log
system/uptime.log
Saving archive to cloud
Copying file:///tmp/tmp.7raWGkmARy/diagnostic.tar [Content-Type=application/x-tar]...
==> NOTE: You are uploading one or more large file(s), which would run
significantly faster if you enable parallel composite uploads. This
feature can be enabled by editing the
"parallel_composite_upload_threshold" value in your .boto
configuration file. However, note that if you do this large files will
be uploaded as `composite objects
<https://cloud.google.com/storage/docs/composite-objects>`_,which
means that any user who downloads such objects will need to have a
compiled crcmod installed (see "gsutil help crcmod"). This is because
without a compiled crcmod, computing checksums on composite objects is
so slow that gsutil disables downloads of composite objects.
/ [1 files][240.9 MiB/240.9 MiB]
Operation completed over 1 objects/240.9 MiB.
Diagnostic results saved in: gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/tasks/task-cddd33c1-1a4a-4e23-8309-54e0fcfb4063/diagnostic.tar
If diagnostic bundle will be shared with Google Support, then
you can grant read permission to Google Support by running:
gsutil acl ch -p viewers-24135231894:R gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/tasks/task-cddd33c1-1a4a-4e23-8309-54e0fcfb4063/diagnostic.tar
-----------------------------------------------
gs://fca-cluster-202/google-cloud-dataproc-metainfo/345dedca-dd2c-4cc3-86b8-53e6a09246c2/tasks/task-cddd33c1-1a4a-4e23-8309-54e0fcfb4063/output
...
Updates are available for some Cloud SDK components. To install them,
please run:
$ gcloud components update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment