Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@gglanzani

gglanzani/gsInstaller-9715.out

Created December 7, 2012 06:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gglanzani/4231334 to your computer and use it in GitHub Desktop.
Save gglanzani/4231334 to your computer and use it in GitHub Desktop.
Download ZIP
My gsInstaller log
Raw
gsInstaller-9715.out
===============================================================================
Grid Stack Installer
===============================================================================
===============================================================================
Installation Details
===============================================================================
===============================================================================
yum -y install pdsh
===============================================================================
Loaded plugins: priorities, security
Repository epel is listed more than once in the configuration
38 packages excluded due to repository priority protections
Setting up Install Process
Package pdsh-2.27-1.el6.rf.x86_64 already installed and latest version
Nothing to do
MYSUPERSECRETIP: root
using pdsh for deploy
CMD -> yum search hadoop
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: ============================= N/S Matched: hadoop ==============================
MYSUPERSECRETIPi386 : Hadoop is a software platform for processing vast amounts of data
MYSUPERSECRETIPx86_64 : Hadoop is a software platform for processing vast amounts of
MYSUPERSECRETIP: : data
MYSUPERSECRETIPi386 : Hadoop installation in pseudo-distributed mode
MYSUPERSECRETIPx86_64 : Hadoop installation in pseudo-distributed mode
MYSUPERSECRETIPi386 : Hadoop Data Node
MYSUPERSECRETIPx86_64 : Hadoop Data Node
MYSUPERSECRETIPi386 : Debug information for package hadoop
MYSUPERSECRETIPx86_64 : Debug information for package hadoop
MYSUPERSECRETIPi386 : Hadoop Documentation
MYSUPERSECRETIPx86_64 : Hadoop Documentation
MYSUPERSECRETIPi386 : Hadoop Job Tracker
MYSUPERSECRETIPx86_64 : Hadoop Job Tracker
MYSUPERSECRETIPi386 : Hadoop Filesystem Library
MYSUPERSECRETIPx86_64 : Hadoop Filesystem Library
MYSUPERSECRETIPi386 : GPL Compression Libraries for Hadoop (jar)
MYSUPERSECRETIPx86_64 : GPL Compression Libraries for Hadoop (jar)
MYSUPERSECRETIPi386 : Debug information for package hadoop-lzo
MYSUPERSECRETIPx86_64 : Debug information for package hadoop-lzo
MYSUPERSECRETIPi386 : GPL Compression Libraries for Hadoop (native)
MYSUPERSECRETIPx86_64 : GPL Compression Libraries for Hadoop (native)
MYSUPERSECRETIPi386 : The Hadoop namenode manages the block locations of HDFS
MYSUPERSECRETIP: : files
MYSUPERSECRETIPx86_64 : The Hadoop namenode manages the block locations of HDFS
MYSUPERSECRETIP: : files
MYSUPERSECRETIPi386 : Native libraries for Hadoop Compression
MYSUPERSECRETIPx86_64 : Native libraries for Hadoop Compression
MYSUPERSECRETIPi386 : Hadoop Pipes Library
MYSUPERSECRETIPx86_64 : Hadoop Pipes Library
MYSUPERSECRETIPi386 : Binaries for secured Hadoop clusters
MYSUPERSECRETIPx86_64 : Binaries for secured Hadoop clusters
MYSUPERSECRETIPi386 : Hadoop Secondary namenode
MYSUPERSECRETIPx86_64 : Hadoop Secondary namenode
MYSUPERSECRETIPi386 : Source code for Hadoop
MYSUPERSECRETIPx86_64 : Source code for Hadoop
MYSUPERSECRETIPi386 : Hadoop Task Tracker
MYSUPERSECRETIPx86_64 : Hadoop Task Tracker
MYSUPERSECRETIPnoarch : Flume is a reliable, scalable, and manageable distributed log
MYSUPERSECRETIP: : collection application for collecting data such as logs and
MYSUPERSECRETIP
MYSUPERSECRETIPnoarch : HBase is the Hadoop database. Use it when you need random,
MYSUPERSECRETIP This project's goal
MYSUPERSECRETIP: : is the hosting of very large tables -- billions of rows X
MYSUPERSECRETIP
MYSUPERSECRETIPnoarch : The Hadoop HBase master Server.
MYSUPERSECRETIPnoarch : The Hadoop HBase RegionServer server.
MYSUPERSECRETIPnoarch : The Hadoop HBase Thrift Interface
MYSUPERSECRETIPnoarch : Hcatalog is a data warehouse infrastructure built on top of
MYSUPERSECRETIP: : Hadoop
MYSUPERSECRETIPnoarch : Hive is a data warehouse infrastructure built on top of Hadoop
MYSUPERSECRETIPx86_64 : Hadoop Monitoring Framework
MYSUPERSECRETIPnoarch : Oozie is a system that runs workflows of Hadoop jobs.
MYSUPERSECRETIPnoarch : Sqoop allows easy imports and exports of data sets between
MYSUPERSECRETIP
MYSUPERSECRETIPnoarch : Templeton provides a REST-like web API for HCatalog and
MYSUPERSECRETIP
MYSUPERSECRETIPnoarch : The Hadoop Zookeeper server
MYSUPERSECRETIP:
MYSUPERSECRETIP
hdfsuser : hdfs
mapreduser : mapred
installhbase : yes
hbaseuser : hbase
zkuser : zookeeper
installhive : no
installtempleton : no
installoozie : no
installhcat : no
installpig : no
installsqoop : no
smoke_test_user : hdptestuser
enablesecurity : yes
enablemon : yes
package : rpm
===============================================================================
===============================================================================
Cluster Details
===============================================================================
Namenode host : MYSUPERSECRETIPeu-west-1.compute.internal
Jobtracker host : MYSUPERSECRETIPeu-west-1.compute.internal
SNamenode host : MYSUPERSECRETIPeu-west-1.compute.internal
Gateway host : MYSUPERSECRETIPeu-west-1.compute.internal
Hbase Master : MYSUPERSECRETIPeu-west-1.compute.internal
===============================================================================
===============================================================================
check your Install logs at : /tmp/gsinstaller-9715.out
===============================================================================
===============================================================================
Checking for non empty directories
===============================================================================
CMD -> ls /hdp/1/hadoop/hdfs/namenode | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/2/hadoop/hdfs/namenode | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/hdfs/snamenode | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/2/hadoop/hdfs/snamenode | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/hdfs/data | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/2/hadoop/hdfs/data | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/mapred | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/2/hadoop/mapred | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/hdfs/data | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/2/hadoop/hdfs/data | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/mapred | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/2/hadoop/mapred | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/hdfs/data | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/2/hadoop/hdfs/data | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/mapred | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/2/hadoop/mapred | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/zookeeper | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/zookeeper | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
CMD -> ls /hdp/1/hadoop/zookeeper | wc -l | grep '^0$'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: 0
===============================================================================
Download all necessary Artifacts
===============================================================================
CMD -> yum -y install wget curl yum-plugin-priorities
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP12-1.4.el6.x86_64 already installed and latest version
MYSUPERSECRETIP12-1.4.el6.x86_64 already installed and latest version
MYSUPERSECRETIP19.7-26.el6_2.4.x86_64 already installed and latest version
MYSUPERSECRETIP19.7-26.el6_2.4.x86_64 already installed and latest version
MYSUPERSECRETIP1.30-14.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP1.30-14.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP12-1.4.el6.x86_64 already installed and latest version
MYSUPERSECRETIP19.7-26.el6_2.4.x86_64 already installed and latest version
MYSUPERSECRETIP1.30-14.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
===============================================================================
Installing JDK
===============================================================================
===============================================================================
Deploying Hadoop
===============================================================================
===============================================================================
Deploying Hadoop RPMs
===============================================================================
CMD -> yum -y install hadoop hadoop-libhdfs hadoop-native hadoop-pMYSUPERSECRETIP6.0_31 || -L /usr/hadoop-jdk1.6.0_31 ]] && rm -f /usr/hadoop-jdk1.6.0_31; ln -fs /usr/java/jdk1.6.0_31 /usr/hadoop-jdk1.6.0_31
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP0.0-25.el6_3.1.x86_64 already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP0.0-25.el6_3.1.x86_64 already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.0-25.el6_3.1.x86_64 already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.3.16-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.0-25.el6_3.1.x86_64 already installed and latest version
MYSUPERSECRETIP: Nothing to do
CMD -> chown root:hadoop /etc/hadoop/conf/taskcontroller.cfg ; chmod 400 /etc/hadoop/conf/taskcontroller.cfg ; chown root:hadoop /usr/lib/hadoop/bin/task-controller ; chmod 6050 /usr/lib/hadoop/bin/task-controller;
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
===============================================================================
Deploying Hadoop Configs
===============================================================================
===============================================================================
Installing Hadoop Configs
===============================================================================
SCP to nodes -> MYSUPERSECRETIPeu-west-1.compute.internal MYSUPERSECRETIPeu-west-1.compute.internal MYSUPERSECRETIPeu-west-1.compute.internal
Artifacts -> /tmp/HDP-artifacts-9715/hadoop-conf.tar.gz
Destination -> /tmp/HDP
CMD -> rm -rf /etc/hadoop/conf/* ; mkdir -p /etc/hadoop/conf ; tar xvzf /tmp/HDP/hadoop-conf.tar.gz --strMYSUPERSECRETIP
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPsh
MYSUPERSECRETIPinclude
MYSUPERSECRETIPexclude
MYSUPERSECRETIP: hadoop-conf/health_check
MYSUPERSECRETIPxml
MYSUPERSECRETIPxml
MYSUPERSECRETIP: hadoop-conf/slaves
MYSUPERSECRETIPxml
MYSUPERSECRETIPxml
MYSUPERSECRETIPexclude
MYSUPERSECRETIPinclude
MYSUPERSECRETIPcfg
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPsh
MYSUPERSECRETIPinclude
MYSUPERSECRETIPexclude
MYSUPERSECRETIP: hadoop-conf/health_check
MYSUPERSECRETIPxml
MYSUPERSECRETIPxml
MYSUPERSECRETIP: hadoop-conf/slaves
MYSUPERSECRETIPxml
MYSUPERSECRETIPxml
MYSUPERSECRETIPexclude
MYSUPERSECRETIPinclude
MYSUPERSECRETIPcfg
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPsh
MYSUPERSECRETIPinclude
MYSUPERSECRETIPexclude
MYSUPERSECRETIP: hadoop-conf/health_check
MYSUPERSECRETIPxml
MYSUPERSECRETIPxml
MYSUPERSECRETIP: hadoop-conf/slaves
MYSUPERSECRETIPxml
MYSUPERSECRETIPxml
MYSUPERSECRETIPexclude
MYSUPERSECRETIPinclude
MYSUPERSECRETIPcfg
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
===============================================================================
Installing Snappy
===============================================================================
CMD -> yum -y install snappy snappy-devel; ln -sf /usr/lib64/libsnappy.so /usr/lib/hadoop/lib/native/Linux-amd64-64/.
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP0.5-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.5-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.5-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP0.5-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP0.5-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.5-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP0.5-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP0.5-1.el6.x86_64 already installed and latest version
MYSUPERSECRETIP: Nothing to do
===============================================================================
Deploying Zookeeper RPMs
===============================================================================
CMD -> yum -y install zookeeper ; [[ ! -f /usr/zookeeper-jdk1.6.0_31 || -L /usr/zookeeper-jdk1.6.0_31 ]] && rm -f /usr/zookeeper-jdk1.6.0_31; ln -fs /usr/java/jdk1.6.0_31 /usr/zookeeper-jdk1.6.0_31
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP3.4.15-1.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP3.4.15-1.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP3.4.15-1.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
===============================================================================
Deploying Zookeeper Configs
===============================================================================
===============================================================================
Installing Zookeeper Configs
===============================================================================
SCP to nodes -> MYSUPERSECRETIPeu-west-1.compute.internal MYSUPERSECRETIPeu-west-1.compute.internal MYSUPERSECRETIPeu-west-1.compute.internal
Artifacts -> /tmp/HDP-artifacts-9715/zk-conf.tar.gz
Destination -> /tmp/HDP
CMD -> rm -rf /etc/zookeeper/conf/* ; mkdir -p /etc/zookeeper/conf ; tar xvzf /tmp/HDP/zk-conf.tar.gz --strMYSUPERSECRETIP
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxsl
MYSUPERSECRETIPcfg
MYSUPERSECRETIPsh
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxsl
MYSUPERSECRETIPcfg
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxsl
MYSUPERSECRETIPcfg
MYSUPERSECRETIPsh
MYSUPERSECRETIPsh
===============================================================================
Deploying HBase RPMs
===============================================================================
===============================================================================
Deploying Hbase
===============================================================================
CMD -> yum -y install hbase
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: Loaded plugins: priorities, security
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP92.1.15-1.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP92.1.15-1.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
MYSUPERSECRETIP: 38 packages excluded due to repository priority protections
MYSUPERSECRETIP: Setting up Install Process
MYSUPERSECRETIP92.1.15-1.el6.noarch already installed and latest version
MYSUPERSECRETIP: Nothing to do
CMD -> [[ ! -f /usr/hbase-jdk1.6.0_31 || -L /usr/hbase-jdk1.6.0_31 ]] && rm -f /usr/hbase-jdk1.6.0_31; ln -fs /usr/java/jdk1.6.0_31 /usr/hbase-jdk1.6.0_31;
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
===============================================================================
Deploying HBase Configs
===============================================================================
===============================================================================
Installing Hbase Configs for Master
===============================================================================
SCP to nodes -> MYSUPERSECRETIPeu-west-1.compute.internal MYSUPERSECRETIPeu-west-1.compute.internal
Artifacts -> /tmp/HDP-artifacts-9715/hbase-conf.tar.gz
Destination -> /tmp/HDP
CMD -> rm -rf /etc/hbase/conf/*; mkdir -p /etc/hbase/conf; tar xvzf /tmp/HDP/hbase-conf.tar.gz --strMYSUPERSECRETIP
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPsh
MYSUPERSECRETIP: hbase-conf/regionservers
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPsh
MYSUPERSECRETIP: hbase-conf/regionservers
MYSUPERSECRETIPxml
===============================================================================
Installing Hbase Configs for RegionServers
===============================================================================
SCP to nodes -> MYSUPERSECRETIPeu-west-1.compute.internal MYSUPERSECRETIPeu-west-1.compute.internal MYSUPERSECRETIPeu-west-1.compute.internal
Artifacts -> /tmp/HDP-artifacts-9715/hbase-conf.tar.gz
Destination -> /tmp/HDP
CMD -> rm -rf /etc/hbase/conf/*; mkdir -p /etc/hbase/conf; tar xvzf /tmp/HDP/hbase-conf.tar.gz --strMYSUPERSECRETIP
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPsh
MYSUPERSECRETIP: hbase-conf/regionservers
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPsh
MYSUPERSECRETIP: hbase-conf/regionservers
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPxml
MYSUPERSECRETIPproperties
MYSUPERSECRETIPsh
MYSUPERSECRETIP: hbase-conf/regionservers
MYSUPERSECRETIPxml
===============================================================================
Setting up various directories required by hadoop
===============================================================================
CMD -> mkdir -p /var/log/hadoop/mapred; mkdir -p /var/log/hadoop/hdfs; chown -R mapred:hadoop /var/log/hadoop/mapred; chown -R hdfs:hadoop /var/log/hadoop/hdfs; chmod 775 -R /var/log/hadoop; mkdir -p /var/run/hadoop/mapred ; mkdir -p /var/run/hadoop/hdfs ; chown -R mapred:hadoop /var/run/hadoop/mapred; chown -R hdfs:hadoop /var/run/hadoop/hdfs; chmod 775 -R /var/run/hadoop
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
CMD -> rm -rf /hdp/1/hadoop/hdfs/namenode /hdp/2/hadoop/hdfs/namenode; mkdir -p /hdp/1/hadoop/hdfs/namenode /hdp/2/hadoop/hdfs/namenode ; chown -R hdfs:hadoop /hdp/1/hadoop/hdfs/namenode /hdp/2/hadoop/hdfs/namenode; chmod -R 755 /hdp/1/hadoop/hdfs/namenode /hdp/2/hadoop/hdfs/namenode
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
CMD -> rm -rf /hdp/1/hadoop/hdfs/snamenode /hdp/2/hadoop/hdfs/snamenode; mkdir -p /hdp/1/hadoop/hdfs/snamenode /hdp/2/hadoop/hdfs/snamenode ; chown -R hdfs:hadoop /hdp/1/hadoop/hdfs/snamenode /hdp/2/hadoop/hdfs/snamenode; chmod -R 755 /hdp/1/hadoop/hdfs/snamenode /hdp/2/hadoop/hdfs/snamenode
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
CMD -> rm -rf /hdp/1/hadoop/hdfs/data /hdp/2/hadoop/hdfs/data; mkdir -p /hdp/1/hadoop/hdfs/data /hdp/2/hadoop/hdfs/data ; chown -R hdfs:hadoop /hdp/1/hadoop/hdfs/data /hdp/2/hadoop/hdfs/data; chmod -R 750 /hdp/1/hadoop/hdfs/data /hdp/2/hadoop/hdfs/data; mkdir -p /hdp/1/hadoop/mapred /hdp/2/hadoop/mapred; chown -R mapred:hadoop /hdp/1/hadoop/mapred /hdp/2/hadoop/mapred; chmod -R 755 /hdp/1/hadoop/mapred /hdp/2/hadoop/mapred
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
CMD -> mkdir -p /hdp/1/hadoop/mapred /hdp/2/hadoop/mapred; chown -R mapred:hadoop /hdp/1/hadoop/mapred /hdp/2/hadoop/mapred; chmod -R 755 /hdp/1/hadoop/mapred /hdp/2/hadoop/mapred
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
===============================================================================
Setting up various directories required by zookeeper
===============================================================================
CMD -> mkdir -p /var/log/zookeeper /var/run/zookeeper; chmod 775 -R /var/log/zookeeper /var/run/zookeeper ; chown -R zookeeper /var/log/zookeeper /var/run/zookeeper; rm -rf /hdp/1/hadoop/zookeeper ; mkdir -p /hdp/1/hadoop/zookeeper; chmod 755 -R /hdp/1/hadoop/zookeeper; chown -R zookeeper:hadoop /hdp/1/hadoop/zookeeper
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
===============================================================================
Setting up various directories required by Hbase
===============================================================================
CMD -> mkdir -p /var/log/hbase /var/run/hbase ; chown -R hbase:hadoop /var/log/hbase /var/run/hbase ; chmod 775 -R /var/log/hbase /var/run/hbase
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
===============================================================================
Starting All Hadoop Services
===============================================================================
CMD -> su - hdfs -c 'yes Y | hadoop --config /etc/hadoop/conf namenode -format';
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
CMD -> su - hdfs -c '/usr/lib/hadoop/bin/hadoop-daemon.sh --config /etc/hadoop/conf start namenode '
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP Stop it first.
CMD -> su - hdfs -c '/usr/lib/hadoop/bin/hadoop-daemon.sh --config /etc/hadoop/conf start secondarynamenode '
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP Stop it first.
CMD -> su - root -c '/usr/lib/hadoop/bin/hadoop-daemon.sh --config /etc/hadoop/conf start datanode'
In nodes: MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal,MYSUPERSECRETIPeu-west-1.compute.internal
MYSUPERSECRETIP Stop it first.
MYSUPERSECRETIP Stop it first.
MYSUPERSECRETIP Stop it first.
===============================================================================
Waiting 600 seconds for namenode to come out of safe mode
===============================================================================
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:44:30 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:30 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:44:30 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:44:33 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:33 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:38 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:38 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:39 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:39 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:41 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:41 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:43 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:43 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:45 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:45 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:46 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:46 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:51 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:51 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:52 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:52 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:56 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:56 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:44:59 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:44:59 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:02 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:02 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:06 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:06 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:08 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:08 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:08 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:08 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:45:08 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:45:19 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:19 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:45:19 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:45:24 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:24 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:28 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:28 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:28 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:28 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:31 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:31 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:31 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:31 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:36 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:36 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:39 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:39 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:41 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:41 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:43 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:43 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:45 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:45 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:45 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:45 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:47 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:47 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:51 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:51 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:52 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:52 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:45:53 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:45:53 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:45:53 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:46:04 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:04 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:46:04 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:46:08 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:08 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:11 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:11 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:12 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:12 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:16 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:16 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:18 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:18 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:21 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:21 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:24 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:24 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:26 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:26 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:28 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:28 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:31 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:31 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:36 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:36 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:40 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:40 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:43 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:43 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:44 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:44 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:46:47 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:47 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:46:47 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:46:58 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:46:58 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:46:58 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:47:02 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:02 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:05 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:05 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:09 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:09 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:13 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:13 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:16 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:16 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:18 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:18 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:20 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:20 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:22 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:22 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:26 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:26 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:30 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:30 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:32 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:32 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:32 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:32 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:35 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:35 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:40 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:40 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:47:43 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:43 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:47:43 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:47:54 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:54 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:47:54 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:47:58 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:47:58 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:01 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:01 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:02 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:02 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:06 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:06 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:07 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:07 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:07 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:07 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:12 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:12 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:14 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:14 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:19 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:19 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:21 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:21 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:26 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:26 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:28 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:28 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:32 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:32 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:34 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:34 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:34 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:34 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:48:34 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:48:45 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:45 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:48:45 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:48:49 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:49 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:54 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:54 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:55 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:55 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:48:55 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:48:55 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:00 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:00 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:03 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:03 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:07 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:07 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:11 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:11 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:14 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:14 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:17 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:17 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:20 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:20 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:22 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:22 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:27 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:27 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:27 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:27 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:30 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:30 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:49:30 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:49:40 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:40 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:49:40 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:49:45 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:45 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:46 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:46 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:50 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:50 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:52 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:52 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:56 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:56 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:57 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:57 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:49:59 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:49:59 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:01 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:01 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:02 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:02 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:04 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:04 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:05 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:05 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:07 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:07 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:07 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:07 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:09 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:09 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:11 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:11 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:50:11 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:50:22 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:22 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:50:22 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:50:27 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:27 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:27 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:27 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:28 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:28 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:31 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:31 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:35 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:35 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:36 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:36 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:37 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:37 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:40 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:40 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:43 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:43 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:47 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:47 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:50 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:50 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:54 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:54 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:56 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:56 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:50:58 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:50:58 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:00 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:00 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:51:00 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:51:11 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:11 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:51:11 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:51:12 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:12 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:13 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:13 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:14 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:14 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:15 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:15 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:15 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:15 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:16 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:16 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:20 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:20 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:24 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:24 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:29 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:29 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:31 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:31 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:35 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:35 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:36 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:36 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:36 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:36 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:36 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:36 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:39 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:39 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:51:39 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:51:49 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:49 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:51:49 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:51:50 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:50 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:52 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:52 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:55 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:55 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:51:57 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:51:57 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:01 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:01 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:02 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:02 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:04 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:04 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:06 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:06 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:09 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:09 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:11 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:11 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:11 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:11 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:15 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:15 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:18 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:18 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:22 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:22 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:25 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:25 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:52:25 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:52:36 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:36 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:52:36 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:52:36 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:36 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:40 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:40 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:44 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:44 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:47 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:47 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:49 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:49 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:49 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:49 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:54 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:54 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:54 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:54 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:55 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:55 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:52:59 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:52:59 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:04 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:04 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:08 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:08 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:10 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:10 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:13 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:13 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:17 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:17 WARN MYSUPERSECRETIPClient: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
12/12/05 14:53:17 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:java.io.IOException: Couldn't setup connection for hdfs@HADOOP.MYDOMAIN to nn/MYSUPERSECRETIPeu-west-1.compute.internal@HADOOP.MYDOMAIN
Bad connection to DFS... command aborted.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
on MYSUPERSECRETIPeu-west-1.compute.internal
on MYSUPERSECRETIPeu-west-1.compute.internal running ssh -o ConnectTimeOut=3 -q -t -i /root/.ssh/id_rsa root@MYSUPERSECRETIPeu-west-1.compute.internal "su - hdfs -c '/usr/bin/kinit -kt /tmp/hdfs.headless.keytab hdfs; hadoop --config /etc/hadoop/conf dfsadmin -safemode get'"
Output from gwhost safe mode command: 12/12/05 14:53:28 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:28 INFO security.UserGroupInformation: Initiating logout for hdfs@HADOOP.MYDOMAIN
12/12/05 14:53:28 INFO security.UserGroupInformation: Initiating re-login for hdfs@HADOOP.MYDOMAIN
12/12/05 14:53:29 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:29 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:29 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:29 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:32 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:32 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:33 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:33 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
12/12/05 14:53:35 ERROR security.UserGroupInformation: PriviledgedActionException as:hdfs@HADOOP.MYDOMAIN cause:org.apache.hadoop.MYSUPERSECRETIPRemoteException: GSS initiate failed
12/12/05 14:53:35 WARN security.UserGroupInformation: Not attempting to re-login since the last re-login was attempted less than 600 seconds before.
^C
Session terminated, killing shell...^C ...killed.
Found MYSUPERSECRETIPeu-west-1.compute.internal with safe mode ON
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment