Yuan-Kuei Hsueh phstudy
phstudy
/ bigtop-standby-NN-jps-analysis
Last active
August 29, 2015 14:01
bigtop-standby-NN-jps-analysis
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@bigtop1 usr]# jps | |
| 12751 QuorumPeerMain | |
| 15524 Jps | |
| 13192 NameNode | |
| 13758 DataNode | |
| [root@bigtop2 vagrant]# jps | |
| 9224 Jps | |
| 12640 NameNode |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [vagrant@bigtop1 ~]$ sudo jps | |
| 9040 QuorumPeerMain | |
| 9431 SecondaryNameNode | |
| 13624 HRegionServer | |
| 14495 ThriftServer | |
| 9888 NameNode | |
| 14043 HMaster | |
| 14909 Jps | |
| 10315 DataNode |
phstudy
/ bigtop-cluster-package-analysis
Created
May 12, 2014 15:12
bigtop-cluster-package-analysis
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [vagrant@bigtop1 ~]$ yum list installed | grep @Bigtop | |
| bigtop-jsvc.x86_64 1.0.10.5-1.el6 @Bigtop | |
| bigtop-utils.noarch 0.7.0.5-1.el6 @Bigtop | |
| hadoop.x86_64 2.0.6.5-1.el6 @Bigtop | |
| hadoop-client.x86_64 2.0.6.5-1.el6 @Bigtop | |
| hadoop-debuginfo.x86_64 2.0.6.5-1.el6 @Bigtop | |
| hadoop-doc.x86_64 2.0.6.5-1.el6 @Bigtop | |
| hadoop-hdfs.x86_64 2.0.6.5-1.el6 @Bigtop | |
| 2.0.6.5-1.el6 @Bigtop | |
| hadoop-hdfs-fuse.x86_64 2.0.6.5-1.el6 @Bigtop |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Inspector ran on all 4 hosts. | |
| The following failures were observed in checking hostnames... | |
| No errors were found while looking for conflicting init scripts. | |
| No errors were found while checking /etc/hosts. | |
| All hosts resolved localhost to 127.0.0.1. | |
| All hosts checked resolved each other's hostnames correctly and in a timely manner. | |
| Host clocks are approximately in sync (within ten minutes). | |
| Host time zones are consistent across the cluster. | |
| No users or groups are missing. | |
| No conflicts detected between packages and parcels. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <property> | |
| <name>dfs.nameservices</name> | |
| <value>study</value> | |
| </property> | |
| <property> | |
| <name>dfs.ha.namenodes.study</name> | |
| <value>nn1,nn2</value> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| sudo mkdir -p /dfs/jn | |
| sudo chown -R hdfs:hdfs /dfs/jn |
phstudy
/ Bighadoop_read_TestDFSIO_results.log
Created
May 16, 2014 03:43
Bighadoop Single Node - 32GB NN, SNN, DN (without Hive & HBase)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ----- TestDFSIO ----- : read | |
| Date & time: Fri May 16 11:33:04 CST 2014 | |
| Number of files: 10 | |
| Total MBytes processed: 10000.0 | |
| Throughput mb/sec: 291.97932786358723 | |
| Average IO rate mb/sec: 328.28546142578125 | |
| IO rate std deviation: 105.94441144516256 | |
| Test exec time sec: 49.525 | |
| ----- TestDFSIO ----- : read |
phstudy
/ HA_read_TestDFSIO_results.log
Created
May 16, 2014 05:43
HA 4 Nodes - 8GB/Node 2NN, 3DN, 3NM
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ----- TestDFSIO ----- : read | |
| Date & time: Fri May 16 13:21:58 CST 2014 | |
| Number of files: 10 | |
| Total MBytes processed: 10000.0 | |
| Throughput mb/sec: 15.751603513237647 | |
| Average IO rate mb/sec: 26.165821075439453 | |
| IO rate std deviation: 16.41244162412394 | |
| Test exec time sec: 177.314 | |
| ----- TestDFSIO ----- : read |
phstudy
/ HA_1DN_3NM_read_TestDFSIO_results.log
Created
May 16, 2014 10:43
HA 4 Nodes - 8GB/Node 2NN, 1DN, 3NM
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ----- TestDFSIO ----- : read | |
| Date & time: Fri May 16 14:33:46 CST 2014 | |
| Number of files: 10 | |
| Total MBytes processed: 10000.0 | |
| Throughput mb/sec: 15.215165259517466 | |
| Average IO rate mb/sec: 15.31158447265625 | |
| IO rate std deviation: 1.1998894009063248 | |
| Test exec time sec: 176.768 | |
| ----- TestDFSIO ----- : read |
phstudy
/ HA_1DN_1NM_read_TestDFSIO_results.log
Created
May 16, 2014 10:43
HA 4 Nodes - 8GB/Node 2NN, 1DN, 1NM
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ----- TestDFSIO ----- : read | |
| Date & time: Fri May 16 17:43:43 CST 2014 | |
| Number of files: 10 | |
| Total MBytes processed: 10000.0 | |
| Throughput mb/sec: 80.46800189904485 | |
| Average IO rate mb/sec: 81.07865905761719 | |
| IO rate std deviation: 7.228150704445114 | |
| Test exec time sec: 187.542 | |
| ----- TestDFSIO ----- : read |