bugcy013
/ Hbase in CDH4.txt
Last active
December 10, 2015 19:48
— forked from nipra/notes.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode in MRv1 | |
| # https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode#InstallingCDH4onaSingleLinuxNodeinPseudo-distributedMode-InstallingCDH4withMRv1onaSingleLinuxNodeinPseudodistributedmode | |
| # Installing CDH4 with MRv1 on a Single Linux Node in Pseudo-distributed mode | |
| # On Ubuntu Precise | |
| tech@tech-VirtualBox:~$ wget -cv http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb | |
| tech@tech-VirtualBox:~$ sudo dpkg -i cdh4-repository_1.0_all.deb # Adds /etc/apt/sources.list.d/cloudera-cdh4.list ?? | |
| tech@tech-VirtualBox:~$ dpkg -L cdh4-repository # To view the files on Ubuntu systems | |
| /. | |
| /etc |
bugcy013
/ gist:4484295
Created
January 8, 2013 14:47
— forked from develash/gist:4243381
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Assumes you have Sun JDK installed already and JAVA_HOME set to that for root | |
| # This is all basically a summary of various parts of https://ccp.cloudera.com/display/CDH4DOC/CDH4+Documentation | |
| # Add Cloudera RPM-GPG-KEY and repo | |
| rpm --import http://archive.cloudera.com/cdh4/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera | |
| rpm -ivh http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm | |
| # Install CDH4 Base | |
| yum install hadoop-0.20-conf-pseudo |
bugcy013
/ hadoop_1.1.1_setup.sh
Created
January 8, 2013 14:48
— forked from develash/hadoop_1.1.1_setup.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # hdfs-site.xml | |
| # <property> | |
| # <name>dfs.replication</name> | |
| # <value>1</value> | |
| # </property> | |
| # core-site.xml |
bugcy013
/ hadoop_0.23.5_single_node.sh
Created
January 8, 2013 14:48
— forked from develash/hadoop_0.23.5_single_node.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # hdfs-site.hml | |
| #<property> | |
| # <name>dfs.replication</name> | |
| # <value>1</value> | |
| # </property> | |
| # <property> | |
| # <name>dfs.namenode.name.dir</name> | |
| # <value>file:/opt/yarn_data/hdfs/namenode</value> | |
| # </property> | |
| # <property> |
bugcy013
/ Linux Static IP
Created
January 14, 2013 10:58
— forked from fernandoaleman/Linux Static IP
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## Configure eth0 | |
| # | |
| # vi /etc/sysconfig/network-scripts/ifcfg-eth0 | |
| DEVICE="eth0" | |
| NM_CONTROLLED="yes" | |
| ONBOOT=yes | |
| HWADDR=A4:BA:DB:37:F1:04 | |
| TYPE=Ethernet | |
| BOOTPROTO=static |
bugcy013
/ mrv1 cdh4
Created
February 14, 2013 10:10
— forked from SwathiMystery/mrv1 cdh4
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Cluster name goes here | |
| whirr.cluster-name=testcluster | |
| # Change the number of machines in the cluster here | |
| # Using 3 DN and TT and 1JT and NN# Ganglia is configured | |
| whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode+ganglia-monitor+ganglia-metad,3 hadoop-datanode+hadoop-tasktracker+ganglia-monitor | |
| # Install JAVA | |
| whirr.java.install-function=install_openjdk | |
| whirr.java.install-function=install_oab_java |
bugcy013
/ initdfs.sh
Created
February 19, 2013 07:28
— forked from Voronenko/initdfs.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| sudo -u hdfs hadoop fs -mkdir /tmp | |
| sudo -u hdfs hadoop fs -chmod -R 1777 /tmp | |
| sudo -u hdfs hadoop fs -mkdir /var | |
| sudo -u hdfs hadoop fs -mkdir /var/lib | |
| sudo -u hdfs hadoop fs -mkdir /var/lib/hadoop-hdfs | |
| sudo -u hdfs hadoop fs -mkdir /var/lib/hadoop-hdfs/cache | |
| sudo -u hdfs hadoop fs -mkdir /var/lib/hadoop-hdfs/cache/mapred | |
| sudo -u hdfs hadoop fs -mkdir /var/lib/hadoop-hdfs/cache/mapred/mapred | |
| sudo -u hdfs hadoop fs -mkdir /var/lib/hadoop-hdfs/cache/mapred/mapred/staging |
bugcy013
/ Add LZO to Hadoop
Created
February 19, 2013 07:31
— forked from tovbinm/Add LZO to Hadoop
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| On Ubuntu: | |
| 1.sudo apt-get install lzop liblzo2-dev | |
| 2.download and build: https://github.com/kevinweil/hadoop-lzo | |
| 3.copy the resulted jar to: <yourhadoop>/lib/, typically: /usr/lib/hadoop/lib/ | |
| 4.download: http://code.google.com/a/apache-extras.org/p/hadoop-gpl-compression/ | |
| 5.cp ./hadoop-gpl-compression-0.1.0/lib/native/Linux-<your_acrh_type>/*.* /usr/lib/hadoop/lib/native/Linux-<your_acrh_type>/ | |
| 6.Add the following properties to core-site.xml: | |
| <property> | |
| <name>io.compression.codecs</name> | |
| <value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.BZip2Codec</value> |
bugcy013
/ gist:4984024
Created
February 19, 2013 08:27
— forked from tovbinm/gist:3977493
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| echo "sun-java6-jdk shared/accepted-sun-dlj-v1-1 boolean true" | debconf-set-selections | |
| DEBIAN_FRONTEND=noninteractive aptitude install -y -f sun-java6-jre sun-java6-bin sun-java6-jdk |
bugcy013
/ notes.txt
Created
February 21, 2013 17:35
— forked from nipra/notes.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode | |
| # https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode | |
| # Installing CDH4 with MRv1 on a Single Linux Node in Pseudo-distributed mode | |
| # On Ubuntu and other Debian systems | |
| nipra@lambda:Downloads$ wget -cv http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb | |
| nipra@lambda:Downloads$ sudo dpkg -i cdh4-repository_1.0_all.deb # Adds /etc/apt/sources.list.d/cloudera-cdh4.list ?? | |
| nipra@lambda:Downloads$ dpkg -L cdh4-repository # To view the files on Ubuntu systems | |
| # Install CDH4 |
OlderNewer