Skip to content

Instantly share code, notes, and snippets.

Avatar

Joey Andres JoeyAndres

View GitHub Profile
@JoeyAndres
JoeyAndres / mapred-site.xml
Created Oct 30, 2017
mapred-site.xml for cluster config.
View mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
@JoeyAndres
JoeyAndres / yarn-site.xml
Created Oct 30, 2017
yarn-site.xml cluster configuration.
View yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>linux-01</value>
<description>The hostname of the RM.</description>
@JoeyAndres
JoeyAndres / hdfs-site.xml
Created Oct 30, 2017
hdfs-site.xml for cluster with 2 nodes.
View hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/opt/hadoop/namenode</value>
</property>
<property>
@JoeyAndres
JoeyAndres / core-site.xml
Last active Oct 30, 2017
core-site.xml for cluster configuration
View core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://linux-01</value>
</property>
</configuration>
@JoeyAndres
JoeyAndres / hosts
Created Oct 30, 2017
Ubuntu modified /etc/hosts, based on the Hadoop's requirement.
View hosts
127.0.0.1 localhost
192.168.1.1 linux-01
192.168.1.2 linux-02
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
@JoeyAndres
JoeyAndres / hosts
Last active Oct 30, 2017
Ubuntu default /etc/hosts for linux-01 hostname
View hosts
127.0.0.1 localhost
127.0.1.1 linux-01
# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
@JoeyAndres
JoeyAndres / core-site.xml
Created Oct 30, 2017
Multi-node core-site.xml
View core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:9000</value>
</property>
</configuration>
@JoeyAndres
JoeyAndres / hdfs-site.xml
Created Oct 30, 2017
Pseudo-distributed hdfs-site.xml
View hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
@JoeyAndres
JoeyAndres / core-site.xml
Created Oct 30, 2017
Pseudo-distributed core-site.xml
View core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
@JoeyAndres
JoeyAndres / interfaces
Last active Oct 27, 2017
/etc/network/interfaces static ip setup
View interfaces
auto eth0
iface eth0 inet static
address 192.168.1.1
netmask 255.255.0.0
gateway 192.168.0.1
dns-nameservers 8.8.8.8 8.8.4.4
You can’t perform that action at this time.