flume_install_ubuntu.md

Install flume on ubuntu

http://hadooptutorial.info/apache-flume-installation/

###Step 1 : Загружаем bin файл флюма http://flume.apache.org/download.html ###Step 2 : поочередно выполняем нижеприведенные строки кода: $ sudo mkdir /usr/lib/flume $ sudo chmod -R 777 /usr/lib/flume

###Step 3 : заходим в папку, в которую загрузили apache-flume-1.6.0-bin.tar.gz и перекопируем этот файл в /usr/lib/flume

	$ cp apache-flume-1.6.0-bin.tar.gz /usr/lib/flume
	$ cd /usr/lib/flume/
	$ tar -xzf apache-flume-1.6.0-src.tar.gz

###Step 4 : $ sudo nano ~/.bashrc

###Step 5 : export FLUME_HOME=/usr/local/flume export FLUME_CONF_DIR=$FLUME_HOME/conf export FLUME_CLASS_PATH=$FLUME_CONF_DIR export PATH=$FLUME_HOME/bin:$PATH

###Step 6 : $ cd /usr/local/flume $ sudo cp conf/flume-env.sh.template conf/flume-env.sh

###Step 7 : check java version and java path $ which java $ echo $JAVA_HOME $ java -version

#step 8 : $ sudo nano conf/flume-env.sh $ export JAVA_HOME=/usr/local/java/jdk1.8.0_65 $ export JAVA_OPTS="-Xms100m -Xmx200m -Dcom.sun.management.jmxremote" $ FLUME_CLASSPATH=/usr/local/flume/lib/native

###step 9 : $ source ~/.bashrc

###step 10 : $ cd /usr/local/flume $ sudo mkdir logs $ sudo chmod 777 logs

###step 11 : $ cd /usr/local/flume/logs $ sudo nano flume.log $ sudo chmod 777 flume.log

###step 12 : for kafka download zookeeper-3.4.5-cdh5.4.4.jar from http://grepcode.com/snapshot/repository.cloudera.com/content/repositories/releases/org.apache.zookeeper/zookeeper/3.4.5-cdh5.4.4

###step 13 : for hdfs download hadoop-hdfs-2.6.0-cdh5.4.2.jar from http://grepcode.com/snapshot/repository.cloudera.com/content/repositories/releases/org.apache.hadoop/hadoop-hdfs/2.6.0-cdh5.4.2

###step 14 : $ sudo cp <YOUR_DOWNLOAD_FOLDER>/zookeeper-3.4.5-cdh5.4.4.jar /usr/local/flume/lib

###step 15 : $ sudo cp <YOUR_DOWNLOAD_FOLDER>/hadoop-hdfs-2.6.0-cdh5.4.2.jar /usr/local/flume/lib

###step 16 : add folder native, which contain hdfs-libs

	$ sudo cp -rf /mnt/nfs/galactic/lib/native /usr/local/flume/lib/

###step 17 : $ cd /usr/local/flume $ sudo cp conf/flume-conf.properties.template conf/flume.conf

###step 18 : $ sudo nano conf/flume.conf

see flume_config_examples: