tcarland

Keybase proof

I hereby claim:

• I am tcarland on github.
• I am tcarland (https://keybase.io/tcarland) on keybase.
• I have a public key ASArEJi3ufKa92N7uTJG5w0LXWWI5y-2b5-ariGWBrIMCAo

To claim this, I am signing this object:

tcarland

Hadoop v2 to v3

Stop all Application and Services other than HDFS

Run Fsck

hdfs fsck / -files -blocks -locations > dfs-fsck.log

tcarland

Configuring Vim for ctags and Scala

This document configures Vim or GVim for auto-completion using Vim OmniComplete (built-in) and exuberant-ctags.

It should be noted that the exuberant-ctags project seems a bit dated, and potentially Universal Ctags is a better choice ctags.io.

tcarland

Configuring MySQL for Replication

Configure Mysql:

Additional mysql options common to all instances is provided below. The following are the necessary options for enabling bin-log format necessary for replication. 'server-id' should be unique across all mysql instances.

server-id=1

tcarland

Pseudo-Distributed Hadoop Environment

A document describing the configuration of a local, apache-based hadoop distribution running in pseudo-distributed mode. While there are useful VM's provided by various hadoop vendors, running natively provides better performance and more control over the environment for testing purposes (such as running multiple versions). For developers interested in underlying details of the hadoop stack, having a native version based on compiled apache projects is much more clear versus trying to make sense of Cloudera's internal versions.

tcarland

Hadoop Node Prerequisites

Configuring root ssh

There are various methods of automation for applying these nodes requisites, including distributing CDH agents, but it is still very useful to have an administration tool that allows interaction with all nodes with proper feedback and diff capabilities. Clustershell works brilliantly for this and is a must for managing clusters without opening too many windows. If not just for clustershell, configuring root ssh is also useful

tcarland

Building Hadoop and Various Ecosystem Components

A guide for building hadoop and other ecosystem components from source.

Building Hadoop (v2.7.4)

Prerequisites:

• Oracle JDK 1.8