Skip to content

Instantly share code, notes, and snippets.

@philippeback
Created November 15, 2016 15:00
Show Gist options
  • Save philippeback/31007c9edc70c93c235209fb950ea9ea to your computer and use it in GitHub Desktop.
Save philippeback/31007c9edc70c93c235209fb950ea9ea to your computer and use it in GitHub Desktop.
ambari setup

Ambari uses a local postgres db by default.This page describes how to use ambari-server with remote postgres server.

Install

Ambari is installed on centos 6.4 with the following command:

curl -so /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari.repo
yum repolist
yum -y install ambari-server

Configuration

The default configuration can be done as:

ambari-server setup --silent

All the configuration is stored in: /etc/ambari-server/conf/ambari.properties

significant properties are:

server.persistence.type=local
server.jdbc.database=ambari
server.jdbc.user.name=ambari
server.jdbc.user.passwd=/etc/ambari-server/conf/password.dat

Start

The normal

ambari-server start

Under The Hood

The start script /usr/sbin/ambari-server is a shell script, that set environment variables and kicks a python script which kicks a java process similar to:

/usr/sbin/ambari-server

# where to look for ambari.properties
export AMBARI_CONF_DIR=/etc/ambari-server/conf

# sets AMBARI_JVM_ARGS like memory and java security
. /var/lib/ambari-server/ambari-env.sh

/usr/sbin/ambari-server.py
/usr/jdk64/jdk1.6.0_31/bin/java -server  \
  -XX:NewRatio=2 -XX:+UseConcMarkSweepGC -Xms512m -Xmx2048m \
  -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf\
  -Djava.security.krb5.conf=/etc/krb5.conf \
  -Djavax.security.auth.useSubjectCredsOnly=false\
  -Xdebug -Xrunjdwp:transport=dt_socket,address=5005,server=y,suspend=n \
  -cp /etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/ambari-server/*\
  org.apache.ambari.server.controller.AmbariServer &

Directories

  • /var/lib/ambari-server/
    • ambari-env.sh sets jvm args/security
  • /var/lib/ambari-server/resources/
    • Ambari-DDL-*.sql create/drop scripts for mysql/oracle/postgres
    • jdk-6u31-linux-x64.bin jdk installer
    • role_command_order.json describes service INSTALL/START/CHECK dependencies in a stange format: blockedRole-blockedCommand: [blockerRole1-blockerCommand1, blockerRole2-blockerCommand2, ...]
  • /var/lib/ambari-server/resources/scripts
    • configs.sh pokes ambari API with curl for configurtion get/set/delete
    • hadoop.sh init.d type start/stop/status/meta-data script
    • add-host-component.sh pokes ambari API with curl
    • relocate_host_components.py python script via API calls (with urllib2, doesn't uses ambari-cli python lib)
    • install_flume.sh pokes ambari API with curl to install flume
  • /var/lib/ambari-server/resources/stacks/HDP/
    • 1.2.0
    • 1.3.2
    • 1.3.3
    • 2.0.6
      • metainfo.xml as simple as <active>true</active>
      • role_command_order.json
      • repos
      • services
        • GANGLIA
        • HBASE
        • HDFS
          • metainfo.xml describes <configuration-dependencies> and <components> like: NAMENODE/MASTER DATANODE/SLAVE HDFS_CLIENT/CLIENT
          • metrics.json ganglia and jmx metrics defined as java.lang:type=Memory.NonHeapMemoryUsage[used]
          • configuration
            • global.xml
            • core-site.xml
            • hdfs-site.xml
            • hadoop-policy.xml

Trobleshooting

You can start ambari in debug mode to get more detailed aoutput via:

ambari-server start --verbose --debug
# or for short
ambari-server start -v -g

Significant files/directories:

  • /var/log/ambari-server/ambari-server.log (for ambari server)
  • /var/log/ambari-agent/ambari-agent.log (for ambari agens)
  • /var/lib/ambari-agent/data/ (server action output)

SQL scripts to initialize psql DB

  • /var/lib/ambari-server/resources/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment