###修改 hostname
##修改 master 的 hostname
# vim /etc/hostname
master
# hostname -b master
## 退出重新登陆 master, 可以看到prompt里面的 hostname 已经变成 master了
##同样去修改 slave01 的 hostname
# vim /etc/hostname
slave01
# hostname -b slave01
## 退出重新登陆 slave01, 可以看到prompt里面的 hostname 已经变成 slave01 了###修改 /etc/hosts, 包含如下内容
ambari 要求使用 FQDN 来指定服务器, 所以在hosts文件里面的第二列写上 FQDN, 注意是第二列.
10.10.245.83 master.example.com master
10.10.245.85 slave01.example.com slave01
127.0.0.1 localhost配置 ubuntu 源, 这里使用 aliyun 的源:
deb http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse注: 如果使用 163 的源, 可能会在 $ sudo apt-get update 出现错误, 这会
导致后面的网页部署过程无法进行下去. 当然如果执行 update 的时候, 没有报
错, 使用 163 的源也是ok的.
配置 ambari, HDP, HDP-UTILS 的源:
在目录 /etc/apt/sources.list.d 下新建一个文件 ambari.list, 添加如
下内容:
### ambari repo
deb http://10.10.245.224/hdp/ambari/ubuntu14/2.x/updates/2.1.2 Ambari main
deb http://10.10.245.224/hdp/HDP-UTILS-1.1.0.20/repos/ubuntu14 HDP-UTILS main
deb http://10.10.245.224/hdp/HDP/ubuntu14/2.x/updates/2.3.2.0 HDP main其中 http://10.10.245.224/hdp 是之前配置的 ambari 本地镜像服务地址.
注意: 必须严格按照这个文件路径操作, ambari-server 的程序 里面写死了从这个路径把 ambari.list 拷贝到slave节点的对应目录.
###配置 ntp 服务, 略.
###外网依赖
安装过程中, 还是会依赖外网的一些资源, 目前暂时无法做到无网络的情况下安 装完成.
依赖的资源记录如下:
# sudo apt-get install ambari-server
Get:1 http://10.10.245.224/hdp/ambari/ubuntu14/2.x/updates/2.1.2/ Ambari/main ambari-server amd64 2.1.2-377 [400 MB]
Get:2 http://mirrors.163.com/ubuntu/ trusty-security/main libpq5 amd64 9.3.10-0ubuntu0.14.04 [81.5 kB]
Get:3 http://mirrors.163.com/ubuntu/ trusty-updates/main postgresql-client-common all 154ubuntu1 [25.4 kB]
Get:4 http://mirrors.163.com/ubuntu/ trusty-security/main postgresql-client-9.3 amd64 9.3.10-0ubuntu0.14.04 [783 kB]
Get:5 http://mirrors.163.com/ubuntu/ trusty/main ssl-cert all 1.0.33 [16.6 kB]
Get:6 http://mirrors.163.com/ubuntu/ trusty-updates/main postgresql-common all 154ubuntu1 [103 kB]
Get:7 http://mirrors.163.com/ubuntu/ trusty-security/main postgresql-9.3 amd64 9.3.10-0ubuntu0.14.04 [2,669 kB]
Get:8 http://mirrors.163.com/ubuntu/ trusty-updates/main postgresql all 9.3+154ubuntu1 [5,038 B]###配置 mysql
创建新的数据库 ambari, 创建新的用户 amabri:
# mysql -uroot -p
mysql> CREATE DATABASE ambari;
mysql> GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'%' IDENTIFIED BY
'ambaripw' WITH GRANT OPTION;
mysql> flush privileges;修改 mysql 配置文件, 允许从外部连接 mysql:
# vim /etc/mysql/my.cnf
## 将下面这项改成 0.0.0.0
bind-address = 0.0.0.0
## 重启mysql
# service mysql stop
# service mysql start测试从另一台机器能够登陆 mysql, 否则后面的安装过程中, 会报这个错误: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
###ambari-server setup
会从 hortonworks 镜像下载 JDK (135.8MB), 当然也可以选择自定义JDK, 不过我没有尝试 这个选项.
下载jdk, 安装 mysql jdbc, 用户, 建立数据库表.
# ambari-server setup
Using python /usr/bin/python2.7
Setup ambari-server
Checking SELinux...
WARNING: Could not run /usr/sbin/sestatus: OK
Customize user account for ambari-server daemon [y/n] (n)? n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 2
To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)? y
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-7u67-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
jdk-7u67-linux-x64.tar.gz... 100% (135.8 MB of 135.8 MB)E
Successfully downloaded JDK distribution to /var/lib/ambari-server/resources/jdk-7u67-linux-x64.tar.gz
Installing JDK to /usr/jdk64/
Successfully installed JDK to /usr/jdk64/
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/UnlimitedJCEPolicyJDK7.zip to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
Successfully downloaded JCE Policy archive to /var/lib/ambari-server/resources/UnlimitedJCEPolicyJDK7.zip
Installing JCE policy...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
input not recognized, please try again:
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database name (ambari): ambari
Username (ambari): ambari
Enter Database Password (bigdata):
Re-enter password:
Configuring ambari database...
WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
Press <enter> to continue.
Copying JDBC drivers to server resources...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
..ambari-admin-2.1.2.377.jar
....
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.# ambari-server start
Using python /usr/bin/python2.7
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start....................
Ambari Server 'start' completed successfully.如果报错 java.lang.ClassNotFoundException: com.mysql.jdbc.Driver, 需要查看如下几个目录, 确保 JDBC 在这几个目录下面:
- /var/lib/ambari-server/resources
- /usr/lib/ambari-server
如果安装成功, 用浏览器访问: http://10.10.245.83:8080/, 如果无法访问,
查看日志 /var/log/ambari-server/ambari-server.log 找原因.
###查看 ambari 的安装目录
# find / -iname "*ambari*" -type d
/etc/ambari-server
/var/lib/ambari-server
/var/lib/ambari-server/resources/views/work/ADMIN_VIEW{2.1.2}/META-INF/maven/org.apache.ambari
/var/lib/ambari-server/resources/views/work/ADMIN_VIEW{2.1.2}/META-INF/maven/org.apache.ambari/ambari-admin
/var/lib/ambari-server/resources/views/work/ADMIN_VIEW{2.1.2}/views/ambariViews
/var/lib/ambari-server/resources/common-services/AMBARI_METRICS
/var/lib/ambari-server/resources/stacks/HDP/2.0.6/services/AMBARI_METRICS
/var/lib/mysql/ambari
/var/log/ambari-server
/run/ambari-server
/usr/lib/ambari-server
/usr/lib/ambari-server/lib/ambari_simplejson
/usr/lib/ambari-server/lib/ambari_commons
/usr/lib/ambari-server/lib/ambari_jinja2
/usr/lib/ambari-server/web/data/ambari_components
/usr/lib/ambari-server/web/data/services/metrics/ambari_metrics
/usr/lib/python2.6/site-packages/ambari_server登陆 http://10.10.245.83:8080/, 用户名密码为: admin:admin
####Name your cluster: ambariQy
####Select Stack: HDP 2.3
Advanced Repository Options:
取消勾选其他发行版的源, 修改 ubuntu14 的源如下:
HDP-2.3 对应的url原本为
http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.3.4.0,
改为 http://10.10.245.224/hdp/HDP/ubuntu14/2.x/updates/2.3.2.0
HDP-UTILS-1.1.0.20 对应的url原本为
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu12,
改为 http://10.10.245.224/hdp/HDP-UTILS-1.1.0.20/repos/ubuntu14
Next
####Install Options
Target Hosts
master.example.com
slave01.example.com
Provide your SSH Private Key to automatically register hosts:
SSH User Account: root
Register and Confirm:
####Confirm Hosts
不行就查看日志, 解决错误以后, Retry. 常见的错误如下:
Trouble shooting:
#####错误1:没有找到 ambari.list 文件
Command start time 2015-12-30 15:10:51
chmod: cannot access ‘/var/lib/ambari-agent/data’: No such file or directory
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2015-12-30 15:10:51
-------------------
Command start time 2015-12-30 15:10:54
/etc/apt/sources.list.d/ambari.list: No such file or directory
scp /etc/apt/sources.list.d/ambari.list
host=master.example.com, exitcode=1
Command end time 2015-12-30 15:10:54
-------------------
Command start time 2015-12-30 15:10:54
mv: cannot stat ‘/var/lib/ambari-agent/tmp/ambari1451459454.list’: No such file or directory
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=1
Command end time 2015-12-30 15:10:55
----------------
Command start time 2015-12-30 15:10:55
chmod: cannot access ‘/etc/apt/sources.list.d/ambari.list’: No such file or directory
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=1
Command end time 2015-12-30 15:10:55
出现这个问题的原始是没有 /etc/apt/sources.list.d/ambari.list 文件,
按照之前说明的去创建这个文件.
#####错误2: 执行 apt-get update 出错
==========================
Update apt cache of repository...
==========================
Command start time 2015-12-30 15:18:20
0% [Working]
Hit http://10.10.245.224 Ambari InRelease
0% [Waiting for headers]
Hit http://10.10.245.224 HDP-UTILS InRelease
Hit http://10.10.245.224 HDP InRelease
0% [Working]
0% [InRelease gpgv 3,187 B]
30% [Working]
30% [InRelease gpgv 7,439 B]
Hit http://10.10.245.224 Ambari/main amd64 Packages
30% [InRelease gpgv 7,439 B]
38% [Working]
99% [Working]
99% [InRelease gpgv 7,391 B] [Waiting for headers]
99% [Waiting for headers]
Hit http://10.10.245.224 HDP-UTILS/main amd64 Packages
99% [Waiting for headers]
100% [Waiting for headers]
Err http://10.10.245.224 Ambari/main i386 Packages
404 Not Found
100% [Waiting for headers]
Hit http://10.10.245.224 HDP/main amd64 Packages
100% [Waiting for headers]
100% [Packages 119 kB] [Waiting for headers]
100% [Waiting for headers]
Err http://10.10.245.224 HDP-UTILS/main i386 Packages
404 Not Found
100% [Working]
Err http://10.10.245.224 HDP/main i386 Packages
404 Not Found
100% [Waiting for headers]
Ign http://10.10.245.224 Ambari/main Translation-en_US
100% [Waiting for headers]
Ign http://10.10.245.224 Ambari/main Translation-en
100% [Waiting for headers]
Ign http://10.10.245.224 HDP-UTILS/main Translation-en_US
100% [Working]
Ign http://10.10.245.224 HDP-UTILS/main Translation-en
100% [Working]
Ign http://10.10.245.224 HDP/main Translation-en_US
100% [Working]
Ign http://10.10.245.224 HDP/main Translation-en
100% [Working]
W: Failed to fetch http://10.10.245.224/hdp/ambari/ubuntu14/2.x/updates/2.1.2/dists/Ambari/main/binary-i386/Packages 404 Not Found
W: Failed to fetch http://10.10.245.224/hdp/HDP-UTILS-1.1.0.20/repos/ubuntu14/dists/HDP-UTILS/main/binary-i386/Packages 404 Not Found
W: Failed to fetch http://10.10.245.224/hdp/HDP/ubuntu14/2.x/updates/2.3.2.0/dists/HDP/main/binary-i386/Packages 404 Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=100
Command end time 2015-12-30 15:18:21
这个错误的原始是执行 $ sudo apt-get update 出现了问题.
问题的原因:
- 163 的源有问题, 改成 aliyun 的源
- 自己做的镜像里面, 没有 HDP-UTILS i386 架构的包, 禁用 apt-get 同步 i386 的包
以下为具体过程:
把 163 的源改成 aliyun 的源:
deb http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse
那个 i386 not found 的问题, 让执行 apt-get update 的时候, 不要更新 i386 的索引
# dpkg --remove-architecture i386
可能需要重新签名:
# apt-key adv --recv-keys --keyserver keyserver.ubuntu.com B9733A7A07513CAD
手动测试下 $ sudo apt-get update 是否还会报这种错误:
W: Failed to fetch
http://10.10.245.224/hdp/ambari/ubuntu14/2.x/updates/2.1.2/dists/Ambari/main/binary-i386/Packages
404 Not Foun
确保不再报错, 否则安装无法进行下去.
每台要安装的服务器, 都需执行上述两个步骤.
Retry
Success
Next
#####Choose Services:
根据需要选择
#####Assign Masters:
将 hive 相关的服务放到 slave01 上, 因为 ambari 不允许 hive meta store 安装到 ambari 所在机器的 mysql 服务器上面.
如果把hive相关服务配置到 master机器上, 安装界面执行到后面的时候, 会提 示如下错误:
New MySQL Database for Hive Conflict
You cannot install a "New MySQL Database" for Hive on the same host as
the Ambari Server MySQL Server. Please go back to <b>Assign
Masters</b> and reassign Hive to another host <b>OR</b> choose
"Existing MySQL Database" option to specify the Database Credentials
and URL for the Ambari Server MySQL Server instance. If you choose
"Existing MySQL Database" option, you need to perform the Hive
prerequisite steps to prepare the MySQL Server instance for Hive.
无法忽略这个提醒, 只好在 slave01 上也安装个 mysql.
mysql 安装方法同前述.
# mysql -uroot -p
mysql> GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hivepw' WITH GRANT OPTION;
mysql> flush privileges;注: 正常情况下, 使用 ambari 来搭建 Hadoop 平台的时候, 不会把 ambari 的 master 节点作为 Hadoop 的 master 节点. 这里我只是尝试下安装 ambari, 所 以这样配置了.
#####Assign Slaves and Clients:
#####Customize Services:
Hive 标签上有个通知, 点进去, 因为需要设置 mysql 连接参数, 进入设置下, 密码设置成 hivepw, 需要手动去创建新的用户 hive, 密码为 hivepw:
Review -> Next -> Deploy
等待安装完成
访问 http://10.10.245.83:8080/#/main/dashboard/metrics 查看集群状态.
完.