因?yàn)镃loudera的使用場(chǎng)景較多是私有云，本文介紹的是如何在離線環(huán)境下進(jìn)行部署的parcles方式。

硬件環(huán)境

本文不討論Hadoop角色分布，只描述如何部署Cloudera本身。

Cloudera角色分布：

軟件版本

OS: CentOS 7.2

JDK: 1.8.0_73

Cloudera: 5.8.2

依賴資源

• CDH-5.8.2-1.cdh5.8.2.p0.3-el7.parcel
• CDH-5.8.2-1.cdh5.8.2.p0.3-el7.parcel.sha1
• cloudera-manager-centos7-cm5.8.2_x86_64.tar.gz
• cm5.8.2-centos7.tar.gz
• manifest.json
• mysql-connector-java-5.1.34-bin.jar
• jdk-8u73-linux-x64.rpm

假設(shè)所有依賴包都已經(jīng)上傳至/home/admin/soft下，記得下載完CDH-5.8.2-1.cdh5.8.2.p0.3-el7.parcel.sha1后把文件名改為CDH-5.8.2-1.cdh5.8.2.p0.3-el7.parcel.sha，因?yàn)楹罄m(xù)進(jìn)行 parcel 版本比對(duì)的時(shí)候，只會(huì)對(duì) repo 目錄下的 *.sha 文件內(nèi)容進(jìn)行提取，比對(duì) parcel 的 sha 值，以確認(rèn)我們的 parcel 文件是正確可用沒有損毀的。

部署步驟

Step1. SSH免密打通

假設(shè)所有步驟都在admin賬戶下執(zhí)行，所有服務(wù)器的admin賬戶密碼統(tǒng)一，需要打通Cloudera Server至所有agent的SSH免密登錄，假設(shè)list_agents已經(jīng)包含了所有agent機(jī)器的列表：

192.168.1.1
192.168.1.3
192.168.1.4
192.168.1.5
192.168.1.6
192.168.1.7
192.168.1.8
192.168.1.9
192.168.1.10
192.168.1.11
192.168.1.12
192.168.1.13
192.168.1.14
192.168.1.15
192.168.1.16
192.168.1.17

在192.168.1.1上進(jìn)行執(zhí)行：

ssh 192.168.1.1
ssh-keygen
for agent in `cat list_agents`;do ssh-copy-id -i /home/admin/.ssh/id_rsa.pub admin@${agent};done;

當(dāng)然，我們也可以用expect來(lái)做自動(dòng)化，這里過(guò)程省略。

Step2. JDK安裝

cd /home/admin/soft
pscp -h list_agents jdk-8u73-linux-x64.rpm ~/
pssh -h list_agents -P "rpm -ivh /home/admin/soft/jdk-8u73-linux-x64.rpm"

修改環(huán)境變量：

# .bash_profile 
# Get the aliases and functions 
if [ -f ~/.bashrc ]; then 
    . ~/.bashrc 
fi 
# User specific environment and startup programs 
JAVA_HOME=/usr/java/jdk1.8.0_73 
PATH=$JAVA_HOME/bin:$PATH 
export PATH JAVA_HOME

分發(fā)環(huán)境變量配置：

pscp -h list_agents ~/.bash_profile /tmp 
pssh -h list_agents "sudo cp /tmp/.bash_profile ~/"

Step3. MySQL安裝并初始化數(shù)據(jù)庫(kù)，默認(rèn)用戶root無(wú)密碼

安裝MySQL，我們使用 mariadb：

ssh 192.168.1.2
sudo yum install mariadb-server

修改 /etc/my.cnf 以支持utf-8：

[mysql] 
default-character-set=utf8 
[mysqld] 
character_set_server=utf8 
init_connect='SET NAMES utf8'

啟動(dòng) mysql：

sudo systemctl start mariadb
sudo systemctl enable mariadb

初始化Cloudera Manager數(shù)據(jù)庫(kù)：

mysql -uroot
MariaDB > grant all privileges on *.* to 'root'@'%' identified by 'root'; 
MariaDB > grant all privileges on *.* to 'root'@'localhost' identified by 'root'; 
MariaDB > use mysql; 
MariaDB > update user set password=password('root') where user='root'; 
MariaDB > create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci; 
MariaDB > create database reports DEFAULT CHARSET utf8 COLLATE utf8_general_ci; 
MariaDB > create database navigator DEFAULT CHARSET utf8 COLLATE utf8_general_ci; 
MariaDB > create database navigatormeta DEFAULT CHARSET utf8 COLLATE utf8_general_ci; 
MariaDB > create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci; 
MariaDB > create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci; 
MariaDB > create database sentry DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

Step4. 離線資源準(zhǔn)備

在192.168.1.1上對(duì)離線包進(jìn)行解壓，并且分發(fā)：

sudo tar zxvf /home/admin/soft/cloudera-manager-centos7-cm5.8.2_x86_64.tar.gz -C /opt/
sudo cp /home/admin/soft/mysql-connector-java-5.1.34.jar /opt/cm-5.8.2/share/cmf/lib/

創(chuàng)建Cloudera Manager的初始化數(shù)據(jù)，mysql密碼在之前初始化了，是root：

sudo /opt/cm-5.8.2/share/cmf/schema/scm_prepare_database.sh mysql cm -h127.0.0.1 -uroot -p --port 3306 --scm-host 127.0.0.1 scm scm scm 
mysql -uroot -p
MariaDB > use cm; 
MariaDB > grant all PRIVILEGES on cm to scm;

修改 192.168.1.1 上的 /opt/cm-5.8.2/etc/cloudera-scm-agent/config.ini的server_host修改為主節(jié)點(diǎn)hostname(v001001.idc.domain.com)，將離線包中的CDH5相關(guān)的parcel包放到主節(jié)點(diǎn)192.168.1.1/opt/cloudera/parcel-repo/目錄中：

sudo mkdir -p /opt/cloudera/parcel-repo/ 
sudo cp /home/admin/soft/cdh5.8.2/CDH-5.8.2-1.cdh5.8.2.p0.3-el7.parcel /opt/cloudera/parcel-repo/ 
sudo cp /home/admin/soft/cdh5.8.2/CDH-5.8.2-1.cdh5.8.2.p0.3-el7.parcel.sha /opt/cloudera/parcel-repo/ 
sudo cp /home/admin/soft/cdh5.8.2/manifest.json /opt/cloudera/parcel-repo/

在主節(jié)點(diǎn)192.168.1.1上創(chuàng)建storage目錄：

sudo mkdir -p /var/lib/cloudera-scm-server

Step5. 啟動(dòng)Cloudera Server服務(wù)

在192.168.1.1上啟動(dòng)Cloudera Server：

sudo /opt/cm-5.8.2/etc/init.d/cloudera-scm-server start

在agent上把mysql-connector-java-5.1.34.jar拷貝至Cloudera Manager的lib目錄下：

pscp -h list_agents /home/admin/soft/mysql-connector-java-5.1.34.jar /tmp 
pssh -h list_agents -P "sudo mkdir -p /usr/share/cmf/lib"
pssh -h list_agents -P "sudo mkdir -p /usr/share/java" 
pssh -h list_agents -P "sudo cp /tmp/mysql-connector-java-5.1.34.jar /usr/share/cmf/lib"
pssh -h list_agents -P "sudo cp /tmp/mysql-connector-java-5.1.34.jar /usr/share/java/mysql-connector-java.jar"

Step6. 搭建臨時(shí)httpd服務(wù)器

搭建httpd服務(wù)器的目的是為了實(shí)現(xiàn)離線本地Repo安裝，需要把之前從cm5.8.2-centos7.tar.gz解壓縮至Root Dir下，假設(shè)Repo地址為http://192.168.1.1/cm/5.8.2/。

sudo yum install -y httpd
sudo tar zxvf cm5.8.2-centos7.tar.gz -C /var/www/html/
sudo rm -rf /var/run/yum.pid

Step7. 進(jìn)入圖形化界面進(jìn)行部署

默認(rèn)的Cloudera控制臺(tái)界面為 http://192.168.1.1:7180
初次登錄會(huì)要求輸入默認(rèn)的管理員賬戶名密碼，請(qǐng)切記。進(jìn)入圖形化界面后按照以下步驟進(jìn)行agent發(fā)現(xiàn)和集群初始化：

• 選擇 Yes, I accept the End User License Terms and Conditions.，然后一路 Continue；
• 在 Specify hosts for your CDH cluster installation. 界面輸入以下機(jī)器，并點(diǎn)擊 Search：

192.168.1.1
192.168.1.3
192.168.1.4
192.168.1.5
192.168.1.6
192.168.1.7
192.168.1.8
192.168.1.9
192.168.1.10
192.168.1.11
192.168.1.12
192.168.1.13
192.168.1.14
192.168.1.15
192.168.1.16
192.168.1.17

• 全選機(jī)器后，點(diǎn)擊 Continue；
• 選擇Use Parcels (Recommended)；
• 在More Options中去掉所有遠(yuǎn)程Repo配置；
• Select the version of CDH選擇CDH-5.8.2-1.cdh5.8.2.p0.3；
• Select the specific release of the Cloudera Manager Agent you want to install on your hosts.選擇Custom Repository，并且使用局域網(wǎng)httpd，地址為：http://192.168.1.1/cm/5.8.2/
• Install Oracle Java SE Development Kit (JDK)不勾選；
• Single User Mode不勾選；
• Login To All Hosts As:選擇admin，并錄入CentOS的admin密碼；

正常情況下會(huì)自動(dòng)進(jìn)行Cloudera安裝包分發(fā)，以及agent進(jìn)程啟動(dòng)。

Q: 安裝過(guò)程中如果遇到：/opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p0.3/meta/parcel.json 文件無(wú)法找到
A: 請(qǐng)把進(jìn)程停止后清除 uuid 之后再走一遍流程：

sudo /bin/systemctl stop cloudera-scm-agent
sudo rm /var/lib/cloudera-scm-agent/*
sudo /bin/systemctl start cloudera-scm-agent

如果無(wú)效，請(qǐng)?jiān)诎惭b向?qū)У?More Options 中重新指定 parcel repo 目錄，如果 repo 目錄空間不足也會(huì)導(dǎo)致 parcel.json 文件安裝失敗。重新指定的方法 Hosts -> Parcels -> Configuration -> Local Parcel Repository Path 修改為其他地址。并且查看 Remote Parcel Repository URLs 配置是否指向本地 httpd 服務(wù)器對(duì)應(yīng)目錄。
Q: 遇到 ProtocolError: <ProtocolError for 127.0.0.1/RPC2: 401 Unauthorized>
A: 請(qǐng)把 supervisor 殺掉后重試：

pid=`ps aux | grep "/usr/lib64/cmf/agent/build/env/bin/supervisord" | grep -v grep | awk '{print$2}'`
sudo kill -9 ${pid}

Step8.進(jìn)行Hadoop服務(wù)部署

這塊不在這里進(jìn)行描述，按照您的需要進(jìn)行部署。但是有一塊要注意，記得在部署服務(wù)前創(chuàng)建相應(yīng) lib 目錄并且把 mysql-connector 拷貝至對(duì)應(yīng)服務(wù)的 lib 目錄：

pscp -h list_agents /home/admin/soft/mysql-connector-java-5.1.34.jar /tmp
pssh -h list_agents "sudo mkdir -p /opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p0.3/lib/hive/lib/ /opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p0.3/lib/oozie/lib/ /var/lib/oozie" 
pssh -h list_agents "sudo cp /tmp/mysql-connector-java-5.1.34.jar /opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p0.3/lib/hive/lib/" 
pssh -h list_agents "sudo cp /tmp/mysql-connector-java-5.1.34.jar /opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p0.3/lib/oozie/lib/" 
pssh -h list_agents "sudo mkdir -p  /opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p0.3/lib/sentry/lib"
pssh -h list_agents "sudo cp /tmp/mysql-connector-java-5.1.34.jar /opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p0.3/lib/sentry/lib/" 
pssh -h list_agents "sudo cp /tmp/mysql-connector-java-5.1.34.jar /var/lib/oozie"
pssh -h list_agents "sudo rm -f /tmp/mysql-connector-java-5.1.34.jar"

Q: 部署 Oozie 的時(shí)候報(bào)錯(cuò)：mkdir: cannot create directory '/var/lib/oozie/tomcat-deployment': No such file or directory
A: 在 MySQL 中重建 oozie 這個(gè) database，然后 執(zhí)行：

sudo chown oozie:oozie /var/lib/oozie

Step9. 修改HUE和Oozie的時(shí)區(qū)

修改時(shí)區(qū)是針對(duì)國(guó)內(nèi)用戶的，如果不進(jìn)行HUE時(shí)區(qū)修改，會(huì)造成HUE中無(wú)法使用sqoop，報(bào)Sqoop error: Could not get connectors.；如果不進(jìn)行Oozie時(shí)區(qū)修改，那么所看到的Oozie日志時(shí)間戳?xí)e(cuò)誤。

HUE修改很簡(jiǎn)單，只需要在Cloudera控制臺(tái)HUE Service的configuration中對(duì)如下屬性進(jìn)行修改，并重啟服務(wù)即可：

time_zone = Asia/Shanghai

Oozie修改的話，也是在Cloudera控制臺(tái)HUE Service的configuration找到oozie-env.sh進(jìn)行修改，添加如下屬性：

<property> 
    <name>oozie.processing.timezone</name> 
    <value>GMT+0800</value> 
</property>

然后在每臺(tái)運(yùn)行Oozie的服務(wù)器上執(zhí)行，完成后重啟Oozie服務(wù)：

cd /opt/cloudera/parcels/CDH/lib/oozie/libext 
unzip ext-2.2.zip 
chown oozie:oozie -R ext-2.2

最后在Oozie Web Console中的Settings下修改Timezone為CST(Asia/Shanghai)。

附 完全卸載腳本

#!/bin/bash 
RELEASE_VERSION=5.8.2
RPM_CMA_VERSION=5.8.2-1.cm582.p0.17.el7.x86_64
RPM_CMD_VERSION=5.8.2-1.cm582.p0.17.el7.x86_64   
pssh -h list -P "sudo /bin/systemctl stop cloudera-scm-agent"

sudo /opt/cm-${RELEASE_VERSION}/etc/init.d/cloudera-scm-server stop 
pssh -h list -P "sudo umount /run/cloudera-scm-agent/process" 
pssh -h list -P "sudo rm -rf /usr/share/cmf /var/lib/cloudera* /var/cache/yum/x86_64/6/cloudera* /var/log/cloudera* /var/run/cloudera* /etc/cloudera* /usr/lib64/cmf /etc/init.d/cloudera-scm-agent /etc/rc3.d/S90cloudera-scm-agent /etc/cloudera-scm-agent" 
pssh -h list -P "sudo rpm -e --noscripts --nodeps cloudera-manager-agent-${RPM_CMA_VERSION}" 
pssh -h list -P "sudo rpm -e --noscripts --nodeps cloudera-manager-daemons-${RPM_CMD_VERSION}" 
pssh -h list -P "sudo rm -rf /var/lib/hadoop-* /var/lib/impala /var/lib/solr /var/lib/zookeeper /var/lib/hue /var/lib/oozie /var/lib/pgsql /var/lib/sqoop2 /data/dfs/ /data/impala/ /data/yarn/ /dfs/ /impala/ /yarn/ /var/run/hadoop-*/ /var/run/hdfs-*/ /usr/bin/hadoop* /usr/bin/zookeeper* /usr/bin/hbase* /usr/bin/hive* /usr/bin/hdfs /usr/bin/mapred /usr/bin/yarn /usr/bin/sqoop* /usr/bin/oozie /etc/hadoop* /etc/zookeeper* /etc/hive* /etc/hue /etc/impala /etc/sqoop* /etc/oozie /etc/hbase* /etc/hcatalog /var/lib/flume-ng /var/lib/hadoop* /var/lib/hue /var/lib/navigator /var/lib/oozie /var/lib/solr /var/lib/sqoop* /var/lib/zookeeper /var/lib/hbase /var/lib/hive /var/lib/impala /var/lib/spark" 
pssh -h list -P "sudo rm -rf /opt/cloudera" 
pssh -h list -P "sudo userdel -r cloudera-scm" 
pssh -h list -P "sudo rm -rf /usr/share/cmf /var/lib/cloudera* /var/cache/yum/x86_64/6/cloudera* /var/log/cloudera* /var/run/cloudera* /etc/cloudera* /usr/lib64/cmf" 
pssh -h list -P "sudo rm -rf /opt/cloudera /opt/cm-${RELEASE_VERSION}/" 

附 Agent 卸載腳本

/sbin/service cloudera-scm-agent stop
/bin/sed -e s/\(server_host=\).*/\1localhost/ -i /etc/cloudera-scm-agent/config.ini
/bin/yum -y erase cloudera-manager-agent
/bin/rm -rf /var/log/cloudera-scm-agent/
/bin/rm -rf /etc/cloudera-scm-agent/