ambari-2.6.2本地編譯

環(huán)境搭建

安裝jdk

(ambari 2.7以上必須使用jdk1.8編譯,其余的必須使用jdk1.7編譯)

tar -zxvf jdk1.7.0_60.tgz
ln -s jdk1.7.0_60 java 

安裝maven

Download maven(ambari要求maven版本3.3.9以上)本次使用的是3.5.4

tar -zxvf apache-maven-3.5.4-bin.tar.gz
ln -s apache-maven-3.5.4 maven

安裝nodejs

tar -xvf node-v8.11.4-linux-x64.tar.xz
ln -s node-v8.11.4-linux-x64 node

安裝ant

# https://ant.apache.org/bindownload.cgi
# 1.10.x 必須要jdk1.8版本,所以使用1.9.x
tar -zxvf apache-ant-1.9.13-bin.tar.gz
ln -s apache-ant-1.9.13 ant

設(shè)置jdk和maven的環(huán)境變量

export userEnvi=/home/compile/software
export JAVA_HOME=$userEnvi/java
export MAVEN_HOME=$userEnvi/maven
export NODE_HOME=$userEnvi/node
export ANT_HOME=$userEnvi/ant
export PATH=$JAVA_HOME/bin:$MAVEN_HOME/bin:$NODE_HOME/bin:$ANT_HOME/bin:$PATH

安裝python

Centos7自帶python2.7.5(ambari2.7以上需要python2.7以上環(huán)境,其余最低需要python2.6)

安裝git

yum install git

安裝setuptools

• python2.6 Download setuptools and run

sh setuptools-0.6c11-py2.6.egg

• python2.7 Download setuptools and run

sh setuptools-0.6c11-py2.7.egg

根據(jù)主機(jī)安裝的python版本進(jìn)行選擇

安裝rpmbuild

rpm -qa | grep rpm-build #檢測是否已安裝rpmbuild
yum install rpm-build    #如果沒安裝則手動(dòng)安裝

安裝g++

rpm -qa | grep gcc-c++ #檢測是否已安裝g++
yum install gcc-c++    #如果沒安裝則手動(dòng)安裝

源碼編譯

下載&編譯

從github上下載ambari本次編譯選用ambari2.6.2進(jìn)行編譯

tar -zxvf apache-ambari-2.6.2-src.tar.gz
cd apache-ambari-2.6.2-src
# 版本號必須要5位數(shù)字,因?yàn)閙etric模塊下有版本正則表達(dá)式檢測
mvn versions:set -DnewVersion=2.6.2.0.0

#Note: The ambari-metrics project is not wired up to the main ambari project. However there is a dependency on ambari-metrics-common to build the ambari-server RPM.
#Hence you also need to set ambari-metrics project version as well.

pushd ambari-metrics
mvn versions:set -DnewVersion=2.6.2.0.0
popd

mvn -B clean install package rpm:rpm -DnewVersion=2.6.2.0.0 -DskipTests -Dpython.ver="python >= 2.6"

# -rf : xxx 每次問題解決后，執(zhí)行下面命令可以繼續(xù)編譯
mvn -B install package rpm:rpm -DnewVersion=2.6.2.0.0 -DskipTests -Dpython.ver="python >= 2.6" -rf :ambari-server

問題總結(jié)

1. 在ambari-meteics設(shè)置版本時(shí) mvn versions:set -DnewVersion=2.6.2.1 錯(cuò)誤: Error resolving version for plugin 'org.apache.rat:apache-rat-plugin' from the repositories [local(/home/compile/.m2/repository),central(https://repo.maven.apache.org/maven2)]: Plugin not found in any plugin repository -> [Help 1]

# /home/compile/ambari/ambari-release-2.6.2/ambari-metrics/pom.xml 添加版本號 0.11
<groupId>org.apache.rat</groupId>
        <artifactId>apache-rat-plugin</artifactId>
        <version>0.11</version>

2. 在ambari-meteics設(shè)置版本時(shí) mvn versions:set -DnewVersion=2.6.2.1 錯(cuò)誤Error resolving version for plugin 'org.codehaus.mojo:versions-maven-plugin' from the repositories [local (/home/compile/.m2/repository), central (https://repo.maven.apache.org/maven2)]: Plugin not found in any plugin repository -> [Help 1]

<!-- 遇到以上錯(cuò)誤時(shí),可以修改maven的鏡像,設(shè)置為阿里鏡像 -->
<mirror>  
    <id>nexus-aliyun</id>  
    <mirrorOf>central</mirrorOf>    
    <name>Nexus aliyun</name>  
    <url>http://maven.aliyun.com/nexus/content/groups/public</url>  
</mirror>  

3. Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.0.1:rpm (default-cli) on project ambari: RPM build execution returned: '127' executing '/bin/sh -c cd /home/compile/ambari/ambari-release-2.6.2/target/rpm/ambari/SPECS && rpmbuild -bb --buildroot /home/compile/ambari/ambari-release-2.6.2/target/rpm/ambari/buildroot --define '_topdir /home/compile/ambari/ambari-release-2.6.2/target/rpm/ambari' --target noarch-redhat-linux ambari.spec' -> [Help 1]

#rpm 或者 rpm-build不存在需要手動(dòng)安裝
yum install rpm
yum install -y rpm-build.x86_64

4. Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.4:install-node-and-yarn (install node and yarn) on project ambari-web: Could not download Yarn: Could not download https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz: Received fatal alert: protocol_version -> [Help 1]

#在瀏覽器中下載https://github.com/yarnpkg/yarn/releases/download/v0.23.2/yarn-v0.23.2.tar.gz

mkdir -p /home/compile/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./

cp yarn-v0.23.2.tar.gz /home/compile/.m2/repository/com/github/eirslett/yarn/0.23.2/yarn-0.23.2./

5. Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:exec (Bower install) on project ambari-admin: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]

# 安裝git即可解決

6. around Ant part ...<exec dir="/home/compile/ambari/ambari-release-2.6.2/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/home/compile/ambari/ambari-release-2.6.2/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap" failonerror="true">... @ 4:299 in /home/compile/ambari/ambari-release-2.6.2/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml

# 安裝ant即可解決

7. around Ant part ...<get dest="/home/compile/ambari/ambari-release-2.6.2/ambari-metrics/ambari-metrics-timelineservice/target/embedded/hbase.tar.gz" src="https://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.4.0/tars/hbase/hbase-1.1.2.2.6.4.0-91.tar.gz" usetimestamp="true"/>... @ 5:270 in /home/compile/ambari/ambari-release-2.6.2/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download HBase.xml

# https://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.4.0/tars/hbase/hbase-1.1.2.2.6.4.0-91.tar.gz
# https://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.4.0/tars/phoenix/phoenix-4.7.0.2.6.4.0-91.tar.gz
# https://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.6.4.0/tars/hadoop/hadoop-2.7.3.2.6.4.0-91.tar.gz
# https://grafanarel.s3.amazonaws.com/builds/grafana-2.6.0.linux-x64.tar.gz

# 手動(dòng)下載以上tar.gz放入相應(yīng)位置,修改ambari-metric->pom.xml使用本地文件模式
 <hbase.tar>file:///home/compile/bak/hbase-1.1.2.2.6.4.0-91.tar.gz</hbase.tar>
 <hadoop.tar>file:///home/compile/bak/hadoop-2.7.3.2.6.4.0-91.tar.gz</hadoop.tar>
 <phoenix.tar>file:///home/compile/bak/phoenix-4.7.0.2.6.4.0-91.tar.gz</phoenix.tar>
 <grafana.tar>file:///home/compile/bak/grafana-2.6.0.linux-x64.tar.gz</grafana.tar>

8. around Ant part ...<exec dir="/home/compile/ambari/ambari-release-2.6.2/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/home/compile/ambari/ambari-release-2.6.2/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap" failonerror="true">... @ 4:299 in /home/compile/ambari/ambari-release-2.6.2/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml

# 缺少python  psutils模塊
wget https://pypi.python.org/packages/source/p/psutil/psutil-2.0.0.tar.gz
tar -xzvf psutil-2.0.0.tar.gz
cd psutil-2.0.0
python setup.py install
# 安裝python setup.py install時(shí)出現(xiàn)如下錯(cuò)誤 psutil/_psutil_linux.c:12:20: error: Python.h: No such file or directory
yum -y install python-devel

9. 編譯時(shí)出現(xiàn)maven-antrun-plugin Error excute ant tasks PermGen space

# 進(jìn)入maven安裝目錄 bin/mvn
export MAVEN_OPTS="-Xmx512m -XX:MaxPermSize=128m"

相關(guān)博客參考

ambari2.5.0源碼編譯
基于ubuntu的源碼編譯和安裝
ambari官方編譯環(huán)境