編譯 apache Oozie 4.3.0

文章開(kāi)始前先吐槽下Oozie居然要自己編譯。官方文檔

1.環(huán)境準(zhǔn)備

  • Unix box

  • Java JDK 1.7+(本文用的是1.8)

  • maven (現(xiàn)在一般是3.3.9)

  • apache Oozie 4.3.0

    官方文檔有要求安裝pig,其實(shí)沒(méi)什么卵用。

2.MAVEN配置

maven安裝過(guò)程這里就省略了。接下來(lái)配置maven mirror,可以配置成阿里云的。

<mirror>
     <id>nexus</id>
     <mirrorOf>*</mirrorOf> 
     <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
</mirror>

3.編譯

3.1.解壓Oozie

tar -xzvf oozie-4.3.0.tar.gz

3.2.修改pom.xml

我的java版本是1.8的,Oozie默認(rèn)的java版本是1.7

cd oozie-4.3.0
vi pom.xml
<targetJavaVersion>1.7</targetJavaVersion>      ->      <targetJavaVersion>1.8</targetJavaVersion>
<sourceJavaVersion>1.7</sourceJavaVersion>      ->      <sourceJavaVersion>1.8</sourceJavaVersion> 

3.3.準(zhǔn)備缺少的dependency

${mvn_repo}代表本地maven倉(cāng)庫(kù)地址

3.3.1 pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
mkdir -p ${mvn_repo}/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde

cd ${mvn_repo}/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde

wget http://repo.spring.io/plugins-release/org/pentaho/pentaho-aggdesigner-\
algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
3.3.2 doxia-module-twiki-1.0-alpha-9.2y.jar
mkdir -p ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y
                         
cd ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y
                         
wget https://repository.cloudera.com/content/repositories/releases/org/\
apache/maven/doxia/doxia-module-twiki/1.0-alpha-9.2y/doxia-module-twiki-1.0-alpha-9.2y.jar --no-check-certificate
3.3.3 doxia-core-1.0-alpha-9.2y.jar
mkdir -p ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki

cd ${mvn_repo}/org/apache/maven/doxia/doxia-module-twiki

wget https://repository.cloudera.com/content/repositories\
/releases/org/apache/maven/doxia/doxia-core/1.0-alpha-9.2y/doxia-core-1.0-alpha-9.2y.jar --no-check-certificate

3.4.編譯Oozie

cd oozie-4.3.0
bin/mkdistro.sh  -DskipTests -Puber -Phadoop-2 \
-Dhadoop.version=2.7.2 \
-Dhadoop.auth.version=2.7.2 \
-Ddistcp.version=2.7.2 \
-Dsqoop.version=1.4.3 \
-Dhive.version=1.2.0 \
-Dhbase.version=0.94.27

hadoop version需要改成自己安裝hadoop version,其他的也是,沒(méi)安裝也沒(méi)關(guān)系。

經(jīng)過(guò)漫長(zhǎng)的等待。。。

[INFO] Apache Oozie Main .................................. SUCCESS [  0.601 s]
[INFO] Apache Oozie Hadoop Utils hadoop-2-4.3.0 ........... SUCCESS [  1.371 s]
[INFO] Apache Oozie Hadoop Distcp hadoop-2-4.3.0 .......... SUCCESS [  0.061 s]
[INFO] Apache Oozie Hadoop Auth hadoop-2-4.3.0 Test ....... SUCCESS [  0.191 s]
[INFO] Apache Oozie Hadoop Libs ........................... SUCCESS [  0.011 s]
[INFO] Apache Oozie Client ................................ SUCCESS [  8.851 s]
[INFO] Apache Oozie Share Lib Oozie ....................... SUCCESS [  1.710 s]
[INFO] Apache Oozie Share Lib HCatalog .................... SUCCESS [  1.460 s]
[INFO] Apache Oozie Share Lib Distcp ...................... SUCCESS [  0.443 s]
[INFO] Apache Oozie Core .................................. SUCCESS [ 29.701 s]
[INFO] Apache Oozie Share Lib Streaming ................... SUCCESS [  2.492 s]
[INFO] Apache Oozie Share Lib Pig ......................... SUCCESS [  2.425 s]
[INFO] Apache Oozie Share Lib Hive ........................ SUCCESS [  3.291 s]
[INFO] Apache Oozie Share Lib Hive 2 ...................... SUCCESS [  2.734 s]
[INFO] Apache Oozie Share Lib Sqoop ....................... SUCCESS [  1.063 s]
[INFO] Apache Oozie Examples .............................. SUCCESS [  2.499 s]
[INFO] Apache Oozie Share Lib Spark ....................... SUCCESS [  3.877 s]
[INFO] Apache Oozie Share Lib ............................. SUCCESS [ 12.781 s]
[INFO] Apache Oozie Docs .................................. SUCCESS [  5.929 s]
[INFO] Apache Oozie WebApp ................................ SUCCESS [ 23.219 s]
[INFO] Apache Oozie Tools ................................. SUCCESS [  1.896 s]
[INFO] Apache Oozie MiniOozie ............................. SUCCESS [  0.693 s]
[INFO] Apache Oozie Distro ................................ SUCCESS [ 17.542 s]
[INFO] Apache Oozie ZooKeeper Security Tests .............. SUCCESS [  1.894 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

編譯好的包在 oozie-4.3.0/distro/target/oozie-4.3.0-distro.tar.gz

3.5.可能出現(xiàn)的錯(cuò)誤

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (default) on project\
 oozie-distro: An Ant BuildException has occured: java.net.ConnectException: 連接超時(shí) (Connection timed out) -> [Help]

這里需要下載tomcat,可能因?yàn)楣镜木W(wǎng)有代理導(dǎo)致java連接超時(shí),那就手動(dòng)下載一下吧。。。

cd oozie-4.3.0/distro/downloads
wget http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.47/bin/apache-tomcat-6.0.47.tar.gz
mv apache-tomcat-6.0.47.tar.gz tomcat-6.0.47.tar.gz

需要重命名下文件名,然后修改oozie-distro.xml,刪了下面三行代碼,然后再編譯一次吧。

vi oozie-4.3.0/distro/oozie-distro.xml

<!--<mkdir dir="downloads"/>
<get src="http://archive.apache.org/dist/tomcat/tomcat-6/v${tomcat.version}/bin/apache-tomcat-${tomcat.version}.tar.gz"
     dest="downloads/tomcat-${tomcat.version}.tar.gz" verbose="true" skipexisting="true"/>-->
最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書(shū)系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容