前言
dolphinscheduler安裝版本：3.1.8
hive版本：2.1.1-cdh6.3.2
發(fā)現(xiàn)問題&分析問題
- 添加數(shù)據(jù)源報錯
[ERROR] 2021-01-08 10:55:40.553 org.apache.dolphinscheduler.api.service.DataSourceService:
[433] - Could not establish connection to jdbc:hive2://10.0.4.78:10000/vfs_dw: Required field 'client_protocol' is unset!
Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=vfs_dw})
java.sql.SQLException: Could not establish connection to jdbc:hive2://10.0.4.78:10000/vfs_dw:
Required field 'client_protocol' is unset!
Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=vfs_dw})
2.運行任務報錯
1.Dolphinscheduler任務啟動報錯，協(xié)議錯誤
2.Dolphinscheduler任務全部啟動異常，所有實例都變成提交成功，但是沒有日志，也不會提交
解決辦法
解決1：將hive-jdbc-2.1.1-cdh6.3.2-standalone.jar 放到api和work的lib目錄，刪除多余hive jar包
發(fā)現(xiàn)Hive數(shù)據(jù)可以配置成功，但是ds所有工作流啟動不了了，啟動就變成提交成功，沒有日志。
解決2：將cdh所有hive-*.jar都放到api和work的lib目錄，刪除多余hive jar包
發(fā)現(xiàn)work啟動異常，日志出現(xiàn)hive-jdbc-2.1.1-cdh6.3.2-standalone相關(guān)報錯
Exception in thread "Worker-Server" java.lang.IllegalArgumentException:
LoggerFactory is not a Logback LoggerContext but Logback is on the classpath.
Either remove Logback or the competing implementation
(class org.apache.logging.slf4j.Log4jLoggerFactory loaded from file:/opt/software/dolphinscheduler-3.1.8/worker-server/libs/hive-jdbc-2.1.1-cdh6.3.2-standalone.jar).
If you are using WebLogic you will need to add 'org.slf4j' to prefer-application-packages in WEB-INF/weblogic.xml: org.apache.logging.slf4j.Log4jLoggerFactory
解決3：將cdh hive包選擇性替換到api和work的lib目錄
成功解決
#api-server
#備份libs
cp -ar /opt/software/dolphinscheduler-3.1.8/api-server/libs/ /opt/software/dolphinscheduler-3.1.8/api-server/libs-bac
#復制到api-server
cp /opt/cloudera/parcels/CDH/jars/hive-jdbc-2.1.1-cdh6.3.2-standalone.jar /opt/software/dolphinscheduler-3.1.8/api-server/libs/
#權(quán)限改為dolphinscheduler
chown dolphinscheduler:dolphinscheduler /opt/software/dolphinscheduler-3.1.8/api-server/libs/hive-*.jar
#刪除ds自帶的所有hive錯誤版本相關(guān)驅(qū)動依賴
rm -rf /opt/software/dolphinscheduler-3.1.8/api-server/libs/hive-*2.3.9.jar
#check
ll /opt/software/dolphinscheduler-3.1.8/api-server/libs/|grep hive
#備份libs
cp -ar /opt/software/dolphinscheduler-3.1.8/worker-server/libs/ /opt/software/dolphinscheduler-3.1.8/worker-server/libs-bac
#worker-server
#刪除ds自帶的所有hive錯誤版本相關(guān)驅(qū)動依賴
rm -rf /opt/software/dolphinscheduler-3.1.8/worker-server/libs/hive-*.jar
#復制到worker-server
cp -n /opt/cloudera/parcels/CDH/jars/hive-common*.jar /opt/software/dolphinscheduler-3.1.8/worker-server/libs/
cp -n /opt/cloudera/parcels/CDH/jars/hive-jdbc-2.1.1-cdh6.3.2.jar /opt/software/dolphinscheduler-3.1.8/worker-server/libs/
cp -n /opt/cloudera/parcels/CDH/jars/hive-metastore*.jar /opt/software/dolphinscheduler-3.1.8/worker-server/libs/
cp -n /opt/cloudera/parcels/CDH/jars/hive-serde*.jar /opt/software/dolphinscheduler-3.1.8/worker-server/libs/
cp -n /opt/cloudera/parcels/CDH/jars/hive-service*.jar /opt/software/dolphinscheduler-3.1.8/worker-server/libs/
cp -n /opt/cloudera/parcels/CDH/jars/hive-service-rpc*.jar /opt/software/dolphinscheduler-3.1.8/worker-server/libs/
cp -n /opt/cloudera/parcels/CDH/jars/hive-storage-api*.jar /opt/software/dolphinscheduler-3.1.8/worker-server/libs/
#權(quán)限改為dolphinscheduler
chown dolphinscheduler:dolphinscheduler /opt/software/dolphinscheduler-3.1.8/worker-server/libs/hive-*.jar
#check
ll /opt/software/dolphinscheduler-3.1.8/worker-server/libs/|grep hive
