hadoop常見錯(cuò)誤

錯(cuò)誤一:本地運(yùn)行Java 代碼時(shí)出出現(xiàn):

2018-09-14 19:45:07,266 DEBUG org.apache.hadoop.util.Shell.checkHadoopHome() - Failed to detect a valid hadoop home directory

java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.

2018-09-14 19:45:07,275 ERROR org.apache.hadoop.util.Shell.getWinUtilsPath() - Failed to locate the winutils binary in the hadoop binary path

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

解決:把hadoop安裝目錄下的hadoop.dll考到system32目錄下


錯(cuò)誤二:運(yùn)行mapreduce

Job job_1536830416611_0021 failed with state FAILED due to: Application application_1536830416611_0021 failed 2 times due to Error launching appattempt_1536830416611_0021_000002. Got exception: org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.

This token is expired. current time is 1536878358609 found 1536837804741

Note: System times on machines may be out of sync. Check system time and time zones.

沒有同步時(shí)間

解決:

在namenode 和datanode所在的服務(wù)器上都輸入:

cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

ntpdate pool.ntp.org? 進(jìn)行時(shí)間同步

如果在ntpdate pool.ntp.org? 時(shí) 出現(xiàn)? the NTP socket is in use, exiting

用 ps -ef|grep ntp 查出進(jìn)程后 殺死


錯(cuò)誤三:

啟動(dòng)zookeeper 服務(wù)時(shí)

zkServer.sh status

ZooKeeper JMX enabled by default

Using config: /usr/local/zookeeper-3.4.10/bin/../conf/zoo.cfg

Error contacting service. It is probably not running.

刪除 /etc/hosts 中的 127.0.0.0拿一行

錯(cuò)誤四:

linux的shell腳本運(yùn)行出現(xiàn)失敗

shell 腳本在/usr/local/bin 下 不管在哪執(zhí)行都可以不用寫路徑

-bash: /usr/local/bin/xcall.sh: /bin/bash^M: bad interpreter:

在行末模式下輸入set ff回車,查看當(dāng)前的模式,如果時(shí)doc就通過set fileformat=unix修改成unix


錯(cuò)誤五:

Connection refused

? at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)

? at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)

? at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)

? at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)

18/09/19 19:41:53 INFO ipc.Client: Retrying connect to server: hadoop03/192.168.136.130:8485. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)(所有journalnode都得開啟,才能format namenode)

錯(cuò)誤六:

2018-09-19 20:22:04,275 FATAL org.apache.hadoop.ha.ZKFailoverController: Unable to start failover controller. Parent znode does not exist.

Run with -formatZK flag to initialize ZooKeeper.

(要保證所有設(shè)置的zookeper集群的zookeeper服務(wù)都開啟 才能 格式化 )

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容