hadoop安裝遇到的一些障礙

一.start-dfs.sh(開啟不成功)

原因:1.dadoop? ?文件所屬者 與 執(zhí)行時 登陸者不一致(權(quán)限不足)

? ? ????????????報錯:

WARNING: HADOOP_SECURE_DN_USER has been replaced by HDFS_DATANODE_SECURE_USER. Using value of HADOOP_SECURE_DN_USER.

Starting namenodes on [centos7]

centos7: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

Starting datanodes

localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

Starting secondary namenodes [centos7]

centos7: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

log4j:ERROR setFile(null,true) call failed.

java.io.FileNotFoundException: /hadoop/hadoop-3.2.0/logs/fairscheduler-statedump.log (權(quán)限不夠)

at java.io.FileOutputStream.open0(Native Method)

at java.io.FileOutputStream.open(FileOutputStream.java:270)

at java.io.FileOutputStream.<init>(FileOutputStream.java:213)

at java.io.FileOutputStream.<init>(FileOutputStream.java:133)

at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)

at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)

at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)

at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)

at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)

at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)

at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)

at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)

at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)

at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)

at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)

at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)

at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)

at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)

at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)

at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)

at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)

at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)

at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)

at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)

at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)

at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:227)

at org.apache.hadoop.hdfs.tools.GetConf.<clinit>(GetConf.java:131)

情況一

二.使用root配置的hadoop并啟動會出現(xiàn)報錯

錯誤:

??????? Starting namenodes on [master]

ERROR: Attempting to operate on hdfs namenode as root


ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting operation.


Starting datanodes

ERROR: Attempting to operate on hdfs datanode as root


ERROR: but there is no HDFS_DATANODE_USER defined. Aborting operation.

??????? Starting secondary namenodes [slave1]

??????? ERROR: Attempting to operate on hdfs secondarynamenode as root

??????? ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting operation.

解決方法:

在/hadoop/sbin路徑下:?

將start-dfs.sh,stop-dfs.sh兩個文件頂部添加以下參數(shù)

HDFS_DATANODE_USER=root

HADOOP_SECURE_DN_USER=hdfs

HDFS_NAMENODE_USER=root

HDFS_SECONDARYNAMENODE_USER=root

start-yarn.sh,stop-yarn.sh頂部也需添加以下

YARN_RESOURCEMANAGER_USER=root

HADOOP_SECURE_DN_USER=yarn

YARN_NODEMANAGER_USER=root

2、添加1后出現(xiàn)以下錯誤

WARNING: HADOOP_SECURE_DN_USER has been replaced by HDFS_DATANODE_SECURE_USER. Using value of HADOOP_SECURE_DN_USER.

Starting namenodes on [mylinux_1]

mylinux_1: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

Starting datanodes

localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

Starting secondary namenodes [mylinux_1]

mylinux_1: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

2018-11-26 09:32:18,082 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Starting resourcemanager

Starting nodemanagers

localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

解決辦法:配置免密登錄(注意:對本機(jī)也需要配置)

ssh-keygen -t rsa

ssh-copy-id -i ~/.ssh/id_rsa.pub root@localhost


三.執(zhí)行jps 沒有 nodemanager

原因是:yarn-site.xml 配置出現(xiàn)了問題

在yarn-site.xml添加

<property>

? ? <name>yarn.nodemanager.resource.cpu-vcores</name>

? ? <value>你的cpu個數(shù)</value>

</property>

? ? ? ? <property>

? ? ? ? <name> yarn.nodemanager.resource.memory-mb </name>

? ? ? ? <value> 你的內(nèi)存</value>

? ? </property>


?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容