一、Hadoop3.x介紹

Hadoop是用Java語言實(shí)現(xiàn)的，允許用戶使用簡(jiǎn)單的編程模型實(shí)現(xiàn)跨機(jī)器集群對(duì)海量數(shù)據(jù)進(jìn)行分布式計(jì)算處理的框架。Hadoop3.x主要包含如下三個(gè)組件：

• Hadoop HDFS，分布式文件存儲(chǔ)系統(tǒng)，用來解決海量數(shù)據(jù)的存儲(chǔ)，確保存儲(chǔ)具有可擴(kuò)展性、高效及可靠性。
• Hadoop YARN，集群資源管理和任務(wù)調(diào)度框架，用來解決資源任務(wù)調(diào)度的問題。
• Hadoop MapReduce，分布式計(jì)算框架，用來解決海量數(shù)據(jù)的高效計(jì)算問題。

Hadoop是整個(gè)大數(shù)據(jù)生態(tài)的底層支撐技術(shù)，MapReduce作為第一代離線計(jì)算框架，因?yàn)槠湫实膯栴}，現(xiàn)在幾乎沒有公司使用了，但是其思想和編程模型是學(xué)習(xí)大數(shù)據(jù)不能繞開的，HDFS和YARN，則是大數(shù)據(jù)生態(tài)的核心技術(shù)，無論計(jì)算框架如何更替，這兩者始終是大數(shù)據(jù)生態(tài)的基石。

二、Hadoop3.x集群安裝

Hadoop集群的安裝其實(shí)就是HDFS集群和YARN集群的安裝，這兩個(gè)集群在邏輯上分離，但在物理上通常是在一起的。

HDFS集群中的角色有：

• NameNode，主要負(fù)責(zé)元數(shù)據(jù)的存放；
• SecondNameNode，協(xié)助NameNode做一些持久化的工作；
• DataNode，主要用來存放數(shù)據(jù)；

YARN集群中的角色有：

• ResoureManager，資源管理器，負(fù)責(zé)統(tǒng)一調(diào)度資源；
• NodeManager，負(fù)責(zé)具體每個(gè)節(jié)點(diǎn)上任務(wù)的資源啟停和報(bào)告；

關(guān)于集群中每個(gè)角色的作用和詳細(xì)工作原理，此處做簡(jiǎn)單了解即可，詳細(xì)內(nèi)容后續(xù)有文章會(huì)專門介紹。所以Hadoop集群就等于HDFS集群+YARN集群，而MapReduce只是作為一個(gè)計(jì)算框架，其程序本質(zhì)上就是Java程序，會(huì)被分發(fā)到集群中的節(jié)點(diǎn)上去參與計(jì)算而已。

我們專門在阿里云上購買了三臺(tái)服務(wù)器，都是1C2G Linux CentOS7.5 X64的系統(tǒng)，分別命名為node1（172.24.38.209），node2（172.24.38.210），node3（172.24.38.211），確保三臺(tái)服務(wù)器相互都能聯(lián)通，然后我們對(duì)于這三臺(tái)服務(wù)器組件集群的規(guī)劃如下：

Hadoop集群節(jié)點(diǎn)規(guī)劃

第一步，我們需要在每個(gè)節(jié)點(diǎn)上安裝JDK運(yùn)行環(huán)境，假設(shè)我們下載的JDK放在/root/soft目錄下。

#進(jìn)入JDK所在目錄
cd /root/soft

#解壓
tar zxvf jdk-8u65-linux-x64.tar.gz

#配置環(huán)境變量
vim /etc/profile
export JAVA_HOME=/root/soft/jdk1.8.0_241
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
source /etc/profile

#驗(yàn)證
[root@iZuf6gmsvearrd5uc3emkyZ soft]# java -version
java version "1.8.0_241"
Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode)

第二步，我們需要在每個(gè)節(jié)點(diǎn)上確保時(shí)間同步和關(guān)閉防火墻。

#時(shí)間同步
ntpdate ntp5.aliyun.com

# 防火墻關(guān)閉
#查看防火墻狀態(tài)
firewall-cmd --state
#停止firewalld服務(wù)
systemctl stop firewalld.service  
#開機(jī)禁用firewalld服務(wù)
systemctl disable firewalld.service  

第三步，我們需要開通node1到node1、node2、node3的ssh免密連接服務(wù)。

# 生成公鑰密鑰，全部回車下一步即可
ssh-keygen -t rsa
ssh-keygen -t dsa

#開通node1到node1的免密ssh
[root@iZuf6gmsvearrd5uc3emkyZ soft]# ssh-copy-id 172.24.38.209
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@172.24.38.209's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '172.24.38.209'"
and check to make sure that only the key(s) you wanted were added.

#開通node1到node2的免密ssh
[root@iZuf6gmsvearrd5uc3emkyZ soft]# ssh-copy-id 172.24.38.210
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '172.24.38.210 (172.24.38.210)' can't be established.
ECDSA key fingerprint is SHA256:ah4dSYvdlmiJv/Q8aJ5Vdm/PtYGCLE61/hl8waEeeSg.
ECDSA key fingerprint is MD5:4b:53:93:61:2b:a7:6d:79:67:c4:54:ca:24:11:86:26.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@172.24.38.210's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '172.24.38.210'"
and check to make sure that only the key(s) you wanted were added.

#開通node1到node3的免密ssh
[root@iZuf6gmsvearrd5uc3emkyZ soft]# ssh-copy-id 172.24.38.211
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '172.24.38.211 (172.24.38.211)' can't be established.
ECDSA key fingerprint is SHA256:F1oP0hFY+V3VHUL5rOSLEeTCv3m+y92u5RCW6RpBNDI.
ECDSA key fingerprint is MD5:ed:c7:a5:0b:f2:25:71:7b:fc:a8:e1:ce:fd:eb:19:7b.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@172.24.38.211's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '172.24.38.211'"
and check to make sure that only the key(s) you wanted were added.

#node1免密ssh到node1實(shí)驗(yàn)
[root@iZuf6gmsvearrd5uc3emkyZ soft]# ssh 172.24.38.209
Last login: Wed Nov  2 15:31:22 2022 from 172.24.38.209

Welcome to Alibaba Cloud Elastic Compute Service !

[root@iZuf6gmsvearrd5uc3emkyZ ~]# exit
logout
Connection to 172.24.38.209 closed.

#node1免密ssh到node2實(shí)驗(yàn)
[root@iZuf6gmsvearrd5uc3emkyZ soft]# ssh 172.24.38.210
Last login: Wed Nov  2 15:13:46 2022 from 122.193.199.200

Welcome to Alibaba Cloud Elastic Compute Service !

[root@iZuf6gmsvearrd5uc3emkzZ ~]# exit
logout
Connection to 172.24.38.210 closed.

#node1免密ssh到node3實(shí)驗(yàn)
[root@iZuf6gmsvearrd5uc3emkyZ soft]# ssh 172.24.38.211
Last login: Wed Nov  2 14:50:32 2022 from 122.193.199.232

Welcome to Alibaba Cloud Elastic Compute Service !

[root@iZuf6gmsvearrd5uc3eml0Z ~]# exit
logout
Connection to 172.24.38.211 closed.

第四步，我們需要在每個(gè)節(jié)點(diǎn)上安裝Hadoop，假設(shè)我們下載的Hadoop放在/root/soft目錄下，可以先在一臺(tái)服務(wù)器上完成修改后，再將Hadoop文件夾scp到其它兩臺(tái)服務(wù)器。

#進(jìn)入目錄
cd /root/soft

#解壓
tar zxvf hadoop-3.3.4.tar.gz

#修改hadoop配置文件hadoop-env.sh
cd /root/soft/hadoop-3.3.4/etc/hadoop
vim hadoop-env.sh
export HDFS_NAMENODE_USER=root
export HDFS_DATANODE_USER=root
export HDFS_SECONDARYNAMENODE_USER=root
export YARN_RESOURCEMANAGER_USER=root
export YARN_NODEMANAGER_USER=root 
#改成你自己的JAVA_HOME地址
export JAVA_HOME=/root/soft/jdk1.8.0_241

#設(shè)置HADOOP環(huán)境變量
vim /etc/profile
export HADOOP_HOME=/root/soft/hadoop-3.3.4
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
source /etc/profile

然后再修改core-site.xml配置文件：

<!-- 設(shè)置默認(rèn)使用的文件系統(tǒng) Hadoop支持file、HDFS、GFS、ali|Amazon云等文件系統(tǒng) -->
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://172.24.38.209:8020</value>
</property>

<!-- 設(shè)置Hadoop本地保存數(shù)據(jù)路徑 -->
<property>
    <name>hadoop.tmp.dir</name>
    <value>/root/data/hadoop</value>
</property>

<!-- 設(shè)置HDFS web UI用戶身份 -->
<property>
    <name>hadoop.http.staticuser.user</name>
    <value>root</value>
</property>

<!-- 整合hive 用戶代理設(shè)置 -->
<property>
    <name>hadoop.proxyuser.root.hosts</name>
    <value>*</value>
</property>

<property>
    <name>hadoop.proxyuser.root.groups</name>
    <value>*</value>
</property>

<!-- 文件系統(tǒng)垃圾桶保存時(shí)間 -->
<property>
    <name>fs.trash.interval</name>
    <value>1440</value>
</property>

然后再修改hdfs-site.xml配置文件：

<!-- 設(shè)置SecondNameNode進(jìn)程運(yùn)行機(jī)器位置信息 -->
<property>
    <name>dfs.namenode.secondary.http-address</name>
    <value>172.24.38.210:9868</value>
</property>

然后再修改mapred-site.xml配置文件：

<!-- 設(shè)置MR程序默認(rèn)運(yùn)行模式： yarn集群模式 local本地模式 -->
<property>
  <name>mapreduce.framework.name</name>
  <value>yarn</value>
</property>

<!-- MR程序歷史服務(wù)地址 -->
<property>
  <name>mapreduce.jobhistory.address</name>
  <value>172.24.38.209:10020</value>
</property>
 
<!-- MR程序歷史服務(wù)器web端地址 -->
<property>
  <name>mapreduce.jobhistory.webapp.address</name>
  <value>172.24.38.209:19888</value>
</property>

<property>
  <name>yarn.app.mapreduce.am.env</name>
  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>

<property>
  <name>mapreduce.map.env</name>
  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>

<property>
  <name>mapreduce.reduce.env</name>
  <value>HADOOP_MAPRED_HOME=${HADOOP_HOME}</value>
</property>

然后再修改yarn-site.xml配置文件：

<!-- 設(shè)置YARN集群主角色運(yùn)行機(jī)器位置 -->
<property>
    <name>yarn.resourcemanager.hostname</name>
    <value>172.24.38.209</value>
</property>

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
</property>

<!-- 是否將對(duì)容器實(shí)施物理內(nèi)存限制 -->
<property>
    <name>yarn.nodemanager.pmem-check-enabled</name>
    <value>false</value>
</property>

<!-- 是否將對(duì)容器實(shí)施虛擬內(nèi)存限制。 -->
<property>
    <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
</property>

<!-- 開啟日志聚集 -->
<property>
  <name>yarn.log-aggregation-enable</name>
  <value>true</value>
</property>

<!-- 設(shè)置yarn歷史服務(wù)器地址 -->
<property>
    <name>yarn.log.server.url</name>
    <value>http://172.24.38.209:19888/jobhistory/logs</value>
</property>

<!-- 歷史日志保存的時(shí)間 7天 -->
<property>
  <name>yarn.log-aggregation.retain-seconds</name>
  <value>604800</value>
</property>

然后再修改workers配置文件：

172.24.38.209
172.24.38.210
172.24.38.211

然后將修改好的hadoop-3.3.4文件夾遠(yuǎn)程拷貝到其它兩臺(tái)服務(wù)器：

scp -r hadoop-3.3.4 root@172.24.38.210:/root/soft
scp -r hadoop-3.3.4 root@172.24.38.211:/root/soft

最后，我們需要在每一臺(tái)服務(wù)器上配置Hadoop的環(huán)境變量：

vim /etc/profile
export HADOOP_HOME=/root/soft/hadoop-3.3.4
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
source /etc/profile

第五步，我們首次啟動(dòng)時(shí)，需要在node1上進(jìn)行namenode的格式化：

hdfs namenode -format

#格式化成功的部分日志
2022-11-02 16:40:49,064 INFO common.Storage: Storage directory /root/data/hadoop/dfs/name has been successfully formatted.

第六步，啟動(dòng)我們的hadoop hdfs集群：

#啟動(dòng)HDFS集群，停止命令為stop-dfs.sh
[root@iZuf6gmsvearrd5uc3emkyZ hadoop]# start-dfs.sh
Starting namenodes on [iZuf6gmsvearrd5uc3emkyZ]
Last login: Wed Nov  2 16:51:29 CST 2022 on pts/1
Starting datanodes
Last login: Wed Nov  2 16:53:30 CST 2022 on pts/1
172.24.38.210: WARNING: /root/soft/hadoop-3.3.4/logs does not exist. Creating.
172.24.38.211: WARNING: /root/soft/hadoop-3.3.4/logs does not exist. Creating.
Starting secondary namenodes [172.24.38.210]
Last login: Wed Nov  2 16:53:33 CST 2022 on pts/1

#在node1上查看java進(jìn)程，檢驗(yàn)計(jì)劃的hdfs角色是否都啟動(dòng)了
[root@iZuf6gmsvearrd5uc3emkyZ hadoop]# jps
8055 DataNode
8333 Jps
7919 NameNode

#在node2上查看java進(jìn)程，檢驗(yàn)計(jì)劃的hdfs角色是否都啟動(dòng)了
[root@iZuf6gmsvearrd5uc3emkzZ hadoop]# jps
1793 Jps
1738 SecondaryNameNode
1643 DataNode

#在node3上查看java進(jìn)程，檢驗(yàn)計(jì)劃的hdfs角色是否都啟動(dòng)了
[root@iZuf6gmsvearrd5uc3eml0Z hadoop]# jps
1605 DataNode
1671 Jps

結(jié)果顯示，和我們計(jì)劃的集群角色一致，hdfs集群?jiǎn)?dòng)成功。

第七步，啟動(dòng)我們的hadoop yarn集群：

#啟動(dòng)YARN集群，停止命令為stop-yarn.sh
[root@iZuf6gmsvearrd5uc3emkyZ hadoop]# start-yarn.sh
Starting resourcemanager
Last login: Wed Nov  2 16:53:43 CST 2022 on pts/1
Starting nodemanagers
Last login: Wed Nov  2 17:02:08 CST 2022 on pts/1

#在node1上查看java進(jìn)程，檢驗(yàn)計(jì)劃的hdfs和yarn角色是否都啟動(dòng)了
[root@iZuf6gmsvearrd5uc3emkyZ hadoop]# jps
8947 Jps
8487 ResourceManager
8055 DataNode
8617 NodeManager
7919 NameNode

#在node2上查看java進(jìn)程，檢驗(yàn)計(jì)劃的hdfs和yarn角色是否都啟動(dòng)了
[root@iZuf6gmsvearrd5uc3emkzZ hadoop]# jps
1875 NodeManager
1973 Jps
1738 SecondaryNameNode
1643 DataNode

#在node3上查看java進(jìn)程，檢驗(yàn)計(jì)劃的hdfs和yarn角色是否都啟動(dòng)了
[root@iZuf6gmsvearrd5uc3eml0Z hadoop]# jps
1605 DataNode
1835 Jps
1741 NodeManager

結(jié)果顯示，和我們計(jì)劃的集群角色一致，hdfs集群和yarn集群都啟動(dòng)成功。如果嫌棄兩個(gè)集群需要一個(gè)個(gè)啟動(dòng)麻煩，也可以使用一鍵啟動(dòng)命令：

#一鍵啟動(dòng)HDFS和YARN集群
start-all.sh
#一鍵停止HDFS和YARN集群
stop-all.sh

hdfs和yarn都為用戶提供了WebUI界面，因?yàn)槟J(rèn)情況下，我們?cè)诎⒗镌苿?chuàng)建的服務(wù)器是不對(duì)外開放不需要的接口的，所以想要能在公網(wǎng)訪問WebUI界面，需要在阿里云的控制臺(tái)安全組里面開放服務(wù)器的hdfs和yarn端口：

阿里云服務(wù)器開放hdfs和yarn的web UI端口

然后訪問node1（NameNode和ResourceManger所在的服務(wù)器）的公網(wǎng)域名加端口就能使用HDFS和YARN了。

最后，我們使用官方自帶的MapReduce案例來計(jì)算一下圓周率的值：

# 進(jìn)入案例所在目錄
[root@iZuf6gmsvearrd5uc3emkyZ ~]# cd /root/soft/hadoop-3.3.4/share/hadoop/mapreduce/
[root@iZuf6gmsvearrd5uc3emkyZ mapreduce]# ls
hadoop-mapreduce-client-app-3.3.4.jar     hadoop-mapreduce-client-hs-plugins-3.3.4.jar       hadoop-mapreduce-client-shuffle-3.3.4.jar   lib-examples
hadoop-mapreduce-client-common-3.3.4.jar  hadoop-mapreduce-client-jobclient-3.3.4.jar        hadoop-mapreduce-client-uploader-3.3.4.jar  sources
hadoop-mapreduce-client-core-3.3.4.jar    hadoop-mapreduce-client-jobclient-3.3.4-tests.jar  hadoop-mapreduce-examples-3.3.4.jar
hadoop-mapreduce-client-hs-3.3.4.jar      hadoop-mapreduce-client-nativetask-3.3.4.jar       jdiff
# 執(zhí)行計(jì)算任務(wù)
[root@iZuf6gmsvearrd5uc3emkyZ mapreduce]# hadoop jar hadoop-mapreduce-examples-3.3.4.jar pi 2 4
Number of Maps  = 2
Samples per Map = 4
Wrote input for Map #0
Wrote input for Map #1
Starting Job
# 由Yarn的ResourceManager來分配資源
2022-11-02 17:31:41,735 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /172.24.38.209:8032
2022-11-02 17:31:42,738 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/root/.staging/job_1667379747044_0002
2022-11-02 17:31:43,046 INFO input.FileInputFormat: Total input files to process : 2
2022-11-02 17:31:43,953 INFO mapreduce.JobSubmitter: number of splits:2
2022-11-02 17:31:44,735 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1667379747044_0002
2022-11-02 17:31:44,736 INFO mapreduce.JobSubmitter: Executing with tokens: []
2022-11-02 17:31:45,110 INFO conf.Configuration: resource-types.xml not found
2022-11-02 17:31:45,110 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2022-11-02 17:31:45,264 INFO impl.YarnClientImpl: Submitted application application_1667379747044_0002
2022-11-02 17:31:45,349 INFO mapreduce.Job: The url to track the job: http://iZuf6gmsvearrd5uc3emkyZ:8088/proxy/application_1667379747044_0002/
2022-11-02 17:31:45,350 INFO mapreduce.Job: Running job: job_1667379747044_0002
2022-11-02 17:31:59,929 INFO mapreduce.Job: Job job_1667379747044_0002 running in uber mode : false
2022-11-02 17:31:59,931 INFO mapreduce.Job:  map 0% reduce 0%
2022-11-02 17:32:14,551 INFO mapreduce.Job:  map 50% reduce 0%
2022-11-02 17:32:15,568 INFO mapreduce.Job:  map 100% reduce 0%
2022-11-02 17:32:23,754 INFO mapreduce.Job:  map 100% reduce 100%
2022-11-02 17:32:25,819 INFO mapreduce.Job: Job job_1667379747044_0002 completed successfully
2022-11-02 17:32:26,168 INFO mapreduce.Job: Counters: 54
        File System Counters
                FILE: Number of bytes read=50
                FILE: Number of bytes written=830664
                FILE: Number of read operations=0
                FILE: Number of large read operations=0
                FILE: Number of write operations=0
                HDFS: Number of bytes read=536
                HDFS: Number of bytes written=215
                HDFS: Number of read operations=13
                HDFS: Number of large read operations=0
                HDFS: Number of write operations=3
                HDFS: Number of bytes read erasure-coded=0
        Job Counters
                Launched map tasks=2
                Launched reduce tasks=1
                Data-local map tasks=2
                Total time spent by all maps in occupied slots (ms)=24774
                Total time spent by all reduces in occupied slots (ms)=6828
                Total time spent by all map tasks (ms)=24774
                Total time spent by all reduce tasks (ms)=6828
                Total vcore-milliseconds taken by all map tasks=24774
                Total vcore-milliseconds taken by all reduce tasks=6828
                Total megabyte-milliseconds taken by all map tasks=25368576
                Total megabyte-milliseconds taken by all reduce tasks=6991872
        Map-Reduce Framework
                Map input records=2
                Map output records=4
                Map output bytes=36
                Map output materialized bytes=56
                Input split bytes=300
                Combine input records=0
                Combine output records=0
                Reduce input groups=2
                Reduce shuffle bytes=56
                Reduce input records=4
                Reduce output records=0
                Spilled Records=8
                Shuffled Maps =2
                Failed Shuffles=0
                Merged Map outputs=2
                GC time elapsed (ms)=781
                CPU time spent (ms)=2500
                Physical memory (bytes) snapshot=537804800
                Virtual memory (bytes) snapshot=8220102656
                Total committed heap usage (bytes)=295051264
                Peak Map Physical memory (bytes)=213385216
                Peak Map Virtual memory (bytes)=2737229824
                Peak Reduce Physical memory (bytes)=113819648
                Peak Reduce Virtual memory (bytes)=2745643008
        Shuffle Errors
                BAD_ID=0
                CONNECTION=0
                IO_ERROR=0
                WRONG_LENGTH=0
                WRONG_MAP=0
                WRONG_REDUCE=0
        File Input Format Counters
                Bytes Read=236
        File Output Format Counters
                Bytes Written=97
Job Finished in 44.671 seconds
# 計(jì)算結(jié)果
Estimated value of Pi is 3.50000000000000000000

如果任務(wù)執(zhí)行時(shí)間較久，也可以手動(dòng)停止：

# 查看運(yùn)行中任務(wù)列表
[root@iZuf6gmsvearrd5uc3emkyZ ~]# yarn application -list -appStates RUNNING
2022-11-02 17:29:29,506 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /172.24.38.209:8032
Total number of applications (application-types: [], states: [RUNNING] and tags: []):1
                Application-Id      Application-Name        Application-Type          User           Queue                   State             Final-State             Progress                       Tracking-URL
application_1667379747044_0001       QuasiMonteCarlo               MAPREDUCE          root         default                 RUNNING               UNDEFINED                27.5%http://iZuf6gmsvearrd5uc3emkyZ:36895

# 停止指定的任務(wù)
[root@iZuf6gmsvearrd5uc3emkyZ ~]# yarn application -kill application_1667379747044_0001
2022-11-02 17:29:58,805 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /172.24.38.209:8032
Killing application application_1667379747044_0001
2022-11-02 17:30:00,183 INFO impl.YarnClientImpl: Killed application application_1667379747044_0001

至此，一個(gè)完整的Hadoop集群搭建教程已經(jīng)完成。

在企業(yè)的實(shí)際應(yīng)用中，更多的是直接使用云廠商提供的Hadoop集群，在創(chuàng)建MapReduce集群的時(shí)候就可以選擇對(duì)應(yīng)的集群類型，無需用戶自己搭建，非常便捷，但是收費(fèi)也比較高。因此，對(duì)于個(gè)人學(xué)習(xí)還是需要自己搭建一下，既能方便后續(xù)的學(xué)習(xí)，也學(xué)習(xí)了Hadoop的搭建，提高了自己的動(dòng)手能力。