ZooKeeper的偽分布式集群搭建以及真分布式集群搭建

zk集群的一些基本概念

zookeeper集群搭建：

  • zk集群，主從節(jié)點(diǎn)，心跳機(jī)制（選舉模式）
  • 配置數(shù)據(jù)文件 myid 1/2/3 對(duì)應(yīng) server.1/2/3
  • 通過 zkCli.sh -server [ip]:[port] 命令檢測(cè)集群是否配置成功

和其他大多數(shù)集群結(jié)構(gòu)一樣，zookeeper集群也是主從結(jié)構(gòu)。搭建集群時(shí)，機(jī)器數(shù)量最低也是三臺(tái)，因?yàn)樾∮谌_(tái)就無法進(jìn)行選舉。選舉就是當(dāng)集群中的master節(jié)點(diǎn)掛掉之后，剩余的兩臺(tái)機(jī)器會(huì)進(jìn)行選舉，在這兩臺(tái)機(jī)器中選舉出一臺(tái)來做master節(jié)點(diǎn)。而當(dāng)原本掛掉的master恢復(fù)正常后，也會(huì)重新加入集群當(dāng)中。但是不會(huì)再作為master節(jié)點(diǎn)，而是作為slave節(jié)點(diǎn)。如下：


image.png

單機(jī)偽分布式搭建zookeeper集群

本節(jié)介紹單機(jī)偽分布式的zookeeper安裝，官方下載地址如下：

https://archive.apache.org/dist/zookeeper/

我這里使用的是3.4.11版本，所以找到相應(yīng)的版本點(diǎn)擊進(jìn)去，復(fù)制到.tar.gz的下載鏈接到Linux上進(jìn)行下載。命令如下：

[root@study-01 ~]# cd /usr/local/src/
[root@study-01 /usr/local/src]# wget https://archive.apache.org/dist/zookeeper/zookeeper-3.4.11/zookeeper-3.4.11.tar.gz

下載完成之后將其解壓到/usr/local/目錄下：

[root@study-01 /usr/local/src]# tar -zxvf zookeeper-3.4.11.tar.gz -C /usr/local/
[root@study-01 /usr/local/src]# cd ../zookeeper-3.4.11/
[root@study-01 /usr/local/zookeeper-3.4.11]# ls
bin        dist-maven       lib          README_packaging.txt  zookeeper-3.4.11.jar.asc
build.xml  docs             LICENSE.txt  recipes               zookeeper-3.4.11.jar.md5
conf       ivysettings.xml  NOTICE.txt   src                   zookeeper-3.4.11.jar.sha1
contrib    ivy.xml          README.md    zookeeper-3.4.11.jar
[root@study-01 /usr/local/zookeeper-3.4.11]#

然后給目錄重命名一下：

[root@study-01 ~]# cd /usr/local/
[root@study-01 /usr/local]# mv zookeeper-3.4.11/ zookeeper00

接著進(jìn)行一系列的配置：

[root@study-01 /usr/local]# cd zookeeper00/
[root@study-01 /usr/local/zookeeper00]# cd conf/
[root@study-01 /usr/local/zookeeper00/conf]# cp zoo_sample.cfg zoo.cfg  # 拷貝官方提供的模板配置文件
[root@study-01 /usr/local/zookeeper00/conf]# vim zoo.cfg  # 增加或修改成如下內(nèi)容
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper00/dataDir
dataLogDir=/usr/local/zookeeper00/dataLogDir
clientPort=2181
4lw.commands.whitelist=*
server.1=192.168.190.129:2888:3888  # master節(jié)點(diǎn)，ip后面跟的是集群通信的端口
server.2=192.168.190.129:2889:3889
server.3=192.168.190.129:2890:3890
[root@study-01 /usr/local/zookeeper00/conf]# cd ../
[root@study-01 /usr/local/zookeeper00]# mkdir {dataDir,dataLogDir}
[root@study-01 /usr/local/zookeeper00]# cd dataDir/
[root@study-01 /usr/local/zookeeper00/dataDir]# vim myid  # 配置該節(jié)點(diǎn)的id
1
[root@study-01 /usr/local/zookeeper00/dataDir]#

配置完之后，拷貝多個(gè)目錄出來，因?yàn)槭菃螜C(jī)的偽分布式所以需要在一臺(tái)機(jī)器上安裝多個(gè)zookeeper：

[root@study-01 /usr/local]# cp zookeeper00 zookeeper01 -rf
[root@study-01 /usr/local]# cp zookeeper00 zookeeper02 -rf

配置 zookeeper01：

[root@study-01 /usr/local]# cd zookeeper01/conf/
[root@study-01 /usr/local/zookeeper01/conf]# vim zoo.cfg  # 修改內(nèi)容如下
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper01/dataDir
dataLogDir=/usr/local/zookeeper01/dataLogDir
clientPort=2182  # 端口號(hào)必須要修改
4lw.commands.whitelist=*
server.1=192.168.190.129:2888:3888
server.2=192.168.190.129:2889:3889
server.3=192.168.190.129:2890:3890
[root@study-01 /usr/local/zookeeper01/conf]# cd ../dataDir/
[root@study-01 /usr/local/zookeeper01/dataDir]# vim myid
2
[root@study-01 /usr/local/zookeeper01/dataDir]#

配置 zookeeper02：

[root@study-01 /usr/local]# cd zookeeper02/conf/
[root@study-01 /usr/local/zookeeper02/conf]# vim zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper02/dataDir
dataLogDir=/usr/local/zookeeper02/dataLogDir
clientPort=2183  # 端口號(hào)必須要修改
4lw.commands.whitelist=*
server.1=192.168.190.129:2888:3888
server.2=192.168.190.129:2889:3889
server.3=192.168.190.129:2890:3890
[root@study-01 /usr/local/zookeeper02/conf]# cd ../dataDir/
[root@study-01 /usr/local/zookeeper02/dataDir]# vim myid
3
[root@study-01 /usr/local/zookeeper02/dataDir]#

以上就在單機(jī)上配置好三個(gè)zookeeper集群節(jié)點(diǎn)了，現(xiàn)在我們來測(cè)試一下，這個(gè)偽分布式的zookeeper集群能否正常運(yùn)作起來：

[root@study-01 ~]# cd /usr/local/zookeeper00/bin/
[root@study-01 /usr/local/zookeeper00/bin]# ./zkServer.sh start  # 啟動(dòng)第一個(gè)節(jié)點(diǎn)
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper00/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@study-01 /usr/local/zookeeper00/bin]# netstat -lntp |grep java  # 查看監(jiān)聽的端口
tcp6       0      0 192.168.190.129:3888    :::*                    LISTEN      3191/java  # 集群通信的端口
tcp6       0      0 :::44793                :::*                    LISTEN      3191/java
tcp6       0      0 :::2181                 :::*                    LISTEN      3191/java 
[root@study-01 /usr/local/zookeeper00/bin]# cd ../../zookeeper01/bin/
[root@study-01 /usr/local/zookeeper01/bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper01/bin/../conf/zoo.cfg  # 啟動(dòng)第二個(gè)節(jié)點(diǎn)
Starting zookeeper ... STARTED
[root@study-01 /usr/local/zookeeper01/bin]# cd ../../zookeeper02/bin/
[root@study-01 /usr/local/zookeeper02/bin]# ./zkServer.sh start  # 啟動(dòng)第三個(gè)節(jié)點(diǎn)
[root@study-01 /usr/local/zookeeper02/bin]# netstat -lntp |grep java   # 查看監(jiān)聽的端口
tcp6       0      0 192.168.190.129:2889    :::*                    LISTEN      3232/java           
tcp6       0      0 :::48463                :::*                    LISTEN      3232/java           
tcp6       0      0 192.168.190.129:3888    :::*                    LISTEN      3191/java           
tcp6       0      0 192.168.190.129:3889    :::*                    LISTEN      3232/java           
tcp6       0      0 192.168.190.129:3890    :::*                    LISTEN      3286/java           
tcp6       0      0 :::44793                :::*                    LISTEN      3191/java           
tcp6       0      0 :::60356                :::*                    LISTEN      3286/java           
tcp6       0      0 :::2181                 :::*                    LISTEN      3191/java           
tcp6       0      0 :::2182                 :::*                    LISTEN      3232/java           
tcp6       0      0 :::2183                 :::*                    LISTEN      3286/java           
[root@study-01 /usr/local/zookeeper02/bin]# jps  # 查看進(jìn)程
3232 QuorumPeerMain
3286 QuorumPeerMain
3191 QuorumPeerMain
3497 Jps
[root@study-01 /usr/local/zookeeper02/bin]#

如上，可以看到，三個(gè)節(jié)點(diǎn)都正常啟動(dòng)成功了，接下來我們進(jìn)入客戶端，創(chuàng)建一些znode，看看是否會(huì)同步到集群中的其他節(jié)點(diǎn)上去：

[root@study-01 /usr/local/zookeeper02/bin]# ./zkCli.sh -server localhost:2181  # 登錄第一個(gè)節(jié)點(diǎn)的客戶端
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 1] create /data test-data
Created /data
[zk: localhost:2181(CONNECTED) 2] ls /
[zookeeper, data]
[zk: localhost:2181(CONNECTED) 3] quit
[root@study-01 /usr/local/zookeeper02/bin]# ./zkCli.sh -server localhost:2182  # 登錄第二個(gè)節(jié)點(diǎn)的客戶端
[zk: localhost:2182(CONNECTED) 0] ls /  # 可以查看到我們?cè)诘谝粋€(gè)節(jié)點(diǎn)上創(chuàng)建的znode，代表集群中的節(jié)點(diǎn)能夠正常同步數(shù)據(jù)
[zookeeper, data]
[zk: localhost:2182(CONNECTED) 1] get /data  # 數(shù)據(jù)也是一致的
test-data
cZxid = 0x100000002
ctime = Tue Apr 24 18:35:56 CST 2018
mZxid = 0x100000002
mtime = Tue Apr 24 18:35:56 CST 2018
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: localhost:2182(CONNECTED) 2] quit
[root@study-01 /usr/local/zookeeper02/bin]# ./zkCli.sh -server localhost:2183  # 登錄第三個(gè)節(jié)點(diǎn)的客戶端
[zk: localhost:2183(CONNECTED) 0] ls /  # 第三個(gè)節(jié)點(diǎn)也能查看到我們?cè)诘谝粋€(gè)節(jié)點(diǎn)上創(chuàng)建的znode
[zookeeper, data]
[zk: localhost:2183(CONNECTED) 1] get /data  # 數(shù)據(jù)也是一致的
test-data
cZxid = 0x100000002
ctime = Tue Apr 24 18:35:56 CST 2018
mZxid = 0x100000002
mtime = Tue Apr 24 18:35:56 CST 2018
pZxid = 0x100000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: localhost:2183(CONNECTED) 2] quit
[root@study-01 /usr/local/zookeeper02/bin]#

查看集群的狀態(tài)、主從信息需要使用 ./zkServer.sh status 命令，但是多個(gè)節(jié)點(diǎn)的話，逐個(gè)查看有些費(fèi)勁，所以我們寫一個(gè)簡(jiǎn)單的shell腳本來批量執(zhí)行命令。如下：

[root@study-01 ~]# vim checked.sh  # 腳本內(nèi)容如下
#!/bin/bash
/usr/local/zookeeper00/bin/zkServer.sh status
/usr/local/zookeeper01/bin/zkServer.sh status
/usr/local/zookeeper02/bin/zkServer.sh status
[root@study-01 ~]# sh ./checked.sh  # 執(zhí)行腳本
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper00/bin/../conf/zoo.cfg
Mode: follower  # 從節(jié)點(diǎn)
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper01/bin/../conf/zoo.cfg
Mode: leader   # 主節(jié)點(diǎn)
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper02/bin/../conf/zoo.cfg
Mode: follower
[root@study-01 ~]#

到此為止，我們就成功完成了單機(jī)zookeeper偽分布式集群的搭建，并且也測(cè)試成功了。

搭建zookeeper分布式集群

接下來，我們使用三臺(tái)虛擬機(jī)來搭建zookeeper真實(shí)分布式集群，機(jī)器的ip地址如下：

  • 192.168.190.128
  • 192.168.190.129
  • 192.168.190.130

注：三臺(tái)機(jī)器都必須具備java的運(yùn)行環(huán)境，并且關(guān)閉或清空防火墻規(guī)則，不想關(guān)閉防火墻的話，就需要去配置相應(yīng)的防火墻規(guī)則

首先配置一下系統(tǒng)的hosts文件：

[root@localhost ~]# vim /etc/hosts
192.168.190.128 zk000
192.168.190.129 zk001
192.168.190.130 zk002

把之前做偽分布式實(shí)驗(yàn)機(jī)器上的其他zookeeper目錄刪除掉，并把zookeeper目錄使用rsync同步到其他機(jī)器上。如下：

[root@zk001 ~]# cd /usr/local/
[root@zk001 /usr/local]# rm -rf zookeeper01
[root@zk001 /usr/local]# rm -rf zookeeper02
[root@zk001 /usr/local]# mv zookeeper00/ zookeeper
[root@zk001 /usr/local]# rsync -av /usr/local/zookeeper/ 192.168.190.128:/usr/local/zookeeper/
[root@zk001 /usr/local]# rsync -av /usr/local/zookeeper/ 192.168.190.130:/usr/local/zookeeper/

然后逐個(gè)在三臺(tái)機(jī)器上都配置一下環(huán)境變量，如下：

[root@zk001 ~]# vim .bash_profile  # 增加如下內(nèi)容
export ZOOKEEPER_HOME=/usr/local/zookeeper
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin
export PATH
[root@zk001 ~]# source .bash_profile

逐個(gè)修改配置文件，zk000：

[root@zk000 ~]# cd /usr/local/zookeeper/conf/
[root@zk000 /usr/local/zookeeper/conf]# vim zoo.cfg 
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/dataDir
dataLogDir=/usr/local/zookeeper/dataLogDir
clientPort=2181
4lw.commands.whitelist=*
server.1=192.168.190.128:2888:3888  # 默認(rèn)server.1為master節(jié)點(diǎn)
server.2=192.168.190.129:2888:3888
server.3=192.168.190.130:2888:3888
[root@zk000 /usr/local/zookeeper/conf]# cd ../dataDir/
[root@zk000 /usr/local/zookeeper/dataDir]# vim myid 
1
[root@zk000 /usr/local/zookeeper/dataDir]#

zk001：

[root@zk001 ~]# cd /usr/local/zookeeper/conf/
[root@zk001 /usr/local/zookeeper/conf]# vim zoo.cfg 
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/dataDir
dataLogDir=/usr/local/zookeeper/dataLogDir
clientPort=2181
4lw.commands.whitelist=*
server.1=192.168.190.128:2888:3888  # 默認(rèn)server.1為master節(jié)點(diǎn)
server.2=192.168.190.129:2888:3888
server.3=192.168.190.130:2888:3888
[root@zk001 /usr/local/zookeeper/conf]# cd ../dataDir/
[root@zk001 /usr/local/zookeeper/dataDir]# vim myid 
2
[root@zk001 /usr/local/zookeeper/dataDir]#

zk002：

[root@zk002 ~]# cd /usr/local/zookeeper/conf/
[root@zk002 /usr/local/zookeeper/conf]# vim zoo.cfg 
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/usr/local/zookeeper/dataDir
dataLogDir=/usr/local/zookeeper/dataLogDir
clientPort=2181
4lw.commands.whitelist=*
server.1=192.168.190.128:2888:3888  # 默認(rèn)server.1為master節(jié)點(diǎn)
server.2=192.168.190.129:2888:3888
server.3=192.168.190.130:2888:3888
[root@zk002 /usr/local/zookeeper/conf]# cd ../dataDir/
[root@zk002 /usr/local/zookeeper/dataDir]# vim myid 
3
[root@zk002 /usr/local/zookeeper/dataDir]#

配置完成之后，啟動(dòng)三臺(tái)機(jī)器的zookeeper服務(wù)：

[root@zk000 ~]# zkServer.sh start
[root@zk001 ~]# zkServer.sh start
[root@zk002 ~]# zkServer.sh start

啟動(dòng)成功后，查看三個(gè)機(jī)器的集群狀態(tài)信息：

[root@zk000 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@zk000 ~]#
[root@zk001 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@zk001 ~]# 
[root@zk002 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@zk002 ~]#

然后我們來測(cè)試創(chuàng)建znode是否會(huì)同步，進(jìn)入192.168.190.128機(jī)器的客戶端：

[root@zk000 ~]# zkCli.sh -server 192.168.190.128:2181
[zk: 192.168.190.128:2181(CONNECTED) 0] ls /
[zookeeper, data]
[zk: 192.168.190.128:2181(CONNECTED) 1] create /real-culster real-data
Created /real-culster
[zk: 192.168.190.128:2181(CONNECTED) 2] ls /
[zookeeper, data, real-culster]
[zk: 192.168.190.128:2181(CONNECTED) 3] get /real-culster
real-data
cZxid = 0x300000002
ctime = Tue Apr 24 20:48:32 CST 2018
mZxid = 0x300000002
mtime = Tue Apr 24 20:48:32 CST 2018
pZxid = 0x300000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: 192.168.190.128:2181(CONNECTED) 4] quit

進(jìn)入192.168.190.129機(jī)器的客戶端

[root@zk000 ~]# zkCli.sh -server 192.168.190.129:2181
[zk: 192.168.190.129:2181(CONNECTED) 0] ls /
[zookeeper, data, real-culster]
[zk: 192.168.190.129:2181(CONNECTED) 1] get /real-culster
real-data
cZxid = 0x300000002
ctime = Tue Apr 24 20:48:32 CST 2018
mZxid = 0x300000002
mtime = Tue Apr 24 20:48:32 CST 2018
pZxid = 0x300000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: 192.168.190.129:2181(CONNECTED) 2] quit

進(jìn)入192.168.190.130機(jī)器的客戶端

[root@zk000 ~]# zkCli.sh -server 192.168.190.130:2181
[zk: 192.168.190.130:2181(CONNECTED) 0] ls /
[zookeeper, data, real-culster]
[zk: 192.168.190.130:2181(CONNECTED) 1] get /real-culster
real-data
cZxid = 0x300000002
ctime = Tue Apr 24 20:48:32 CST 2018
mZxid = 0x300000002
mtime = Tue Apr 24 20:48:32 CST 2018
pZxid = 0x300000002
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0
[zk: 192.168.190.130:2181(CONNECTED) 2] quit

從以上的測(cè)試可以看到，在zookeeper分布式集群中，我們?cè)谌我庖粋€(gè)節(jié)點(diǎn)創(chuàng)建的znode都會(huì)被同步的集群中的其他節(jié)點(diǎn)上，數(shù)據(jù)也會(huì)被一并同步。所以到此為止，我們的zookeeper分布式集群就搭建成功了。

測(cè)試集群角色以及選舉

以上我們只是測(cè)試了znode的同步，還沒有測(cè)試集群中的節(jié)點(diǎn)選舉，所以本節(jié)就來測(cè)試一下，當(dāng)master節(jié)點(diǎn)掛掉之后看看slave節(jié)點(diǎn)會(huì)不會(huì)通過選舉坐上master的位置。首先我們來把master節(jié)點(diǎn)的zookeeper服務(wù)給停掉：

[root@zk001 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@zk001 ~]# zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@zk001 ~]#

這時(shí)到其他兩臺(tái)機(jī)器上進(jìn)行查看，可以看到有一臺(tái)已經(jīng)成為master節(jié)點(diǎn)了：

[root@zk002 ~]# zkServer.sh status  # 可以看到zk002這個(gè)節(jié)點(diǎn)成為了master節(jié)點(diǎn)
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@zk002 ~]# 
[root@zk000 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@zk000 ~]#

然后再把停掉的節(jié)點(diǎn)啟動(dòng)起來，可以看到，該節(jié)點(diǎn)重新加入了集群，但是此時(shí)是以slave角色存在，而不會(huì)以master角色存在：

[root@zk001 ~]# zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@zk001 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@zk001 ~]#

可以看到，zk002這個(gè)節(jié)點(diǎn)依舊是master角色，不會(huì)被取代，所以只有在選舉的時(shí)候集群中的節(jié)點(diǎn)才會(huì)切換角色：

[root@zk002 ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@zk002 ~]#
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容