zookeeper 是 google 內(nèi)部項(xiàng)目chubby項(xiàng)目的開源實(shí)現(xiàn),其主要功能是實(shí)現(xiàn)分布式系統(tǒng)的分布式協(xié)作。
分布式主要涉及兩個(gè)問題:任務(wù)拆分與節(jié)點(diǎn)分工
分布式解決的是高可用高并發(fā)的問題。
集群解決的是高可用問題。
集群是從物理上來定義,分布式是一種工作方式。
例如:一個(gè)工作任務(wù)需要10個(gè)小時(shí)
分布式:10臺(tái)機(jī)器,只需要1個(gè)小時(shí)就能完成
集群:10臺(tái)機(jī)器,還是需要10個(gè)小時(shí)才能完成。
死鎖與活鎖?
分布式中遇到的問題:
1.節(jié)點(diǎn)必須是高可用的
2.各個(gè)節(jié)點(diǎn)數(shù)據(jù)必須保持一致
3.如何解決通訊異常問題
4.網(wǎng)絡(luò)分區(qū)
zookeeper解決的問題:
1.Master節(jié)點(diǎn)管理:Master高可用,保證節(jié)點(diǎn)唯一
2.配置文件管理:統(tǒng)一把配置文件放到zk上,由zk統(tǒng)一分發(fā)
3.實(shí)現(xiàn)發(fā)布訂閱:生產(chǎn)者將消息放到zk上,供消費(fèi)者獲取
4.分布式鎖:分布式環(huán)境訪問同一個(gè)資源時(shí),由第三方配鎖實(shí)現(xiàn),保持?jǐn)?shù)據(jù)一致性
5.集群的管理
單機(jī)偽分布式zookeeper集群搭建
安裝jdk
root@ubuntu:/app# java -version
java version "1.8.0_201"
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
下載zookeeper,解壓文件夾
root@ubuntu:/app# tar -zxvf -C /user/local/
配置zookeeper的數(shù)據(jù)目錄,并在目錄下創(chuàng)建對(duì)應(yīng)的myid,每個(gè)myid都賦值
[root@localhost conf]# cd /tmp/
[root@localhost tmp]# mkdir zookeeper
[root@localhost zookeeper]# mkdir zk1
[root@localhost zookeeper]# mkdir zk2
[root@localhost zookeeper]# mkdir zk3
[root@localhost zookeeper]# cd zk1
[root@localhost zk1]# touch myid
[root@localhost zk1]# echo 1 >> /tmp/zookeeper/zk1/myid
[root@localhost zk1]# cat /tmp/zookeeper/zk1/myid
1
[root@localhost zookeeper]# cd zk2
[root@localhost zk2]# ls
[root@localhost zk2]# touch myid
[root@localhost zk2]# echo 2 >> /tmp/zookeeper/zk2/myid
[root@localhost zk2]# cat /tmp/zookeeper/zk2/myid
2
[root@localhost zookeeper]# cd zk3
[root@localhost zk3]# ls
[root@localhost zk3]# touch myid
[root@localhost zk3]# ls
myid
[root@localhost zk3]# echo 3 >> /tmp/zookeeper/zk3/myid
[root@localhost zk3]# ls
myid
[root@localhost zk3]# cat /tmp/zookeeper/zk3/myid
3
配置zookeeper
[root@ubuntu /usr/local]# cd zookeeper-3.4.11/
[root@ubuntu /usr/local/zookeeper]# cd conf/
[root@ubuntu /usr/local/zookeeper/conf]# cp zoo_sample.cfg zoo1.cfg # 拷貝官方提供的模板配置文件
[root@ubuntu /usr/local/zookeeper/conf]# cp zoo_sample.cfg zoo2.cfg
[root@ubuntu /usr/local/zookeeper/conf]# cp zoo_sample.cfg zoo3.cfg
[root@ubuntu /usr/local/zookeeper/conf]# vim zoo1.cfg # 增加或修改成如下內(nèi)容
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/tmp/zookeeper/zk1/
clientPort=2181
4lw.commands.whitelist=*
#server.myid值=主機(jī)名:通訊端口:選舉端口
#主機(jī)名查看命令 hostname
server.1=master:2888:3888
server.2=master:2889:3889
server.3=master:2890:3890
quorumListenOnAllIPs=true
[root@ubuntu /usr/local/zookeeper/conf]# cd ../
真分布式
Zookeeper真分布部署在master、slave1、slave2 三臺(tái)不同的服務(wù)器上
conf配置文件不用區(qū)分zoo.cfg只有一份
myid也只有一份 /usr/local/zookeeper/zk
server配置也要多個(gè)主機(jī)地址:
server.1=master:2888:3888
server.2=slave1:2888:3888
server.3=slave2:2888:3888
啟動(dòng)集群
[root@ubuntu ~]# cd /usr/local/zookeeper/bin/
[root@ubuntu /usr/local/zookeeper/bin]# ./zkServer.sh start /usr/local/zookeeper/conf/zoo1.cfg # 啟動(dòng)第一個(gè)節(jié)點(diǎn)
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@ubuntu /usr/local/zookeeper/bin]# netstat -lntp |grep java # 查看監(jiān)聽的端口
[root@ubuntu /usr/local/zookeeper/bin]# ./zkServer.sh start /usr/local/zookeeper/conf/zoo2.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg # 啟動(dòng)第二個(gè)節(jié)點(diǎn)
Starting zookeeper ... STARTED
[root@ubuntu /usr/local/zookeeper/bin]# ./zkServer.sh start /usr/local/zookeeper/conf/zoo3.cfg # 啟動(dòng)第三個(gè)節(jié)點(diǎn)
[root@ubuntu /usr/local/zookeeper/bin]# netstat -lntp |grep java # 查看監(jiān)聽的端口
tcp6 0 0 :::40576 :::* LISTEN 2421/java
tcp6 0 0 :::2181 :::* LISTEN 2282/java
tcp6 0 0 :::2182 :::* LISTEN 2359/java
tcp6 0 0 :::2183 :::* LISTEN 2421/java
tcp6 0 0 :::44808 :::* LISTEN 2282/java
tcp6 0 0 127.0.0.1:2889 :::* LISTEN 2359/java
tcp6 0 0 127.0.0.1:3888 :::* LISTEN 2282/java
tcp6 0 0 127.0.0.1:3889 :::* LISTEN 2359/java
tcp6 0 0 127.0.0.1:3890 :::* LISTEN 2421/java
#通過通訊端口(2889)可以看到哪個(gè)是master服務(wù),如圖通訊端口對(duì)應(yīng)的是2182服務(wù)
[root@study-01 /usr/local/zookeeper/bin]# jps # 查看進(jìn)程
Will not attempt to authenticate using SASL (unknown error)異常
Will not attempt to authenticate using SASL (unknown error)
這種異常需要在 /etc/host里配置下主機(jī)名和IP映射關(guān)系
異常查看
1:請(qǐng)關(guān)閉防火墻 ufw status查看防火墻狀態(tài)
2:如果沒有監(jiān)聽,請(qǐng)查看bin目錄下 zookeeper.out 日志文件
3:./zkServer.sh restart|start|stop|status命令可以 重啟|啟動(dòng)|停止|查看狀態(tài)
三個(gè)節(jié)點(diǎn)正常啟動(dòng)后,進(jìn)入客戶端,創(chuàng)建znode,看是否可以在節(jié)點(diǎn)間同步
root@ubuntu:/usr/local/zookeeper/bin# ./zkCli.sh -server master:2181
Connecting to localhost:2181
2019-03-26 07:43:11,989 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.11-
Welcome to ZooKeeper!
WatchedEvent state:SyncConnected type:None path:null
[zk: localhost:2181(CONNECTED) 0] ls /
[zookeeper]
[zk: localhost:2181(CONNECTED) 1] create /data test-data
Created /data
[zk: localhost:2181(CONNECTED) 2] ls /
[zookeeper, data]
[zk: localhost:2181(CONNECTED) 3] quit
Quitting...
root@ubuntu:/usr/local/zookeeper/bin# ./zkCli.sh -server master:2182 #查看2182節(jié)點(diǎn)是否同步了2181創(chuàng)建的znode
Connecting to localhost:2182
Welcome to ZooKeeper!
[zk: localhost:2182(CONNECTED) 0] ls /
[zookeeper, data]
[zk: localhost:2182(CONNECTED) 1] get /data
test-data
[zk: localhost:2182(CONNECTED) 2] quit
Quitting...
root@ubuntu:/usr/local/zookeeper/bin# ./zkCli.sh -server localhost:2183 #查看節(jié)點(diǎn)數(shù)據(jù)是否同步到2183
[zk: localhost:2183(CONNECTED) 0] ls /
[zookeeper, data]
[zk: localhost:2183(CONNECTED) 1] get /data
test-data
[zk: localhost:2183(CONNECTED) 2] quit
查看主從信息
root@ubuntu:/usr/local# cd zookeeper/bin
root@ubuntu:/usr/local/zookeeper/bin# ./zkServer.sh status /usr/local/zookeeper/conf/zoo1.cfg
ZooKeeper JMX enabled by default
Mode: follower #從節(jié)點(diǎn)
root@ubuntu:/usr/local/zookeeper/bin# ./zkServer.sh status /usr/local/zookeeper/conf/zoo2.cfg
ZooKeeper JMX enabled by default
Mode: leader #主節(jié)點(diǎn)
root@ubuntu:/usr/local/zookeeper/bin# ./zkServer.sh status /usr/local/zookeeper/conf/zoo3.cfg
ZooKeeper JMX enabled by default
Mode: follower #從節(jié)點(diǎn)
root@ubuntu:/usr/local/zookeeper/bin#
測(cè)試選舉
zoo2是master節(jié)點(diǎn)
root@ubuntu:/usr/local/zookeeper/bin# ./zkServer.sh status /usr/local/zookeeper-3.4.11/conf/zoo1.cfg
ZooKeeper JMX enabled by default
Mode: follower #zoo1是slave
root@ubuntu:/usr/local/zookeeper01/bin#./zkServer.sh status /usr/local/zookeeper-3.4.11/conf/zoo2.cfg
ZooKeeper JMX enabled by default
Mode: leader #zoo2是master
root@ubuntu:/usr/local/zookeeper/bin# ./zkServer.sh status /usr/local/zookeeper-3.4.11/conf/zoo3.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper02/bin/../conf/zoo.cfg
Mode: follower #zoo3是slave
關(guān)閉zoo2,重啟就會(huì)發(fā)生一次選舉
[root@localhost bin]# ./zkServer.sh stop /usr/local/zookeeper-3.4.11/conf/zoo2.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.11/conf/zoo2.cfg
Stopping zookeeper ... STOPPED
[root@localhost bin]# ./zkServer.sh start /usr/local/zookeeper-3.4.11/conf/zoo2.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.11/conf/zoo2.cfg
Starting zookeeper ... STARTED
查看節(jié)點(diǎn)狀態(tài)
[root@localhost bin]# ./zkServer.sh status /usr/local/zookeeper-3.4.11/conf/zoo2.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.11/conf/zoo2.cfg
Mode: follower #從節(jié)點(diǎn)
[root@localhost bin]# ./zkServer.sh status /usr/local/zookeeper-3.4.11/conf/zoo1.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.11/conf/zoo1.cfg
Mode: follower #從節(jié)點(diǎn)
[root@localhost bin]# ./zkServer.sh status /usr/local/zookeeper-3.4.11/conf/zoo3.cfg
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper-3.4.11/conf/zoo3.cfg
Mode: leader #新的主節(jié)點(diǎn)