使用docker-compose部署redis集群

集群架構(gòu)：3主3從
redis版本：5.0
網(wǎng)絡(luò)模式：host
根據(jù)官網(wǎng)的描述，需要用host模式。

In order to make Docker compatible with Redis Cluster you need to use the host networking mode of Docker. Please check the --net=host option in the Docker documentation for more information.

在host模式下，container共用宿主機(jī)的ip，因此不需要做端口映射

端口規(guī)劃

圖片.png

文件規(guī)劃

掛載配置文件和日志文件

修改配置文件redis.conf

分別為每個(gè)redis建一個(gè)文件夾，里面放redis.conf和redis.log
修改每個(gè)配置文件

port 6379 #這里填每臺(tái)redis的端口號(hào)
bind 0.0.0.0 #綁定ip 0.0.0.0 指可以外網(wǎng)訪問
cluster-enabled yes #開啟集群
logfile "/usr/local/redis/logs/redis-server.log" #指定日志文件

docker-compose文件內(nèi)容

version: '3'

services:
 master-1:
  container_name: master-1
  image: redis
  command: redis-server /etc/usr/local/redis.conf
  network_mode: "host"
  volumes:
   - /home/jasonlin/redis/master1/redis.conf:/etc/usr/local/redis.conf
   - /home/jasonlin/redis/master1/redis.log:/usr/local/redis/logs/redis-server.log

 master-2:
  container_name: master-2
  command: redis-server /etc/usr/local/redis.conf
  image: redis
  network_mode: "host"
  volumes:
   - /home/jasonlin/redis/master2/redis.conf:/etc/usr/local/redis.conf
   - /home/jasonlin/redis/master2/redis.log:/usr/local/redis/logs/redis-server.log

 master-3:
  container_name: master-3
  image: redis
  command: redis-server /etc/usr/local/redis.conf
  network_mode: "host"
  volumes:
   - /home/jasonlin/redis/master3/redis.conf:/etc/usr/local/redis.conf
   - /home/jasonlin/redis/master3/redis.log:/usr/local/redis/logs/redis-server.log

 slave-1:
  container_name: slave-1
  image: redis
  command: redis-server /etc/usr/local/redis.conf
  network_mode: "host"
  volumes:
   - /home/jasonlin/redis/slave1/redis.conf:/etc/usr/local/redis.conf
   - /home/jasonlin/redis/slave1/redis.log:/usr/local/redis/logs/redis-server.log

 slave-2:
  container_name: slave-2
  image: redis
  command: redis-server /etc/usr/local/redis.conf
  network_mode: "host"
  volumes:
   - /home/jasonlin/redis/slave2/redis.conf:/etc/usr/local/redis.conf
   - /home/jasonlin/redis/slave2/redis.log:/usr/local/redis/logs/redis-server.log
   
 slave-3:
  container_name: slave-3
  image: redis
  command: redis-server /etc/usr/local/redis.conf
  network_mode: "host"
  volumes:
   - /home/jasonlin/redis/slave3/redis.conf:/etc/usr/local/redis.conf
   - /home/jasonlin/redis/slave3/redis.log:/usr/local/redis/logs/redis-server.log

部署redis

在docker-compose.yaml的文件夾下執(zhí)行(root用戶)

docker-compose up -d

root@ubuntu:/home/jasonlin/redis# docker-compose up -d
Creating slave-1 ... 
Creating slave-3 ... 
Creating master-3 ... 
Creating master-2 ... 
Creating master-1 ... 
Creating slave-2 ... 
Creating slave-1
Creating slave-3
Creating master-3
Creating master-1
Creating master-2
Creating master-3 ... done

查看一下是否啟動(dòng)成功。如果容器列表沒有，說明沒啟動(dòng)成功，需要docker logs 容器名稱/ID 看一下錯(cuò)誤情況

root@ubuntu:/home/jasonlin/redis# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
414bdb7a4b04        redis               "docker-entrypoint.s…"   7 seconds ago       Up 5 seconds                            slave-2
2c79aaa1ed74        redis               "docker-entrypoint.s…"   7 seconds ago       Up 5 seconds                            master-2
8e89a3fbb859        redis               "docker-entrypoint.s…"   7 seconds ago       Up 5 seconds                            master-1
f5bfe6a41adb        redis               "docker-entrypoint.s…"   7 seconds ago       Up 5 seconds                            master-3
09fd8166e31f        redis               "docker-entrypoint.s…"   7 seconds ago       Up 5 seconds                            slave-1
16904e456595        redis               "docker-entrypoint.s…"   7 seconds ago       Up 5 seconds                            slave-3

再用rdm或redis-cli連一下看


圖片.png

添加集群

redis部署后，只是單機(jī)部署，需要手動(dòng)創(chuàng)建集群。
接下來隨便連一臺(tái)創(chuàng)建集群，以master1為例

root@ubuntu:/home/jasonlin/redis# docker exec -it master-1 redis-cli --cluster create 192.168.139.134:6379 \
192.168.139.134:6380 \
192.168.139.134:6381 \
192.168.139.134:6382 \
192.168.139.134:6383 \
192.168.139.134:6384 --cluster-replicas 1

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.139.134:6383 to 192.168.139.134:6379
Adding replica 192.168.139.134:6384 to 192.168.139.134:6380
Adding replica 192.168.139.134:6382 to 192.168.139.134:6381
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 69b0a09758c68ffd96f4c869261c6b08fbfed2d3 192.168.139.134:6379
   slots:[0-5460] (5461 slots) master
M: 93d75c3497a9a89c74728e1844dc6326d8aa7a23 192.168.139.134:6380
   slots:[5461-10922] (5462 slots) master
M: 4e17362768c8b7b42f46a7a3682b4e2ff3163840 192.168.139.134:6381
   slots:[10923-16383] (5461 slots) master
S: a6521ef0f0e07cf1ade9516eb455cadc7c05d1df 192.168.139.134:6382
   replicates 69b0a09758c68ffd96f4c869261c6b08fbfed2d3
S: 038a2b1407acdc7d250b0359273cb9c89af9719b 192.168.139.134:6383
   replicates 93d75c3497a9a89c74728e1844dc6326d8aa7a23
S: 4d69e6d54ca0980c03c2d90e0ac5e57114d038b6 192.168.139.134:6384
   replicates 4e17362768c8b7b42f46a7a3682b4e2ff3163840
Can I set the above configuration? (type 'yes' to accept):

輸入yes，確定該配置

Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
........
>>> Performing Cluster Check (using node 192.168.139.134:6379)
M: 69b0a09758c68ffd96f4c869261c6b08fbfed2d3 192.168.139.134:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 4e17362768c8b7b42f46a7a3682b4e2ff3163840 192.168.139.134:6381
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: 93d75c3497a9a89c74728e1844dc6326d8aa7a23 192.168.139.134:6380
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: a6521ef0f0e07cf1ade9516eb455cadc7c05d1df 192.168.139.134:6382
   slots: (0 slots) slave
   replicates 69b0a09758c68ffd96f4c869261c6b08fbfed2d3
S: 038a2b1407acdc7d250b0359273cb9c89af9719b 192.168.139.134:6383
   slots: (0 slots) slave
   replicates 93d75c3497a9a89c74728e1844dc6326d8aa7a23
S: 4d69e6d54ca0980c03c2d90e0ac5e57114d038b6 192.168.139.134:6384
   slots: (0 slots) slave
   replicates 4e17362768c8b7b42f46a7a3682b4e2ff3163840
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

輸入創(chuàng)建集群命令后，會(huì)return節(jié)點(diǎn)的信息以及主從關(guān)系
輸入yes后，
更新節(jié)點(diǎn)配置
分配不同的配置epoch（紀(jì)元）給每個(gè)節(jié)點(diǎn)
發(fā)送MEET消息到每個(gè)節(jié)點(diǎn)加入到集群中
加入后，可以看到
3個(gè)Master，3個(gè)slave，前面一長串的是每個(gè)節(jié)點(diǎn)的ID，后面是IP，端口以及分配的slot

測試集群

從master1中set一個(gè)value，看是否會(huì)redirect到其他節(jié)點(diǎn)上

root@ubuntu:/home/jasonlin/redis# docker exec -it master-1 redis-cli -c
127.0.0.1:6379> set name jasonlin
-> Redirected to slot [5798] located at 192.168.139.134:6380
OK

可以看到，保存到了第5798個(gè)slot里，這個(gè)slot是分在了master2

請(qǐng)注意，集群模式下需要以集群方式使用redis-cli不然在redirect時(shí)候報(bào)錯(cuò)

root@ubuntu:/home/jasonlin/redis# docker exec -it master-1 redis-cli 
127.0.0.1:6379> set key ggg mmm
(error) MOVED 12539 192.168.139.134:6381

總結(jié)

這次使用docker-compose部署redis集群，遇到最大問題還是這個(gè)網(wǎng)絡(luò)模式。
用bridge模式，會(huì)卡在添加集群那一塊，一直在wait。
查了文檔，除了通信用的端口需要開啟，還需要開啟一個(gè)叫做集群總線端口，即通信端口+10000
然后我也做了相應(yīng)的映射，不行。
我也試過添加節(jié)點(diǎn)時(shí)候分別輸入宿主機(jī)IP和container IP、把虛擬網(wǎng)卡添加到防火墻 ，也不行。

另外，也出現(xiàn)了詭異的問題。
查看日志發(fā)現(xiàn)，每個(gè)節(jié)點(diǎn)都做了config Epoch設(shè)置
但在IP address for this node updated時(shí)候，在master1上顯示的是網(wǎng)關(guān)的ip，在master2、3顯示的是container自己的ip

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容