Docker方式部署redis-cluster

前面我們手工部署了redis-cluster集群,可以看到步驟煩多,現(xiàn)在使用docker的方式部署,非常簡(jiǎn)單,共分為二步。
參考鏡像https://github.com/publicisworldwide/docker-stacks/tree/master/oracle-linux/environments/storage/redis-cluster

Docker方式部署redis-cluster步驟
1、redis容器初始化
2、redis容器集群配置

一、redis容器初始化
容器初始化,使用docker-compose方式,先創(chuàng)建一個(gè)docker-compose.yml文件,內(nèi)容如下:

version: '3'

services:
 redis1:
  image: publicisworldwide/redis-cluster
  network_mode: host
  restart: always
  volumes:
   - /data/redis/8001/data:/data
  environment:
   - REDIS_PORT=8001

 redis2:
  image: publicisworldwide/redis-cluster
  network_mode: host
  restart: always
  volumes:
   - /data/redis/8002/data:/data
  environment:
   - REDIS_PORT=8002

 redis3:
  image: publicisworldwide/redis-cluster
  network_mode: host
  restart: always
  volumes:
   - /data/redis/8003/data:/data
  environment:
   - REDIS_PORT=8003

 redis4:
  image: publicisworldwide/redis-cluster
  network_mode: host
  restart: always
  volumes:
   - /data/redis/8004/data:/data
  environment:
   - REDIS_PORT=8004

 redis5:
  image: publicisworldwide/redis-cluster
  network_mode: host
  restart: always
  volumes:
   - /data/redis/8005/data:/data
  environment:
   - REDIS_PORT=8005

 redis6:
  image: publicisworldwide/redis-cluster
  network_mode: host
  restart: always
  volumes:
   - /data/redis/8006/data:/data
  environment:
   - REDIS_PORT=8006

這里引用了別人的一個(gè)鏡像publicisworldwide/redis-cluster,方便快捷。
這里使用host(主機(jī))網(wǎng)絡(luò)模式,把redis數(shù)據(jù)掛載到本機(jī)目錄/data/redis/800*下。

若不想使用host模式,也可以把network_mode去掉,但就要加ports映射。
redis-cluster的節(jié)點(diǎn)端口共分為2種,一種是節(jié)點(diǎn)提供服務(wù)的端口,如6379;一種是節(jié)點(diǎn)間通信的端口,固定格式為:10000+6379。

version: '3'

services:
 redis1:
  image: publicisworldwide/redis-cluster
  restart: always
  volumes:
   - /data/redis/8001/data:/data
  environment:
   - REDIS_PORT=8001
  ports:
    - '8001:8001'       #服務(wù)端口
    - '18001:18001'   #集群端口

 redis2:
  image: publicisworldwide/redis-cluster
  restart: always
  volumes:
   - /data/redis/8002/data:/data
  environment:
   - REDIS_PORT=8002
  ports:
    - '8002:8002'
    - '18002:18002'

 redis3:
  image: publicisworldwide/redis-cluster
  restart: always
  volumes:
   - /data/redis/8003/data:/data
  environment:
   - REDIS_PORT=8003
  ports:
    - '8003:8003'
    - '18003:18003'

 redis4:
  image: publicisworldwide/redis-cluster
  restart: always
  volumes:
   - /data/redis/8004/data:/data
  environment:
   - REDIS_PORT=8004
  ports:
    - '8004:8004'
    - '18004:18004'

 redis5:
  image: publicisworldwide/redis-cluster
  restart: always
  volumes:
   - /data/redis/8005/data:/data
  environment:
   - REDIS_PORT=8005
  ports:
    - '8005:8005'
    - '18005:18005'

 redis6:
  image: publicisworldwide/redis-cluster
  restart: always
  volumes:
   - /data/redis/8006/data:/data
  environment:
   - REDIS_PORT=8006
  ports:
    - '8006:8006'
    - '18006:18006'

創(chuàng)建文件后,直接啟動(dòng)服務(wù)

窗口模式
docker-compose up
后臺(tái)進(jìn)程
docker-compose up -d

查看啟動(dòng)的進(jìn)程

[root@localhost redis-cluster]# docker-compose ps
        Name                       Command               State   Ports
----------------------------------------------------------------------
rediscluster_redis1_1   /usr/local/bin/entrypoint. ...   Up           
rediscluster_redis2_1   /usr/local/bin/entrypoint. ...   Up           
rediscluster_redis3_1   /usr/local/bin/entrypoint. ...   Up           
rediscluster_redis4_1   /usr/local/bin/entrypoint. ...   Up           
rediscluster_redis5_1   /usr/local/bin/entrypoint. ...   Up           
rediscluster_redis6_1   /usr/local/bin/entrypoint. ...   Up           

狀態(tài)為Up,說(shuō)明服務(wù)均已啟動(dòng),鏡像無(wú)問(wèn)題。
注意:以上鏡像不能設(shè)置永久密碼,其實(shí)redis一般是內(nèi)網(wǎng)訪問(wèn),可以不需密碼。

二、redis容器集群配置
上面只是啟動(dòng)了6個(gè)redis容器,并沒(méi)有設(shè)置集群,通過(guò)下面的命令可以設(shè)置集群。

docker run --rm -it inem0o/redis-trib create --replicas 1 172.19.165.222:8001 172.19.165.222:8002 172.19.165.222:8003 172.19.165.222:8004 172.19.165.222:8005 172.19.165.222:8006

這里同樣使用了另一個(gè)鏡像inem0o/redis-trib,執(zhí)行時(shí)會(huì)自動(dòng)下載。
日志如下:

[root@localhost disconf]# docker run --rm -it inem0o/redis-trib create --replicas 1 172.19.165.222:8001 172.19.165.222:8002 172.19.165.222:8003 172.19.165.222:8004 172.19.165.222:8005 172.19.165.222:8006
Unable to find image 'inem0o/redis-trib:latest' locally
latest: Pulling from inem0o/redis-trib
a2b2998a36ab: Pull complete 
a3ed95caeb02: Pull complete 
46ab6b64c08e: Pull complete 
3d82c3ac2025: Pull complete 
Digest: sha256:0b89d25b387f70ef1c54605bdf061dd86e0833dbc0e2149390570b8b372278f8
Status: Downloaded newer image for inem0o/redis-trib:latest
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.19.165.222:8001
172.19.165.222:8002
172.19.165.222:8003
Adding replica 172.19.165.222:8004 to 172.19.165.222:8001
Adding replica 172.19.165.222:8005 to 172.19.165.222:8002
Adding replica 172.19.165.222:8006 to 172.19.165.222:8003
M: 67d9a6bb6875f3a0f9a53e5bb05ddeca8e656950 172.19.165.222:8001
   slots:0-5460 (5461 slots) master
M: 206626063f31dcd7e69010ce13c258e786197f1e 172.19.165.222:8002
   slots:5461-10922 (5462 slots) master
M: e9924018d95772b8ff535f6bc0605a6630837069 172.19.165.222:8003
   slots:10923-16383 (5461 slots) master
S: 548f4e65fbab8dcde8aac187849d50983d68599d 172.19.165.222:8004
   replicates 67d9a6bb6875f3a0f9a53e5bb05ddeca8e656950
S: 0a5c799c1f8fed083c50902639fc354e4c25aa8c 172.19.165.222:8005
   replicates 206626063f31dcd7e69010ce13c258e786197f1e
S: 94e2530ddd05b0e9eb3e71a9616342bd6647a5e6 172.19.165.222:8006
   replicates e9924018d95772b8ff535f6bc0605a6630837069
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
>>> Performing Cluster Check (using node 172.19.165.222:8001)
M: 67d9a6bb6875f3a0f9a53e5bb05ddeca8e656950 172.19.165.222:8001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 94e2530ddd05b0e9eb3e71a9616342bd6647a5e6 172.19.165.222:8006@18006
   slots: (0 slots) slave
   replicates e9924018d95772b8ff535f6bc0605a6630837069
S: 0a5c799c1f8fed083c50902639fc354e4c25aa8c 172.19.165.222:8005@18005
   slots: (0 slots) slave
   replicates 206626063f31dcd7e69010ce13c258e786197f1e
M: e9924018d95772b8ff535f6bc0605a6630837069 172.19.165.222:8003@18003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 206626063f31dcd7e69010ce13c258e786197f1e 172.19.165.222:8002@18002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 548f4e65fbab8dcde8aac187849d50983d68599d 172.19.165.222:8004@18004
   slots: (0 slots) slave
   replicates 67d9a6bb6875f3a0f9a53e5bb05ddeca8e656950
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

可以看到設(shè)置了3個(gè)Master,3個(gè)Slave,若想測(cè)試下集群,可以參考【Redis-Cluster集群

只需二步就完成了集群的部署,但上面是使用了別人的鏡像,若是要自己創(chuàng)建鏡像該如何呢?

手動(dòng)創(chuàng)建鏡像
編寫(xiě)Dockerfile

#基礎(chǔ)鏡像
FROM redis
#修復(fù)時(shí)區(qū)
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
RUN echo 'Asia/Shanghai' >/etc/timezone
#環(huán)境變量
ENV REDIS_PORT 8000
#ENV REDIS_PORT_NODE 18000
#暴露變量
EXPOSE $REDIS_PORT
#EXPOSE $REDIS_PORT_NODE
#復(fù)制
COPY entrypoint.sh /usr/local/bin/
COPY redis.conf /usr/local/etc/
#for config rewrite
RUN chmod 777 /usr/local/etc/redis.conf
RUN chmod +x /usr/local/bin/entrypoint.sh
#入口
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
#命令
CMD ["redis-server", "/usr/local/etc/redis.conf"]

編寫(xiě)shell文件entrypoint.sh

#!/bin/sh
#只作用于當(dāng)前進(jìn)程,不作用于其創(chuàng)建的子進(jìn)程
set -e
#$0--Shell本身的文件名 $1--第一個(gè)參數(shù) $@--所有參數(shù)列表
# allow the container to be started with `--user`
if [ "$1" = 'redis-server' -a "$(id -u)" = '0' ]; then
    sed -i 's/REDIS_PORT/'$REDIS_PORT'/g' /usr/local/etc/redis.conf
    chown -R redis .  #改變當(dāng)前文件所有者
    exec gosu redis "$0" "$@"  #gosu是sudo輕量級(jí)”替代品”
fi

exec "$@"

編寫(xiě)redis.conf

#端口
port REDIS_PORT
#開(kāi)啟集群
cluster-enabled yes
#配置文件
cluster-config-file nodes.conf
cluster-node-timeout 5000
#更新操作后進(jìn)行日志記錄
appendonly yes
#設(shè)置主服務(wù)的連接密碼
# masterauth
#設(shè)置從服務(wù)的連接密碼
# requirepass

注意:

  • requirepass和masterauth不能啟用,否則redis-trib創(chuàng)建集群失敗。
  • protected-mode 保護(hù)模式是禁止公網(wǎng)訪問(wèn),但是不能設(shè)置密碼和bind ip。

以上編寫(xiě)了3個(gè)文件,其實(shí)可以創(chuàng)建鏡像了,不過(guò)也可以在docker-compose中自動(dòng)創(chuàng)建。
編寫(xiě)docker-compose.yml文件

version: '3'

services:
 redis1:
  build: ./
  restart: always
  volumes:
   - /data/redis/8001/data:/data
  environment:
   - REDIS_PORT=8001
  ports:
    - '8001:8001'       #服務(wù)端口
    - '18001:18001'   #集群端口

 redis2:
  build: ./
  restart: always
  volumes:
   - /data/redis/8002/data:/data
  environment:
   - REDIS_PORT=8002
  ports:
    - '8002:8002'
    - '18002:18002'

 redis3:
  build: ./
  restart: always
  volumes:
   - /data/redis/8003/data:/data
  environment:
   - REDIS_PORT=8003
  ports:
    - '8003:8003'
    - '18003:18003'

 redis4:
  build: ./
  restart: always
  volumes:
   - /data/redis/8004/data:/data
  environment:
   - REDIS_PORT=8004
  ports:
    - '8004:8004'
    - '18004:18004'

 redis5:
  build: ./
  restart: always
  volumes:
   - /data/redis/8005/data:/data
  environment:
   - REDIS_PORT=8005
  ports:
    - '8005:8005'
    - '18005:18005'

 redis6:
  build: ./
  restart: always
  volumes:
   - /data/redis/8006/data:/data
  environment:
   - REDIS_PORT=8006
  ports:
    - '8006:8006'
    - '18006:18006'

創(chuàng)建鏡像

窗口模式
docker-compose up
后臺(tái)進(jìn)程
docker-compose up -d
重新創(chuàng)建
docker-compose up --build

若創(chuàng)建成功,可以使用命令登錄并查看集群信息。

[root@localhost src]# ./redis-cli -c -p 8003
127.0.0.1:8003> cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0

從上面的日志里看到,當(dāng)前集群狀態(tài)失敗。

與上面相同,重新執(zhí)行上面的第二步,再次查看結(jié)果如下:

127.0.0.1:8003> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:3
cluster_stats_messages_ping_sent:39
cluster_stats_messages_pong_sent:37
cluster_stats_messages_meet_sent:4
cluster_stats_messages_sent:80
cluster_stats_messages_ping_received:34
cluster_stats_messages_pong_received:43
cluster_stats_messages_meet_received:3
cluster_stats_messages_received:80

以上信息顯示集群狀態(tài)ok。

設(shè)置集群密碼
上面提供了publicisworldwide/redis-cluster的鏡像不能設(shè)置永久密碼,是因?yàn)樗鼪](méi)有給redis.conf寫(xiě)入權(quán)限,在我們重創(chuàng)建的鏡像里是有寫(xiě)入權(quán)限的,所以可以創(chuàng)建永久密碼。
集群構(gòu)建完畢后再通過(guò)config set + config rewrite命令逐個(gè)機(jī)器設(shè)置密碼
在之前的鏡像上設(shè)置密碼會(huì)有權(quán)限問(wèn)題,如下:

127.0.0.1:8001> config set masterauth 1111
OK
127.0.0.1:8001> config set requirepass 1111
OK
127.0.0.1:8001> auth
(error) ERR wrong number of arguments for 'auth' command
127.0.0.1:8001> auth 1111
OK
127.0.0.1:8001> config rewrite
(error) ERR Rewriting config file: Permission denied

在新的鏡像上添加密碼,如下:

[root@localhost src]# ./redis-cli -c -p 8001
127.0.0.1:8001> config set masterauth 1111
OK
127.0.0.1:8001> config set requirepass 1111
OK
127.0.0.1:8001> config rewrite
(error) NOAUTH Authentication required.
127.0.0.1:8001> auth 1111
OK
127.0.0.1:8001> config rewrite
OK

當(dāng)重登錄8001機(jī)器并get數(shù)據(jù)時(shí)就會(huì)提示需要密碼了。其它機(jī)器也要一并設(shè)置密碼才行。

查看密碼文件
是否將密碼寫(xiě)入到配置文件redis.conf,可以登錄到容器查看即可。

docker exec -it 容器ID /bin/bash
cat /usr/local/etc/redis.conf

可以看到masterauth和requirepass被追加到文件的最后,即使重啟密碼也還生效。
在單機(jī)上部署意義不大,可以把slave結(jié)點(diǎn)放到不同的服務(wù)器上,再通過(guò)docker的redis-trib容器來(lái)連接,十分靈活和方便。

異常處理
在第三步執(zhí)行可能會(huì)現(xiàn)以下異常,如下:

[root@localhost redis-cluster]# docker run --rm -it inem0o/redis-trib create --replicas 1 172.19.165.222:8001 172.19.165.222:8002 172.19.165.222:8003 172.19.165.222:8004 172.19.165.222:8005 172.19.165.222:8006
>>> Creating cluster
[ERR] Node 172.19.165.222:8001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

這個(gè)異常是因?yàn)樵谀夸浵?code>/data/redis/800*已經(jīng)存在redis的配置文件,要先清除掉。

[root@localhost redis-cluster]# rm /data/redis/800* -rf
最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡(jiǎn)書(shū)系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

友情鏈接更多精彩內(nèi)容