Redis-cluster之在線搭建詳解

一、在線集群安裝:

將以下shell保存至腳本文件redis-install.sh

#!/bin/bash

for ((i=83;i<132;i++)) do

??????? ssh root@192.168.212.$i yum -y install epel-release

??????? ssh root@192.168.212.$i yum -y install redis

done

執(zhí)行腳本:?$?./redis-install.sh ??(執(zhí)行完畢將在從192.168.212.83~192.168.212.131的所有機(jī)器上都安裝了redis)

啟動(dòng):?$?/usr/sbin/redis-server /etc/redis.conf

-------------------------------------華麗的分割線---------------------------------------------

二、在線安裝:

首先安裝依賴:

yum?-y install tcl ruby rubygems

gem install redis

下面我們簡單的使用一下,操作系統(tǒng)是一臺(tái)ubuntu14.04系統(tǒng)(啟動(dòng)6個(gè)實(shí)例,

3 Master/3 Slave):

1.?下載最新版redis

wget 'http://download.redis.io/releases/redis-3.0.4.tar.gz'

2.??編譯安裝

#安裝到/opt/apps/redis-cluster目錄

tar zxvf redis-3.0.4.tar.gz

cd redis-3.0.4/

make PREFIX=/opt/apps/redis-cluster install

#創(chuàng)建配置目錄,拷貝相關(guān)文件

mkdir -p?/opt/apps/redis-cluster/{bin,log,conf,rdb,run,nodes}

cp src/redis-trib.rb /opt/apps/redis-cluster/bin/

3.??安裝其他依賴包

yum install ruby1.9.1

gem install redis ???#?也可離線安裝,后面會(huì)介紹(在只有內(nèi)網(wǎng)的機(jī)器上非常有用)

4.??創(chuàng)建配置文件

看看6379的具體信息:grep?-v?-E?'(^#|^$)' redis-6379.conf

-------------------------------------華麗的分割線---------------------------------------------

daemonize yes

pidfile /opt/apps/redis-cluster/run/redis-6379.pid

port 6379

tcp-backlog 511

timeout 0

tcp-keepalive 0

loglevel notice

logfile "/opt/apps/redis-cluster/log/redis-6379.log"

databases 16

save 900 1

save 300 10

save 60 10000

stop-writes-on-bgsave-error yes

rdbcompression yes

rdbchecksum yes

dbfilename dump-6379.rdb

dir /opt/redis-cluster/rdb

slave-serve-stale-data yes

slave-read-only yes

repl-diskless-sync no

repl-diskless-sync-delay 5

repl-disable-tcp-nodelay no

slave-priority 100

appendonly yes

appendfilename "redis-6379.aof"

appendfsync everysec

no-appendfsync-on-rewrite no

auto-aof-rewrite-percentage 100

auto-aof-rewrite-min-size 64mb

aof-load-truncated yes

lua-time-limit 5000

cluster-enabled yes

cluster-config-file nodes-6379.conf

cluster-node-timeout 15000

slowlog-log-slower-than 10000

slowlog-max-len 128

latency-monitor-threshold 0

notify-keyspace-events ""

hash-max-ziplist-entries 512

hash-max-ziplist-value 64

list-max-ziplist-entries 512

list-max-ziplist-value 64

set-max-intset-entries 512

zset-max-ziplist-entries 128

zset-max-ziplist-value 64

hll-sparse-max-bytes 3000

activerehashing yes

client-output-buffer-limit normal 0 0 0

client-output-buffer-limit slave 256mb 64mb 60

client-output-buffer-limit pubsub 32mb 8mb 60

hz 10

aof-rewrite-incremental-fsync yes

#?其實(shí)大部分都是原有的配置信息,不需要做大的改動(dòng),與端口相關(guān)的都需要調(diào)整,與集群相關(guān)的配置如下:

cluster-enabled yes

cluster-config-file nodes-6379.conf

cluster-node-timeout 15000

#?這里得注意,如果在配置文件里指定了密碼,那么在創(chuàng)建集群的時(shí)候?qū)?huì)出現(xiàn)認(rèn)證錯(cuò)誤,解決辦法下面“FAQ1”會(huì)提到。

masterauth yourpassword

requirepass yourpassword

-------------------------------------華麗的分割線---------------------------------------------

注意其中的nodes-6379.conf這個(gè)文件不需要?jiǎng)?chuàng)建,在初始化集群的時(shí)候會(huì)自動(dòng)創(chuàng)建的。

5.??在啟動(dòng)redis實(shí)例前先修改一些系統(tǒng)級(jí)別的配置

echo never >?/sys/kernel/mm/transparent_hugepage/enabled

#?打開/etc/sysctl.conf,追加如下內(nèi)容

vm.overcommit_memory?= 1

#使配置生效

sysctl -p

6.??啟動(dòng)所有redis實(shí)例

cd?/opt/redis-cluster/bin

./redis-server?/opt/apps/redis-cluster/conf/redis-6379.conf

./redis-server?/opt/apps/redis-cluster/conf/redis-6380.conf

./redis-server?/opt/apps/redis-cluster/conf/redis-6381.conf

./redis-server?/opt/apps/redis-cluster/conf/redis-7379.conf

./redis-server?/opt/apps/redis-cluster/conf/redis-7380.conf

./redis-server?/opt/apps/redis-cluster/conf/redis-7381.conf

7.? redis-trib.rb

??????? redis-trib.rb是一個(gè)官方提供的用來操作cluster的ruby腳本,我們后面管理cluster會(huì)經(jīng)常使用到這個(gè)腳本

Usage: redis-trib ??

? create ???????? host1:port1 ... hostN:portN

??????????????????--replicas

? check ????????? host:port

? fix ??????????? host:port

? reshard ??????? host:port

??????????????????--from

??????????????????--to?

??????????????????--slots?

??????????????????--yes

? add-node ?????? new_host:new_port existing_host:existing_port

??????????????????--slave

??????????????????--master-id

? del-node ?????? host:port node_id

? set-timeout ??? host:port milliseconds

? call ?????????? host:port command arg arg .. arg

? import ???????? host:port

??????????????????--from

? help ???????????(show this help)

For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.

8 ???初始化啟動(dòng)cluster(一定要記得--replicas后面有個(gè) 1

./redis-trib.rb create --replicas1?127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:7379 127.0.0.1:7380 127.0.0.1:7381

??????結(jié)果如下:

>>> Creating cluster

Connecting to node 172.16.1.100:6379: OK

Connecting to node 172.16.1.100:6380: OK

Connecting to node 172.16.1.100:6381: OK

Connecting to node 172.16.1.100:7379: OK

Connecting to node 172.16.1.100:7380: OK

Connecting to node 172.16.1.100:7381: OK

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:????###注意三個(gè)6xxx的都被定義為master的了

172.16.1.100:6379

172.16.1.100:6380

172.16.1.100:6381

Adding replica 172.16.1.100:7379 to 172.16.1.100:6379 ??###注意三個(gè)7xxx的都被定義為相關(guān)的slave了

Adding replica 172.16.1.100:7380 to 172.16.1.100:6380

Adding replica 172.16.1.100:7381 to 172.16.1.100:6381

M: cdb8b1fe29feb9a564fbfed6599aa61dda250eb1 172.16.1.100:6379

?? slots:0-5460 (5461 slots) master ???###6379被分配了0-5460個(gè)slots

M: a188b59b30056f61c1cf55ff5072d60b6f8ce5d7 172.16.1.100:6380

?? slots:5461-10922 (5462 slots) master ??###6380被分配了5461-10922個(gè)slots

M: fada90b0520d5aa3305bc89651cc7bb5de9b2a16 172.16.1.100:6381

?? slots:10923-16383 (5461 slots) master ????###6381被分配了10923-16383個(gè)slots

S: 818fb192631cf8c34bc58f24d2538444abe7d995 172.16.1.100:7379 ???###剩余的三個(gè)slave

?? replicates cdb8b1fe29feb9a564fbfed6599aa61dda250eb1

S: 1cf6ed9f8a5049513fb631e78207c7f0b1ba6674 172.16.1.100:7380

?? replicates a188b59b30056f61c1cf55ff5072d60b6f8ce5d7

S: bc09734e243b9b2e9fe2f64c55aaf72073c6918b 172.16.1.100:7381

?? replicates fada90b0520d5aa3305bc89651cc7bb5de9b2a16

Can I set the above configuration??(type 'yes' to accept): yes ????###是否在nodes配置文件中保存更新配置

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join.....

>>> Performing Cluster Check (using node 172.16.1.100:6379)

M: cdb8b1fe29feb9a564fbfed6599aa61dda250eb1 172.16.1.100:6379

?? slots:0-5460 (5461 slots) master

M: a188b59b30056f61c1cf55ff5072d60b6f8ce5d7 172.16.1.100:6380

?? slots:5461-10922 (5462 slots) master

M: fada90b0520d5aa3305bc89651cc7bb5de9b2a16 172.16.1.100:6381

?? slots:10923-16383 (5461 slots) master

M: 818fb192631cf8c34bc58f24d2538444abe7d995 172.16.1.100:7379

?? slots:?(0 slots) master

?? replicates cdb8b1fe29feb9a564fbfed6599aa61dda250eb1

M: 1cf6ed9f8a5049513fb631e78207c7f0b1ba6674 172.16.1.100:7380

?? slots:?(0 slots) master

?? replicates a188b59b30056f61c1cf55ff5072d60b6f8ce5d7

M: bc09734e243b9b2e9fe2f64c55aaf72073c6918b 172.16.1.100:7381

?? slots:?(0 slots) master

?? replicates fada90b0520d5aa3305bc89651cc7bb5de9b2a16

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

????檢查一下cluster的狀態(tài):redis-trib.rb check 127.0.0.1:6379

9 ?接入cluster,并存入一個(gè)key/value?(name/guo)

redis-cli?-c?-h 127.0.0.1 -p 6379

????注意:redis-cli在接入集群模式時(shí),要使用-c參數(shù)??

????????可以看到我在6379接入cluster中,并存入可一個(gè)name/guodalu鍵值對(duì),但是cluster卻把我保存的值定位到7380去了,而我使用6381接入cluster中后,再次獲取name值時(shí),會(huì)被自動(dòng)的定位到7380上。我們來看看是我是怎么被定位到7380的。

??????? 1 先計(jì)算guodalu的CRC16值是0xA36B,轉(zhuǎn)化成10進(jìn)制是41835

??????? 2 41835對(duì)16384取模是9067,9067 slot是落在6380上的,而7380是6380的slave。

????下面我們把7380停止,可以看到再次檢查cluster的時(shí)候,并不會(huì)報(bào)錯(cuò)。

????我們?cè)?381上接入cluster,獲取name

10 ?恢復(fù)7380,停止6380

??????可以看到這時(shí)候check cluster的時(shí)候,會(huì)提示一個(gè)節(jié)點(diǎn)已經(jīng)故障,我們這時(shí)候再嘗試獲取key

????????可以看到還是可以獲取到的。如果把6380和7380同時(shí)停止呢?

????????如果cluster中的master和slave全部故障的話,則這部分?jǐn)?shù)據(jù)還是不可用的。

11 ???在cluster中新增一個(gè)redis master節(jié)點(diǎn)

????????啟動(dòng)redis 8379實(shí)例

./redis-server /opt/redis-cluster/conf/redis-8379.conf

????????然后把該實(shí)例加入cluster中:第一個(gè)參數(shù)是新加入的redis節(jié)點(diǎn),第二個(gè)參數(shù)是cluster中的任何一個(gè)已有的節(jié)點(diǎn)

./redis-trib.rb add-node 127.0.0.1:8379 127.0.0.1:6379

????這時(shí)候檢查一下cluster狀態(tài)

./redis-trib.rb check 127.0.0.1:6379

????????可以看到8379已經(jīng)加入cluster中了,默認(rèn)是以master加入cluster的,而且副本為1,但是這個(gè)新的master卻沒有被cluster分配slots,所以它里面是沒有數(shù)據(jù)的,當(dāng)有slave服務(wù)器要升級(jí)為master服務(wù)器時(shí),它不能參與選舉。這時(shí)候我們可以使用重新劃分slots的特性給新加入的8379遷移其他節(jié)點(diǎn)上的slots,這個(gè)后面操作。

12 ???給8379添加一個(gè)slave節(jié)點(diǎn)

./redis-server?/opt/redis-cluster/conf/redis-8380.conf

./redis-trib.rb add-node --slave?--master-id 63ec1e7d57b3e19c6e1fa07768673001cfa0d30f 127.0.0.1:8380 127.0.0.1:6379

#或者

./redis-trib.rb add-node --slave 127.0.0.1:8380 127.0.0.1:6379

????????第一個(gè)添加slvae的命令是直接指定把該節(jié)點(diǎn)作為哪個(gè)master的slave加入cluster,而第二個(gè)命令redis-trib 會(huì)添加該新節(jié)點(diǎn)作為一個(gè)具有較少副本的隨機(jī)的主服務(wù)器的副本(在本環(huán)境中顯然是8379的副本了)。

????再次檢查cluster狀態(tài)

./redis-trib.rb check 127.0.0.1:6379

????????可以看到8379已經(jīng)有一個(gè)slave了,但是它的slots還是0,因?yàn)槲覀円恢睕]有給它遷移slots。

13 ?刪除一個(gè)redis slave節(jié)點(diǎn)

我們先刪除8380 slave節(jié)點(diǎn),第一個(gè)參數(shù)是cluster中的任意一個(gè)節(jié)點(diǎn),第二個(gè)參數(shù)要?jiǎng)h除的slave的ID

./redis-trib.rb del-node 172.16.1.100:6379 12decfd0d975ef0dcd151d0ff232da7a2269a1c2

????????這時(shí)候再次檢查cluster的狀態(tài)

./redis-trib.rb check 127.0.0.1:6379

????????注意,紅框中的8379的副本又變?yōu)?了。

14 ?刪除一個(gè)redis master節(jié)點(diǎn)

????????刪除master節(jié)點(diǎn)和刪除slave節(jié)點(diǎn)操作一樣,不過為了移除一個(gè)master節(jié)點(diǎn),它必須是空的。如果master不是空的,你需要先將其數(shù)據(jù)重分片到其他的master節(jié)點(diǎn)。另一種移除master節(jié)點(diǎn)的方式,就是在其從服務(wù)器上執(zhí)行一次手工故障轉(zhuǎn)移,當(dāng)它變?yōu)榱诵碌膍aster的slave以后將其移除。

????????因?yàn)槲覀冞€沒操作slots重新分配,就先不刪除8379了。

15 ?? cluster重新分片

????????經(jīng)過測(cè)試發(fā)現(xiàn)cluster的重新分片并不是在新的cluster內(nèi)再次均勻的分配slots,而是需要是手動(dòng)指定把哪個(gè)節(jié)點(diǎn)的多少slots遷移到另一個(gè)節(jié)點(diǎn)去.......

./redis-trib.rb reshard 127.0.0.1:6379

看看最終結(jié)果

????????在經(jīng)過我一番瞎遷移后,最終成為這樣了。當(dāng)然你也可以使用如下命令進(jìn)行嚴(yán)格的遷移:

./redis-trib.rb reshard :?--from??--to??--slots?--yes

16??刪除8379 master節(jié)點(diǎn)

????????因?yàn)?379的slots都被我遷移到其他節(jié)點(diǎn)去了,而刪除一個(gè)master節(jié)點(diǎn)需要該節(jié)點(diǎn)的數(shù)據(jù)為空

./redis-trib.rb del-node 127.0.0.1:6379 63ec1e7d57b3e19c6e1fa07768673001cfa0d30f

以上redis-trib的這些操作,在redis-cli中也是可以進(jìn)行操作的:

CLUSTER INFO 打印集群的信息??

CLUSTER NODES 列出集群當(dāng)前已知的所有節(jié)點(diǎn)(node),以及這些節(jié)點(diǎn)的相關(guān)信息。??

CLUSTER RESET? reset

CLUSTER SAVECONFIG ?強(qiáng)制節(jié)點(diǎn)保存集群當(dāng)前狀態(tài)到磁盤上。

CLUSTER SLOTS 獲得slot在節(jié)點(diǎn)上的映射關(guān)系

CLUSTER MEET ??將 ip 和 port 所指定的節(jié)點(diǎn)添加到集群當(dāng)中,讓它成為集群的一份子。??

CLUSTER FORGET ?從集群中移除 node_id 指定的節(jié)點(diǎn)。??

CLUSTER REPLICATE ?將當(dāng)前節(jié)點(diǎn)設(shè)置為 node_id 指定的節(jié)點(diǎn)的從節(jié)點(diǎn)。????

CLUSTER SLAVES ??列出該slave節(jié)點(diǎn)的master節(jié)點(diǎn)

CLUSTER ADDSLOTS ?[slot?...]?將一個(gè)或多個(gè)槽(slot)指派(assign)給當(dāng)前節(jié)點(diǎn)。??

CLUSTER DELSLOTS ?[slot?...]?移除一個(gè)或多個(gè)槽對(duì)當(dāng)前節(jié)點(diǎn)的指派。??

CLUSTER FLUSHSLOTS 移除指派給當(dāng)前節(jié)點(diǎn)的所有槽,讓當(dāng)前節(jié)點(diǎn)變成一個(gè)沒有指派任何槽的節(jié)點(diǎn)。??

CLUSTER SETSLOT NODE ?將槽 slot 指派給 node_id 指定的節(jié)點(diǎn),如果槽已經(jīng)指派給另一個(gè)節(jié)點(diǎn),那么先讓另一個(gè)節(jié)點(diǎn)刪除該槽>,然后再進(jìn)行指派。??

CLUSTER SETSLOT MIGRATING ?將本節(jié)點(diǎn)的槽 slot 遷移到 node_id 指定的節(jié)點(diǎn)中。??

CLUSTER SETSLOT IMPORTING ?從 node_id 指定的節(jié)點(diǎn)中導(dǎo)入槽 slot 到本節(jié)點(diǎn)。??

CLUSTER SETSLOT STABLE 取消對(duì)槽 slot 的導(dǎo)入(import)或者遷移(migrate)。???

CLUSTER KEYSLOT ?計(jì)算鍵 key 應(yīng)該被放置在哪個(gè)槽上。??

CLUSTER COUNTKEYSINSLOT ?返回槽 slot 目前包含的鍵值對(duì)數(shù)量。??

CLUSTER GETKEYSINSLOT ??返回 count 個(gè) slot 槽中的鍵

READONLY??在集群中的salve節(jié)點(diǎn)開啟只讀模式

READWRITE??禁止讀取請(qǐng)求跳轉(zhuǎn)到集群中的salve節(jié)點(diǎn)上。

今天先看到這里.....

FAQ1:當(dāng)在redis-xx.conf中設(shè)置了密碼,使用redis-trib.rb創(chuàng)建集群時(shí)直接報(bào)報(bào)錯(cuò):?

Sorry, can't connect to node 10.111.178.58:6379 ????這個(gè)錯(cuò)誤提示很不準(zhǔn)確,很容易誤導(dǎo)人的。

1.??首先執(zhí)行搜索 find /?-name client.rb 找到前面 gem install redis 安裝的模塊(默認(rèn)路徑為/usr/lib/ruby/gems/1.8/gems/redis-3.3.3/lib/redis/client.rb),

2.??然后再編輯client.rb,找到如下部分:

class Redis

? class Client

??? DEFAULTS =?{

??????:url?=> lambda { ENV["REDIS_URL"]?},

??????:scheme =>?"redis",

??????:host?=>?"127.0.0.1",

??????:port?=> 6379,

??????:path => nil,

??????:timeout => 5.0,

?:password => nil,

??????:db => 0,

??????:driver => nil,

??????:id => nil,

??????:tcp_keepalive => 0,

??????:reconnect_attempts => 1,

??????:inherit_socket => false

????}

只需將文中?:password => nill 改為?:password?=>?"yourpassword"?即可,再執(zhí)行一遍 redis-trib.rb 創(chuàng)建一次集群有不會(huì)報(bào)錯(cuò)了

參考文獻(xiàn)1

親測(cè)可用配置下載

Windows集群redis搭建

安裝可能出錯(cuò)(大天朝局域網(wǎng))

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡書系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

友情鏈接更多精彩內(nèi)容