一、在線集群安裝:
將以下shell保存至腳本文件redis-install.sh
#!/bin/bash
for ((i=83;i<132;i++)) do
??????? ssh root@192.168.212.$i yum -y install epel-release
??????? ssh root@192.168.212.$i yum -y install redis
done
執(zhí)行腳本:?$?./redis-install.sh ??(執(zhí)行完畢將在從192.168.212.83~192.168.212.131的所有機(jī)器上都安裝了redis)
啟動(dòng):?$?/usr/sbin/redis-server /etc/redis.conf
-------------------------------------華麗的分割線---------------------------------------------
二、在線安裝:
首先安裝依賴:
yum?-y install tcl ruby rubygems
gem install redis
下面我們簡單的使用一下,操作系統(tǒng)是一臺(tái)ubuntu14.04系統(tǒng)(啟動(dòng)6個(gè)實(shí)例,
3 Master/3 Slave):
1.?下載最新版redis
wget 'http://download.redis.io/releases/redis-3.0.4.tar.gz'
2.??編譯安裝
#安裝到/opt/apps/redis-cluster目錄
tar zxvf redis-3.0.4.tar.gz
cd redis-3.0.4/
make PREFIX=/opt/apps/redis-cluster install
#創(chuàng)建配置目錄,拷貝相關(guān)文件
mkdir -p?/opt/apps/redis-cluster/{bin,log,conf,rdb,run,nodes}
cp src/redis-trib.rb /opt/apps/redis-cluster/bin/
3.??安裝其他依賴包
yum install ruby1.9.1
gem install redis ???#?也可離線安裝,后面會(huì)介紹(在只有內(nèi)網(wǎng)的機(jī)器上非常有用)
4.??創(chuàng)建配置文件
看看6379的具體信息:grep?-v?-E?'(^#|^$)' redis-6379.conf
-------------------------------------華麗的分割線---------------------------------------------
daemonize yes
pidfile /opt/apps/redis-cluster/run/redis-6379.pid
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 0
loglevel notice
logfile "/opt/apps/redis-cluster/log/redis-6379.log"
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump-6379.rdb
dir /opt/redis-cluster/rdb
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly yes
appendfilename "redis-6379.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
#?其實(shí)大部分都是原有的配置信息,不需要做大的改動(dòng),與端口相關(guān)的都需要調(diào)整,與集群相關(guān)的配置如下:
cluster-enabled yes
cluster-config-file nodes-6379.conf
cluster-node-timeout 15000
#?這里得注意,如果在配置文件里指定了密碼,那么在創(chuàng)建集群的時(shí)候?qū)?huì)出現(xiàn)認(rèn)證錯(cuò)誤,解決辦法下面“FAQ1”會(huì)提到。
masterauth yourpassword
requirepass yourpassword
-------------------------------------華麗的分割線---------------------------------------------
注意其中的nodes-6379.conf這個(gè)文件不需要?jiǎng)?chuàng)建,在初始化集群的時(shí)候會(huì)自動(dòng)創(chuàng)建的。
5.??在啟動(dòng)redis實(shí)例前先修改一些系統(tǒng)級(jí)別的配置
echo never >?/sys/kernel/mm/transparent_hugepage/enabled
#?打開/etc/sysctl.conf,追加如下內(nèi)容
vm.overcommit_memory?= 1
#使配置生效
sysctl -p
6.??啟動(dòng)所有redis實(shí)例
cd?/opt/redis-cluster/bin
./redis-server?/opt/apps/redis-cluster/conf/redis-6379.conf
./redis-server?/opt/apps/redis-cluster/conf/redis-6380.conf
./redis-server?/opt/apps/redis-cluster/conf/redis-6381.conf
./redis-server?/opt/apps/redis-cluster/conf/redis-7379.conf
./redis-server?/opt/apps/redis-cluster/conf/redis-7380.conf
./redis-server?/opt/apps/redis-cluster/conf/redis-7381.conf
7.? redis-trib.rb
??????? redis-trib.rb是一個(gè)官方提供的用來操作cluster的ruby腳本,我們后面管理cluster會(huì)經(jīng)常使用到這個(gè)腳本
Usage: redis-trib ??
? create ???????? host1:port1 ... hostN:portN
??????????????????--replicas
? check ????????? host:port
? fix ??????????? host:port
? reshard ??????? host:port
??????????????????--from
??????????????????--to?
??????????????????--slots?
??????????????????--yes
? add-node ?????? new_host:new_port existing_host:existing_port
??????????????????--slave
??????????????????--master-id
? del-node ?????? host:port node_id
? set-timeout ??? host:port milliseconds
? call ?????????? host:port command arg arg .. arg
? import ???????? host:port
??????????????????--from
? help ???????????(show this help)
For check, fix, reshard, del-node, set-timeout you can specify the host and port of any working node in the cluster.
8 ???初始化啟動(dòng)cluster(一定要記得--replicas后面有個(gè) 1)
./redis-trib.rb create --replicas1?127.0.0.1:6379 127.0.0.1:6380 127.0.0.1:6381 127.0.0.1:7379 127.0.0.1:7380 127.0.0.1:7381
??????結(jié)果如下:
>>> Creating cluster
Connecting to node 172.16.1.100:6379: OK
Connecting to node 172.16.1.100:6380: OK
Connecting to node 172.16.1.100:6381: OK
Connecting to node 172.16.1.100:7379: OK
Connecting to node 172.16.1.100:7380: OK
Connecting to node 172.16.1.100:7381: OK
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:????###注意三個(gè)6xxx的都被定義為master的了
172.16.1.100:6379
172.16.1.100:6380
172.16.1.100:6381
Adding replica 172.16.1.100:7379 to 172.16.1.100:6379 ??###注意三個(gè)7xxx的都被定義為相關(guān)的slave了
Adding replica 172.16.1.100:7380 to 172.16.1.100:6380
Adding replica 172.16.1.100:7381 to 172.16.1.100:6381
M: cdb8b1fe29feb9a564fbfed6599aa61dda250eb1 172.16.1.100:6379
?? slots:0-5460 (5461 slots) master ???###6379被分配了0-5460個(gè)slots
M: a188b59b30056f61c1cf55ff5072d60b6f8ce5d7 172.16.1.100:6380
?? slots:5461-10922 (5462 slots) master ??###6380被分配了5461-10922個(gè)slots
M: fada90b0520d5aa3305bc89651cc7bb5de9b2a16 172.16.1.100:6381
?? slots:10923-16383 (5461 slots) master ????###6381被分配了10923-16383個(gè)slots
S: 818fb192631cf8c34bc58f24d2538444abe7d995 172.16.1.100:7379 ???###剩余的三個(gè)slave
?? replicates cdb8b1fe29feb9a564fbfed6599aa61dda250eb1
S: 1cf6ed9f8a5049513fb631e78207c7f0b1ba6674 172.16.1.100:7380
?? replicates a188b59b30056f61c1cf55ff5072d60b6f8ce5d7
S: bc09734e243b9b2e9fe2f64c55aaf72073c6918b 172.16.1.100:7381
?? replicates fada90b0520d5aa3305bc89651cc7bb5de9b2a16
Can I set the above configuration??(type 'yes' to accept): yes ????###是否在nodes配置文件中保存更新配置
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.....
>>> Performing Cluster Check (using node 172.16.1.100:6379)
M: cdb8b1fe29feb9a564fbfed6599aa61dda250eb1 172.16.1.100:6379
?? slots:0-5460 (5461 slots) master
M: a188b59b30056f61c1cf55ff5072d60b6f8ce5d7 172.16.1.100:6380
?? slots:5461-10922 (5462 slots) master
M: fada90b0520d5aa3305bc89651cc7bb5de9b2a16 172.16.1.100:6381
?? slots:10923-16383 (5461 slots) master
M: 818fb192631cf8c34bc58f24d2538444abe7d995 172.16.1.100:7379
?? slots:?(0 slots) master
?? replicates cdb8b1fe29feb9a564fbfed6599aa61dda250eb1
M: 1cf6ed9f8a5049513fb631e78207c7f0b1ba6674 172.16.1.100:7380
?? slots:?(0 slots) master
?? replicates a188b59b30056f61c1cf55ff5072d60b6f8ce5d7
M: bc09734e243b9b2e9fe2f64c55aaf72073c6918b 172.16.1.100:7381
?? slots:?(0 slots) master
?? replicates fada90b0520d5aa3305bc89651cc7bb5de9b2a16
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
????檢查一下cluster的狀態(tài):redis-trib.rb check 127.0.0.1:6379
9 ?接入cluster,并存入一個(gè)key/value?(name/guo)
redis-cli?-c?-h 127.0.0.1 -p 6379
????注意:redis-cli在接入集群模式時(shí),要使用-c參數(shù)??
????????可以看到我在6379接入cluster中,并存入可一個(gè)name/guodalu鍵值對(duì),但是cluster卻把我保存的值定位到7380去了,而我使用6381接入cluster中后,再次獲取name值時(shí),會(huì)被自動(dòng)的定位到7380上。我們來看看是我是怎么被定位到7380的。
??????? 1 先計(jì)算guodalu的CRC16值是0xA36B,轉(zhuǎn)化成10進(jìn)制是41835
??????? 2 41835對(duì)16384取模是9067,9067 slot是落在6380上的,而7380是6380的slave。
????下面我們把7380停止,可以看到再次檢查cluster的時(shí)候,并不會(huì)報(bào)錯(cuò)。
????我們?cè)?381上接入cluster,獲取name
10 ?恢復(fù)7380,停止6380
??????可以看到這時(shí)候check cluster的時(shí)候,會(huì)提示一個(gè)節(jié)點(diǎn)已經(jīng)故障,我們這時(shí)候再嘗試獲取key
????????可以看到還是可以獲取到的。如果把6380和7380同時(shí)停止呢?
????????如果cluster中的master和slave全部故障的話,則這部分?jǐn)?shù)據(jù)還是不可用的。
11 ???在cluster中新增一個(gè)redis master節(jié)點(diǎn)
????????啟動(dòng)redis 8379實(shí)例
./redis-server /opt/redis-cluster/conf/redis-8379.conf
????????然后把該實(shí)例加入cluster中:第一個(gè)參數(shù)是新加入的redis節(jié)點(diǎn),第二個(gè)參數(shù)是cluster中的任何一個(gè)已有的節(jié)點(diǎn)
./redis-trib.rb add-node 127.0.0.1:8379 127.0.0.1:6379
????這時(shí)候檢查一下cluster狀態(tài)
./redis-trib.rb check 127.0.0.1:6379
????????可以看到8379已經(jīng)加入cluster中了,默認(rèn)是以master加入cluster的,而且副本為1,但是這個(gè)新的master卻沒有被cluster分配slots,所以它里面是沒有數(shù)據(jù)的,當(dāng)有slave服務(wù)器要升級(jí)為master服務(wù)器時(shí),它不能參與選舉。這時(shí)候我們可以使用重新劃分slots的特性給新加入的8379遷移其他節(jié)點(diǎn)上的slots,這個(gè)后面操作。
12 ???給8379添加一個(gè)slave節(jié)點(diǎn)
./redis-server?/opt/redis-cluster/conf/redis-8380.conf
./redis-trib.rb add-node --slave?--master-id 63ec1e7d57b3e19c6e1fa07768673001cfa0d30f 127.0.0.1:8380 127.0.0.1:6379
#或者
./redis-trib.rb add-node --slave 127.0.0.1:8380 127.0.0.1:6379
????????第一個(gè)添加slvae的命令是直接指定把該節(jié)點(diǎn)作為哪個(gè)master的slave加入cluster,而第二個(gè)命令redis-trib 會(huì)添加該新節(jié)點(diǎn)作為一個(gè)具有較少副本的隨機(jī)的主服務(wù)器的副本(在本環(huán)境中顯然是8379的副本了)。
????再次檢查cluster狀態(tài)
./redis-trib.rb check 127.0.0.1:6379
????????可以看到8379已經(jīng)有一個(gè)slave了,但是它的slots還是0,因?yàn)槲覀円恢睕]有給它遷移slots。
13 ?刪除一個(gè)redis slave節(jié)點(diǎn)
我們先刪除8380 slave節(jié)點(diǎn),第一個(gè)參數(shù)是cluster中的任意一個(gè)節(jié)點(diǎn),第二個(gè)參數(shù)要?jiǎng)h除的slave的ID
./redis-trib.rb del-node 172.16.1.100:6379 12decfd0d975ef0dcd151d0ff232da7a2269a1c2
????????這時(shí)候再次檢查cluster的狀態(tài)
./redis-trib.rb check 127.0.0.1:6379
????????注意,紅框中的8379的副本又變?yōu)?了。
14 ?刪除一個(gè)redis master節(jié)點(diǎn)
????????刪除master節(jié)點(diǎn)和刪除slave節(jié)點(diǎn)操作一樣,不過為了移除一個(gè)master節(jié)點(diǎn),它必須是空的。如果master不是空的,你需要先將其數(shù)據(jù)重分片到其他的master節(jié)點(diǎn)。另一種移除master節(jié)點(diǎn)的方式,就是在其從服務(wù)器上執(zhí)行一次手工故障轉(zhuǎn)移,當(dāng)它變?yōu)榱诵碌膍aster的slave以后將其移除。
????????因?yàn)槲覀冞€沒操作slots重新分配,就先不刪除8379了。
15 ?? cluster重新分片
????????經(jīng)過測(cè)試發(fā)現(xiàn)cluster的重新分片并不是在新的cluster內(nèi)再次均勻的分配slots,而是需要是手動(dòng)指定把哪個(gè)節(jié)點(diǎn)的多少slots遷移到另一個(gè)節(jié)點(diǎn)去.......
./redis-trib.rb reshard 127.0.0.1:6379
看看最終結(jié)果
????????在經(jīng)過我一番瞎遷移后,最終成為這樣了。當(dāng)然你也可以使用如下命令進(jìn)行嚴(yán)格的遷移:
./redis-trib.rb reshard :?--from??--to??--slots?--yes
16??刪除8379 master節(jié)點(diǎn)
????????因?yàn)?379的slots都被我遷移到其他節(jié)點(diǎn)去了,而刪除一個(gè)master節(jié)點(diǎn)需要該節(jié)點(diǎn)的數(shù)據(jù)為空
./redis-trib.rb del-node 127.0.0.1:6379 63ec1e7d57b3e19c6e1fa07768673001cfa0d30f
以上redis-trib的這些操作,在redis-cli中也是可以進(jìn)行操作的:
CLUSTER INFO 打印集群的信息??
CLUSTER NODES 列出集群當(dāng)前已知的所有節(jié)點(diǎn)(node),以及這些節(jié)點(diǎn)的相關(guān)信息。??
CLUSTER RESET? reset
CLUSTER SAVECONFIG ?強(qiáng)制節(jié)點(diǎn)保存集群當(dāng)前狀態(tài)到磁盤上。
CLUSTER SLOTS 獲得slot在節(jié)點(diǎn)上的映射關(guān)系
CLUSTER MEET ??將 ip 和 port 所指定的節(jié)點(diǎn)添加到集群當(dāng)中,讓它成為集群的一份子。??
CLUSTER FORGET ?從集群中移除 node_id 指定的節(jié)點(diǎn)。??
CLUSTER REPLICATE ?將當(dāng)前節(jié)點(diǎn)設(shè)置為 node_id 指定的節(jié)點(diǎn)的從節(jié)點(diǎn)。????
CLUSTER SLAVES ??列出該slave節(jié)點(diǎn)的master節(jié)點(diǎn)
CLUSTER ADDSLOTS ?[slot?...]?將一個(gè)或多個(gè)槽(slot)指派(assign)給當(dāng)前節(jié)點(diǎn)。??
CLUSTER DELSLOTS ?[slot?...]?移除一個(gè)或多個(gè)槽對(duì)當(dāng)前節(jié)點(diǎn)的指派。??
CLUSTER FLUSHSLOTS 移除指派給當(dāng)前節(jié)點(diǎn)的所有槽,讓當(dāng)前節(jié)點(diǎn)變成一個(gè)沒有指派任何槽的節(jié)點(diǎn)。??
CLUSTER SETSLOT NODE ?將槽 slot 指派給 node_id 指定的節(jié)點(diǎn),如果槽已經(jīng)指派給另一個(gè)節(jié)點(diǎn),那么先讓另一個(gè)節(jié)點(diǎn)刪除該槽>,然后再進(jìn)行指派。??
CLUSTER SETSLOT MIGRATING ?將本節(jié)點(diǎn)的槽 slot 遷移到 node_id 指定的節(jié)點(diǎn)中。??
CLUSTER SETSLOT IMPORTING ?從 node_id 指定的節(jié)點(diǎn)中導(dǎo)入槽 slot 到本節(jié)點(diǎn)。??
CLUSTER SETSLOT STABLE 取消對(duì)槽 slot 的導(dǎo)入(import)或者遷移(migrate)。???
CLUSTER KEYSLOT ?計(jì)算鍵 key 應(yīng)該被放置在哪個(gè)槽上。??
CLUSTER COUNTKEYSINSLOT ?返回槽 slot 目前包含的鍵值對(duì)數(shù)量。??
CLUSTER GETKEYSINSLOT ??返回 count 個(gè) slot 槽中的鍵
READONLY??在集群中的salve節(jié)點(diǎn)開啟只讀模式
READWRITE??禁止讀取請(qǐng)求跳轉(zhuǎn)到集群中的salve節(jié)點(diǎn)上。
今天先看到這里.....
FAQ1:當(dāng)在redis-xx.conf中設(shè)置了密碼,使用redis-trib.rb創(chuàng)建集群時(shí)直接報(bào)報(bào)錯(cuò):?
Sorry, can't connect to node 10.111.178.58:6379 ????這個(gè)錯(cuò)誤提示很不準(zhǔn)確,很容易誤導(dǎo)人的。
1.??首先執(zhí)行搜索 find /?-name client.rb 找到前面 gem install redis 安裝的模塊(默認(rèn)路徑為/usr/lib/ruby/gems/1.8/gems/redis-3.3.3/lib/redis/client.rb),
2.??然后再編輯client.rb,找到如下部分:
class Redis
? class Client
??? DEFAULTS =?{
??????:url?=> lambda { ENV["REDIS_URL"]?},
??????:scheme =>?"redis",
??????:host?=>?"127.0.0.1",
??????:port?=> 6379,
??????:path => nil,
??????:timeout => 5.0,
?:password => nil,
??????:db => 0,
??????:driver => nil,
??????:id => nil,
??????:tcp_keepalive => 0,
??????:reconnect_attempts => 1,
??????:inherit_socket => false
????}
只需將文中?:password => nill 改為?:password?=>?"yourpassword"?即可,再執(zhí)行一遍 redis-trib.rb 創(chuàng)建一次集群有不會(huì)報(bào)錯(cuò)了