一、簡述LVS四種集群特點及使用場景
1、LVS-NAT
多目標(biāo)IP的DNAT,通過將請求報文中的目標(biāo)地址和目標(biāo)端口修改為某挑選出的RS的RIP和PORT實現(xiàn)轉(zhuǎn)發(fā)
(1) RIP和DIP必須在同一個IP網(wǎng)絡(luò),且應(yīng)該使用私網(wǎng)地址;RS的網(wǎng)關(guān)要指向DIP;
(2) 請求報文和響應(yīng)報文都必須經(jīng)由Director轉(zhuǎn)發(fā);Director易于成為系統(tǒng)瓶頸;
(3) 支持端口映射,可修改請求報文的目標(biāo)PORT;
(4) VS必須是Linux系統(tǒng),RS可以是任意系統(tǒng);
應(yīng)用場景:由于配置簡單,節(jié)省IP的特點,一般用在并發(fā)量不大的中小企業(yè);
2、LVS-DR
直接路由;通過為請求報文重新封裝一個MAC首部進(jìn)行轉(zhuǎn)發(fā),源MAC是DIP所在接口的MAC,目標(biāo)MAC是某挑選出的RS的RIP所在接口的MAC地址;源IP/PORT以及目標(biāo)IP/PORT均保持不變;
(1) 確保前端路由器將目標(biāo)IP的VIP的請求報文發(fā)往Director;
(a) 在前端網(wǎng)關(guān)做靜態(tài)綁定;
(b) 在RS上使用arptables;
(c) 在RS上修改內(nèi)核參數(shù)以限制arp通告及應(yīng)答級別;arp_ignore與arp_announce
(2) RS的RIP可以使用私網(wǎng)地址,也可以是公網(wǎng)地址;RIP與DIP在同一IP網(wǎng)絡(luò);RIP的網(wǎng)關(guān)不指向DIP,以確保響應(yīng)報文不會經(jīng)由Director;
(3) RS跟Director要在同一個物理網(wǎng)絡(luò);
(4) 請求報文要經(jīng)由Director,但響應(yīng)不能經(jīng)由Director,而是由RS直接發(fā)往Client;
(5) 不支持端口映射;
應(yīng)用場景:并發(fā)量非常大的情況下會用到此類型,DR模型的并發(fā)處理量能達(dá)到硬件級別的能力;
3、LVS-TUN
轉(zhuǎn)發(fā)方式:不修改請求報文的IP首部(源IP為CIP,目標(biāo)IP為VIP),而是在原IP報文之外再封裝一個IP首部(源IP是DIP,目標(biāo)IP是RIP),將報文發(fā)往挑選出的目標(biāo)RS;RS直接響應(yīng)給客戶端(源IP是VIP,目標(biāo)IP是CIP);
(1) DIP、VIP、RIP都應(yīng)該是公網(wǎng)地址;
(2) RS的網(wǎng)關(guān)不能也不可指向DIP;
(3) 請求報文要經(jīng)由Director,但響應(yīng)不能經(jīng)由Director;
(4) 不支持端口映射;
(5) RS的OS得支持隧道功能;
應(yīng)用場景:如果環(huán)境要求DIP與RIP不在同一物理網(wǎng)絡(luò)(如災(zāi)備)時,就需要用到lvs-tun模型;
4、LVS-FULLNAT
通過同時修改請求報文的源IP地址和目標(biāo)IP地址進(jìn)行轉(zhuǎn)發(fā);
(1) VIP是公網(wǎng)地址,RIP和DIP是私網(wǎng)地址,且通常不在同一IP網(wǎng)絡(luò);因此,RIP的網(wǎng)關(guān)一般不會指向DIP;
(2) RS收到的請求報文源地址是DIP,因此,只能響應(yīng)給DIP;但Director還要將其發(fā)往Client;
(3) 請求和響應(yīng)報文都經(jīng)由Director;
(4) 支持端口映射;
應(yīng)用場景:與lvs-nat類似,解決了跨越網(wǎng)段部署lvs的問題
二、描術(shù)LVS-DR工作原理,并配置實現(xiàn)
主機:四臺,一臺VS服務(wù)器,二臺RS服務(wù)器,一臺客戶端服務(wù)器
網(wǎng)絡(luò)配置:VS服務(wù)器 DIP:192.168.27.7(eth0),RS1服務(wù)器RIP:192.168.27.17(eth0),RS2服務(wù)器RIP:192.168.27.27(eth0),VIP:192.168.27.100(lo:1),客戶端服務(wù)器CIP:192.168.27.37(eth1)
軟件包:keepalived,ipvsadm,httpd(光盤yum源)
1、在VS服務(wù)器上配置
[root@VS ~]# yum install -y ipvsadm
[root@VS ~]# ifconfig eth0:1 192.168.27.100/32
[root@VS ~]# ipvsadm -A -t 192.168.27.100:80 -s wrr
[root@VS ~]# ipvsadm -a -t 192.168.27.100:80 -r 192.168.27.17 -g -w 1
[root@VS ~]# ipvsadm -a -t 192.168.27.100:80 -r 192.168.27.27 -g -w 1
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.27.100:80 wrr
-> 192.168.27.17:80 Route 1 0 0
-> 192.168.27.27:80 Route 1 0 0
2、在RS1服務(wù)器上配置
[root@RS1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@RS1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@RS1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@RS1 ~]# ifconfig lo:1 192.168.27.100/32
[root@RS1 ~]# yum install -y httpd
[root@RS1 ~]# echo 192.168.27.17 RS1 > /var/www/html/index.html
[root@RS1 ~]# systemctl start httpd
3、在RS2服務(wù)器上配置
[root@RS2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS2 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@RS2 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@RS2 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@RS2 ~]# ifconfig lo:1 192.168.27.100/32
[root@RS2 ~]# yum install -y httpd
[root@RS2 ~]# echo 192.168.27.27 RS2 > /var/www/html/index.html
[root@RS2 ~]# systemctl start httpd
4、在客戶端服務(wù)器上測試
[root@client ~]# while true; do curl 192.168.27.100 ;sleep 1; done
192.168.27.17 RS1
192.168.27.27 RS2
192.168.27.17 RS1
192.168.27.27 RS2
192.168.27.17 RS1
192.168.27.27 RS2
192.168.27.17 RS1
192.168.27.27 RS2
三、實現(xiàn)LVS+Keepalived高可用
主機:四臺,兩臺LVS+Keepalived的主備服務(wù)器(lvs1:192.168.27.7,lvs2:192.168.27.17),兩臺RS服務(wù)器(RS1:192.168.27.37,RS2:192.168.27.47)
軟件包:keepalived,ipvsadm,httpd(光盤yum源)
1、兩臺LVS+Keepalived的主備服務(wù)器安裝ipvsadm與keepalived
[root@lvs1 ~]# yum install -y ipvsadm keepalived
[root@lvs2 ~]# yum install -y ipvsadm keepalived
2、 配置keepalived主備與lvs
[root@lvs1 ~]# cp /etc/keepalived/keepalived.conf{,.bak} #先備份
[root@lvs1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1
vrrp_mcast_group4 224.100.100.100
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 66
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
192.168.27.100 dev eth0 label eth0:1
}
}
virtual_server 192.168.27.100 80 {
delay_loop 6
lb_algo wrr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.27.27 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.27.37 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
#從節(jié)點配置與以上大致一樣,只需修改三項
# router_id node1 ----> router_id node2
# state MASTER ----> state BACKUP
# priority 100 ----> priority 80
3、配置RS1與RS2服務(wù)器,先安裝httpd服務(wù),再配置RS服務(wù)器的VIP與內(nèi)核參數(shù)
#RS1
[root@rs1 ~]# yum install -y httpd
[root@rs1 ~]# echo 192.168.27.27 RS1 > /var/www/html/index.html
[root@rs1 ~]# systemctl start httpd
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs1 ~]# ifconfig lo:1 192.168.27.100/32
#RS2
[root@rs2 ~]# yum install -y httpd
[root@rs2 ~]# echo 192.168.27.37 RS1 > /var/www/html/index.html
[root@rs2 ~]# systemctl start httpd
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs2 ~]# ifconfig lo:1 192.168.27.100/32
4、在Keepalived主節(jié)點與備節(jié)點啟動keepalived服務(wù),使用ipvsadm查看LVS集群,并查看VIP的綁定情況
[root@lvs1 ~]# systemctl start keepalived
[root@lvs2 ~]# systemctl start keepalived
[root@lvs1 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.27.100:80 wrr
-> 192.168.27.27:80 Route 1 0 0
-> 192.168.27.37:80 Route 1 0 0
[root@lvs1 ~]# ip a |grep 192.168.27.100 #VIP綁定在主節(jié)點上
inet 192.168.27.100/32 scope global eth0:1
5、 在客戶端測試LVS的調(diào)度情況及故障轉(zhuǎn)移情況
[root@client ~]# while true;do curl 192.168.27.100 ;sleep 1;done
192.168.27.27 RS1
192.168.27.37 RS2
192.168.27.27 RS1
192.168.27.37 RS2
192.168.27.27 RS1
192.168.27.37 RS2
192.168.27.27 RS1
192.168.27.37 RS2
...
#下面先測試VS服務(wù)器(keepalived)的主備故障轉(zhuǎn)移
[root@lvs1 ~]# systemctl stop keepalived
[root@lvs1 ~]# ip a |grep 192.168.27.100 #主節(jié)點上的VIP已轉(zhuǎn)移
[root@lvs2 ~]# ip a |grep 192.168.27.100 #VIP已綁定在備節(jié)點上,而且訪問也未斷
inet 192.168.27.100/32 scope global eth0:1
[root@lvs1 ~]# systemctl start keepalived #重新啟動主節(jié)點
[root@lvs1 ~]# ip a |grep 192.168.27.100 #主節(jié)點又取得VIP
inet 192.168.27.100/32 scope global eth0:1
[root@lvs2 ~]# ip a |grep 192.168.27.100 #備節(jié)點VIP已釋放
#下面測試RS服務(wù)器故障時,lvs的調(diào)度情況
#一開始是輪詢的調(diào)度的,現(xiàn)在關(guān)掉RS1的httpd服務(wù)
[root@rs1 ~]# systemctl stop httpd
#短暫的失敗后,后續(xù)的訪問全調(diào)度給RS2了
[root@client ~]# while true;do curl 192.168.27.100 ;sleep 1;done
192.168.27.27 RS1
192.168.27.37 RS2
192.168.27.27 RS1
192.168.27.37 RS2
curl: (7) Failed connect to 192.168.27.100:80; Connection refused
192.168.27.37 RS2
curl: (7) Failed connect to 192.168.27.100:80; Connection refused
192.168.27.37 RS2
curl: (7) Failed connect to 192.168.27.100:80; Connection refused
192.168.27.37 RS2
curl: (7) Failed connect to 192.168.27.100:80; Connection refused
192.168.27.37 RS2
curl: (7) Failed connect to 192.168.27.100:80; Connection refused
192.168.27.37 RS2
curl: (7) Failed connect to 192.168.27.100:80; Connection refused
192.168.27.37 RS2
192.168.27.37 RS2
#恢復(fù)RS1的httpd服務(wù)
[root@rs1 ~]# systemctl start httpd
#等RS1重新連接正常后,可以看到后續(xù)也參與了調(diào)度
[root@client ~]# while true;do curl 192.168.27.100 ;sleep 1;done
192.168.27.37 RS2
192.168.27.37 RS2
192.168.27.37 RS2
192.168.27.37 RS2
192.168.27.37 RS2
192.168.27.37 RS2
192.168.27.37 RS2
192.168.27.37 RS2
192.168.27.37 RS2
192.168.27.27 RS1
192.168.27.37 RS2
192.168.27.27 RS1
192.168.27.37 RS2
192.168.27.27 RS1
192.168.27.37 RS2
...