HA Cluster 高可用集群keepalived

一、簡述HA Cluster原理

高可用集群,英文原文為High Availability Cluster,簡稱HA Cluster;集群(cluster)就是一組計(jì)算機(jī),它們作為一個整體向用戶提供一組網(wǎng)絡(luò)資源。這些單個的計(jì)算機(jī)系統(tǒng) 就是集群的節(jié)點(diǎn)(node)。高可用性集群(HA cluster)是指如單系統(tǒng)一樣地運(yùn)行并支持(計(jì)算機(jī))持續(xù)正常運(yùn)行的一個主機(jī)群。

高可用集群的出現(xiàn)是為了使集群的整體服務(wù)盡可能可用,從而減少由計(jì)算機(jī)硬件和軟件易錯性所帶來的損 失。如果某個節(jié)點(diǎn)失效,它的備援節(jié)點(diǎn)將在幾 秒鐘的時間內(nèi)接管它的職責(zé)。因此,對于用戶而言,集群永遠(yuǎn)不會停機(jī)。高可用集群軟件的主要作用就是實(shí)現(xiàn)故障檢查和業(yè)務(wù)切換的自動化。

簡單說高可用集群就是為了解決集群中的單點(diǎn)故障(SPoF),保證服務(wù)不間斷運(yùn)行的冗余(redundant)手段。

  • SPoF:Single Point of Failure;單點(diǎn)故障
  • 冗余(redundant): 在兩個節(jié)點(diǎn)上裝一個軟件程序,根據(jù)判斷狀態(tài)完成資源轉(zhuǎn)移;
高可用集群的衡量標(biāo)準(zhǔn)

通常用平均無故障時間(MTTF)來度量系統(tǒng)的可靠性,用平均故障維修時間(MTTR)來度量系統(tǒng)的可維護(hù)性。于是可用性被定義為:HA=MTTF/(MTTF+MTTR)*100%

  • 可用性衡量指標(biāo):
    • 基本可用性:2個9;99%;年度停機(jī)時間 87.6小時
    • 較高可用性:3個9;99.9%;年度停機(jī)時間 8.8小時
    • 具有故障自動恢復(fù)能力的可用性:4個9;99.99%;年度停機(jī)時間53分鐘
    • 極高可用性:5個9;99.999%;年度停機(jī)時間5分鐘

二、keepalived

  1. KeepAlived主要有兩個功能:
  • (1).能夠?qū)ealServer進(jìn)行健康狀況檢查,支持4層、5層和7層協(xié)議進(jìn)行健康檢查;
  • (2).對負(fù)載均衡調(diào)度器實(shí)現(xiàn)高可用,防止Director單點(diǎn)故障。
  1. KeepAlived工作過程:
    keepalived實(shí)現(xiàn)故障轉(zhuǎn)移的功能是通過VRRP(virtual router redundancy protocol虛擬路由器冗余協(xié)議)協(xié)議來實(shí)現(xiàn)的。 在keepalived正常工作的時候,主節(jié)點(diǎn)(master)會不斷的發(fā)送心跳信息給備節(jié)點(diǎn)(backup),當(dāng)備節(jié)點(diǎn)不能在一定時間內(nèi)收到主節(jié)點(diǎn)的心跳信息時,備節(jié)點(diǎn)會認(rèn)為主節(jié)點(diǎn)宕了,然后會接管主節(jié)點(diǎn)上的資源,并繼續(xù)向外提供服務(wù)保證其可用性。當(dāng)主節(jié)點(diǎn)恢復(fù)的時候,備節(jié)點(diǎn)會自動讓出資源并再次自動成為備節(jié)點(diǎn)。
  1. keepalived基于vrrp協(xié)議的軟件實(shí)現(xiàn),原生設(shè)計(jì)的目的為了高可用ipvs服務(wù);
  • 基于vrrp協(xié)議完成地址流動;
  • 為vip地址所在的節(jié)點(diǎn)生成ipvs規(guī)則(在配置文件中預(yù)先定義);
  • 為ipvs集群的各RS做健康狀態(tài)檢測;
  • 基于腳本調(diào)用接口通過執(zhí)行腳本完成腳本中定義的功能,進(jìn)而影響集群事務(wù);
  1. HA Cluster的配置前提:
    (1) 各節(jié)點(diǎn)時間必須同步; ntp, chrony
    (2) 確保iptables及selinux不會成為阻礙;
    (3) 各節(jié)點(diǎn)之間可通過主機(jī)名互相通信(對KA并非必須);
    建議使用/etc/hosts文件實(shí)現(xiàn);
    (4) 確保各節(jié)點(diǎn)的用于集群服務(wù)的接口支持MULTICAST通信;
    D類:224-239;
  2. keepalived安裝配置:
    在CentOS6.4以后,keepalivd隨base倉庫提供;
  • 程序環(huán)境:
    主配置文件:/etc/keepalived/keepalived.conf
    主程序文件:/usr/sbin/keepalived
    nit File:keepalived.service
    Unit File的環(huán)境配置文件:/etc/sysconfig/keepalived

  • 配置文件組件部分:
    TOP HIERACHY
    - GLOBAL CONFIGURATION
    - Global definitions
    - Static routes/addresses
    - VRRPD CONFIGURATION
    - VRRP synchronization group(s):vrrp同步組;
    - VRRP instance(s):每個vrrp instance即一個vrrp路由器;
    - LVS CONFIGURATION
    - Virtual server group(s)
    - Virtual server(s):ipvs集群的vs和rs;

  • 配置語法:

    • 配置虛擬路由器:
      vrrp_instance <STRING> { .... }

    • 專用參數(shù):
      state MASTER|BACKUP:當(dāng)前節(jié)點(diǎn)在此虛擬路由器上的初始狀態(tài);只能有一個是MASTER,余下的都應(yīng)該為BACKUP;
      interface IFACE_NAME:綁定為當(dāng)前虛擬路由器使用的物理接口;
      virtual_router_id VRID:當(dāng)前虛擬路由器的惟一標(biāo)識,范圍是0-255;
      priority 100:當(dāng)前主機(jī)在此虛擬路徑器中的優(yōu)先級;范圍1-254;
      advert_int 1:vrrp通告的時間間隔;

                    authentication {
                              auth_type AH|PASS  
                            auth_pass <PASSWORD>
                    }```
                    `virtual_ipaddress {
                        <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>
                        192.168.200.17/24 dev eth1
                        192.168.200.18/24 dev eth2 label eth2:1
                    }
                     track_interface {
                        eth0
                        eth1
                        ...
                    }`  # 配置要監(jiān)控的網(wǎng)絡(luò)接口,一旦接口出現(xiàn)故障,則轉(zhuǎn)為FAULT狀態(tài);
      
                    `nopreempt:`定義工作模式為非搶占模式;
                    `preempt_delay 300:`搶占式模式下,節(jié)點(diǎn)上線后觸發(fā)新選舉操作的延遲時長;                       
      
  • 定義通知腳本:

                          notify_master <STRING>|<QUOTED-STRING>:當(dāng)前節(jié)點(diǎn)成為主節(jié)點(diǎn)時觸發(fā)的腳本;
                          notify_backup <STRING>|<QUOTED-STRING>:當(dāng)前節(jié)點(diǎn)轉(zhuǎn)為備節(jié)點(diǎn)時觸發(fā)的腳本;
                          notify_fault <STRING>|<QUOTED-STRING>:當(dāng)前節(jié)點(diǎn)轉(zhuǎn)為“失敗”狀態(tài)時觸發(fā)的腳本;                           
                          notify <STRING>|<QUOTED-STRING>:通用格式的通知觸發(fā)機(jī)制,一個腳本可完成以上三種狀態(tài)的轉(zhuǎn)換時的通知;
    
  • 虛擬服務(wù)器:
    配置參數(shù):

        virtual_server IP port |
        virtual_server fwmark int 
        {
            ...
            real_server {
                ...
            }
            ...
        }
    

常用參數(shù):
delay_loop <INT>:服務(wù)輪詢的時間間隔;
lb_algo rr|wrr|lc|wlc|lblc|sh|dh:定義調(diào)度方法;
lb_kind NAT|DR|TUN:集群的類型;
persistence_timeout <INT>:持久連接時長;
protocol TCP:服務(wù)協(xié)議,僅支持TCP;
sorry_server <IPADDR> <PORT>:備用服務(wù)器地址;

real_server <IPADDR> <PORT>
                {
                     weight <INT>
                     notify_up <STRING>|<QUOTED-STRING>
                     notify_down <STRING>|<QUOTED-STRING>
                     HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK { ... }:定義當(dāng)前主機(jī)的健康狀態(tài)檢測方法;
                }
  • HTTP_GET|SSL_GET:應(yīng)用層檢測

              HTTP_GET|SSL_GET {
                  url {
                      path <URL_PATH>:定義要監(jiān)控的URL;
                      status_code <INT>:判斷上述檢測機(jī)制為健康狀態(tài)的響應(yīng)碼;
                      digest <STRING>:判斷上述檢測機(jī)制為健康狀態(tài)的響應(yīng)的內(nèi)容的校驗(yàn)碼;
                  }
                  nb_get_retry <INT>:重試次數(shù);
                  delay_before_retry <INT>:重試之前的延遲時長;
                  connect_ip <IP ADDRESS>:向當(dāng)前RS的哪個IP地址發(fā)起健康狀態(tài)檢測請求
                  connect_port <PORT>:向當(dāng)前RS的哪個PORT發(fā)起健康狀態(tài)檢測請求
                  bindto <IP ADDRESS>:發(fā)出健康狀態(tài)檢測請求時使用的源地址;
                  bind_port <PORT>:發(fā)出健康狀態(tài)檢測請求時使用的源端口;
                  connect_timeout <INTEGER>:連接請求的超時時長;
              }
    
  • TCP_CHECK:傳輸層檢測

               TCP_CHECK {
                  connect_ip <IP ADDRESS>:向當(dāng)前RS的哪個IP地址發(fā)起健康狀態(tài)檢測請求
                  connect_port <PORT>:向當(dāng)前RS的哪個PORT發(fā)起健康狀態(tài)檢測請求
                  bindto <IP ADDRESS>:發(fā)出健康狀態(tài)檢測請求時使用的源地址;
                  bind_port <PORT>:發(fā)出健康狀態(tài)檢測請求時使用的源端口;
                  connect_timeout <INTEGER>:連接請求的超時時長;
              }
    

三、keepalived實(shí)現(xiàn)主從、主主架構(gòu)

  1. 主從配置:
    準(zhǔn)備2個節(jié)點(diǎn):node1:192.168.1.108;node2:192.168.1.109
    同步時間:[root@node1 ~]# ntpdate 192.168.1.10
    安裝配置keepalived:
    在node1如下配置

     [root@node1 ~]# yum -y install keepalived         #安裝keepalived
     [root@node1 ~]# cd /etc/keepalived/
     [root@node1 keepalived]# cp keepalived.conf{,.bak}    #備份keepalived原始配置文件
     [root@node1 keepalived]# vim keepalived.conf
     #在打開的文件中配置如下內(nèi)容
         ! Configuration File for keepalived
    
             global_defs {
                    notification_email {
                     root@localhost
                }
                 
                    notification_email_from keepalived@localhost
                    smtp_server 127.0.0.1
                    smtp_connect_timeout 30
                    router_id node1
                     vrrp_mcast_group4 224.1.105.33
                 }
             
             vrrp_instance VI_1 {
                     state MASTER   #當(dāng)前節(jié)點(diǎn)在此虛擬路由器上的初始狀態(tài);只能有一個是MASTER,余下的都應(yīng)該為BACKUP;
                     interface ens33
                     virtual_router_id 33
                     priority 100
                     advert_int 1
                     authentication {
                         auth_type PASS
                         auth_pass 1111
                     }
                     virtual_ipaddress {
                         192.168.1.99 dev ens33 label ens33:0
                     }
                 }
    

在node2節(jié)點(diǎn)上如下配置:

    [root@node2 ~]# yum -y install keepalived         #安裝keepalived
    [root@node2 ~]# cd /etc/keepalived/
    [root@node2 keepalived]# cp keepalived.conf{,.bak}    #備份keepalived原始配置文件
    [root@node2 keepalived]# vim keepalived.conf
    #在打開的文件中配置如下內(nèi)容
        ! Configuration File for keepalived

            global_defs {
                   notification_email {
                    root@localhost
               }
                
                   notification_email_from keepalived@localhost
                   smtp_server 127.0.0.1
                   smtp_connect_timeout 30
                   router_id node2
                    vrrp_mcast_group4 224.1.105.33
                }
            
            vrrp_instance VI_1 {
                    state BACKUP
                    interface ens33
                    virtual_router_id 33
                    priority 96
                    advert_int 1
                    authentication {
                        auth_type PASS
                        auth_pass 1111
                    }
                    virtual_ipaddress {
                        192.168.1.99 dev ens33 label ens33:0
                    }
                }

啟動node2節(jié)點(diǎn)keepalived測試

[root@node2 ~]# systemctl start keepalived
[root@node2 ~]# ifconfig
...
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.99  netmask 255.255.255.255  broadcast 0.0.0.0
    ether 00:0c:29:f0:be:9c  txqueuelen 1000  (Ethernet)
[root@node2 ~]# systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2018-08-26 09:37:52 CST; 7s ago
  Process: 15928 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 15929 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─15929 /usr/sbin/keepalived -D
           ├─15930 /usr/sbin/keepalived -D
           └─15931 /usr/sbin/keepalived -D

Aug 26 09:37:52 node2 Keepalived_vrrp[15931]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Aug 26 09:37:56 node2 Keepalived_vrrp[15931]: VRRP_Instance(VI_1) Transition to MASTER STATE
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: VRRP_Instance(VI_1) Entering MASTER STATE
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: VRRP_Instance(VI_1) setting protocol VIPs.
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ...1.99
Aug 26 09:37:57 node2 Keepalived_vrrp[15931]: Sending gratuitous ARP on ens33 for 192.168.1.99

#在node1節(jié)點(diǎn)上抓包測試
[root@node1 ~]# tcpdump -i ens33 -nn host 224.1.105.33
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
09:37:56.221751 IP 192.168.1.109 > 224.1.105.33: VRRPv2, Advertisement, vrid 33, prio 96, authtype simple, intvl 1s, length 20
09:37:57.227332 IP 192.168.1.109 > 224.1.105.33: VRRPv2, Advertisement, vrid 33, prio 96, authtype simple, intvl 1s, length 20

啟動node1節(jié)點(diǎn)keepalived:

[root@node1 ~]# systemctl start keepalived
[root@node1 ~]# ifconfig
...
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 192.168.1.99  netmask 255.255.255.255  broadcast 0.0.0.0
    ether 00:0c:29:21:8d:06  txqueuelen 1000  (Ethernet)
[root@node1 ~]# systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Sun 2018-08-26 09:42:23 CST; 1min 9s ago
  Process: 15076 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 15077 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─15077 /usr/sbin/keepalived -D
           ├─15078 /usr/sbin/keepalived -D
           └─15079 /usr/sbin/keepalived -D

Aug 26 09:42:25 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:25 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:25 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:25 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: VRRP_Instance(VI_1) Sending/queueing gratuitous ....99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Aug 26 09:42:30 www.ilinux.com Keepalived_vrrp[15079]: Sending gratuitous ARP on ens33 for 192.168.1.99
Hint: Some lines were ellipsized, use -l to show in full.

#node1節(jié)點(diǎn)抓包測試
[root@node1 ~]# tcpdump -i ens33 -nn host 224.1.105.33
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), capture size 262144 bytes
09:43:18.304748 IP 192.168.1.108 > 224.1.105.33: VRRPv2, Advertisement, vrid 33, prio 100, authtype simple, intvl 1s, length 20
09:43:19.305917 IP 192.168.1.108 > 224.1.105.33: VRRPv2, Advertisement, vrid 33, prio 100, authtype simple, intvl 1s, length 20
  1. 雙主模式配置

     #node1節(jié)點(diǎn)上修改keepalived.conf配置文件,在最后添加如下內(nèi)容
     vrrp_instance VI_ 2{
         state BACKUP
         interface ens33
         virtual_router_id 34
         priority 96
         advert_int 1
         authentication {
             auth_type PASS
             auth_pass XXXX1111
         }
         virtual_ipaddress {
             192.168.1.98 dev ens33 label ens33:0
         }
     }
    
     #node2節(jié)點(diǎn)上修改keepalived.conf配置文件,在最后添加如下內(nèi)容
     vrrp_instance VI_2 {
         state MASTER
         interface ens33
         virtual_router_id 34
         priority 100
         advert_int 1
         authentication {
             auth_type PASS
             auth_pass XXXX1111
         }
         virtual_ipaddress {
             192.168.1.98 dev ens33 label ens33:0
         }
     }
      #停止keepalived服務(wù),再重新啟動
      [root@node2 ~]# systemctl stop keepalived  
     [root@node2 ~]# systemctl start keepalived
     [root@node2 ~]# ip a l
     ...
     2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
         link/ether 00:0c:29:f0:be:9c brd ff:ff:ff:ff:ff:ff
         inet 192.168.1.109/24 brd 192.168.1.255 scope global ens33
            valid_lft forever preferred_lft forever
         inet 192.168.1.98/32 scope global ens33:0
            valid_lft forever preferred_lft forever
         inet6 fe80::7221:3e5e:6c0:4c1c/64 scope link 
            valid_lft forever preferred_lft forever
     ...
     [root@node2 ~]# systemctl status keepalived
     ● keepalived.service - LVS and VRRP High Availability Monitor
        Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
        Active: active (running) since Sun 2018-08-26 10:17:01 CST; 1min 8s ago
       Process: 16217 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
      Main PID: 16218 (keepalived)
        CGroup: /system.slice/keepalived.service
                ├─16218 /usr/sbin/keepalived -D
                ├─16219 /usr/sbin/keepalived -D
                └─16220 /usr/sbin/keepalived -D
    
     Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
     Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
     Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
     Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ...1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Hint: Some lines were ellipsized, use -l to show in full
    
     #重新啟動node1節(jié)點(diǎn)的keepalived服務(wù)
      [root@node1 keepalived]# systemctl start keepalived
     [root@node1 keepalived]# ip a l
     ...
     2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
         link/ether 00:0c:29:21:8d:06 brd ff:ff:ff:ff:ff:ff
         inet 192.168.1.108/24 brd 192.168.1.255 scope global dynamic ens33
            valid_lft 2403sec preferred_lft 2403sec
         inet 192.168.1.99/32 scope global ens33:0
            valid_lft forever preferred_lft forever
         inet6 fe80::959:d8ab:dd39:b1b/64 scope link 
            valid_lft forever preferred_lft forever
     ...
     [root@node1 keepalived]# systemctl status keepalived
     ● keepalived.service - LVS and VRRP High Availability Monitor
        Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
        Active: active (running) since Sun 2018-08-26 10:19:19 CST; 1min 24s ago
       Process: 15146 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
      Main PID: 15147 (keepalived)
        CGroup: /system.slice/keepalived.service
                ├─15147 /usr/sbin/keepalived -D
                ├─15148 /usr/sbin/keepalived -D
                └─15149 /usr/sbin/keepalived -D
    
     Aug 26 10:19:22 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:22 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:22 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:22 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: VRRP_Instance(VI_1) Sending/queueing gratuitous ....99
     Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:27 www.ilinux.com Keepalived_vrrp[15149]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Hint: Some lines were ellipsized, use -l to show in full.
    
     #   在node2節(jié)點(diǎn)上status查看狀態(tài)
     [root@node2 ~]# systemctl status keepalived
     ● keepalived.service - LVS and VRRP High Availability Monitor
        Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
        Active: active (running) since Sun 2018-08-26 10:17:01 CST; 4min 59s ago
       Process: 16217 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
      Main PID: 16218 (keepalived)
        CGroup: /system.slice/keepalived.service
                ├─16218 /usr/sbin/keepalived -D
                ├─16219 /usr/sbin/keepalived -D
                └─16220 /usr/sbin/keepalived -D
    
     Aug 26 10:17:08 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.98
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) Sending/queueing gratuitous ARPs on ...1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:17:11 node2 Keepalived_vrrp[16220]: Sending gratuitous ARP on ens33 for 192.168.1.99
     Aug 26 10:19:21 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) Received advert with higher priority...s 96
     Aug 26 10:19:21 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) Entering BACKUP STATE
     Aug 26 10:19:21 node2 Keepalived_vrrp[16220]: VRRP_Instance(VI_1) removing protocol VIPs.
    
  2. 通知腳本使用方式

     #編輯通知腳本
     [root@node1 ~]# cd /etc/keepalived/
     [root@node1 keepalived]# vim notify.sh
             #!/bin/bash
         #keepalived 郵件通知腳本
     #date: 2018-8-26
     contact = 'root@localhost'
     notify () {
         local mailsubject="$(hostname) to be $1 vip floating"
         local mailbody="$(data + '%F %T'): vrrp transition, $(hostname) changed to be $1"
         echo "$mailbody" | mail -s "$mailsubject" $contact
     }
     case $1 in
     master)
         notify master
         ;;
     backup)
         notify backup
         ;;
     fault)
         notify fault
         ;;
     *)
         echo "Usage: $(basename $0) {master|backup|fault}"
         exit 1
         ;;
     esac
     #在keepalived.conf中的vrrp實(shí)例中添加如下內(nèi)容
     vrrp_instance VI_1 {
                     state BACKUP
                     interface ens33
                     virtual_router_id 33
                     priority 96
                     advert_int 1
                     authentication {
                         auth_type PASS
                         auth_pass 1111
                     }
                     virtual_ipaddress {
                         192.168.1.99 dev ens33 label ens33:0
                     }
                     notify_master "/etc/keepalived/notify.sh master"
                     notify_backuo "/etc/keepalived/notify.sh backup"
                     notify_fault "/etc/keepalived/notify.sh fault"
                 }
    
  3. 以dr集群架構(gòu)配置示例


    dr架構(gòu).png
[root@node1 ~]# yum -y install ipvsadm #安裝ipvsadm以便查看生成的規(guī)則
#編輯keepalived.conf為node1和node2生成規(guī)則
[root@node1 keepalived]# vim keepalived.conf
    ! Configuration File for keepalived

        global_defs {
           notification_email {
            root@localhost
           }
            
           notification_email_from keepalived@localhost
           smtp_server 127.0.0.1
           smtp_connect_timeout 30
           router_id node1
            vrrp_mcast_group4 224.1.105.33
        }
        
        vrrp_instance VI_1 {
            state MASTER
            interface ens33
            virtual_router_id 33
            priority 100
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass XXXX1111
            }
            virtual_ipaddress {
                192.168.1.99 dev ens33 label ens33:0
            }
             notify_master "/etc/keepalived/notify.sh master"
             notify_backup "/etc/keepalived/notify.sh backup"
             notify_fault "/etc/keepalived/notify.sh fault"
        }
        virtual_server 192.168.1.99 80 {
            delay_loop 1
            lb_algo wrr
            lb_kind DR
            protocol TCP
            sorry_server 127.0.0.1 80
        
            real_server 192.168.1.111 80 {
                weight 1
                HTTP_GET {
                    url {
                        path /index.html
                        status_code 200
                        }
                    nb_get_retry 3
                    delay_before_retry 2
                    connect_timeout 3
                    }
            }
            real_server 192.168.1.122 80 {
                weight 1
                HTTP_GET {
                    url {
                        path /index.html
                        status_code 200
                        }
                    nb_get_retry 3
                    delay_before_retry 2
                    connect_timeout 3
                    }
            }
                    
        }
#將此配置文件拷貝到node2節(jié)點(diǎn),并修改以下幾行
    router_id node2
    state BACKUP
    priority 96
#重新啟動node2節(jié)點(diǎn)的keepalived服務(wù)
[root@node2 ~]# systemctl stop keepalived
[root@node2 ~]# systemctl start keepalived
[root@node2 ~]# ifconfig
...
ens33:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.99  netmask 255.255.255.255  broadcast 0.0.0.0
        ether 00:0c:29:f0:be:9c  txqueuelen 1000  (Ethernet)

...

[root@node2 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.99:80 wrr
  -> 192.168.1.111:80             Route   1      0          0         
  -> 192.168.1.122:80             Route   1      0          0     
  
#使用client訪問測試正常
[root@localhost ~]# curl http://192.168.1.99
<h1>RealServer 1</h1>
[root@localhost ~]# curl http://192.168.1.99
<h1>RealServer 2</h1>

#啟動node1的keepalived服務(wù),通過下面查看ip和status后看到node1已經(jīng)成功上線
[root@node1 keepalived]# systemctl start keepalived
[root@node1 keepalived]# ifconfig
[root@node1 keepalived]# systemctl status keepalived
#使用client訪問服務(wù)正常
[root@localhost ~]# curl http://192.168.1.99
<h1>RealServer 1</h1>
[root@localhost ~]# curl http://192.168.1.99
<h1>RealServer 2</h1>

四、高可用nginx配置示例

高可用nginx架構(gòu).png

@以上圖的架構(gòu)配置實(shí)驗(yàn)網(wǎng)絡(luò)環(huán)境;node1和node2雙網(wǎng)卡,dip橋接,私網(wǎng)地址VMent1,rs1和rs2為后端主機(jī),配置兩個訪問index.html為RealServer 1和RealServer 2;

  • 在node1節(jié)點(diǎn)上安裝配置keepalived服務(wù)

    [root@node1 ~]# yum -y install keepalived
    [root@node1 ~]# vim /etc/keepalived/keepalived.conf
    #按如下內(nèi)容編輯配置文件
          ! Configuration File for keepalived
    
          global_defs {
                 notification_email {
                  root@localhost
             }                    
                 notification_email_from keepalived@localhost
                 smtp_server 127.0.0.1
                 smtp_connect_timeout 30
                 router_id node1
                  vrrp_mcast_group4 224.1.105.33
              }
          
          vrrp_script chk_down {
                  script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
                  weight -10
                  interval 1
                  fall 1
                  rise 1
              }   
          
          
          vrrp_instance VI_1 {
                  state MASTER
                  interface ens33
                  virtual_router_id 33
                  priority 100
                  advert_int 1                    
                  authentication {
                      auth_type PASS
                      auth_pass XXXX1111
                  }
                  virtual_ipaddress {
                      192.168.1.99/24 dev ens33 label ens33:0
                  }
                  track_script {
                      chk_down
                  }
               notify_master "/etc/keepalived/notify.sh master"
               notify_backup "/etc/keepalived/notify.sh backup"
               notify_fault "/etc/keepalived/notify.sh fault"
          }
    
  • 安裝配置nginx服務(wù)

      [root@node1 ~]# yum -y install nginx
      [root@node1 ~]# vim /etc/nginx/nginx.conf
          #在配置文件中http{...}段中添加如下內(nèi)容
          upstream websrvs {
                  server 192.168.10.111:80;
                  server 192.168.10.122:80;
          }
          #在location段中配置反向代理讀物
          location / {
                  proxy_pass http://websrvs;
          }
    
  • 修改通知腳本

      [root@node1 ~]# vim /etc/keepalived/notify.sh 
          #!/bin/bash
          #keepalived 郵件通知腳本
          #2018-8-26
          contact = 'root@localhost'
          notify () {
              local mailsubject="$(hostname) to be $1 vip floating"
              local mailbody="$(data + '%F %T'): vrrp transition, $(hostname) changed to be $1"
              echo "$mailbody" | mail -s "$mailsubject" $contact
          }
          case $1 in
          master)
              systemctl start nginx
              notify master
              ;;
          backup)
              systemctl start nginx
              notify backup
              ;;
          fault)
              systemctl start nginx
              notify fault
              ;;
          *)
              echo "Usage: $(basename $0) {master|backup|fault}"
              exit 1
              ;;
          esac
    
  • 在keepalived配置中定義nginx腳本并調(diào)用

  • 配置雙主模式的keepalived服務(wù),在keepalived.conf配置如下內(nèi)容

      vrrp_instance VI_2 {
                  state BACKUP
                  interface ens33
                  virtual_router_id 43
                  priority 96
                  advert_int 1
               authentication {
                      auth_type PASS
                      auth_pass XXXX1111
                  }   
                  virtual_ipaddress {
                      192.168.1.98/24 dev ens33 label ens33:0
                  }   
                      track_script {
                              chk_down
                              chk_ngx
                      }       
                       notify_master "/etc/keepalived/notify.sh master"
                       notify_backup "/etc/keepalived/notify.sh backup"
                       notify_fault "/etc/keepalived/notify.sh fault"
              }        
    
  • 配置檢測網(wǎng)卡狀態(tài),在keepalived.conf配置文件最后添加如下內(nèi)容

      track_interface {
          ens33
          ens37
      }
    
  • 至此node1節(jié)點(diǎn)配置完成,下面配置node2節(jié)點(diǎn)

      [root@node1 ~]# scp /etc/keepalived/keepalived.conf /etc/nginx/nginx.conf root@192.168.1.109:      #復(fù)制keepalived和nginx服務(wù)的配置文件到node2
      [root@node2 ~]# mv nginx.conf /etc/nginx/     
      [root@node2 ~]# nginx -t
      [root@node2 ~]# systemctl start nginx
      [root@node2 ~]# mv keepalived.conf /etc/keepalived/ 
      [root@node2 ~]# vim /etc/keepalived/keepalived.conf
          #修改如下內(nèi)容
          router_id node2
          state BACKUP
          priority 96
          vrrp_instance VI_2 {
              state MASTER
              virtual_router_id 43
              priority 100
    
  • 至此配置完成,啟動keepalived服務(wù)后可以使用client訪問測試

      [root@localhost ~]# curl http://www.ilinux.io
      <h1>RealServer 1</h1>
      [root@localhost ~]# curl http://www.ilinux.io
      <h1>RealServer 2</h1>
      [root@localhost ~]# curl http://www.ilinux.io
      <h1>RealServer 1</h1>
    
最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點(diǎn),簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容