zabbix高可用設(shè)計(jì)目標(biāo)
1、keepalived服務(wù)優(yōu)先級(jí)選擇切換機(jī)制：對(duì)于zabbix服務(wù)器來說，只要zabbix存活和mysql存活，就能夠正常記錄數(shù)據(jù)，不會(huì)丟失數(shù)據(jù)，php和httpd只是web頁面的訪問而已，所以我在這里定義mysql和zabbix為主要服務(wù)，php和httpd為次要服務(wù)，為了實(shí)現(xiàn)主要服務(wù)存在，次要服務(wù)掛了；次要服務(wù)器存在，主要服務(wù)器掛了，keepalived會(huì)優(yōu)先選擇主要服務(wù)存在的一方作為master，讓keepalived切換更為更合理。
2、數(shù)據(jù)庫主主同步：不管切到哪一邊都需要保持?jǐn)?shù)據(jù)一致性，不可出現(xiàn)丟數(shù)據(jù)或者數(shù)據(jù)重復(fù)，保證數(shù)據(jù)庫的高可用。
3、文件雙向同步：
? ? ? ? （1）web文件同步：任何做web文件得配置和修改，保證兩邊一致，確保切換不會(huì)發(fā)生任何變化。
? ? ? ? （2）zabbix服務(wù)文件同步：對(duì)zabbix_servr的配置文件做的任何修改，和腳本的修改進(jìn)行同步，也是為了保證兩邊的服務(wù)一致性，沒有落后情況，達(dá)到無需人工干預(yù)自動(dòng)切換服務(wù)可正常時(shí)候。
? ? ? ? （3）到此完成了zabbix得無縫切換，無故障時(shí)間！
一、實(shí)驗(yàn)環(huán)境
操作系統(tǒng)： CentOS7.2 Minial
zabbixA： 192.168.1.102
zabbixB： ?192.168.1.103?
vip： ? ? ? ? ?192.168.1.120
二、軟件安裝
在zabbixA和zabbixB
1.安裝zabbix組件
# yum ?-y install epel-release
# rpm -hiv? ?http://repo.zabbix.com/zabbix/3.5/rhel/7/x86_64/zabbix-release-3.5-1.el7.noarch.rpm
# yum -y install zabbix-server-mysql zabbix-web.noarch zabbix-web-mysql.noarch ?zabbix-java-gateway ?zabbix-get
# yum ?-y install ?zabbix-agent ?zabbix-sender?
2. 安裝數(shù)據(jù)庫組件
# rpm -ivh? ? ?https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm
# yum repolist enabled | grep "mysql.*-community.*"
# yum -y install mysql-community-server ?mysql-community-client
3.安裝php httpd組件
# yum -y install ?php php-gd php-mysql php-bcmath php-mbstring php-xml ?httpd
4.安裝文件同步組件
# ?yum ?-y install rsync ?inotify-tools
5. 雙機(jī)互信，實(shí)現(xiàn)免密登錄
# ssh-keygen -t rsa -P '' -q ?-f ~/.ssh/id_rsa
# ssh-copy-id ?root@$ip
6. 設(shè)置selinux模式和防火墻
#?sysctl?-w?net.ipv4.ip_nonlocal_bind=1
#?echo?"net.ipv4.ip_nonlocal_bind=1"?>>?/etc/sysctl.conf
# setenforce 0
# sed -i 's/^SELINUX=.*/SELINUX=permissive/g' ? /etc/selinux/config
# iptables -F
三、zabbixA軟件配置
1.數(shù)據(jù)庫配置
# systemctl start mysqld.service
# systemctl enable mysqld.service
# cat /var/log/mysqld.log | grep password?
# mysql -u root -p"XXX"
mysql> set global validate_password_policy=0;
mysql> set global validate_password_length=1;
mysql> set password = password('mysql123');
mysql> create database zabbix character set utf8 collate utf8_bin;
mysql> grant all privileges on zabbix.* to zabbix@localhost identified by 'zabbix123';
2.zabbix-server配置
# sed -i ? '/^# SourceIP/s/=.*$/=192.168.1.120/' ? ? ? ? ? ? ? ? ?/etc/zabbix/zabbix_server.conf
# sed -i ? '/^# DBHost/s/=.*$/=192.168.1.120/' ? ? ? ? ? ? ? ? ? ? /etc/zabbix/zabbix_server.conf
# sed -i ?'/^DBName/s/=.*$/=zabbix/' ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/etc/zabbix/zabbix_server.conf
# sed -i ?'/^# DBPassword/s/.*$/DBPassword=zabbix123/' ?/etc/zabbix/zabbix_server.conf
3.php配置
# sed -i ? ? ?'s/post_max_size = 8M/post_max_size = 32M/g' ? ? ? ? ? ? ? ? ? ? ? ? ?/etc/php.ini
# sed -i ? ? ?'s/upload_max_filesize = 2M/upload_max_filesize = 50M/g' ? ? ? ?/etc/php.ini
# sed -i ? ? ? 's/\;date.timezone =/date.timezone = Asia\/Shanghai/' ? ? ? ? ? ? ? /etc/php.ini
# sed -i ? ? ?'s/max_execution_time = 30/max_execution_time = 600/g' ? ? ? ? ?/etc/php.ini
# sed -i ? ? 's/max_input_time = 60/max_input_time = 600/g' ? ? ? ? ? ? ? ? ? ? ? /etc/php.ini
# sed -i ? ? 's/memory_limit = 128M/memory_limit = 256M/g' ? ? ? ? ? ? ? ? ? ? ?/etc/php.ini?
# systemctl restart zabbix-server
# systemctl enable zabbix-server
# systemctl restart httpd
# systemctl enable httpd
4.keepalived配置
# vim /etc/keepalived/keepalived.conf
############################################
! Configuration File for keepalived
global_defs {
? ?router_id LVS_DEVEL
}
vrrp_script check {?
? ? script "/etc/keepalived/check.sh"?
? ? interval 5 ? ??
} ??
vrrp_instance VI_1 {
? ? state BACKUP
? ? interface eno16777736
? ? virtual_router_id 100
? ? priority 100
? ? advert_int 1
? ? nopreempt
? ? authentication {
? ? ? ? auth_type PASS
? ? ? ? auth_pass 1111
? ? }
? ? track_script { ??
? ? ? check
? ? } ??
? ? virtual_ipaddress {
? ? ? ? 192.168.1.120
? ? }
#notify_master "/etc/keepalived/zabbix.sh start"
#notify_backup "/etc/keepalived/zabbix.sh stop"
}
##############################################
# vim /etc/keepalived/check.sh
##############################################
#!/bin/bash
zabbix_server_status1=$(ps -C zabbix_server --no-heading|wc -l)
if [ "${zabbix_server_status1}" = "0" ]; then
? systemctl start zabbix_server.service
? sleep 3
? zabbix_server_status2=$(ps -C zabbix_server --no-heading|wc -l)
? if [ "${zabbix_server_status2}" = "0" ]; then
? ? systemctl restart keepalived.service
? fi
fi
mysqld_status1=$(ps -C mysqld --no-heading|wc -l)
if [ "${mysqld_status1}" = "0" ]; then
? systemctl start mysqld.service
? sleep 3
? mysqld_status2=$(ps -C mysqld --no-heading|wc -l)
? if [ "${mysqld_status2}" = "0" ]; then
? ? systemctl restart keepalived.service
? fi
fi
httpd_status1=$(ps -C httpd --no-heading|wc -l)
if [ "${httpd_status1}" = "0" ]; then
? systemctl start httpd.service
? sleep 3
? httpd_status2=$(ps -C httpd --no-heading|wc -l)
? if [ "${httpd_status2}" = "0" ]; then
? ? systemctl restart keepalived.service
? fi
fi
##########################################
# chmod +x?/etc/keepalived/check.sh
5. 文件同步設(shè)置
# vim /opt/data_realtime_rsync.sh
####################################################################
#!/bin/bash
# File name:data_realtime_sync.sh
# File path: /opt/data_realtime_sync.sh
src_dir="/etc/zabbix"
dest_dir="/etc/zabbix"
src_ip="192.168.1.102"
dest_ip="192.168.1.103"
. /etc/init.d/functions
cd ${src_dir}
inotifywait -mrq -e modify,attrib,close_write,move,create,delete --format '%e %w%f' ./ |
while read file;do
? ? INO_EVENT=$(echo $file | awk '{print $1}') ? ? ?
? ? INO_FILE=$(echo $file ?| awk '{print $2}') ? ? ?
? ? if [[ $INO_EVENT =~ 'CREATE' ]] || [[ $INO_EVENT =~ 'MODIFY' ]] || [[ $INO_EVENT =~ 'CLOSE_WRITE' ]] || [[ $INO_EVENT =~ 'MOVED_TO' ]];then
? ? ? ? for ip in ${dest_ip};do
? ? ? ? ? ? rsync -avzcR -e ssh ?$(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}
? ? ? ? done
? ? fi
? ? if [[ $INO_EVENT =~ 'ATTRIB' ]];then
? ? ? ? if [ ! -d "$INO_FILE" ];then
? ? ? ? ? ? for ip in ${dest_ip};do
? ? ? ? ? ? ? ? rsync -avzcR ?-e ssh ?$(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}
? ? ? ? ? ? done
? ? ? ? fi
? ? fi
? ? if [[ $INO_EVENT =~ 'DELETE' ]] || [[ $INO_EVENT =~ 'MOVED_FROM' ]];then
? ? ? ? for ip in ${dest_ip};do
? ? ? ? ? ? rsync -avzcR --delete ?-e ssh ?$(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}
? ? ? ? done
? ? fi
done
#####################################################################
# chmod +x /opt/data_realtime_rsync.sh
# vim /etc/systemd/system/rsync-inotity.service
###############################################
[Unit]
Description=File instant rsync script
[Service]
ExecStart=/usr/bin/nohup /opt/data_realtime_rsync.sh ?&
[Install]
WantedBy=multi-user.target
#################################################
6.數(shù)據(jù)庫主主同步配置
# vim /etc/my.cnf
######################################################
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
server-id=1 ? ? ? ? ? ? ? ? ? ? ??
log-bin=mysql-bin ? ? ? ? ? ? ? ?
auto-increment-increment = 2 ? ?
auto-increment-offset = 1 ? ? ?
binlog_format = mixed ? ? ? ??
sync_binlog=1 ? ? ? ? ? ? ? ?
binlog-do-db=zabbix ? ? ? ??
binlog-ignore-db=mysql ? ??
binlog-ignore-db=sys
binlog-ignore-db=performance_schema
binlog-ignore-db=information_schema
replicate-do-db=zabbix ? ? ? ? ? ?
replicate-ignore-db=mysql ? ? ? ?
replicate-ignore-db=sys
replicate-ignore-db=performance_schema
replicate-ignore-db=information_schema
######################################################
# systemctl restart mysqld
登錄zabbixB的MySQL
創(chuàng)建數(shù)據(jù)同步賬號(hào)
# mysql -u root -p"mysql123"
mysql> GRANT REPLICATION SLAVE ON *.* TO 'repluser'@'%' IDENTIFIED BY 'Repl@123';
mysql> grant all privileges on zabbix.* to zabbix@192.168.1.120 identified by 'zabbix123';
mysql> FLUSH PRIVILEGES;
# exit
獲取zabbixB二進(jìn)制日志名和位置
#mysql -uroot -p"mysql123" ? ?--execute='show master status;
# mysql -uroot -p$"mysql123" --execute='show master status;'
在zabbixA 上
# ?mysql -uroot -p"mysql123"
mysql>?CHANGE?MASTER?TO
????->?MASTER_HOST='192.168.1.103',
????->?MASTER_USER='repluser',
????->?MASTER_PASSWORD='Repl@123',
????->?MASTER_LOG_FILE='mysql-bin.000002',
????->?MASTER_LOG_POS=50334;
mysql> START SLAVE;
mysql> SHOW SLAVE STATUS\G
四、zabbixB軟件配置
1.數(shù)據(jù)庫配置
# systemctl start mysqld.service
# systemctl enable mysqld.service
# cat /var/log/mysqld.log | grep password?
# mysql -u root -p"XXX"
mysql> set global validate_password_policy=0;
mysql> set global validate_password_length=1;
mysql> set password = password('mysql123');
mysql> create database zabbix character set utf8 collate utf8_bin;
mysql> grant all privileges on zabbix.* to zabbix@localhost identified by 'zabbix123';
2.zabbix-server配置
# sed -i ? '/^ SourceIP/s/=.*$/=192.168.1.120/' ? ? ? ? ? ? ? ? ? ? /etc/zabbix/zabbix_server.conf
# sed -i ? '/^# DBHost/s/=.*$/=192.168.1.120/' ? ? ? ? ? ? ? ??? ? /etc/zabbix/zabbix_server.conf
# sed -i ?'/^DBName/s/=.*$/=zabbix/' ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/etc/zabbix/zabbix_server.conf
# sed -i ?'/^# DBPassword/s/.*$/DBPassword=zabbix123/' ? /etc/zabbix/zabbix_server.conf
3.php配置
# sed -i ? ? ?'s/post_max_size = 8M/post_max_size = 32M/g' ? ? ? ? ? ? ? ? ? ? ? ? ?/etc/php.ini
# sed -i ? ? ?'s/upload_max_filesize = 2M/upload_max_filesize = 50M/g' ? ? ? ?/etc/php.ini
# sed -i ? ? ? 's/\;date.timezone =/date.timezone = Asia\/Shanghai/' ? ? ? ? ? ? ? /etc/php.ini
# sed -i ? ? ?'s/max_execution_time = 30/max_execution_time = 600/g' ? ? ? ? ?/etc/php.ini
# sed -i ? ? 's/max_input_time = 60/max_input_time = 600/g' ? ? ? ? ? ? ? ? ? ? ? /etc/php.ini
# sed -i ? ? 's/memory_limit = 128M/memory_limit = 256M/g' ? ? ? ? ? ? ? ? ? ? ?/etc/php.ini?
# systemctl restart zabbix-server
# systemctl enable zabbix-server
# systemctl restart httpd
# systemctl enable httpd
4.keepalived配置
# vim /etc/keepalived/keepalived.conf
############################################
! Configuration File for keepalived
global_defs {
? ?router_id LVS_DEVEL
}
vrrp_script check {?
? ? script "/etc/keepalived/check.sh"?
? ? interval 5 ? ??
} ??
vrrp_instance VI_1 {
? ? state BACKUP
? ? interface eno16777736
? ? virtual_router_id 90
? ? priority 100
? ? advert_int 1
? ? nopreempt
? ? authentication {
? ? ? ? auth_type PASS
? ? ? ? auth_pass 1111
? ? }
? ? track_script { ??
? ? ? check
? ? } ??
? ? virtual_ipaddress {
? ? ? ? 192.168.1.120
? ? }
#notify_master "/etc/keepalived/zabbix.sh start"
#notify_backup "/etc/keepalived/zabbix.sh stop"
}
##############################################
# vim /etc/keepalived/check.sh
##############################################
#!/bin/bash
zabbix_server_status1=$(ps -C zabbix_server --no-heading|wc -l)
if [ "${zabbix_server_status1}" = "0" ]; then
? systemctl start zabbix_server.service
? sleep 3
? zabbix_server_status2=$(ps -C zabbix_server --no-heading|wc -l)
? if [ "${zabbix_server_status2}" = "0" ]; then
? ? systemctl restart keepalived.service
? fi
fi
mysqld_status1=$(ps -C mysqld --no-heading|wc -l)
if [ "${mysqld_status1}" = "0" ]; then
? systemctl start mysqld.service
? sleep 3
? mysqld_status2=$(ps -C mysqld --no-heading|wc -l)
? if [ "${mysqld_status2}" = "0" ]; then
? ? systemctl restart keepalived.service
? fi
fi
httpd_status1=$(ps -C httpd --no-heading|wc -l)
if [ "${httpd_status1}" = "0" ]; then
? systemctl start httpd.service
? sleep 3
? httpd_status2=$(ps -C httpd --no-heading|wc -l)
? if [ "${httpd_status2}" = "0" ]; then
? ? systemctl restart keepalived.service
? fi
fi
##########################################
# chmod +x?/etc/keepalived/check.sh
5. 文件同步設(shè)置
# vim /opt/data_realtime_rsync.sh
####################################################################
#!/bin/bash
# File name:data_realtime_sync.sh
# File path: /opt/data_realtime_sync.sh
src_dir="/etc/zabbix"
dest_dir="/etc/zabbix"
src_ip="192.168.1.103"
dest_ip="192.168.1.102"
. /etc/init.d/functions
cd ${src_dir}
inotifywait -mrq -e modify,attrib,close_write,move,create,delete --format '%e %w%f' ./ |
while read file;do
? ? INO_EVENT=$(echo $file | awk '{print $1}') ? ? ?
? ? INO_FILE=$(echo $file ?| awk '{print $2}') ? ? ?
? ? if [[ $INO_EVENT =~ 'CREATE' ]] || [[ $INO_EVENT =~ 'MODIFY' ]] || [[ $INO_EVENT =~ 'CLOSE_WRITE' ]] || [[ $INO_EVENT =~ 'MOVED_TO' ]];then
? ? ? ? for ip in ${dest_ip};do
? ? ? ? ? ? rsync -avzcR -e ssh ?$(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}
? ? ? ? done
? ? fi
? ? if [[ $INO_EVENT =~ 'ATTRIB' ]];then
? ? ? ? if [ ! -d "$INO_FILE" ];then
? ? ? ? ? ? for ip in ${dest_ip};do
? ? ? ? ? ? ? ? rsync -avzcR ?-e ssh ?$(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}
? ? ? ? ? ? done
? ? ? ? fi
? ? fi
? ? if [[ $INO_EVENT =~ 'DELETE' ]] || [[ $INO_EVENT =~ 'MOVED_FROM' ]];then
? ? ? ? for ip in ${dest_ip};do
? ? ? ? ? ? rsync -avzcR --delete ?-e ssh ?$(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}
? ? ? ? done
? ? fi
done
#####################################################################
# chmod +x /opt/data_realtime_rsync.sh
# vim /etc/systemd/system/rsync-inotity.service
###############################################
[Unit]
Description=File instant rsync script
[Service]
ExecStart=/usr/bin/nohup /opt/data_realtime_rsync.sh ?&
[Install]
WantedBy=multi-user.target
#################################################
?6.數(shù)據(jù)庫主主同步配置
# vim /etc/my.cnf
######################################################
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
server-id=2 ? ? ? ? ? ? ? ? ? ? ??
log-bin=mysql-bin ? ? ? ? ? ? ? ?
auto-increment-increment = 2 ? ?
auto-increment-offset = 2 ? ? ?
binlog_format = mixed ? ? ? ??
sync_binlog=1 ? ? ? ? ? ? ? ?
binlog-do-db=zabbix ? ? ? ??
binlog-ignore-db=mysql ? ??
binlog-ignore-db=sys
binlog-ignore-db=performance_schema
binlog-ignore-db=information_schema
replicate-do-db=zabbix ? ? ? ??
replicate-ignore-db=mysql ? ??
replicate-ignore-db=sys
replicate-ignore-db=performance_schema
replicate-ignore-db=information_schema
######################################################
# systemctl restart mysqld
登錄zabbixA的MySQL
創(chuàng)建數(shù)據(jù)同步賬號(hào)
# mysql -u root -p"mysql123"
mysql> GRANT REPLICATION SLAVE ON *.* TO 'repluser'@'%' IDENTIFIED BY 'Repl@123';
mysql>?grant all privileges on zabbix.* to zabbix@192.168.1.120 identified by 'zabbix123';
mysql> FLUSH PRIVILEGES;
# exit
獲取zabbixA二進(jìn)制日志名和位置
#mysql -uroot -p"mysql123" ? ?--execute='show master status;
# mysql -uroot -p$"mysql123" --execute='show master status;'
在zabbixB上
#?mysql -uroot -p"mysql123"
mysql>?CHANGE?MASTER?TO
????->?MASTER_HOST='192.168.1.102',
????->?MASTER_USER='repluser',
????->?MASTER_PASSWORD='Repl@123',
????->?MASTER_LOG_FILE='mysql-bin.000002',
????->?MASTER_LOG_POS=50334;
mysql> START SLAVE;
mysql> SHOW SLAVE STATUS\G
?五、zabbixA ?zabbixB服務(wù)啟動(dòng)
# systemctl start ?zabbix-server
# systemctl enable zabbix-server
# systemctl start ?rsync-inotify
# systemctl enable ?rsync-inotify
# cat /etc/zabbix/zabbix_server.conf | grep 120
# cat /etc/zabbix/zabbix_server.conf | grep -Ev "^#|^$"
# vim ?/etc/zabbix/web/zabbix.conf.php
六、訪問測(cè)試
七、參考
inotify+rsync+mysql主主復(fù)制+keepalived實(shí)現(xiàn)zabbix高可用
http://blog.51cto.com/3241766/2137303
Keepalived中Master和Backup主備切換機(jī)制淺析
http://blog.51cto.com/3241766/2097483
Zabbix高可用，實(shí)現(xiàn)zabbix的無縫切換，無故障時(shí)間
http://blog.51cto.com/yigemeng/1738174
實(shí)現(xiàn)Zabbix的高可用
https://www.longlong.asia/2016/10/29/zabbix-ha.html
keepalived + rsync +inotify 實(shí)現(xiàn)真正的高效數(shù)據(jù)實(shí)時(shí)同步
https://fandenggui.com/post/keepalived-rsync-inotify.html
真正的inotify+rsync實(shí)時(shí)同步 徹底告別同步慢
http://www.ttlsa.com/web/let-infotify-rsync-fast
Zabbix HA Requirment
https://kasperdeng.github.io/zabbix-ha-on-aws
Configuring the Zabbix server for high availability
