實(shí)時(shí)同步概述

• 實(shí)時(shí)同步概念說(shuō)明
  實(shí)時(shí)同步的概念：是一種只要當(dāng)前目錄發(fā)生變化則會(huì)觸發(fā)一個(gè)事件，事件出發(fā)后會(huì)將變化的目錄同步至遠(yuǎn)程服務(wù)器。

• 實(shí)時(shí)同步的作用：
  保證數(shù)據(jù)的連續(xù)性；
  減少人力維護(hù)成本。

• 數(shù)據(jù)備份的方法：
  定時(shí)任務(wù)備份：主要是內(nèi)部人員備份數(shù)據(jù)，備份周期最短是1分鐘。
  實(shí)時(shí)任務(wù)備份：主要是外部人員備份數(shù)據(jù)，一般是用戶保存數(shù)據(jù)，當(dāng)用戶在網(wǎng)站提交保存數(shù)據(jù)時(shí)，沒(méi)有同步等待時(shí)間。

• 實(shí)時(shí)同步的原理：
  1.監(jiān)控存儲(chǔ)服務(wù)器，查看數(shù)據(jù)的信息變化，主要用到的工具為inotify；
  2.利用同步傳輸數(shù)據(jù)軟件，將變化的數(shù)據(jù)增量傳輸給遠(yuǎn)程服務(wù)器；
  3.實(shí)現(xiàn)實(shí)時(shí)同步。

實(shí)時(shí)同步的工具：sersync+rsync，inotify+rsync
Inotify是一個(gè)通知接口，用來(lái)監(jiān)控文件系統(tǒng)的各種變化，如果文件存取，刪除，移動(dòng)?？梢苑浅７奖愕貙?shí)現(xiàn)文件異動(dòng)告警，增量備份功能，并針對(duì)目錄或文件的變化及時(shí)作出響應(yīng)。Inotify+rsync可以實(shí)出發(fā)式實(shí)時(shí)同步增量備份。
inotify的作用：監(jiān)控目錄中數(shù)據(jù)信息變化；
sersync是國(guó)人基于rsync+inotify-tools開(kāi)發(fā)的工具，不僅保留了inotify的優(yōu)點(diǎn)同時(shí)還強(qiáng)化了實(shí)時(shí)監(jiān)控，文件過(guò)濾，簡(jiǎn)化配置等功能，幫助用戶提高了運(yùn)行效率，節(jié)省時(shí)間和網(wǎng)絡(luò)資源。

sersync服務(wù)安裝部署

• 第一步：安裝軟件，將軟件推送到指定的目錄

[root@nfs01 ~]# yum install rsync inotify -y
[root@nfs01-server ~]# cd /server/tools/
[root@nfs01-server tools]# ls
sersync_installdir_64bit.zip
[root@nfs01-server tools]# mv sersync_installdir_64bit.zip  /usr/local/
[root@nfs01-server tools]# unzip /usr/local/sersync_installdir_64bit.zip 
Archive:  /usr/local/sersync_installdir_64bit.zip
   creating: sersync_installdir_64bit/
   creating: sersync_installdir_64bit/sersync/
   creating: sersync_installdir_64bit/sersync/bin/
  inflating: sersync_installdir_64bit/sersync/bin/sersync  
   creating: sersync_installdir_64bit/sersync/conf/
  inflating: sersync_installdir_64bit/sersync/conf/confxml.xml  
   creating: sersync_installdir_64bit/sersync/logs/

• 第二步：修改配置文件信息

<inotify>
 13         <delete start="true"/>
 14         <createFolder start="true"/>
 15         <createFile start="false"/>
 16         <closeWrite start="true"/>
 17         <moveFrom start="true"/>
 18         <moveTo start="true"/>
 19         <attrib start="false"/>
 20         <modify start="false"/>
 21     </inotify>
 <sersync>
 24         <localpath watch="/backup"> ## 目錄信息
 25           <remote ip="172.16.1.41" name="backup"/> ## 備份服務(wù)器IP，backup模塊信息
 26             <!--<remote ip="192.168.8.39" name="tongbu"/>-->
 27             <!--<remote ip="192.168.8.40" name="tongbu"/>-->
 28         </localpath>
 29         <rsync>
 30             <commonParams params="-avz"/> ## 選項(xiàng)參數(shù)，默認(rèn)有-delete
 31             <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>                 ## 開(kāi)啟認(rèn)證用戶功能   
 32             <userDefinedPort start="false" port="874"/><!-- port=874 -->
 33             <timeout start="false" time="100"/><!-- timeout=100 -->
 34             <ssh start="false"/>
 35         </rsync>

[root@nfs01-server local]# ll /usr/local/sersync_installdir_64bit/sersync/bin/sersync        
-rwxr--r-- 1 root root 1810128 Oct 26  2011 /usr/local/sersync_installdir_64bit/sersync/bin/sersync

[root@nfs01-server bin]# ./sersync -dro  /usr/local/sersync_installdir_64bit/sersync/conf/confxml.xml 
set the system param
execute：echo 50000000 > /proc/sys/fs/inotify/max_user_watches
execute：echo 327679 > /proc/sys/fs/inotify/max_queued_events
parse the command param ## 命令參數(shù)解釋說(shuō)明
option: -d      run as a daemon
option: -r      rsync all the local files to the remote servers before the sersync work
option: -o      config xml name：  /usr/local/sersync_installdir_64bit/sersync/conf/confxml.xml
daemon thread num: 10
parse xml config file
host ip : localhost     host port: 8008
daemon start，sersync run behind the console 
use rsync password-file :
user is rsync_backup
passwordfile is         /etc/rsync.password
config xml parse success
please set /etc/rsyncd.conf max connections=0 Manually
sersync working thread 12  = 1(primary thread) + 1(fail retry thread) + 10(daemon sub threads) 
Max threads numbers is: 22 = 12(Thread pool nums) + 10(Sub threads)
please according your cpu ，use -n param to adjust the cpu rate
------------------------------------------
rsync the directory recursivly to the remote servers once
working please wait...
execute command: cd /backup && rsync -avz -R --delete ./ rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password >/dev/null 2>&1 
run the sersync: 
watch path is: /backup ## 成功??！

實(shí)現(xiàn)實(shí)時(shí)同步的腳本

[root@nfs01 scripts]# cat sersync.sh

#!/bin/bash
#1. jiankong date
inotifywait -mrq /backup --format "%w%f" -e create,delete,close_write,move | while read line

#2. push data
do
rsync -avz --delete /backup rsync_backup@172.16.1.41::backup --password-file=/etc/rsync.password
done