mysql環(huán)境配置
步驟 1: 開(kāi)啟 binlog
[root@node01 ~]# vim /etc/my.cnf
server_id=1
log-bin=mysql-bin
binlog_format=row
步驟 2: 賦權(quán)限
[root@node01 ~]# mysql -uroot -p
注：由于5.7版本MySQL有密碼設(shè)置的規(guī)范?？梢允褂胷oot用戶登陸后 降低MySQL密碼安全級(jí)別，或者使用復(fù)雜的密碼創(chuàng)建canal用戶
mysql> SHOW VARIABLES LIKE 'validate_password%';
+--------------------------------------+--------+
| Variable_name | Value |
+--------------------------------------+--------+
| validate_password_check_user_name | OFF |
| validate_password_dictionary_file | |
| validate_password_length | 8 |
| validate_password_mixed_case_count | 1 |
| validate_password_number_count | 1 |
| validate_password_policy | MEDIUM |
| validate_password_special_char_count | 1 |
+--------------------------------------+--------+
7 rows in set (0.02 sec)
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)
mysql> SHOW VARIABLES LIKE 'validate_password%';
+--------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------+-------+
| validate_password_check_user_name | OFF |
| validate_password_dictionary_file | |
| validate_password_length | 4 |
| validate_password_mixed_case_count | 1 |
| validate_password_number_count | 1 |
| validate_password_policy | LOW |
| validate_password_special_char_count | 1 |
+--------------------------------------+-------+
7 rows in set (0.01 sec)
mysql> alter user 'canal'@'%' identified by 'canal';
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE USER canal IDENTIFIED BY 'canal';
mysql> GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%';
mysql> FLUSH PRIVILEGES;
步驟 3: 重啟 mysql 使 binlog 生效
[root@node01 ~]# systemctl restart mysqld
步驟 4: 檢查 binlog 是否生效
進(jìn)入 mysql
mysql> show variables like'log_%';
+----------------------------------------+--------------------------------+
| Variable_name | Value |
+----------------------------------------+--------------------------------+
| log_bin | ON |
| log_bin_basename | /var/lib/mysql/mysql-bin |
| log_bin_index | /var/lib/mysql/mysql-bin.index |
| log_bin_trust_function_creators | OFF |
| log_bin_use_v1_row_events | OFF |
| log_builtin_as_identified_by_password | OFF |
| log_error | /var/log/mysqld.log |
| log_error_verbosity | 3 |
| log_output | FILE |
| log_queries_not_using_indexes | OFF |
| log_slave_updates | OFF |
| log_slow_admin_statements | OFF |
| log_slow_slave_statements | OFF |
| log_statements_unsafe_for_binlog | ON |
| log_syslog | OFF |
| log_syslog_facility | daemon |
| log_syslog_include_pid | ON |
| log_syslog_tag | |
| log_throttle_queries_not_using_indexes | 0 |
| log_timestamps | UTC |
| log_warnings | 2 |
+----------------------------------------+--------------------------------+
21 rows in set (0.01 sec)
安裝 canal 和啟動(dòng)
步驟 1: 下載 Canal
wget https://github.com/alibaba/canal/releases/download/canal-1.1.2/canal.deployer-1.1.2.tar.gz
步驟 2: 解壓
mkdir /opt/module/canal
tar -zxvf canal.deployer-1.1.2.tar.gz -C /opt/module/canal
步驟 3: 配置
- conf/canal.properties canal 的通用配置
- conf/example/instance.propertiesinstance.properties是針對(duì)要追蹤的 mysql 的實(shí)例配置
#################################################
## mysql serverId , v1.0.26+ will autoGen
canal.instance.mysql.slaveId=1
# enable gtid use true/false
canal.instance.gtidon=false
# position info
canal.instance.master.address=node01:3306
canal.instance.master.journal.name=
canal.instance.master.position=
canal.instance.master.timestamp=
canal.instance.master.gtid=
# rds oss binlog
canal.instance.rds.accesskey=
canal.instance.rds.secretkey=
canal.instance.rds.instanceId=
# table meta tsdb info
canal.instance.tsdb.enable=true
#canal.instance.tsdb.url=jdbc:mysql://127.0.0.1:3306/canal_tsdb
#canal.instance.tsdb.dbUsername=canal
#canal.instance.tsdb.dbPassword=canal
#canal.instance.standby.address =
#canal.instance.standby.journal.name =
#canal.instance.standby.position =
#canal.instance.standby.timestamp =
#canal.instance.standby.gtid=
# username/password
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
canal.instance.connectionCharset = UTF-8
canal.instance.defaultDatabaseName =
# enable druid Decrypt database password
canal.instance.enableDruid=false
#canal.instance.pwdPublicKey=MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALK4BUxdDltRRE5/zXpVEVPUgunvscYFtEip3pmLlhrWpacX7y7GCMo2/JM6LeHmiiNdH1FWgGCpUfircSwlWKUCAwEAAQ==
# table regex
canal.instance.filter.regex=.*\\..*
# table black regex
canal.instance.filter.black.regex=
# mq config
canal.mq.topic=example
canal.mq.partition=0
# hash partition config
#canal.mq.partitionsNum=3
#canal.mq.partitionHash=mytest.person:id,mytest.role:id
#################################################
步驟 4: 啟動(dòng) canal
[root@node01 canal]# bin/startup.sh
[root@node01 canal]# jps
3520 NameNode
3522 DataNode
483 CanalLauncher
549 Jps
15367 NodeManager
15368 ResourceManager
16296 HistoryServer
15369 JobHistoryServer
32619 Main
32620 EventCatcherService
13677 Main
32621 AlertPublisher
16595 RunJar
17492 EmbeddedOozieServer
16597 RunJar
32599 Main
18335 YarnSessionClusterEntrypoint
步驟 5: 查看日志
[root@node01 canal]# ls logs/
canal example
