目录
5-01 mysql 复制功能介绍
5-02 mysql 二进制日志
5-03 mysql 二进制日志格式对复制的影响
对于数据库事务型的要求,所以不能简单的通过增加数据库服务器来分担读写负载,而Mysql的读复制功能提供分担读负载功能。
动图演示:
分析:可以使用复制功能对数据库服务器进行水平扩展,为数据库服务器增加一个或者多个备库,而备库可以用于分担主数据库的读负载,同时复制功能也为高可用、灾难恢复、备份功能提供更多的选择。
Mysql的复制是基于主库的二进制日志,然后在备库上存放日志的方式完成,所以Mysql的复制是异步的,也就意味着同一时间点上备库数据和主库数据的数据不一致和网络延迟。
复制解决了什么问题
(一)基于段的日志格式
mysql> show variables like "%bin%";
+-----------------------------------------+--------------------------------+
| Variable_name | Value |
+-----------------------------------------+--------------------------------+
| bind_address | * |
| binlog_cache_size | 32768 |
| binlog_checksum | CRC32 |
| binlog_direct_non_transactional_updates | OFF |
| binlog_error_action | IGNORE_ERROR |
| binlog_format | STATEMENT |
| binlog_gtid_simple_recovery | OFF |
| binlog_max_flush_queue_time | 0 |
| binlog_order_commits | ON |
| binlog_row_image | FULL |
| binlog_rows_query_log_events | OFF |
| binlog_stmt_cache_size | 32768 |
| binlogging_impossible_mode | IGNORE_ERROR |
| innodb_api_enable_binlog | OFF |
| innodb_locks_unsafe_for_binlog | OFF |
| log_bin | OFF |
| log_bin_basename | /var/lib/mysql/mysql-bin |
| log_bin_index | /var/lib/mysql/mysql-bin.index |
| log_bin_trust_function_creators | OFF |
| log_bin_use_v1_row_events | OFF |
| max_binlog_cache_size | 18446744073709547520 |
| max_binlog_size | 1073741824 |
| max_binlog_stmt_cache_size | 18446744073709547520 |
| simplified_binlog_gtid_recovery | OFF |
| sql_log_bin | ON |
| sync_binlog | 0 |
+-----------------------------------------+--------------------------------+
26 rows in set (0.00 sec)
log-bin=mysql-bin
service mysqld restart
mysql> show variables like 'binlog_format';
+---------------+-----------+
| Variable_name | Value |
+---------------+-----------+
| binlog_format | STATEMENT |
+---------------+-----------+
1 row in set (0.00 sec)
# 在演示系统中默认是STATEMENT(基于段)格式日志
# 修改成ROW格式
mysql> show variables like 'binlog_format';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| binlog_format | ROW |
+---------------+-------+
1 row in set (0.00 sec)
# 刷新binlog文件
mysql> show binary logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000001 | 120 |
+------------------+-----------+
1 row in set (0.00 sec)
mysql> flush logs;
mysql> create database crn;
Query OK, 1 row affected (0.01 sec)
mysql> use crn;
Database changed
mysql> create table t(id int,c1 varchar(10));
Query OK, 0 rows affected (0.12 sec)
mysql> insert into t values(1,'aa'),(2,'bb');
Query OK, 2 rows affected (0.01 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> update t set c1='dd' where id=2;
Query OK, 1 row affected (0.10 sec)
Rows matched: 1 Changed: 1 Warnings: 0
[root@localhost ~]# cd /var/lib/mysql
# 仅查看mysql-bin开头的文件
[root@localhost mysql]# ls mysql-bin*
mysql-bin.000001 mysql-bin.index
# error场景:利用mysqlbinlog命令查看内容(以下是为OFF时,所有的记录是没有记录下来的)
[root@localhost mysql]# mysqlbinlog mysql-bin.000001
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#190225 14:59:18 server id 1 end_log_pos 120 CRC32 0x3dab86f6 Start: binlog v 4, server v 5.6.43-log created 190225 14:59:18 at startup
# Warning: this binlog is either in use or was not closed properly.
ROLLBACK/*!*/;
BINLOG '
RpJzXA8BAAAAdAAAAHgAAAABAAQANS42LjQzLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAABGknNcEzgNAAgAEgAEBAQEEgAAXAAEGggAAAAICAgCAAAACgoKGRkAAfaG
qz0=
'/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@localhost mysql]#
# success场景:利用mysqlbinlog命令查看内容
.
.
.
SET @@session.collation_database=DEFAULT/*!*/;
drop database crn
/*!*/;
# at 209
#190225 15:21:15 server id 1 end_log_pos 300 CRC32 0x5bfa68a6 Query thread_id=4 exec_time=0 error_code=0
SET TIMESTAMP=1551079275/*!*/;
create database crn
/*!*/;
# at 300
#190225 15:22:09 server id 1 end_log_pos 409 CRC32 0x34c142d6 Query thread_id=4 exec_time=0 error_code=0
use `crn`/*!*/;
SET TIMESTAMP=1551079329/*!*/;
create table t(id int,c1 varchar(10))
/*!*/;
# at 409
#190225 15:22:18 server id 1 end_log_pos 486 CRC32 0x55aa3a59 Query thread_id=4 exec_time=0 error_code=0
SET TIMESTAMP=1551079338/*!*/;
BEGIN
/*!*/;
# at 486
#190225 15:22:18 server id 1 end_log_pos 595 CRC32 0xbfe76ef0 Query thread_id=4 exec_time=0 error_code=0
SET TIMESTAMP=1551079338/*!*/;
insert into t values(1,'aa'),(2,'bb')
/*!*/;
# at 595
#190225 15:22:18 server id 1 end_log_pos 626 CRC32 0xcb8ecaee Xid = 26
COMMIT/*!*/;
# at 626
#190225 15:22:24 server id 1 end_log_pos 703 CRC32 0x285bf2af Query thread_id=4 exec_time=0 error_code=0
SET TIMESTAMP=1551079344/*!*/;
BEGIN
/*!*/;
# at 703
#190225 15:22:24 server id 1 end_log_pos 806 CRC32 0x3110a17f Query thread_id=4 exec_time=0 error_code=0
SET TIMESTAMP=1551079344/*!*/;
update t set c1='dd' where id=2
/*!*/;
# at 806
.
.
.
问:什么叫做基于段的复制? 答:可以清晰的知道数据的操作是什么,这个就是基于段的复制。
(二)基于行的日志格式
问:基于段、基于行的日志区别?
答:
问:使用row格式的好处?
答:
mysql> set session binlog_format='row';
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like 'binlog_format';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| binlog_format | ROW |
+---------------+-------+
1 row in set (0.00 sec)
mysql> flush logs;
Query OK, 0 rows affected (0.01 sec)
mysql> show variables like 'binlog_row_image';
+------------------+-------+
| Variable_name | Value |
+------------------+-------+
| binlog_row_image | FULL |
+------------------+-------+
1 row in set (0.00 sec)
mysql> use crn;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from t;
+------+------+
| id | c1 |
+------+------+
| 1 | aa |
| 2 | dd |
+------+------+
2 rows in set (0.00 sec)
mysql> alter table t add c2 text;
Query OK, 0 rows affected (0.03 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> selec * from t;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'selec * from t' at line 1
mysql> select * from t;
+------+------+------+
| id | c1 | c2 |
+------+------+------+
| 1 | aa | NULL |
| 2 | dd | NULL |
+------+------+------+
2 rows in set (0.00 sec)
mysql> delete from t where id=1;
Query OK, 1 row affected (0.00 sec)
mysql>
[root@localhost mysql]# ls mysql-bin*
mysql-bin.000001 mysql-bin.000002 mysql-bin.index
[root@localhost mysql]# mysqlbinlog mysql-bin.000002
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!40019 SET @@session.max_insert_delayed_threads=0*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#190225 15:37:56 server id 1 end_log_pos 120 CRC32 0xb241631d Start: binlog v 4, server v 5.6.43-log created 190225 15:37:56
# Warning: this binlog is either in use or was not closed properly.
BINLOG '
VJtzXA8BAAAAdAAAAHgAAAABAAQANS42LjQzLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAAXAAEGggAAAAICAgCAAAACgoKGRkAAR1j
QbI=
'/*!*/;
# at 120
#190225 15:39:00 server id 1 end_log_pos 217 CRC32 0x244d05d7 Query thread_id=5 exec_time=0 error_code=0
use `crn`/*!*/;
SET TIMESTAMP=1551080340/*!*/;
SET @@session.pseudo_thread_id=5/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1075838976/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
alter table t add c2 text
/*!*/;
# at 217
#190225 15:39:40 server id 1 end_log_pos 288 CRC32 0x380d872e Query thread_id=5 exec_time=0 error_code=0
SET TIMESTAMP=1551080380/*!*/;
BEGIN
/*!*/;
# at 288
#190225 15:39:40 server id 1 end_log_pos 336 CRC32 0x16dac56c Table_map: `crn`.`t` mapped to number 72
# at 336
#190225 15:39:40 server id 1 end_log_pos 379 CRC32 0x48b1085c Delete_rows: table id 72 flags: STMT_END_F
BINLOG '
vJtzXBMBAAAAMAAAAFABAAAAAEgAAAAAAAEAA2NybgABdAADAw/8AwoAAgdsxdoW
vJtzXCABAAAAKwAAAHsBAAAAAEgAAAAAAAEAAgAD//wBAAAAAmFhXAixSA==
'/*!*/;
# at 379
#190225 15:39:40 server id 1 end_log_pos 410 CRC32 0x9e44c6ce Xid = 43
COMMIT/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@localhost mysql]#
[root@localhost mysql]# mysqlbinlog -vv mysql-bin.000002
- 1 如果是minimal格式
mysql> select * from t; +------+------+--------+ | id | c1 | c2 | +------+------+--------+ | 2 | dd | NULL | +------+------+--------+ 1 row in set (0.00 sec) mysql> set session binlog_row_image=minimal; Query OK, 0 rows affected (0.00 sec) mysql> update t set c2='this 2' where id=2; Query OK, 1 row affected (0.01 sec) Rows matched: 1 Changed: 1 Warnings: 0 mysql> select * from t; +------+------+--------+ | id | c1 | c2 | +------+------+--------+ | 2 | dd | this 2 | +------+------+--------+ 1 row in set (0.00 sec)
- 2 文件保存
[root@localhost mysql]# mysqlbinlog -vv mysql-bin.000002 > 1.txt
- 3 1.txt 文件内容
分析:值记录下更新的第3列修改前和修改后的值,修改前的值是NULL修改后的值是(没有记录第一第二列修改后的值,因为没有发生变化)
- 1 如果是noblob格式
mysql> select * from t; +------+------+--------+ | id | c1 | c2 | +------+------+--------+ | 2 | dd | this 2 | +------+------+--------+ 1 row in set (0.00 sec) mysql> set session binlog_row_image=noblob; Query OK, 0 rows affected (0.00 sec) mysql> update t set c1='fff' where id=2; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 # 查询数据库表 t 结构 mysql> show columns from t; +-------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+-------------+------+-----+---------+-------+ | id | int(11) | YES | | NULL | | | c1 | varchar(10) | YES | | NULL | | | c2 | text | YES | | NULL | | +-------+-------------+------+-----+---------+-------+ 3 rows in set (0.00 sec)
- 2 noblob更新非text列的表现
分析:由于没有更新blob或者text列,所以text列(即c2)没有出现在SET从句后面,只是出现了第一列和第二列。
总结:如果我们是使用row格式的二进制格式的话,把二进制格式改成minimal或者noblob可以大大减少二进制日志的大小。
(三)基于行的日志格式
建议:使用row格式二进制日志,同时使用minimal短小日志,减少I/O对Mysql的负载。
根据参与复制的主数据服务器所使用的二进制日志格式的不同,复制可以基于sql语句的复制和基于行的复制。基于sql语句的复制就是主数据库服务器的二进制格式利用的是statement格式,而基于行的复制指的是主库的复制方式是基于行的格式,如果选择的是混合则会根据实际内容,两者之间进行切换。
----------------------------------------------------------------------------------------