在mysql主从同步的时候,最初读写都在主库,从库仅用于同步;
但是随着业务的增加,为了减轻MASTER的巨大压力,一般都会进行读写分离——写在主库,读在从库
可还有一个问题,主从有一定的延时,就是对于实时性要求比较高的业务,例如注册用户后马上就要登陆,如果采用读写分离的话,那边注册了,写在主库;由于主从延时2min;注册成功后无法登陆,就会影响用户体验,这在互联网企业是无法容忍的;
为什么主从会有延时呢?这就要说到主从同步的详细过程,步骤如下:
Step1: Client 连接MASTER,并发起一个写入请求
Step2: MASTER 本地执行,并返回成功消息给Client;
Step3: MASTER 产生binlog在本地;
Step4: SLAVE 的IO Thread通过MASTER的DUMP Thread,读取binlog,保存至本地relay log
Step5: Slave 的SQL_Thread 读取本地的relay log进程,写入到本地mysql中
通常情况下,当Client接收到成功消息,即认为操作执行成功了,这就是传统的异步复制;
而后,不管Slave什么时候读取binlog都是可以的;哪怕Slave停机备份,等起来的时候也会去读取MASTER的BINLOG,进而同步;
但如果对实时性要求比较高的业务,这样就不行了;所以就有个同步复制;
同步复制即要求所有的SLAVE都已接收到MASTER的BINLOG,并返回OK的消息;(是否应用到本地不管,但必须要接收完毕)然后MASTER才会返回Client OK的消息
步骤如下:
Step1: Client 连接MASTER,并发起一个写入请求
Step2: MASTER 本地执行;
Step3: MASTER 产生binlog在本地;
Step4: SLAVE 的IO Thread通过MASTER的DUMP Thread,读取binlog,保存至本地relay log后返回MASTER OK消息
Step5: 待MASTER接收到所有SLAVE的 OK消息后,返回Client OK;
这样只要是成功写入的,基本从库都能读取到数据,大大减缓了主从之间的延时;因为IO_thread还是比较快的,但是如果网络质量比较差,或者服务器IO不是很好,SLAVE较多的话,那么写入的时候效率影响还是很大的,可能用户执行要过比较长的时间才能收到成功的消息;
这个时候有一个折中的办法,就是半同步复制,步骤如下:即只要有一个SLAVE接收到BINLOG,即返回CLIENT成功
Step1: Client 连接MASTER,并发起一个写入请求
Step2: MASTER 本地执行;
Step3: MASTER 产生binlog在本地;
Step4: SLAVE 的IO Thread通过MASTER的DUMP Thread,读取binlog,保存至本地relay log后返回MASTER OK消息
Step5: 只要MASTER接收到1个SLAVE的 OK消息后,返回Client OK;
这样就在一定程度上增加了效率执行效率,而且万一主库down了,这个Slave可以切换为Master,保障了数据的不丢失;
具体配置:
半同步复制需要安装插件plugin;一般都是自带的
show variables like '%plugin%';
mysql> show variables like '%plugin%';
+-------------------------------+-----------------------------------+
| Variable_name | Value |
+-------------------------------+-----------------------------------+
| default_authentication_plugin | mysql_native_password |
| plugin_dir | /export/servers/mysql/lib/plugin/ |
+-------------------------------+-----------------------------------+
2 rows in set (0.01 sec)
/export/servers/mysql/lib/plugin/ |
+-------------------------------+-----------------------------------+
2 rows in set (0.01 sec)
然后进入到查检目录看下找到semisync_master.so 和semisync_slave.so 文件,一个是用于安装在master端,一个用于安装在Slave侧;
master的安装命令:
INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';
slave安装命令:
INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';
安装完毕,show plugins在最后一行就看到了改功能;
mysql> show plugins;
+----------------------------+----------+--------------------+--------------------+---------+
| Name | Status | Type | Library | License |
+----------------------------+----------+--------------------+--------------------+---------+
| binlog | ACTIVE | STORAGE ENGINE | NULL | GPL |
| mysql_native_password | ACTIVE | AUTHENTICATION | NULL | GPL |
| sha256_password | ACTIVE | AUTHENTICATION | NULL | GPL |
| CSV | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MyISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MRG_MYISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
| PERFORMANCE_SCHEMA | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MEMORY | ACTIVE | STORAGE ENGINE | NULL | GPL |
| InnoDB | ACTIVE | STORAGE ENGINE | NULL | GPL |
| INNODB_TRX | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_LOCKS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_LOCK_WAITS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMP | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMP_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMPMEM | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMPMEM_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMP_PER_INDEX | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMP_PER_INDEX_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_BUFFER_PAGE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_BUFFER_PAGE_LRU | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_BUFFER_POOL_STATS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_TEMP_TABLE_INFO | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_METRICS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_DEFAULT_STOPWORD | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_DELETED | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_BEING_DELETED | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_CONFIG | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_INDEX_CACHE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_INDEX_TABLE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_TABLES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_TABLESTATS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_INDEXES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_COLUMNS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_FIELDS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_FOREIGN | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_FOREIGN_COLS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_TABLESPACES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_DATAFILES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_VIRTUAL | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| partition | ACTIVE | STORAGE ENGINE | NULL | GPL |
| ARCHIVE | ACTIVE | STORAGE ENGINE | NULL | GPL |
| FEDERATED | DISABLED | STORAGE ENGINE | NULL | GPL |
| BLACKHOLE | ACTIVE | STORAGE ENGINE | NULL | GPL |
| ngram | ACTIVE | FTPARSER | NULL | GPL |
| rpl_semi_sync_master | ACTIVE | REPLICATION | semisync_master.so | GPL |
+----------------------------+----------+--------------------+--------------------+---------+
| rpl_semi_sync_master | ACTIVE | REPLICATION | semisync_master.so | GPL |
+----------------------------+----------+--------------------+--------------------+---------+
然后进行启用功能,并设置超时时间3000毫秒,即3s,即Master节点等待Slave响应时间,如果超过3秒无响应则转换为异步复制;
Master
mysql> set global rpl_semi_sync_master_enabled = 1;
mysql> set global rpl_semi_sync_master_timeout=3000;
Slave 启用
mysql> set GLOBAL rpl_semi_sync_slave_enabled=1
/*永久生效需要写入my.cnf*/
可以使用show variables like ‘%semi%’;查看可配置参数信息
mysql> show variables like '%semi%';
+-------------------------------------------+------------+
| Variable_name | Value |
+-------------------------------------------+------------+
| rpl_semi_sync_master_enabled | ON |
| rpl_semi_sync_master_timeout | 3000 |
| rpl_semi_sync_master_trace_level | 32 |
| rpl_semi_sync_master_wait_for_slave_count | 1 |
| rpl_semi_sync_master_wait_no_slave | ON |
| rpl_semi_sync_master_wait_point | AFTER_SYNC |
+-------------------------------------------+------------
rpl_semi_sync_master_enabled 0关闭,1打开 该功能
rpl_semi_sync_master_timeout master等待Slave响应的时间,单位毫秒,3000即3s,超过该时间未收到响应则转为异步复制
rpl_semi_sync_master_trace_level master侧 debug级别,有4个级别分别是 1| 16 | 32 | 64 级别越高,debug信息越详细
rpl_semi_sync_master_wait_for_slave_count 意味着Master必须在timeout前收到几个Slave的OK消息,默认1,即必须至少有一个Slave返回OK消息
rpl_semi_sync_master_wait_no_slave ON表示即使没有slave,也要等到rpl_semi_sync_master_timeout 时间结束,才会转为异步方式;如果该功能改为OFF,只要SLAVE的数量小于rpl_semi_sync_master_wait_for_slave_count配置的数量就会立即转为异步方式;
rpl_semi_sync_master_wait_point AFTER_SYNC(default) | AFTER_COMMIT
AFTER_SYNC: master将事物写入本地binlog,且发送给slave,并且同步binlog至磁盘,master同步后等待Slave确认事物接收的OK消息,只要收到确认消息,主机就会提交事物至存储引擎,并且返回结果给客户端;
AFTER_COMMIT: master写事物到二进制日志和Slave,同步二进制日志,并且提交事物到存储引擎,master在commit后等待slave OK消息,收到了slave确认消息后,master返回成功消息给client;
安装设置后,要stop/start slave IO_thread 让slave和master重新连接,来是使生效;
mysql> stop slave io_thread;
Query OK, 0 rows affected (0.01 sec)
mysql> start slave io_thread;
Query OK, 0 rows affected (0.00 sec)
查看状态
mysql> show status like '%semi%';
+--------------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients | 1 |
| Rpl_semi_sync_master_net_avg_wait_time | 0 |
| Rpl_semi_sync_master_net_wait_time | 0 |
| Rpl_semi_sync_master_net_waits | 2 |
| Rpl_semi_sync_master_no_times | 0 |
| Rpl_semi_sync_master_no_tx | 0 |
| Rpl_semi_sync_master_status | ON |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 533 |
| Rpl_semi_sync_master_tx_wait_time | 1066 |
| Rpl_semi_sync_master_tx_waits | 2 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 2 |
+--------------------------------------------+-------+
Rpl_semi_sync_master_clients 显示当前处于半同步复制的slave数量
Rpl_semi_sync_master_net_avg_wait_time 等待slave反馈的平均时间,微秒,功能已撤销,一直是0
Rpl_semi_sync_master_net_wait_time 等待的总时间,功能已撤销,一直是0
Rpl_semi_sync_master_net_waits master等待slave确认消息的次数,应该等于yes_tx+no_tx
Rpl_semi_sync_master_no_times master关闭半同步复制的次数
Rpl_semi_sync_master_no_tx 当前未成功发给Slave的事物数量
Rpl_semi_sync_master_status 当前Master是否启用了半同步模式
Rpl_semi_sync_master_timefunc_failures master调用时间函数失败的次数,如gettimeofday()
Rpl_semi_sync_master_tx_avg_wait_time 半同步复制每个事物等待的平均时间
Rpl_semi_sync_master_tx_wait_time 半同步复制等待的总时间
Rpl_semi_sync_master_tx_waits master等待事物的总次数
Rpl_semi_sync_master_wait_pos_backtraverse master等待的总次数,后来的先到;先的是大事物,后的是小事物的时候
Rpl_semi_sync_master_wait_sessions 当前有几个会话正在等待slave响应
Rpl_semi_sync_master_yes_tx 当前成功发给Slave的数量
=======
测试,当我们关闭slave的io时,此时在master侧建表;
mysql> stop slave io_thread;
Query OK, 0 rows affected (0.00 sec)
mysql> create table j select * from a;
Query OK, 512 rows affected (3.02 sec)
Records: 512 Duplicates: 0 Warnings: 0
mysql>
mysql> show status like '%semi%';
+--------------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients | 0 |
| Rpl_semi_sync_master_net_avg_wait_time | 0 |
| Rpl_semi_sync_master_net_wait_time | 0 |
| Rpl_semi_sync_master_net_waits | 2 |
| Rpl_semi_sync_master_no_times | 1 |
| Rpl_semi_sync_master_no_tx | 1 |
| Rpl_semi_sync_master_status | OFF |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 533 |
| Rpl_semi_sync_master_tx_wait_time | 1066 |
| Rpl_semi_sync_master_tx_waits | 2 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 2 |
+--------------------------------------------+-------+
14 rows in set (0.01 sec)
mysql>
Query OK, 512 rows affected (3.02 sec)
Records: 512 Duplicates: 0 Warnings: 0
mysql>
mysql> show status like '%semi%';
+--------------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients | 0 |
| Rpl_semi_sync_master_net_avg_wait_time | 0 |
| Rpl_semi_sync_master_net_wait_time | 0 |
| Rpl_semi_sync_master_net_waits | 2 |
| Rpl_semi_sync_master_no_times | 1 |
| Rpl_semi_sync_master_no_tx | 1 |
| Rpl_semi_sync_master_status | OFF |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 533 |
| Rpl_semi_sync_master_tx_wait_time | 1066 |
| Rpl_semi_sync_master_tx_waits | 2 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 2 |
+--------------------------------------------+-------+
14 rows in set (0.01 sec)
mysql>
由上面结果可以看到建表花了3s的timeout; 而后 Rpl_semi_sync_master_status变为了OFF Rpl_semi_sync_master_no_tx 变成了1;
由于没有在rpl_semi_sync_master_timeout设定的时间内得到SLAVE的确认消息(slave的IO被stop了,肯定无法收到),所以半同步状态自动切换为了异步模式;
mysql> create table k select * from a;
Query OK, 512 rows affected (0.06 sec)
Records: 512 Duplicates: 0 Warnings: 0
512 rows affected (0.06 sec)
Records: 512 Duplicates: 0 Warnings: 0
此时再建表发现就不用等候超时时间了;
这个时候我们mysql> start slave io_thread; 启用slave的IO_thread
去master上看到半同步模式也自动变为了ON;
mysql> show status like '%semi%';
+--------------------------------------------+-------+
| Variable_name | Value |
+--------------------------------------------+-------+
| Rpl_semi_sync_master_clients | 1 |
| Rpl_semi_sync_master_net_avg_wait_time | 0 |
| Rpl_semi_sync_master_net_wait_time | 0 |
| Rpl_semi_sync_master_net_waits | 3 |
| Rpl_semi_sync_master_no_times | 1 |
| Rpl_semi_sync_master_no_tx | 2 |
| Rpl_semi_sync_master_status | ON |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 533 |
| Rpl_semi_sync_master_tx_wait_time | 1066 |
| Rpl_semi_sync_master_tx_waits | 2 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 2 |
+--------------------------------------------+-------+
Rpl_semi_sync_master_status | ON |
| Rpl_semi_sync_master_timefunc_failures | 0 |
| Rpl_semi_sync_master_tx_avg_wait_time | 533 |
| Rpl_semi_sync_master_tx_wait_time | 1066 |
| Rpl_semi_sync_master_tx_waits | 2 |
| Rpl_semi_sync_master_wait_pos_backtraverse | 0 |
| Rpl_semi_sync_master_wait_sessions | 0 |
| Rpl_semi_sync_master_yes_tx | 2 |
+--------------------------------------------+-------+