seteor

[MySQL 5.6] MySQL 5.6 online ddl 使用、测试及关键函数栈

本文主要分为三个部分，第一部分是看文档时的笔记；第二部分使用sysbench简单测试了下性能损耗；第三部分阐述了关键函数栈，但未做深入

前言

Online DDL是MySQL 5.6的重要特性之一，特别是对于不可间断的互联网服务而言意义非凡。尽管我们已经通过工具来实现了在线DDL，但由于借助了触发器来获取增量数据，很难保证不会触发BUG，我们在5.1版本上广泛使用了内部开发的myddl，曾经触发了mysql6个以上的bug。

Innodb允许你通过设置LOCK=EXCLUSIVE | SHARED | DEFAULT/NONE 来进行完全阻塞的DDL、只阻塞DML不阻塞查询、以及完全在线DDL，这有助于你能够在性能和速度之间进行权衡

以下是从官方文档拷贝的一张关于Online ddl对于当前ddl操作的支持：

OPERATION	IN-PLACE?	COPIES TABLE?	ALLOWS CONCURRENT DML?	ALLOWS CONCURRENT QUERY?	NOTES
`CREATE INDEX`,`ADD INDEX`	Yes*	No*	Yes	Yes	Some restrictions for `FULLTEXT` index; see next row. Currently, the operation is not in-place (that is, it copies the table) if the same index being created was also dropped by an earlier clause in the same`ALTER TABLE` statement.
`ADD FULLTEXT INDEX`	Yes	No*	No	Yes	Creating the first `FULLTEXT` index for a table involves a table copy, unless there is a user-supplied `FTS_DOC_ID` column. Subsequent `FULLTEXT` indexes on the same table can be created in-place.
`DROP INDEX`	Yes	No	Yes	Yes
Set default value for a column	Yes	No	Yes	Yes	Modifies `.frm` file only, not the data file.
Change auto-increment value for a column	Yes	No	Yes	Yes	Modifies a value stored in memory, not the data file.
Add a foreign key constraint	Yes*	No*	Yes	Yes	To avoid copying the table, disable`foreign_key_checks` during constraint creation.
Drop a foreign key constraint	Yes	No	Yes	Yes	The `foreign_key_checks` option can be enabled or disabled.
Rename a column	Yes*	No*	Yes*	Yes	To allow concurrent DML, keep the same data type and only change the column name.
Add a column	Yes	Yes	Yes*	Yes	Concurrent DML is not allowed when adding an auto-increment column. Although `ALGORITHM=INPLACE` is allowed, the data is reorganized substantially, so it is still an expensive operation.
Drop a column	Yes	Yes	Yes	Yes	Although `ALGORITHM=INPLACE` is allowed, the data is reorganized substantially, so it is still an expensive operation.
Reorder columns	Yes	Yes	Yes	Yes	Although `ALGORITHM=INPLACE` is allowed, the data is reorganized substantially, so it is still an expensive operation.
Change`ROW_FORMAT`property	Yes	Yes	Yes	Yes	Although `ALGORITHM=INPLACE` is allowed, the data is reorganized substantially, so it is still an expensive operation.
Change`KEY_BLOCK_SIZE`property	Yes	Yes	Yes	Yes	Although `ALGORITHM=INPLACE` is allowed, the data is reorganized substantially, so it is still an expensive operation.
Make column`NULL`	Yes	Yes	Yes	Yes	Although `ALGORITHM=INPLACE` is allowed, the data is reorganized substantially, so it is still an expensive operation.
Make column `NOT NULL`	Yes*	Yes	Yes	Yes	When `SQL_MODE` includes`strict_all_tables` or`strict_all_tables`, the operation fails if the column contains any nulls. Although `ALGORITHM=INPLACE` is allowed, the data is reorganized substantially, so it is still an expensive operation.
Change data type of column	No	Yes	No	Yes
Add primary key	Yes*	Yes	Yes	Yes	Although `ALGORITHM=INPLACE` is allowed, the data is reorganized substantially, so it is still an expensive operation. `ALGORITHM=INPLACE` is not allowed under certain conditions if columns have to be converted to `NOT NULL`. See Example 5.9, “Creating and Dropping the Primary Key”.
Drop primary keyand add another	Yes	Yes	Yes	Yes	`ALGORITHM=INPLACE` is only allowed when you add a new primary key in the same `ALTER TABLE`; the data is reorganized substantially, so it is still an expensive operation.
Drop primary key	No	Yes	No	Yes	Restrictions apply when you drop a primary key primary key without adding a new one in the same `ALTER TABLE`statement.
Convert character set	No	Yes	No	Yes	Rebuilds the table if the new character encoding is different.
Specify character set	No	Yes	No	Yes	Rebuilds the table if the new character encoding is different.
Rebuild with`FORCE` option	No	Yes	No	Yes	Acts like the `ALGORITHM=COPY` clause or the setting `old_alter_table=1`.

从官方提供的这个表格来看，还是有很多操作不支持完全的在线DDL，包括增加一个全文索引，修改列的数据类型，删除一个主键，修改表的字符集等。

但对于大多数我们日常常用的DDL而言，是可以做到在线DDL的。

通常情况下，可以使用默认的语法来进行在线DDL，但你也可以通过选项来改变DDL的行为，有两个选项

LOCK=

ALGORITHM=[INPLACE|COPY]

官方文档给出了一些使用的例子

另外有一个参数

innodb_online_alter_log_max_size  需要注意，它表示在做在线DDL的过程中，并发DML产生的日志最大允许的大小。如果负载很高，这个值应该尽量的调大，否则可能导致DDL失败。

当对主键进行操作时，可以选择ALGORITHM=INPLACE 比设置为COPY更有效率，因为前者不会去记录UNDO LOG或者为其记录REDO LOG；二级索引被预先排序，能够进行有序的加载；change buffer也没有被使用到，因为没有涉及到对二级索引记录的随机插入操作

你可以通过观察执行完DDL后的输出： XX rows affected，来判断是IN-PLACE 还是COPY数据，为0的话就是in-place。

关于ONLINE DDL的具体使用，这里不做阐述，可以看看文档；这里只是简要阐述下其涉及到的函数堆栈

性能损耗

这里使用sysbench来测试,配置如下：

innodb_sort_buffer_size=2M

innodb_online_alter_log_max_size=2G

sysbench command：

sysbench/sysbench –debug=off –test=sysbench/tests/db/update_index.lua –oltp-tables-count=1 –oltp-point-selects=0 –oltp-table-size=1000000 –num-threads=20 –max-requests=10000000000 –max-time=7200 –oltp-auto-inc=off –mysql-engine-trx=yes –mysql-table-engine=innodb –oltp-test-mod=complex –mysql-db=sbtest –mysql-host=$HOST –mysql-port=$PORT –mysql-user=xx run

alter table sbtest1 drop key k;

tps :20,200

alter table sbtest1 add key(k);

tps:大部分聚集在11,000~13,000，有抖动到7,000~9,000;最后出现12秒左右的TPS降低为0

time cost:4 min 8.13 sec)

完成DDL后，TPS稳定在13,000~14,000

alter table sbtest1 drop key k; //TPS恢复至20，200

set session old_alter_table = 1;

alter table sbtest1 add key(k);

tps:0

time cost:28.39 sec

总结：

1. online ddl耗时问题，相比老的ddl方式要更耗时

2. 存在性能抖动，最后阶段的锁表时间可能比较长，这取决于具体的负载，sysbench本身的压力已经比较高了，正常情况下的线上实例不会有这么大压力。

无压力负载测试：

mysql> set session old_alter_table = OFF;

Query OK, 0 rows affected (0.00 sec)

mysql> alter table sbtest1 add key (k);

Query OK, 0 rows affected (10.44 sec)