准备数据
4张100w的表
sysbench oltp_common --mysql-socket=tmp/mysql.sock --mysql-user=root --mysql-db=server_234_db1 --db-driver=mysql --tables=4 --table-size=5000000 --report-interval=1 --threads=8 prepare
select concat(round(sum(data_length/1024/1024),2),'MB') as size,table_schema
from information_schema.tables
group by table_schema;
+------------+--------------------+
| size | table_schema |
+------------+--------------------+
| 3990.00MB | server_234_db1 |
+------------+--------------------+
SELECT CONCAT(table_schema,'.',TABLE_NAME) AS 'Table Name',
CONCAT(ROUND(table_rows/1000000,4),'M') AS 'Number of Rows',
CONCAT(ROUND(data_length/(1024*1024*1024),4),'G') AS 'Data Size',
CONCAT(ROUND(index_length/(1024*1024*1024),4),'G') AS 'Index Size',
CONCAT(ROUND((data_length+index_length)/(1024*1024*1024),4),'G') AS'Total'
FROM information_schema.TABLES
WHERE table_schema LIKE 'server_234_db1'
and TABLE_NAME like 'sbtest%'
ORDER BY 3 desc, 5 desc;
+------------------------+----------------+-----------+------------+---------+
| Table Name | Number of Rows | Data Size | Index Size | Total |
+------------------------+----------------+-----------+------------+---------+
| server_234_db1.sbtest3 | 4.8076M | 1.0322G | 0.0728G | 1.1051G |
| server_234_db1.sbtest4 | 4.8214M | 0.9580G | 0.0728G | 1.0309G |
| server_234_db1.sbtest1 | 4.8219M | 0.9561G | 0.0728G | 1.0289G |
| server_234_db1.sbtest2 | 4.8229M | 0.9502G | 0.0728G | 1.0230G |
+------------------------+----------------+-----------+------------+---------+
[server_234_db1]$ du -sh *
4.0K db.opt
12K sbtest1.frm
1.2G sbtest1.ibd
12K sbtest2.frm
1.2G sbtest2.ibd
12K sbtest3.frm
1.2G sbtest3.ibd
12K sbtest4.frm
1.2G sbtest4.ibd
select TABLE_NAME,DATA_LENGTH,INDEX_LENGTH,DATA_FREE
from information_schema.tables
where table_schema='server_234_db1';
+------------+-------------+--------------+-----------+
| TABLE_NAME | DATA_LENGTH | INDEX_LENGTH | DATA_FREE |
+------------+-------------+--------------+-----------+
| sbtest1 | 1026555904 | 78217216 | 4194304 | -- 1G, 78M, 4M
| sbtest2 | 1020264448 | 78217216 | 4194304 |
| sbtest3 | 1108344832 | 78217216 | 4194304 |
| sbtest4 | 1028653056 | 78217216 | 4194304 |
+------------+-------------+--------------+-----------+
DATA_FREE
The number of allocated but unused bytes.
InnoDB tables report the free space of the tablespace to which the table belongs. For a table located in the shared tablespace, this is the free space of the shared tablespace.
(!!!)If you are using multiple tablespaces and the table has its own tablespace, the free space is for only that table. Free space means the number of bytes in completely free extents minus a safety margin. Even if free space displays as 0, it may be possible to insert rows as long as new extents need not be allocated.
删除测试
opti过程
-- 删除sbtest1中的100w数据(共500w)
-- sbtest1.ibd 1.2G -----> 1.2G
-- DATA_LENGTH 1026555904 -----> 1124073472
-- INDEX_LENGTH 78217216 -----> 78217216
-- DATA_FREE 4194304 -----> 4194304
mysql> delete from server_234_db1.sbtest1 where id < 1000001;
Query OK, 1000000 rows affected (47.41 sec)
-- 再删除sbtest1中的200w数据(共500w)
-- sbtest1.ibd 1.2G -----> 1.2G
-- DATA_LENGTH 1124073472 -----> 900022272 (1.1G-->900M)
-- INDEX_LENGTH 78217216 -----> 78217216 (78M-->78M)
-- DATA_FREE 4194304 -----> 309329920 (4M-->309M)
mysql> delete from server_234_db1.sbtest1 where id < 3000001;
Query OK, 2000000 rows affected (2 min 6.07 sec)
-- sbtest1.ibd 1.2G -----> 533M +650M
-- DATA_LENGTH 900022272 -----> 513703936(900M-->513M) -300M
-- INDEX_LENGTH 78217216 -----> 32030720 (78M-->32M) -40M
-- DATA_FREE 309329920 -----> 2097152 (309M-->2M) -300M
mysql> optimize local table server_234_db1.sbtest1;
2 rows in set (47.09 sec)
-- step1:rebuild表文件 533M #sql-ib201-396037690.ibd
-- step1:交换名字 533M #sql-ib201-396037690.ibd <-----> 1.2G sbtest1.ibd
-- step3:删除1.2G #sql-ib201-396037690.ibd
-- 至少需要额外空间533M
直接回收
mysql> delete from server_234_db1.sbtest3 where id < 3000001;
mysql> alter table server_234_db1.sbtest3 algorithm=inplace, lock=none, ENGINE = InnoDB;
Query OK, 0 rows affected (55.03 sec)
onlineddl
An
ALTER TABLE
operation that uses theCOPY
algorithm prevents concurrent DML operations. Concurrent queries are still allowed. That is, a table-copying operation always includes at least the concurrency restrictions ofLOCK=SHARED
(allow queries but not DML). You can further restrict concurrency for operations that support theLOCK
clause by specifyingLOCK=EXCLUSIVE
, which prevents DML and queries. For more information, see Concurrency Control.
mysql> show global variables like 'innodb_temp_data_file_path';
+----------------------------+-----------------------+
| Variable_name | Value |
+----------------------------+-----------------------+
| innodb_temp_data_file_path | ibtmp1:12M:autoextend |
+----------------------------+-----------------------+
限制大小
set global innodb_temp_data_file_path = 'ibtmp1:12M:autoextend:max:5G';
只能重启回收
MySQL 5.6中开始支持把undo log分离到独立的表空间,并放到单独的文件目录下;这给我们部署不同IO类型的文件位置带来便利,对于并发写入型负载,我们可以把undo文件部署到单独的高速存储设备上。增加了如下几个参数来控制该行为。
mysql> show global variables like '%undo%';
+--------------------------+----------+
| Variable_name | Value |
+--------------------------+----------+
| innodb_max_undo_log_size | 10485760 |
| innodb_undo_directory | ./ |
| innodb_undo_log_truncate | ON |
| innodb_undo_logs | 128 |
| innodb_undo_tablespaces | 0 |
+--------------------------+----------+
innochecksum --verbose=FALSE --log=/tmp/innocheck.log
常用参数:
--help/info
查看帮助信息;
--verbose
打印详细信息,可以通过--verbose=FALSE关闭;
--count/-c
只打印页的数量信息;
--start-page=NUM/-s NUM; --end-page=NUM/-e NUM; --page=NUM/-p NUM
指定开始、结束页,或者只查看指定的页;
--strict-check/-C
指定checksum算法,通常有innodb、crc32、none,默认是从三者中选择,否则强制指定;
--page-type-summary/-S
打印文件中页的统计信息,包括了总页类型以及数量;
--page-type-dump=file-name/-D file-name
打印各个页的详细信息,将其输出到一个文件中;
常用示例:
----- 检查系统表空间,也可以使用table-name.ibd,默认出错时才会输出异常
innodbchecksum ibdata1
----- 保存文件中各个页的信息,并在最后打印统计信息
innodbchecksum -S -D /tmp/page.info schema/*.ibd
使用流程
cp ibdata1 ibdata1.tmp
innochecksum -S ibdata1.tmp
File::ibdata1.tmp
================PAGE TYPE SUMMARY==============
#PAGE_COUNT PAGE_TYPE
===============================================
580 Index page
5838 Undo log page
67 Inode page
228 Insert buffer free list page
2146 Freshly allocated page
1 Insert buffer bitmap
98 System page
1 Transaction system page
1 File Space Header
0 Extent descriptor page
0 BLOB page
0 Compressed BLOB page
0 Other type of page
===============================================
Additional information:
Undo page type: 38 insert, 5800 update, 0 other
Undo page state: 9 active, 44 cached, 0 to_free, 5785 to_purge, 0 prepared, 0 other
当 ibdata1 文件增长太快,通常是 MySQL 里长时间运行的被遗忘的事务引起的。
尝试去解决问题越快越好(提交或者杀死事务),回收空间必须重建实例。
MySQL 5.7引入了新的参数,innodb_undo_log_truncate
,开启后可在线收缩拆分出来的undo表空间。
在满足以下2个条件下,undo表空间文件可在线收缩:
innodb_undo_tablespaces>=2
:因为truncate undo表空间时,该文件处于inactive状态,如果只有1个undo表空间,那么整个系统在此过程中将处于不可用状态。为了尽可能降低truncate对系统的影响,建议将该参数最少设置为3;innodb_undo_logs>=35
(默认128):因为在MySQL 5.7中,第一个undo log永远在系统表空间中,另外32个undo log分配给了临时表空间,即ibtmp1,至少还有2个undo log才能保证2个undo表空间中每个里面至少有1个undo log;满足以上2个条件后,把innodb_undo_log_truncate设置为ON即可开启undo表空间的自动truncate,这还跟如下2个参数有关:
innodb_max_undo_log_size
:undo表空间文件超过此值即标记为可收缩,默认1G,可在线修改;innodb_purge_rseg_truncate_frequency
:指定purge操作被唤起多少次之后才释放rollback segments。当undo表空间里面的rollback segments被释放时,undo表空间才会被truncate。由此可见,该参数越小,undo表空间被尝试truncate的频率越高。