Mysql空间回收总结

准备数据

4张100w的表

sysbench oltp_common --mysql-socket=tmp/mysql.sock --mysql-user=root --mysql-db=server_234_db1 --db-driver=mysql  --tables=4 --table-size=5000000 --report-interval=1 --threads=8 prepare

select concat(round(sum(data_length/1024/1024),2),'MB') as size,table_schema 
from information_schema.tables 
group by table_schema;
+------------+--------------------+
| size       | table_schema       |
+------------+--------------------+
| 3990.00MB  | server_234_db1     |
+------------+--------------------+

SELECT CONCAT(table_schema,'.',TABLE_NAME) AS 'Table Name',
       CONCAT(ROUND(table_rows/1000000,4),'M') AS 'Number of Rows',
       CONCAT(ROUND(data_length/(1024*1024*1024),4),'G') AS 'Data Size',
       CONCAT(ROUND(index_length/(1024*1024*1024),4),'G') AS 'Index Size',
       CONCAT(ROUND((data_length+index_length)/(1024*1024*1024),4),'G') AS'Total'
FROM information_schema.TABLES
WHERE table_schema LIKE 'server_234_db1'
and TABLE_NAME like 'sbtest%'
ORDER BY 3 desc, 5 desc;
+------------------------+----------------+-----------+------------+---------+
| Table Name             | Number of Rows | Data Size | Index Size | Total   |
+------------------------+----------------+-----------+------------+---------+
| server_234_db1.sbtest3 | 4.8076M        | 1.0322G   | 0.0728G    | 1.1051G |
| server_234_db1.sbtest4 | 4.8214M        | 0.9580G   | 0.0728G    | 1.0309G |
| server_234_db1.sbtest1 | 4.8219M        | 0.9561G   | 0.0728G    | 1.0289G |
| server_234_db1.sbtest2 | 4.8229M        | 0.9502G   | 0.0728G    | 1.0230G |
+------------------------+----------------+-----------+------------+---------+

[server_234_db1]$ du -sh *
4.0K	db.opt
12K	sbtest1.frm
1.2G	sbtest1.ibd
12K	sbtest2.frm
1.2G	sbtest2.ibd
12K	sbtest3.frm
1.2G	sbtest3.ibd
12K	sbtest4.frm
1.2G	sbtest4.ibd

select TABLE_NAME,DATA_LENGTH,INDEX_LENGTH,DATA_FREE 
from information_schema.tables 
where table_schema='server_234_db1';
+------------+-------------+--------------+-----------+
| TABLE_NAME | DATA_LENGTH | INDEX_LENGTH | DATA_FREE |
+------------+-------------+--------------+-----------+
| sbtest1    |  1026555904 |     78217216 |   4194304 | -- 1G, 78M, 4M
| sbtest2    |  1020264448 |     78217216 |   4194304 |
| sbtest3    |  1108344832 |     78217216 |   4194304 |
| sbtest4    |  1028653056 |     78217216 |   4194304 |
+------------+-------------+--------------+-----------+

DATA_FREE

The number of allocated but unused bytes.

InnoDB tables report the free space of the tablespace to which the table belongs. For a table located in the shared tablespace, this is the free space of the shared tablespace. 

(!!!)If you are using multiple tablespaces and the table has its own tablespace, the free space is for only that table. Free space means the number of bytes in completely free extents minus a safety margin. Even if free space displays as 0, it may be possible to insert rows as long as new extents need not be allocated.

1 大表opti空间回收

删除测试

  • 至少需要额外空间=无碎片的表大小

opti过程

  • step1:rebuild表文件 533M #sql-ib201-396037690.ibd
  • step2:交换名字 533M #sql-ib201-396037690.ibd <-----> 1.2G sbtest1.ibd
  • step3:删除1.2G #sql-ib201-396037690.ibd:至少需要额外空间533M
-- 删除sbtest1中的100w数据(共500w)
-- sbtest1.ibd  1.2G	      ----->  1.2G
-- DATA_LENGTH  1026555904  ----->  1124073472
-- INDEX_LENGTH 78217216    ----->  78217216
-- DATA_FREE    4194304     ----->  4194304
mysql> delete from server_234_db1.sbtest1 where id < 1000001;
Query OK, 1000000 rows affected (47.41 sec)

-- 再删除sbtest1中的200w数据(共500w)
-- sbtest1.ibd  1.2G	      ----->  1.2G
-- DATA_LENGTH  1124073472  ----->  900022272 (1.1G-->900M)
-- INDEX_LENGTH 78217216    ----->  78217216  (78M-->78M)
-- DATA_FREE    4194304     ----->  309329920 (4M-->309M)
mysql> delete from server_234_db1.sbtest1 where id < 3000001;
Query OK, 2000000 rows affected (2 min 6.07 sec)


-- sbtest1.ibd  1.2G	     ----->  533M                    +650M
-- DATA_LENGTH  900022272  ----->  513703936(900M-->513M)  -300M
-- INDEX_LENGTH 78217216   ----->  32030720 (78M-->32M)    -40M
-- DATA_FREE    309329920  ----->  2097152  (309M-->2M)    -300M
mysql> optimize local table server_234_db1.sbtest1;
2 rows in set (47.09 sec)

-- step1:rebuild表文件 533M #sql-ib201-396037690.ibd
-- step1:交换名字 533M #sql-ib201-396037690.ibd <-----> 1.2G sbtest1.ibd
-- step3:删除1.2G #sql-ib201-396037690.ibd
-- 至少需要额外空间533M

2 表drop/truncate空间回收

直接回收

3 重建表空间回收

  • alter table server_234_db1.sbtest3 algorithm=inplace, lock=none, ENGINE = InnoDB;
    • 和optimize的原理类似,重建表
    • lock=none只是指定这条命令不给表加任何锁
mysql> delete from server_234_db1.sbtest3 where id < 3000001;
mysql> alter table server_234_db1.sbtest3 algorithm=inplace, lock=none, ENGINE = InnoDB;
Query OK, 0 rows affected (55.03 sec)

onlineddl

An ALTER TABLE operation that uses the COPY algorithm prevents concurrent DML operations. Concurrent queries are still allowed. That is, a table-copying operation always includes at least the concurrency restrictions of LOCK=SHARED (allow queries but not DML). You can further restrict concurrency for operations that support the LOCK clause by specifying LOCK=EXCLUSIVE, which prevents DML and queries. For more information, see Concurrency Control.

4 ibtmp1空间回收

mysql> show global variables like 'innodb_temp_data_file_path';
+----------------------------+-----------------------+
| Variable_name              | Value                 |
+----------------------------+-----------------------+
| innodb_temp_data_file_path | ibtmp1:12M:autoextend |
+----------------------------+-----------------------+

限制大小

set global innodb_temp_data_file_path = 'ibtmp1:12M:autoextend:max:5G';

只能重启回收

5 undo表空间回收ibdata1

1 mysql5.6

MySQL 5.6中开始支持把undo log分离到独立的表空间,并放到单独的文件目录下;这给我们部署不同IO类型的文件位置带来便利,对于并发写入型负载,我们可以把undo文件部署到单独的高速存储设备上。增加了如下几个参数来控制该行为。

mysql> show global variables like '%undo%';
+--------------------------+----------+
| Variable_name            | Value    |
+--------------------------+----------+
| innodb_max_undo_log_size | 10485760 |
| innodb_undo_directory    | ./       |
| innodb_undo_log_truncate | ON       |
| innodb_undo_logs         | 128      |
| innodb_undo_tablespaces  | 0        |
+--------------------------+----------+
  • innodb_undo_logs=128:用来设置rollback segment(回滚段)的个数,默认为128。,该参数用来替换之前版本的参数innodb_rollback_segments。该变量可以动态调整,但是物理上的回滚段不会减少,只是会控制用到的回滚段的个数。
  • innodb_undo_tablespaces=0:用于设定创建的undo表空间的个数,也就是用来构成rollback segment文件的数量,这样rollback segment可以较为平均地分布在多个文件中。在Install db时初始化后,就再也不能被改动了。默认值为0,表示不独立设置undo的tablespace,默认记录到ibdata中;否则,则在undo目录下创建这么多个undo文件。
  • 默认文件名叫做ibdata1

(分析ibdata)innochecksum 使用方法

innochecksum --verbose=FALSE --log=/tmp/innocheck.log
常用参数:
  --help/info
    查看帮助信息;
  --verbose
    打印详细信息,可以通过--verbose=FALSE关闭;
  --count/-c
    只打印页的数量信息;
  --start-page=NUM/-s NUM; --end-page=NUM/-e NUM; --page=NUM/-p NUM
    指定开始、结束页,或者只查看指定的页;
  --strict-check/-C
    指定checksum算法,通常有innodb、crc32、none,默认是从三者中选择,否则强制指定;
  --page-type-summary/-S
    打印文件中页的统计信息,包括了总页类型以及数量;
  --page-type-dump=file-name/-D file-name
    打印各个页的详细信息,将其输出到一个文件中;

常用示例:
  ----- 检查系统表空间,也可以使用table-name.ibd,默认出错时才会输出异常
  innodbchecksum ibdata1
  ----- 保存文件中各个页的信息,并在最后打印统计信息
  innodbchecksum -S -D /tmp/page.info schema/*.ibd

使用流程

cp ibdata1 ibdata1.tmp

innochecksum -S ibdata1.tmp

File::ibdata1.tmp
================PAGE TYPE SUMMARY==============
#PAGE_COUNT	PAGE_TYPE
===============================================
     580	Index page
    5838	Undo log page
      67	Inode page
     228	Insert buffer free list page
    2146	Freshly allocated page
       1	Insert buffer bitmap
      98	System page
       1	Transaction system page
       1	File Space Header
       0	Extent descriptor page
       0	BLOB page
       0	Compressed BLOB page
       0	Other type of page
===============================================
Additional information:
Undo page type: 38 insert, 5800 update, 0 other
Undo page state: 9 active, 44 cached, 0 to_free, 5785 to_purge, 0 prepared, 0 other

当 ibdata1 文件增长太快,通常是 MySQL 里长时间运行的被遗忘的事务引起的。
尝试去解决问题越快越好(提交或者杀死事务),回收空间必须重建实例。

2 mysql5.7

MySQL 5.7引入了新的参数,innodb_undo_log_truncate,开启后可在线收缩拆分出来的undo表空间。

在满足以下2个条件下,undo表空间文件可在线收缩:

  1. innodb_undo_tablespaces>=2:因为truncate undo表空间时,该文件处于inactive状态,如果只有1个undo表空间,那么整个系统在此过程中将处于不可用状态。为了尽可能降低truncate对系统的影响,建议将该参数最少设置为3;
  2. innodb_undo_logs>=35(默认128):因为在MySQL 5.7中,第一个undo log永远在系统表空间中,另外32个undo log分配给了临时表空间,即ibtmp1,至少还有2个undo log才能保证2个undo表空间中每个里面至少有1个undo log;

满足以上2个条件后,把innodb_undo_log_truncate设置为ON即可开启undo表空间的自动truncate,这还跟如下2个参数有关:

  1. innodb_max_undo_log_size:undo表空间文件超过此值即标记为可收缩,默认1G,可在线修改;
  2. innodb_purge_rseg_truncate_frequency:指定purge操作被唤起多少次之后才释放rollback segments。当undo表空间里面的rollback segments被释放时,undo表空间才会被truncate。由此可见,该参数越小,undo表空间被尝试truncate的频率越高。

你可能感兴趣的:(mysql)