Mysql内存管理_Change Buffer
插入缓冲，官方文档：https://dev.mysql.com/doc/refman/5.7/en/innodb-change-buffer.html
阿里内核月报：http://mysql.taobao.org/monthly/2015/07/01/
使用条件：辅助索引(二级索引) 且 非唯一索引
使用范围：包括 insert、update、delete (DML) 的操作
使用目的：会产生随机读写，为了减少随机读写的I/O， 先插入到change buffer缓存 ,合并后一起插入到索引页，减少I/O
注意：Change buffering is not supported for a secondary index if the index contains a descending index column or if the primary key includes a descending index column.
--查看change buffer 的配置
mysql> show variables like 'innodb_change_buffering';
+-------------------------+-------+
| Variable_name | Value |
+-------------------------+-------+
| innodb_change_buffering | all |
+-------------------------+-------+
1 row in set (0.09 sec)
--可配置的值如下
all
The default value: buffer inserts, delete-marking operations, and purges.
none
Do not buffer any operations.
inserts
Buffer insert operations.
deletes
Buffer delete-marking operations.
changes
Buffer both inserts and delete-marking operations.
purges
Buffer physical deletion operations that happen in the background.
--change buffer占整个buffer的百分比
--默认；25%，最高可调至50%
mysql> show variables like 'innodb_change_buffer_max_size';
+-------------------------------+-------+
| Variable_name | Value |
+-------------------------------+-------+
| innodb_change_buffer_max_size | 25 |
+-------------------------------+-------+
1 row in set (0.12 sec)
--测试
--建立测试表，b字段建立辅助索引
mysql> create table test_change_buffer_table (a int ,b varchar(10),primary key (a) ,key(b));
--插入数据
DROP PROCEDURE IF EXISTS p_test_change_buffer_table;
DELIMITER $
CREATE PROCEDURE p_test_change_buffer_table()
BEGIN
DECLARE i INT DEFAULT 1;
WHILE i<=1000000 DO
INSERT INTO test_change_buffer_table(a,b) VALUES(i,i);
SET i = i+1;
END WHILE;
COMMIT;
END $
ROLLBACK;
CALL p_test_change_buffer_table();
--更新数据
UPDATE test_change_buffer_table SET b='b' ;
COMMIT;
--观察上述状态change buffer，如下(INSERT BUFFER AND ADAPTIVE HASH INDEX)
mysql> show engine innodb status \G
*************************** 1. row ***************************
Type: InnoDB
Name:
Status:
=====================================
2018-11-15 13:45:25 0x7fa76dd78700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 8 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 454 srv_active, 0 srv_shutdown, 11659 srv_idle
srv_master_thread log flush and writes: 12113
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 1267
OS WAIT ARRAY INFO: signal count 1116
RW-shared spins 0, rounds 378, OS waits 155
RW-excl spins 0, rounds 4980, OS waits 178
RW-sx spins 87, rounds 2610, OS waits 75
Spin rounds per wait: 378.00 RW-shared, 4980.00 RW-excl, 30.00 RW-sx
------------
TRANSACTIONS
------------
Trx id counter 40165
Purge done for trx's n:o < 40165 undo n:o < 0 state: running but idle
History list length 77
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421832186570576, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 40160, ACTIVE 17 sec updating or deleting
mysql tables in use 1, locked 1
706 lock struct(s), heap size 106704, 354952 row lock(s), undo log entries 354952
MySQL thread id 8, OS thread handle 140356915402496, query id 13573251 192.168.56.1 flyremote updating
update test_change_buffer_table set b='b'
--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
13987 OS file reads, 46024 OS file writes, 2684 OS fsyncs
55.71 reads/s, 16384 avg bytes/read, 289.96 writes/s, 10.57 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 25, free list len 20, seg size 46, 35 merges --size为合并记录页的数量25， seg size是目前change buffer大小：46*16k=736k， 35次merges合并操作(实际读取页的次数)，
merged operations:
insert 1391, delete mark 7607, delete 0 --合并的操作有1391次insert和7607次标记的delete mark，真正delete删除为0次
discarded operations:
insert 0, delete mark 0, delete 0 --discarded表明合并时，表已不存在，所以无需合并做dml处理的次数
Hash table size 26041, node heap has 0 buffer(s)
Hash table size 26041, node heap has 1375 buffer(s)
Hash table size 26041, node heap has 287 buffer(s)
Hash table size 26041, node heap has 0 buffer(s)
Hash table size 26041, node heap has 1 buffer(s)
Hash table size 26041, node heap has 0 buffer(s)
Hash table size 26041, node heap has 0 buffer(s)
Hash table size 26041, node heap has 0 buffer(s)
42571.30 hash searches/s, 29614.30 non-hash searches/s --自适应哈希索引的使用效率
---
LOG
---
Log sequence number 2247327060
Log flushed up to 2246693831
Pages flushed up to 2223747920
Last checkpoint at 2197419801
0 pending log flushes, 0 pending chkp writes
887 log i/o's done, 2.14 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 107380736
Dictionary memory allocated 382802
Buffer pool size 6400
Free buffers 1117
Database pages 3614
Old database pages 1314
Modified db pages 1144
Pending reads 0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 7124, not young 15349893
66.56 youngs/s, 78029.85 non-youngs/s
Pages read 13856, created 22298, written 43636
55.71 reads/s, 88.27 creates/s, 282.39 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 301 / 1000
Pages read ahead 53.85/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 3614, unzip_LRU len: 0
I/O sum[3312]:cur[216], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB
Process ID=2652, Main thread ID=140356928546560, state: sleeping
Number of rows inserted 4524249, updated 354951, deleted 0, read 374965
0.00 inserts/s, 23180.48 updates/s, 0.00 deletes/s, 23180.48 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================
--查看change buffer占整个buffer pool的比例
--目前change buffer为：993*16k 大约15M， buffer pool 一共6400*16k=100M， 占比15%左右
mysql> SELECT (SELECT COUNT(*) FROM INFORMATION_SCHEMA.INNODB_BUFFER_PAGE
WHERE PAGE_TYPE LIKE 'IBUF%') AS change_buffer_pages,
(SELECT COUNT(*) FROM INFORMATION_SCHEMA.INNODB_BUFFER_PAGE) AS total_pages,
(SELECT ((change_buffer_pages/total_pages)*100))
AS change_buffer_page_percentage;
+---------------------+-------------+-------------------------------+
| change_buffer_pages | total_pages | change_buffer_page_percentage |
+---------------------+-------------+-------------------------------+
| 993 | 6400 | 15.5156 |
+---------------------+-------------+-------------------------------+
1 row in set (0.16 sec)
Warning
Querying the INNODB_BUFFER_PAGE table can introduce significant performance overhead.
To avoid impacting performance, reproduce the issue you want to investigate on a test instance and run your queries on the test instance.