【MySQL优化(三)】性能监控分析 - Performance Schema
简介
MySQL的performance schema 用于实时监控MySQL server在一个较低级别的运行过程中的资源消耗、资源等待等情况。
在MySQL5.7版本中，性能模式是默认开启的，如果要关闭需要修改配置文件my.cnf。
开启与关闭
查看performance schema是否开启
mysql> SHOW VARIABLES LIKE 'performance_schema';
+--------------------+-------+
| Variable_name | Value |
+--------------------+-------+
| performance_schema | ON |
+--------------------+-------+
1 row in set (0.00 sec)
关闭performance schema
在配置文件中修改performance_schema的属性值，on表示开启，off表示关闭
[mysqld]
performance_schema=ON
相关概念
instruments：生产者，用于采集MySQL中各种各样的操作产生的事件信息，对应配置表中的配置项我们可以称为监控采集配置项。
consumers：消费者，对应的消费者表用于存储来自instruments采集的数据，对应配置表中的配置项我们可以称为消费存储配置项。
performance schema表分类
语句事件记录表
这些表记录了语句事件信息。
| 表 | 说明 |
|---|---|
| events_statements_current | 当前语句事件表 |
| events_statements_history | 历史语句事件表 |
| events_statements_history_long | 长语句历史事件表 |
| events_*_summary_* | 聚合后的摘要表summary，其中，summary表还可以根据帐号(account)，主机(host)，程序(program)，线程(thread)，用户(user)和全局(global)再进行细分) |
等待事件记录表
与语句事件类型的相关记录表类似
mysql> show tables like '%wait%';
+-----------------------------------------------+
| Tables_in_performance_schema (%wait%) |
+-----------------------------------------------+
| events_waits_current |
| events_waits_history |
| events_waits_history_long |
| events_waits_summary_by_account_by_event_name |
| events_waits_summary_by_host_by_event_name |
| events_waits_summary_by_instance |
| events_waits_summary_by_thread_by_event_name |
| events_waits_summary_by_user_by_event_name |
| events_waits_summary_global_by_event_name |
| table_io_waits_summary_by_index_usage |
| table_io_waits_summary_by_table |
| table_lock_waits_summary_by_table |
+-----------------------------------------------+
12 rows in set (0.00 sec)
阶段事件记录表
记录语句执行的阶段事件的表
mysql> show tables like '%stage%';
+------------------------------------------------+
| Tables_in_performance_schema (%stage%) |
+------------------------------------------------+
| events_stages_current |
| events_stages_history |
| events_stages_history_long |
| events_stages_summary_by_account_by_event_name |
| events_stages_summary_by_host_by_event_name |
| events_stages_summary_by_thread_by_event_name |
| events_stages_summary_by_user_by_event_name |
| events_stages_summary_global_by_event_name |
+------------------------------------------------+
8 rows in set (0.00 sec)
事务事件记录表
记录事务相关的事件的表
mysql> show tables like '%transaction%';
+------------------------------------------------------+
| Tables_in_performance_schema (%transaction%) |
+------------------------------------------------------+
| events_transactions_current |
| events_transactions_history |
| events_transactions_history_long |
| events_transactions_summary_by_account_by_event_name |
| events_transactions_summary_by_host_by_event_name |
| events_transactions_summary_by_thread_by_event_name |
| events_transactions_summary_by_user_by_event_name |
| events_transactions_summary_global_by_event_name |
+------------------------------------------------------+
8 rows in set (0.00 sec)
监控文件系统层调用的表
mysql> show tables like '%file%';
+---------------------------------------+
| Tables_in_performance_schema (%file%) |
+---------------------------------------+
| file_instances |
| file_summary_by_event_name |
| file_summary_by_instance |
+---------------------------------------+
3 rows in set (0.00 sec)
监视内存使用的表
mysql> show tables like '%memory%';
+-----------------------------------------+
| Tables_in_performance_schema (%memory%) |
+-----------------------------------------+
| memory_summary_by_account_by_event_name |
| memory_summary_by_host_by_event_name |
| memory_summary_by_thread_by_event_name |
| memory_summary_by_user_by_event_name |
| memory_summary_global_by_event_name |
+-----------------------------------------+
5 rows in set (0.00 sec)
动态对performance_schema进行配置的配置表
mysql> show tables like '%setup%';
+----------------------------------------+
| Tables_in_performance_schema (%setup%) |
+----------------------------------------+
| setup_actors |
| setup_consumers |
| setup_instruments |
| setup_objects |
| setup_timers |
+----------------------------------------+
5 rows in set (0.00 sec)
修改配置
数据库刚刚初始化并启动时，并非所有instruments和consumers的事件配置项都开启了（配置表中每一项都有一个开关字段，或为YES，或为NO，为YES就表示对应的表保存性能数据，为NO就表示对应的表不保存性能数据），所以默认不会收集所有的事件。
可以使用如下两个语句打开对应的instruments和consumers。
可以通过SQL修改setup开头的表，变更会立即生效，影响数据收集。
–打开等待事件的采集器配置项开关
需要修改setup_instruments配置表中对应的采集器配置项
UPDATE setup_instruments SET ENABLED = 'YES', TIMED = 'YES' where name like 'wait%'
使用示例
显示每个线程的最新监视事件
_current表中每个线程只保留一条记录，一旦线程完成工作，该表中不会再记录该线程的事件信息
mysql> select * from events_waits_current\G
*************************** 1. row ***************************
THREAD_ID: 31 # 事件来自哪个线程，事件编号是多少
EVENT_ID: 115
END_EVENT_ID: 115
EVENT_NAME: idle # 表示检测到的具体的内容
SOURCE: # 表示这个检测代码在哪个源文件中以及行号
TIMER_START: 83661764227000000 # 表示该事件的开始时间
TIMER_END: 83663288012000000 # 表示该事件的结束时间
TIMER_WAIT: 1523785000000 # 表示该事件总的花费时间
SPINS: NULL
OBJECT_SCHEMA: NULL
OBJECT_NAME: NULL
INDEX_NAME: NULL
OBJECT_TYPE: NULL
OBJECT_INSTANCE_BEGIN: 0
NESTING_EVENT_ID: NULL
NESTING_EVENT_TYPE: NULL
OPERATION: idle
NUMBER_OF_BYTES: NULL
FLAGS: NULL
1 row in set (0.00 sec)
记录每个线程应该执行完成的事件信息
*_history表记录每个线程的事件信息只会记录10条，再多就会被覆盖。
*_history_long表中记录所有线程的事件信息，但总记录数量是10000，超过就会被覆盖掉。
mysql> select thread_id,event_id,event_name,timer_wait from events_waits_history order by thread_id limit 10;
+-----------+----------+------------+----------------+
| thread_id | event_id | event_name | timer_wait |
+-----------+----------+------------+----------------+
| 28 | 738 | idle | 60000387000000 |
| 28 | 740 | idle | 60000991000000 |
| 28 | 742 | idle | 60000962000000 |
| 28 | 744 | idle | 60000778000000 |
| 28 | 746 | idle | 60000920000000 |
| 29 | 724 | idle | 59160828000000 |
| 29 | 726 | idle | 60734364000000 |
| 29 | 728 | idle | 59570976000000 |
| 29 | 730 | idle | 62425649000000 |
| 29 | 732 | idle | 59371180000000 |
+-----------+----------+------------+----------------+
10 rows in set (0.00 sec)
summary表提供所有事件的汇总信息
该组中的表以不同的方式汇总事件数据（如：按用户，按主机，按线程等等）。例如：要查看哪些instruments占用最多的时间，可以通过对events_waits_summary_global_by_event_name表的COUNT_STAR或SUM_TIMER_WAIT列进行查询（这两列是对事件的记录数执行COUNT（*）、事件记录的TIMER_WAIT列执行SUM（TIMER_WAIT）统计而来）
mysql> SELECT EVENT_NAME,COUNT_STAR FROM events_waits_summary_global_by_event_name ORDER BY COUNT_STAR DESC LIMIT 10;
+---------------------------------------------+------------+
| EVENT_NAME | COUNT_STAR |
+---------------------------------------------+------------+
| idle | 3249 |
| wait/io/file/innodb/innodb_data_file | 2348 |
| wait/io/file/sql/FRM | 1458 |
| wait/io/file/myisam/dfile | 327 |
| wait/io/table/sql/handler | 140 |
| wait/io/file/myisam/kfile | 37 |
| wait/io/file/sql/dbopt | 24 |
| wait/io/file/innodb/innodb_log_file | 21 |
| wait/io/file/keyring_file/keyring_file_data | 9 |
| wait/io/file/mysys/cnf | 5 |
+---------------------------------------------+------------+
10 rows in set (0.02 sec)
instance表记录了哪些类型的对象会被检测
些对象在被server使用时，在该表中将会产生一条事件记录，例如，file_instances表列出了文件I/O操作及其关联文件名
mysql> select * from file_instances limit 5;
+--------------------------------------------------------------------+---------------------------------------------+------------+
| FILE_NAME | EVENT_NAME | OPEN_COUNT |
+--------------------------------------------------------------------+---------------------------------------------+------------+
| /usr/local/mysql-5.7.31-macos10.14-x86_64/share/english/errmsg.sys | wait/io/file/sql/ERRMSG | 0 |
| /usr/local/mysql-5.7.31-macos10.14-x86_64/share/charsets/Index.xml | wait/io/file/mysys/charset | 0 |
| /usr/local/mysql-5.7.31-macos10.14-x86_64/keyring/keyring | wait/io/file/keyring_file/keyring_file_data | 0 |
| /usr/local/mysql-5.7.31-macos10.14-x86_64/data/ibdata1 | wait/io/file/innodb/innodb_data_file | 3 |
| /usr/local/mysql-5.7.31-macos10.14-x86_64/data/ib_logfile0 | wait/io/file/innodb/innodb_log_file | 2 |
+--------------------------------------------------------------------+---------------------------------------------+------------+
5 rows in set (0.00 sec)
常用配置项的参数说明
1. 启用选项
| 参数 | 值 | 说明 |
|---|---|---|
| performance_schema_consumer_events_statements_current | TRUE | 是否在MySQL Server启动时就开启events_statements_current表的记录功能(该表记录当前的语句事件信息)，启动之后也可以在setup_consumers表中使用UPDATE语句进行动态更新setup_consumers配置表中的events_statements_current配置项，默认值为TRUE |
| performance_schema_consumer_events_statements_history | TRUE | 与performance_schema_consumer_events_statements_current选项类似，但该选项是用于配置是否记录语句事件短历史信息，默认为TRUE |
| performance_schema_consumer_events_stages_history_long | FALSE | 与performance_schema_consumer_events_statements_current选项类似，但该选项是用于配置是否记录语句事件长历史信息，默认为FALSE |
| performance_schema_consumer_global_instrumentation | TRUE | 是否在MySQL Server启动时就开启全局表（如：mutex_instances、rwlock_instances、cond_instances、file_instances、users、hostsaccounts、socket_summary_by_event_name、file_summary_by_instance等大部分的全局对象计数统计和事件汇总统计信息表 ）的记录功能，启动之后也可以在setup_consumers表中使用UPDATE语句进行动态更新全局配置项，默认值为TRUE |
| performance_schema_consumer_statements_digest | TRUE | 是否在MySQL Server启动时就开启events_statements_summary_by_digest 表的记录功能，启动之后也可以在setup_consumers表中使用UPDATE语句进行动态更新digest配置项，默认值为TRUE |
| performance_schema_consumer_thread_instrumentation | TRUE | 是否在MySQL Server启动时就开启events_xxx_summary_by_yyy_by_event_name表的记录功能，启动之后也可以在setup_consumers表中使用UPDATE语句进行动态更新线程配置项，默认值为TRUE |
| performance_schema_instrument[=name] | 否在MySQL Server启动时就启用某些采集器，由于instruments配置项多达数千个，所以该配置项支持key-value模式，还支持%号进行通配等，例如：[=name]可以指定为具体的Instruments名称（但是这样如果有多个需要指定的时候，就需要使用该选项多次），也可以使用通配符，可以指定instruments相同的前缀+通配符，也可以使用%代表所有的instruments | |
| performance-schema-instrument= ‘instrument_name=value’ | 指定开启单个instruments | |
| performance-schema-instrument= ‘wait/synch/cond/%=COUNTED’ | 使用通配符指定开启多个instruments | |
| performance-schema-instrument= ‘%=ON’ | 开关所有的instruments |
注意：
- 除了statement(语句)事件之外，还支持：wait(等待)事件、state(阶段)事件、transaction(事务)事件，他们与statement事件一样都有三个启动项分别进行配置，但这些等待事件默认未启用，如果需要在MySQL Server启动时一同启动，则通常需要写进my.cnf配置文件中
- 这些启动选项要生效的前提是，需要设置performance_schema=ON。另外，这些启动选项虽然无法使用show variables语句查看，但我们可以通过setup_instruments和setup_consumers表查询这些选项指定的值
系统变量
查看相关变量
show variables like '%performance_schema%';
重要的属性解释
| 属性 | 值 | 说明 |
|---|---|---|
| performance_schema | ON | 控制performance_schema功能的开关，要使用MySQL的performance_schema，需要在mysqld启动时启用，以启用事件收集功能。该参数在5.7.x之前支持performance_schema的版本中默认关闭，5.7.x版本开始默认开启。注意：如果mysqld在初始化performance_schema时发现无法分配任何相关的内部缓冲区，则performance_schema将自动禁用，并将performance_schema设置为OFF |
| performance_schema_digests_size | 10000 | 控制events_statements_summary_by_digest表中的最大行数。如果产生的语句摘要信息超过此最大值，便无法继续存入该表，此时performance_schema会增加状态变量 |
| performance_schema_events_statements_history_long_size | 10000 | 控制events_statements_history_long表中的最大行数，该参数控制所有会话在events_statements_history_long表中能够存放的总事件记录数，超过这个限制之后，最早的记录将被覆盖。 |
| performance_schema_events_statements_history_size | 10 | 控制events_statements_history表中单个线程（会话）的最大行数，该参数控制单个会话在events_statements_history表中能够存放的事件记录数，超过这个限制之后，单个会话最早的记录将被覆盖 |
| performance_schema_max_digest_length | 1024 | 用于控制标准化形式的SQL语句文本在存入performance_schema时的限制长度，该变量与max_digest_length变量相关(max_digest_length变量含义请自行查阅相关资料) |
| performance_schema_max_sql_text_length | 1024 | 控制存入events_statements_current，events_statements_history和events_statements_history_long语句事件表中的SQL_TEXT列的最大SQL长度字节数。 超出系统变量performance_schema_max_sql_text_length的部分将被丢弃，不会记录，一般情况下不需要调整该参数，除非被截断的部分与其他SQL比起来有很大差异。降低系统变量performance_schema_max_sql_text_length值可以减少内存使用，但如果汇总的SQL中，被截断部分有较大差异，会导致没有办法再对这些有较大差异的SQL进行区分。 增加该系统变量值会增加内存使用，但对于汇总SQL来讲可以更精准地区分不同的部分 |
重要配置表的相关说明
performance_timers表
记录了server中有哪些可用的事件计时器
mysql> select * from performance_timers;
+-------------+-----------------+------------------+----------------+
| TIMER_NAME | TIMER_FREQUENCY | TIMER_RESOLUTION | TIMER_OVERHEAD |
+-------------+-----------------+------------------+----------------+
| CYCLE | 2588686567 | 1 | 20 |
| NANOSECOND | 1000000000 | 1000 | 112 |
| MICROSECOND | 1000000 | 1 | 98 |
| MILLISECOND | 1041 | 1 | 110 |
| TICK | 102 | 1 | 3530 |
+-------------+-----------------+------------------+----------------+
5 rows in set (0.00 sec)
字段解释：
- timer_name：表示可用计时器名称，CYCLE是基于CPU周期计数器的定时器
- timer_frequency：表示每秒钟对应的计时器单位的数量，CYCLE计时器的换算值与CPU的频率相关
- timer_resolution：计时器精度值，表示在每个计时器被调用时额外增加的值
- timer_overhead：表示在使用定时器获取事件时开销的最小周期值
setup_timers表
记录当前使用的事件计时器信息
mysql> select * from setup_timers;
+-------------+-------------+
| NAME | TIMER_NAME |
+-------------+-------------+
| idle | MICROSECOND |
| wait | CYCLE |
| stage | NANOSECOND |
| statement | NANOSECOND |
| transaction | NANOSECOND |
+-------------+-------------+
5 rows in set, 1 warning (0.00 sec)
字段解释：
- name：计时器类型，对应某个事件类别
- timer_name：计时器类型名称
setup_consumers表
列出了consumers可配置列表项
mysql> select * from setup_consumers;
+----------------------------------+---------+
| NAME | ENABLED |
+----------------------------------+---------+
| events_stages_current | NO |
| events_stages_history | NO |
| events_stages_history_long | NO |
| events_statements_current | YES |
| events_statements_history | YES |
| events_statements_history_long | NO |
| events_transactions_current | NO |
| events_transactions_history | NO |
| events_transactions_history_long | NO |
| events_waits_current | YES |
| events_waits_history | YES |
| events_waits_history_long | YES |
| global_instrumentation | YES |
| thread_instrumentation | YES |
| statements_digest | YES |
+----------------------------------+---------+
15 rows in set (0.00 sec)
字段解释：
- NAME：consumers配置名称
- ENABLED：consumers是否启用，有效值为YES或NO，此列可以使用UPDATE语句修改。
setup_instruments表
列出了instruments 列表配置项，即代表了哪些事件支持被收集
mysql> SELECT * FROM setup_instruments;
+--------------------------------------------------------------------------------+---------+-------+
| NAME | ENABLED | TIMED |
+--------------------------------------------------------------------------------+---------+-------+
| wait/synch/mutex/sql/TC_LOG_MMAP::LOCK_tc | NO | NO |
| wait/synch/mutex/sql/LOCK_des_key_file | NO | NO |
| wait/synch/mutex/sql/MYSQL_BIN_LOG::LOCK_commit | NO | NO |
| wait/synch/mutex/sql/MYSQL_BIN_LOG::LOCK_commit_queue | NO | NO |
| wait/synch/mutex/sql/MYSQL_BIN_LOG::LOCK_done | NO | NO |
| wait/synch/mutex/sql/MYSQL_BIN_LOG::LOCK_flush_queue | NO | NO |
.....
字段解释：
- NAME：instruments名称，instruments名称可能具有多个部分并形成层次结构
- ENABLED：instrumetns是否启用，有效值为YES或NO，此列可以使用UPDATE语句修改。如果设置为NO，则这个instruments不会被执行，不会产生任何的事件信息
- TIMED：instruments是否收集时间信息，有效值为YES或NO，此列可以使用UPDATE语句修改，如果设置为NO，则这个instruments不会收集时间信息
setup_actors表
setup_actors表的初始内容是匹配任何用户和主机，因此对于所有前台线程，默认情况下启用监视和历史事件收集功能
mysql> SELECT * FROM setup_actors;
+------+------+------+---------+---------+
| HOST | USER | ROLE | ENABLED | HISTORY |
+------+------+------+---------+---------+
| % | % | % | YES | YES |
+------+------+------+---------+---------+
1 row in set (0.00 sec)
字段解释：
- HOST：与grant语句类似的主机名，一个具体的字符串名字，或使用“％”表示“任何主机”
- USER：一个具体的字符串名称，或使用“％”表示“任何用户”
- ROLE：当前未使用，MySQL 8.0中才启用角色功能
- ENABLED：是否启用与HOST，USER，ROLE匹配的前台线程的监控功能，有效值为：YES或NO
- HISTORY：是否启用与HOST， USER，ROLE匹配的前台线程的历史事件记录功能，有效值为：YES或NO
setup_objects表
控制performance_schema是否监视特定对象。默认情况下，此表的最大行数为100行
mysql> SELECT * FROM setup_objects;
+-------------+--------------------+-------------+---------+-------+
| OBJECT_TYPE | OBJECT_SCHEMA | OBJECT_NAME | ENABLED | TIMED |
+-------------+--------------------+-------------+---------+-------+
| EVENT | mysql | % | NO | NO |
| EVENT | performance_schema | % | NO | NO |
| EVENT | information_schema | % | NO | NO |
| EVENT | % | % | YES | YES |
| FUNCTION | mysql | % | NO | NO |
| FUNCTION | performance_schema | % | NO | NO |
| FUNCTION | information_schema | % | NO | NO |
| FUNCTION | % | % | YES | YES |
| PROCEDURE | mysql | % | NO | NO |
| PROCEDURE | performance_schema | % | NO | NO |
| PROCEDURE | information_schema | % | NO | NO |
| PROCEDURE | % | % | YES | YES |
| TABLE | mysql | % | NO | NO |
| TABLE | performance_schema | % | NO | NO |
| TABLE | information_schema | % | NO | NO |
| TABLE | % | % | YES | YES |
| TRIGGER | mysql | % | NO | NO |
| TRIGGER | performance_schema | % | NO | NO |
| TRIGGER | information_schema | % | NO | NO |
| TRIGGER | % | % | YES | YES |
+-------------+--------------------+-------------+---------+-------+
20 rows in set (0.00 sec)
字段解释：
- OBJECT_TYPE：instruments类型，有效值为：“EVENT”（事件调度器事件）、“FUNCTION”（存储函数）、“PROCEDURE”（存储过程）、“TABLE”（基表）、“TRIGGER”（触发器），TABLE对象类型的配置会影响表I/O事件（wait/io/table/sql/handler instrument）和表锁事件（wait/lock/table/sql/handler instrument）的收集
- OBJECT_SCHEMA：某个监视类型对象涵盖的数据库名称，一个字符串名称，或“％”(表示“任何数据库”)
- OBJECT_NAME：某个监视类型对象涵盖的表名，一个字符串名称，或“％”(表示“任何数据库内的对象”)
- ENABLED：是否开启对某个类型对象的监视功能，有效值为：YES或NO。此列可以修改
- TIMED：是否开启对某个类型对象的时间收集功能，有效值为：YES或NO，此列可以修改
threads表
对于每个server线程生成一行包含线程相关的信息
mysql> select * from threads;
+-----------+----------------------------------------+------------+----------------+------------------+------------------+--------------------+---------------------+------------------+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+------+--------------+---------+-----------------+--------------+
| THREAD_ID | NAME | TYPE | PROCESSLIST_ID | PROCESSLIST_USER | PROCESSLIST_HOST | PROCESSLIST_DB | PROCESSLIST_COMMAND | PROCESSLIST_TIME | PROCESSLIST_STATE | PROCESSLIST_INFO | PARENT_THREAD_ID | ROLE | INSTRUMENTED | HISTORY | CONNECTION_TYPE | THREAD_OS_ID |
+-----------+----------------------------------------+------------+----------------+------------------+------------------+--------------------+---------------------+------------------+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+------+--------------+---------+-----------------+--------------+
| 1 | thread/sql/main | BACKGROUND | NULL | NULL | NULL | NULL | NULL | 85693 | NULL | NULL | NULL | NULL | YES | YES | NULL | 966 |
...
字段解释：
- THREAD_ID：线程的唯一标识符（ID）
- NAME：与server中的线程检测代码相关联的名称(注意，这里不是instruments名称)
- TYPE：线程类型，有效值为：FOREGROUND、BACKGROUND。分别表示前台线程和后台线程
- PROCESSLIST_ID：对应INFORMATION_SCHEMA.PROCESSLIST表中的ID列。
- PROCESSLIST_USER：与前台线程相关联的用户名，对于后台线程为NULL。
- PROCESSLIST_HOST：与前台线程关联的客户端的主机名，对于后台线程为NULL。
- PROCESSLIST_DB：线程的默认数据库，如果没有，则为NULL。
- PROCESSLIST_COMMAND：对于前台线程，该值代表着当前客户端正在执行的command类型，如果是sleep则表示当前会话处于空闲状态
- PROCESSLIST_TIME：当前线程已处于当前线程状态的持续时间（秒）
- PROCESSLIST_STATE：表示线程正在做什么事情。
- PROCESSLIST_INFO：线程正在执行的语句，如果没有执行任何语句，则为NULL。
- PARENT_THREAD_ID：如果这个线程是一个子线程（由另一个线程生成），那么该字段显示其父线程ID
- ROLE：暂未使用
- INSTRUMENTED：线程执行的事件是否被检测。有效值：YES、NO
- HISTORY：是否记录线程的历史事件。有效值：YES、NO *
- THREAD_OS_ID：由操作系统层定义的线程或任务标识符（ID）：
实践操作
1. 哪类的SQL执行最多
mysql> SELECT DIGEST_TEXT,COUNT_STAR,FIRST_SEEN,LAST_SEEN FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+---------------------+---------------------+
| DIGEST_TEXT | COUNT_STAR | FIRST_SEEN | LAST_SEEN |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+---------------------+---------------------+
| SHOW TABLES LIKE ? | 7 | 2022-11-08 19:39:50 | 2022-11-08 20:34:45 |
| SHOW VARIABLES LIKE ? | 6 | 2022-11-08 07:50:06 | 2022-11-08 08:20:06 |
...
2. 哪类SQL的平均响应时间最多
mysql> SELECT DIGEST_TEXT,AVG_TIMER_WAIT FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+
| DIGEST_TEXT | AVG_TIMER_WAIT |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------+
| SHOW TABLES LIKE ? | 1775571000 |
| SHOW VARIABLES LIKE ? | 2102333000 |
...
3. 哪类SQL排序记录数最多
mysql> SELECT DIGEST_TEXT,SUM_SORT_ROWS FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+
| DIGEST_TEXT | SUM_SORT_ROWS |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+
| SHOW TABLES LIKE ? | 0 |
| SHOW VARIABLES LIKE ? | 0 |
...
4. 哪类SQL扫描记录数最多
mysql> SELECT DIGEST_TEXT,SUM_ROWS_EXAMINED FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------+
| DIGEST_TEXT | SUM_ROWS_EXAMINED |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------+
| SHOW TABLES LIKE ? | 76 |
| SHOW VARIABLES LIKE ? | 6204 |
...
5. 哪类SQL使用临时表最多
mysql> SELECT DIGEST_TEXT,SUM_CREATED_TMP_TABLES,SUM_CREATED_TMP_DISK_TABLES FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------+-----------------------------+
| DIGEST_TEXT | SUM_CREATED_TMP_TABLES | SUM_CREATED_TMP_DISK_TABLES |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------+-----------------------------+
| SHOW TABLES LIKE ? | 7 | 0 |
| SHOW VARIABLES LIKE ? | 6 | 0 |
...
6. 哪类SQL返回结果集最多
mysql> SELECT DIGEST_TEXT,SUM_ROWS_SENT FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC;
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+
| DIGEST_TEXT | SUM_ROWS_SENT |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+
| SHOW TABLES LIKE ? | 76 |
| SHOW VARIABLES LIKE ? | 6 |
...
7. 哪个表物理IO最多
mysql> SELECT file_name,event_name,SUM_NUMBER_OF_BYTES_READ,SUM_NUMBER_OF_BYTES_WRITE FROM file_summary_by_instance ORDER BY SUM_NUMBER_OF_BYTES_READ + SUM_NUMBER_OF_BYTES_WRITE DESC;
+----------------------------------------------------------------------------------------------------------------------------+---------------------------------------------+--------------------------+---------------------------+
| file_name | event_name | SUM_NUMBER_OF_BYTES_READ | SUM_NUMBER_OF_BYTES_WRITE |
+----------------------------------------------------------------------------------------------------------------------------+---------------------------------------------+--------------------------+---------------------------+
| /usr/local/mysql-5.7.31-macos10.14-x86_64/data/ibtmp1 | wait/io/file/innodb/innodb_data_file | 0 | 13467648 |
....
8. 哪个表逻辑IO最多
mysql> SELECT object_name,COUNT_READ,COUNT_WRITE,COUNT_FETCH,SUM_TIMER_WAIT FROM table_io_waits_summary_by_table ORDER BY sum_timer_wait DESC;
+-------------+------------+-------------+-------------+----------------+
| object_name | COUNT_READ | COUNT_WRITE | COUNT_FETCH | SUM_TIMER_WAIT |
+-------------+------------+-------------+-------------+----------------+
| test_lock | 0 | 0 | 0 | 0 |
| test_table1 | 0 | 0 | 0 | 0 |
| test_table2 | 0 | 0 | 0 | 0 |
| test_table3 | 0 | 0 | 0 | 0 |
+-------------+------------+-------------+-------------+----------------+
4 rows in set (0.00 sec)
9. 哪个索引访问最多
mysql> SELECT OBJECT_NAME,INDEX_NAME,COUNT_FETCH,COUNT_INSERT,COUNT_UPDATE,COUNT_DELETE FROM table_io_waits_summary_by_index_usage ORDER BY SUM_TIMER_WAIT DESC;
+-------------+---------------------+-------------+--------------+--------------+--------------+
| OBJECT_NAME | INDEX_NAME | COUNT_FETCH | COUNT_INSERT | COUNT_UPDATE | COUNT_DELETE |
+-------------+---------------------+-------------+--------------+--------------+--------------+
| test_lock | PRIMARY | 0 | 0 | 0 | 0 |
| test_table1 | PRIMARY | 0 | 0 | 0 | 0 |
| test_table1 | source | 0 | 0 | 0 | 0 |
| test_table1 | province_city | 0 | 0 | 0 | 0 |
| test_table1 | check_last | 0 | 0 | 0 | 0 |
| test_table1 | admin_id | 0 | 0 | 0 | 0 |
| test_table1 | is_end | 0 | 0 | 0 | 0 |
| test_table1 | idx_uid_time_mobile | 0 | 0 | 0 | 0 |
| test_table1 | sort_time | 0 | 0 | 0 | 0 |
| test_table1 | is_fast | 0 | 0 | 0 | 0 |
| test_table1 | user_mobile | 0 | 0 | 0 | 0 |
| test_table1 | uuid | 0 | 0 | 0 | 0 |
| test_table1 | idx_count | 0 | 0 | 0 | 0 |
| test_table1 | idx_hash_tel | 0 | 0 | 0 | 0 |
| test_table2 | PRIMARY | 0 | 0 | 0 | 0 |
| test_table2 | uuid | 0 | 0 | 0 | 0 |
| test_table2 | tel | 0 | 0 | 0 | 0 |
| test_table2 | idx_user_id | 0 | 0 | 0 | 0 |
| test_table3 | PRIMARY | 0 | 0 | 0 | 0 |
| test_table3 | end_time | 0 | 0 | 0 | 0 |
| test_table3 | information_id | 0 | 0 | 0 | 0 |
+-------------+---------------------+-------------+--------------+--------------+--------------+
21 rows in set (0.01 sec)
10. 哪个索引从来没有用过
mysql> SELECT OBJECT_SCHEMA,OBJECT_NAME,INDEX_NAME FROM table_io_waits_summary_by_index_usage WHERE INDEX_NAME IS NOT NULL AND COUNT_STAR = 0 AND OBJECT_SCHEMA <> 'mysql' ORDER BY OBJECT_SCHEMA,OBJECT_NAME;
+---------------+-------------+---------------------+
| OBJECT_SCHEMA | OBJECT_NAME | INDEX_NAME |
+---------------+-------------+---------------------+
| yupao | test_lock | PRIMARY |
| yupao | test_table1 | PRIMARY |
| yupao | test_table1 | source |
....
11. 哪个等待事件消耗时间最多
mysql> SELECT EVENT_NAME,COUNT_STAR,SUM_TIMER_WAIT,AVG_TIMER_WAIT FROM events_waits_summary_global_by_event_name WHERE event_name != 'idle' ORDER BY SUM_TIMER_WAIT DESC;
+--------------------------------------------------------------------+------------+----------------+----------------+
| EVENT_NAME | COUNT_STAR | SUM_TIMER_WAIT | AVG_TIMER_WAIT |
+--------------------------------------------------------------------+------------+----------------+----------------+
| wait/io/file/innodb/innodb_data_file | 2566 | 249228307340 | 97126864 |
| wait/io/file/sql/FRM | 1556 | 89719093224 | 57659908 |
| wait/io/file/innodb/innodb_log_file | 93 | 81359305172 | 874831172 |
...
12. 剖析某条SQL的执行情况，包括statement信息，stege信息，wait信息
- 获取EventID
mysql> SELECT EVENT_ID,sql_text FROM events_statements_history WHERE sql_text LIKE '%count(*)%';
+----------+------------------------------------------------------------------------------------------+
| EVENT_ID | sql_text |
+----------+------------------------------------------------------------------------------------------+
| 196 | SELECT EVENT_ID,sql_text FROM events_statements_history WHERE sql_text LIKE '%count(*)%' |
| 198 | select count(*) from test_table1 |
| 202 | select count(*) from yupao.test_table1 |
+----------+------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)
- 查看每个阶段的时间消耗
SELECT event_id,EVENT_NAME,SOURCE,TIMER_END - TIMER_START FROM events_stages_history_long WHERE NESTING_EVENT_ID = 198
- 查看每个阶段的锁等待情况
SELECT event_id,event_name,source,timer_wait,object_name,index_name,operation,nesting_event_id FROM events_waits_history_longWHERE nesting_event_id = 198;
实际操作分析
核心指标
- 连接数
客户端连接完成后，如果没有后续的动作，连接就处于空闲状态，空闲状态超过指定时间后，服务器会主动断开连接，这个时间受参数 wait_timeou 控制， 默认8小时mysql> show variables like 'wait_timeout'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | wait_timeout | 28800 | +---------------+-------+ 1 row in set (0.00 sec)
监控长事务，从数据库端来看：
- 监控 information_schema.Innodb_trx 表，设置长事务阈值，超过就报警 / 或者 kill；
- Percona 的 pt-kill 这个工具不错，推荐使用；
- 在业务功能测试阶段要求输出所有的 general_log，分析日志行为提前发现问题；
- 如果使用的是 MySQL 5.6 或者更新版本，把 innodb_undo_tablespaces 设置成 2（或更大的值）。如果真的出现大事务导致回滚段过大，这样设置后清理起来更方便。
性能指标参考
This group of numbers is from a presentation Jeff Dean gave at a Engineering All-Hands Meeting at Google.
- L1 cache reference 0.5 ns
- Branch mispredict 5 ns
- L2 cache reference 7 ns
- Mutex lock/unlock 100 ns
- Main memory reference 100 ns
- Compress 1K bytes with Zippy 10,000 ns
- Send 2K bytes over 1 Gbps network 20,000 ns
- Read 1 MB sequentially from memory 250,000 ns
- Round trip within same datacenter 500,000 ns
- Disk seek 10,000,000 ns
- Read 1 MB sequentially from network 10,000,000 ns
- Read 1 MB sequentially from disk 30,000,000 ns
- Send packet CA->Netherlands->CA 150,000,000 ns
系列文章
上一篇：【MySQL优化(二)】性能监控分析 - Show Profile
下一篇：【MySQL优化(四)】InnoDB数据类型解析及建表规约