MySQL性能优化
原文地址: https://www.ncucoder.com/2019/06/17/mysql-performance-optimization/
环境搭建
Vagrant + VirtualBox
Vagrant: https://www.vagrantup.com/
VirtualBox: https://www.virtualbox.org/
Ubuntu16安装mysql57
更换阿里源
wget -O install.sh https://gitee.com/hsowan/ausi/raw/master/install.sh && sudo bash install.sh xenial
安装mysql57
sudo apt install mysql-server mysql-client
配置mysql(开启慢查询日志)
mysql -uroot -p # 进入mysql
# 修改root用户的host, 避免在其他主机上无法登录, 即远程登录
update mysql.user set host = '%' where user = 'root' and host = 'localhost';
flush privileges; # 上一步操作后必须使用该命令进行刷新
# 修改root的密码的几种方式
# 第一种
alter user 'root'@'%' identified by '';
# 第二种, 这种方式需要刷新权限
update user set authentication_string = password('') where user = 'root' and host = '%';
flush privileges;
# 第三种
set password for 'root'@'%' = password('');
# 忘记密码, 在配置文件(vi /etc/mysql/my.cnf)中添加以下内容
# 则可以不用密码使用root登录, 再做修改密码的操作
[mysqld]
skip-grant-tables
# 查看mysql读取配置文件的顺序, 靠后的配置文件会覆盖前面的配置文件
/usr/sbin/mysqld --verbose --help | grep -A 1 'Default options'
# /etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf
show variables # 查看mysql的配置参数
set slow_query_log = on; # 开启慢查询日志
set slow_query_log_file = /var/lib/mysql/slow-query.log; # 设置慢查询日志的文件
set log_queries_not_using_indexes = on; # 记录那些未使用索引的查询日志
# 上面的设置只针对当前会话有效
# mysql5可以使用global设置当前mysql服务有效(set global slow_query_log = on;), 意味着重启mysql服务这些将会恢复默认
# mysql8可以使用persist进行持久化设置, 即使重启mysql服务也有效
\s 查看mysql版本信息
\G 输出结果旋转90度
Sakila Sample Database
https://dev.mysql.com/doc/sakila/en/
安装pt-query-digest
sudo apt install perl \
git clone [email protected]:mirrors/percona-toolkit.git \
perl Makefile.PL \
make \
make test \
make install
如果直接使用sudo apt install percona-toolkit, 会安装2.2.16版本的percona-toolkit, 但这个版本存在问题: https://stackoverflow.com/questions/38245395/pipeline-process-5-iteration-caused-an-error-redundant-argument-in-sprintf-at
pt-query-digest: https://www.percona.com/doc/percona-toolkit/LATEST/pt-query-digest.html
使用mysqldumpslow/pt-query-digest进行日志分析
mysqldumpslow
该工具是安装mysql57后就有的, 无需另外安装, 但是分析结果不如pt-query-digest丰富
mysqldumpslow /var/lib/mysql/slow-query.log
Count: 1 Time=0.00s (0s) Lock=0.00s (0s) Rows=200.0 (200), root[root]@localhost
select first_name from sakila.actor
pt-query-digest
pt-query-digest /var/lib/mysql/slow-query.log
# Query 6: 0 QPS, 0x concurrency, ID 0x68A4975F6684AA2CA7BDB4CD86DEDB2C at byte 5816
# This item is included in the report because it matches --limit.
# Scores: V/M = 0.00
# Time range: all events occurred at 2019-06-17T13:39:51
# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 3 1
# Exec time 3 365us 365us 365us 365us 365us 0 365us
# Lock time 4 98us 98us 98us 98us 98us 0 98us
# Rows sent 44 200 200 200 200 200 0 200
# Rows examine 5 200 200 200 200 200 0 200
# Query size 2 19 19 19 19 19 0 19
# String:
# Databases sakila
# Hosts localhost
# Users root
# Query_time distribution
# 1us
# 10us
# 100us ################################################################
# 1ms
# 10ms
# 100ms
# 1s
# 10s+
# Tables
# SHOW TABLE STATUS FROM `sakila` LIKE 'actor'\G
# SHOW CREATE TABLE `sakila`.`actor`\G
# EXPLAIN /*!50100 PARTITIONS*/
select * from actor\G
判断什么样的sql需要进行优化
使用pt-query-digest
- 查询次数(Count)多且占用时间(time)长的sql
- IO(Rows examine)大的sql
- 未命中索引的/索引命中率不高(比较Rows send和Rows examine)的sql, 会进行索引扫描/表扫描
使用explain
mysql> explain select customer_id, first_name, last_name from sakila.customer;
+----+-------------+----------+------------+------+---------------+------+---------+------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+----------+------------+------+---------------+------+---------+------+------+----------+-------+
| 1 | SIMPLE | customer | NULL | ALL | NULL | NULL | NULL | NULL | 599 | 100.00 | NULL |
+----+-------------+----------+------------+------+---------------+------+---------+------+------+----------+-------+
1 row in set, 1 warning (0.01 sec)
优化
count
# 在一条语句中同时查询出2006年和2007年电影的数量, count
select count(release_year = 2006 or null) as '2006年电影数量',
count(release_year = 2007 or null) as '2007年电影数量'
from sakila.film
# count(*) 会计算null的记录
# count(id) 不会计算id为null的记录
max
# 查询最近一次支付的时间
mysql> select max(payment_date) from sakila.payment;
+---------------------+
| max(payment_date) |
+---------------------+
| 2006-02-14 15:16:03 |
+---------------------+
1 row in set (0.01 sec)
# 查看sql的执行计划
mysql> explain select max(payment_date) from sakila.payment;
+----+-------------+---------+------------+------+---------------+------+---------+------+-------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+---------+------------+------+---------------+------+---------+------+-------+----------+-------+
| 1 | SIMPLE | payment | NULL | ALL | NULL | NULL | NULL | NULL | 16086 | 100.00 | NULL |
+----+-------------+---------+------------+------+---------------+------+---------+------+-------+----------+-------+
1 row in set, 1 warning (0.00 sec)
# 很明显这里进行了全表扫描, 所以我们需要在payment_date上建立索引
create index idx_payment_date on sakila.payment(payment_date);
# 再次查看sql的执行计划, 可以看到是直接命中查询结果的
mysql> explain select max(payment_date) from sakila.payment \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: NULL
partitions: NULL
type: NULL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: NULL
filtered: NULL
Extra: Select tables optimized away
1 row in set, 1 warning (0.00 sec)
limit
优化 避免过多的扫描
select film_id, description from sakila.film order by title limit 50, 5
这样会进行表扫描的操作，title应该是没有索引的
filesort
limit 50, 5 就是从第50行开始，取5行记录
51 52 53 54 55
limit 5 offset 50
limit 50, 5
这两个语句是等价的
limit 0, 5
limit 5
这两个语句是等价的
优化
select film_id, description from sakila.film order by film_id limit 50, 5
通过主键进行排序
避免很多io的操作
不再使用文件排序的操作
offset越大扫描的记录越多
select film_id, description from sakila.film where film_id > 55 and film_id <= 60 order by film_id limit 1, 5
子查询
- 使用连接查询
- 连接查询不能排除重复数据(可以使用distinct解决)
group by
先分组再连接
合理的索引
- where, group by, order by, on的从句中出现的字段都有必要建立相应的索引
- 索引可以提高查询效率, 但也会降低写(insert/update/delete)的效率, 所以需要删除那些不再使用或冗余的索引(
pt-index-usage,pt-duplicate-key-checker) - 索引字段越小越好, 数据库存储是以页为单位的, 节省跨多个页进行查找
- 联合索引, 离散度越高(唯一值越多的,
select count(distinct customer_id), (distinct staff_id) from payment)的放前面(index(customer_id, staff_id))