SQL优化(二):MySQL索引失效的六种场景与优化方法

一、概述

  • 以下基于用户订单表t_order和订单清单条目表t_order_item来分析,二者通过order_id来建立外键约束。

    mysql> show create table t_order;
    +---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | Table   | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
    +---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | t_order | CREATE TABLE `t_order` (
      `order_id` int(11) NOT NULL AUTO_INCREMENT,
      `user_id` int(11) NOT NULL,
      `cost` double DEFAULT NULL,
      `buy_date` date NOT NULL,
      PRIMARY KEY (`order_id`),
      KEY `idx_order_buy_date` (`order_id`,`buy_date`),
      KEY `idx_user_id` (`user_id`),
      KEY `idx_user_id_buy_date` (`user_id`,`buy_date`),
      CONSTRAINT `user_refrence` FOREIGN KEY (`user_id`) REFERENCES `t_user` (`id`)
    ) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8 |
    +---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    1 row in set (0.00 sec)
    
    mysql> show create table t_order_item;
    +--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | Table        | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
    +--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    | t_order_item | CREATE TABLE `t_order_item` (
      `id` int(11) NOT NULL AUTO_INCREMENT,
      `product_id` int(11) NOT NULL,
      `price` double NOT NULL,
      `num` double NOT NULL,
      `order_id` int(20) NOT NULL,
      `remark` varchar(64) DEFAULT NULL,
      PRIMARY KEY (`id`),
      KEY `idx_remark` (`remark`),
      KEY `idx_num` (`num`),
      KEY `order_reference` (`order_id`),
      CONSTRAINT `order_reference` FOREIGN KEY (`order_id`) REFERENCES `t_order` (`order_id`)
    ) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8 |
    +--------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
    1 row in set (0.01 sec)
    

二. 最左前戳匹配

  • 最左前戳匹配主要是innodb存储引擎的B+树索引的特性导致的,即对于联合索引中的多个索引列在WHERE中需要从左到右保持联合索引中的列的顺序出现,如(a,b,c),则必须为where a=xx and b=xx等,如果是where b=xx and c=xx则无法使用该联合索引,注意如果是where b=xx and a=xx 则还是可以继续使用索引的,最左前戳匹配只是针对使用的列需要保证从左到右,where中的顺序没有限制,如下:t_order表包含联合索引:KEY idx_user_id_buy_date (user_id,buy_date):前两个SQL均可以使用该索引,最后一个只包含buy_date无法使用。

    mysql> explain select * from t_order where buy_date=curdate() and user_id=1;
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------------+------+----------+-------+
    | id | select_type | table   | partitions | type | possible_keys                    | key                  | key_len | ref         | rows | filtered | Extra |
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------------+------+----------+-------+
    |  1 | SIMPLE      | t_order | NULL       | ref  | idx_user_id,idx_user_id_buy_date | idx_user_id_buy_date | 7       | const,const |    1 |   100.00 | NULL  |
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------------+------+----------+-------+
    1 row in set, 1 warning (0.00 sec)
    
    mysql> explain select * from t_order where user_id=1 and buy_date=curdate();
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------------+------+----------+-------+
    | id | select_type | table   | partitions | type | possible_keys                    | key                  | key_len | ref         | rows | filtered | Extra |
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------------+------+----------+-------+
    |  1 | SIMPLE      | t_order | NULL       | ref  | idx_user_id,idx_user_id_buy_date | idx_user_id_buy_date | 7       | const,const |    1 |   100.00 | NULL  |
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------------+------+----------+-------+
    1 row in set, 1 warning (0.00 sec)
    
    mysql> explain select * from t_order where buy_date=curdate();
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    4 |    25.00 | Using where |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.00 sec)
    
  • 对于普通单列索引,如果是字符串则也可以取字符串的前几个字符而不是整个字符串。

  • 关于最左前戳匹配可参考:MySQL学习(七):Innodb存储引擎索引的实现原理

三. 字符串使用 like %

  • 如果字符串类型的索引列在查询条件中使用了like和通配符%进行模糊查询,跟最左前戳匹配类似,由于索引基于B+Tree实现,故如果%是在最左边则无法使用使用索引,如下:当%位于最左边时无法使用索引,否则可以继续使用索引。

    mysql> explain select * from t_order_item where remark like "%1";
    +----+-------------+--------------+------------+------+---------------+------+---------+------+------+----------+-------------+
    | id | select_type | table        | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+--------------+------------+------+---------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order_item | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    1 |   100.00 | Using where |
    +----+-------------+--------------+------------+------+---------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.01 sec)
    
    mysql> explain select * from t_order_item where remark like "1%";
    +----+-------------+--------------+------------+-------+---------------+------------+---------+------+------+----------+-----------------------+
    | id | select_type | table        | partitions | type  | possible_keys | key        | key_len | ref  | rows | filtered | Extra                 |
    +----+-------------+--------------+------------+-------+---------------+------------+---------+------+------+----------+-----------------------+
    |  1 | SIMPLE      | t_order_item | NULL       | range | idx_remark    | idx_remark | 195     | NULL |    1 |   100.00 | Using index condition |
    +----+-------------+--------------+------------+-------+---------------+------------+---------+------+------+----------+-----------------------+
    1 row in set, 1 warning (0.00 sec)
    

四. 数据类型不匹配,隐式转换

  • 数据类型隐式转换是指查询条件中的索引列对应的值的类型和列的类型不一致,需要MySQL进行隐式转换,如列的类型为字符串而查询条件值为数字,如下:在t_order_item表新增一个remark列,类型为varchar,并加上索引,当查询值为数字1时,type显示ALL,未使用索引,故字符串的列记得加双引号保持字符串类型。不过,反过来,即数字类型的字段用了字符串,则索引是可以正常使用的。

    mysql> alter table t_order_item add column remark varchar(64);
    Query OK, 0 rows affected (0.10 sec)
    Records: 0  Duplicates: 0  Warnings: 0
    mysql> alter table t_order_item add index idx_remark(remark);
    Query OK, 0 rows affected (0.02 sec)
    Records: 0  Duplicates: 0  Warnings: 0
    
    # 值为字符串,索引正常使用
    mysql> explain select * from t_order_item where remark="1";
    +----+-------------+--------------+------------+------+---------------+------------+---------+-------+------+----------+-------+
    | id | select_type | table        | partitions | type | possible_keys | key        | key_len | ref   | rows | filtered | Extra |
    +----+-------------+--------------+------------+------+---------------+------------+---------+-------+------+----------+-------+
    |  1 | SIMPLE      | t_order_item | NULL       | ref  | idx_remark    | idx_remark | 195     | const |    1 |   100.00 | NULL  |
    +----+-------------+--------------+------------+------+---------------+------------+---------+-------+------+----------+-------+
    1 row in set, 1 warning (0.00 sec)
    
    # 值为数字,索引失效
    mysql> explain select * from t_order_item where remark=1;
    +----+-------------+--------------+------------+------+---------------+------+---------+------+------+----------+-------------+
    | id | select_type | table        | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+--------------+------------+------+---------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order_item | NULL       | ALL  | idx_remark    | NULL | NULL    | NULL |    1 |   100.00 | Using where |
    +----+-------------+--------------+------------+------+---------------+------+---------+------+------+----------+-------------+
    1 row in set, 3 warnings (0.01 sec)
    
    
  • 除了字符串之外,日期类型的字段如果使用了字符串,索引也会失效,如下buy_date购买日期,如果使用字符串则失效,如果使用curdate函数获取日期,则正常使用。

    mysql> explain select * from t_order where buy_date = '2019-04-10';
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ALL  | idx_buy_date  | NULL | NULL    | NULL |    4 |   100.00 | Using where |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.00 sec)
    
    mysql> explain select * from t_order where buy_date = curdate();
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    | id | select_type | table   | partitions | type | possible_keys | key          | key_len | ref   | rows | filtered | Extra |
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    |  1 | SIMPLE      | t_order | NULL       | ref  | idx_buy_date  | idx_buy_date | 3       | const |    1 |   100.00 | NULL  |
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    1 row in set, 1 warning (0.00 sec)
    
    mysql> explain select * from t_order where buy_date = DATE_ADD(CURDATE(),INTERVAL 1 DAY) ;
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    | id | select_type | table   | partitions | type | possible_keys | key          | key_len | ref   | rows | filtered | Extra |
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    |  1 | SIMPLE      | t_order | NULL       | ref  | idx_buy_date  | idx_buy_date | 3       | const |    1 |   100.00 | NULL  |
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    1 row in set, 1 warning (0.00 sec)
    

五. OR条件

  • 对应innodb存储引擎来说,如果要获取所有列的数据,即不能使用覆盖索引,则OR两边的列都是不能使用索引的,即使两个列都有索引,或者是同一个列,如下,order_id为主键,buy_date包含有索引:

    mysql> explain select * from t_order where buy_date=date_add(curdate(), interval 1 DAY) or buy_date=curdate();
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ALL  | idx_buy_date  | NULL | NULL    | NULL |    5 |    40.00 | Using where |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.00 sec)
    
    mysql> 
    mysql> explain select * from t_order where user_id=1 or user_id=2;
    +----+-------------+---------+------------+------+----------------------------------+------+---------+------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys                    | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+----------------------------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ALL  | idx_user_id,idx_user_id_buy_date | NULL | NULL    | NULL |    5 |   100.00 | Using where |
    +----+-------------+---------+------------+------+----------------------------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.00 sec)
    
  • 但是如果只需要返回某些列,并且这些类可以通过联合索引覆盖到,则可以继续使用索引,如下:t_order表包含了联合索引(user_id, buy_date)。所以如果必须使用OR,则可以考虑使用覆盖索引来避免索引失效。

    mysql> explain select user_id,buy_date from t_order where user_id=1 or user_id=2;
    +----+-------------+---------+------------+-------+----------------------------------+----------------------+---------+------+------+----------+--------------------------+
    | id | select_type | table   | partitions | type  | possible_keys                    | key                  | key_len | ref  | rows | filtered | Extra                    |
    +----+-------------+---------+------------+-------+----------------------------------+----------------------+---------+------+------+----------+--------------------------+
    |  1 | SIMPLE      | t_order | NULL       | index | idx_user_id,idx_user_id_buy_date | idx_user_id_buy_date | 7       | NULL |    5 |   100.00 | Using where; Using index |
    +----+-------------+---------+------------+-------+----------------------------------+----------------------+---------+------+------+----------+--------------------------+
    1 row in set, 1 warning (0.00 sec)
    
    mysql> explain select user_id, buy_date from t_order where buy_date=date_add(curdate(), interval 1 DAY) or buy_date=curdate();
    +----+-------------+---------+------------+-------+---------------+----------------------+---------+------+------+----------+--------------------------+
    | id | select_type | table   | partitions | type  | possible_keys | key                  | key_len | ref  | rows | filtered | Extra                    |
    +----+-------------+---------+------------+-------+---------------+----------------------+---------+------+------+----------+--------------------------+
    |  1 | SIMPLE      | t_order | NULL       | index | idx_buy_date  | idx_user_id_buy_date | 7       | NULL |    5 |    40.00 | Using where; Using index |
    +----+-------------+---------+------------+-------+---------------+----------------------+---------+------+------+----------+--------------------------+
    1 row in set, 1 warning (0.00 sec)
    
  • 如果不适合创建联合索引来解决问题,如当OR查询需要返回的列较多时,则可以通过UNION来替代OR,如下:user_id和buy_date上面均有索引,当通过UNION时,则可以使用到索引,不过由Using temporary可知,需要额外的内存来汇总结果。

    mysql> explain select * from t_order where user_id=2 union select * from t_order where buy_date=curdate();
    +----+--------------+------------+------------+------+----------------------------------+--------------+---------+-------+------+----------+-----------------+
    | id | select_type  | table      | partitions | type | possible_keys                    | key          | key_len | ref   | rows | filtered | Extra           |
    +----+--------------+------------+------------+------+----------------------------------+--------------+---------+-------+------+----------+-----------------+
    |  1 | PRIMARY      | t_order    | NULL       | ref  | idx_user_id,idx_user_id_buy_date | idx_user_id  | 4       | const |    1 |   100.00 | NULL            |
    |  2 | UNION        | t_order    | NULL       | ref  | idx_buy_date                     | idx_buy_date | 3       | const |    1 |   100.00 | NULL            |
    | NULL | UNION RESULT | <union1,2> | NULL       | ALL  | NULL                             | NULL         | NULL    | NULL  | NULL |     NULL | Using temporary |
    +----+--------------+------------+------------+------+----------------------------------+--------------+---------+-------+------+----------+-----------------+
    3 rows in set, 1 warning (0.00 sec)
    
    mysql> explain select * from t_order where user_id=2 or buy_date=curdate();
    +----+-------------+---------+------------+------+-----------------------------------------------+------+---------+------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys                                 | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+-----------------------------------------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ALL  | idx_user_id,idx_user_id_buy_date,idx_buy_date | NULL | NULL    | NULL |    4 |   100.00 | Using where |
    +----+-------------+---------+------------+------+-----------------------------------------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.00 sec)
    

六. 列参与了MySQL函数的计算

  • 如果SQL查询条件中索引列对应的值是通过函数计算出来而不是某个确定的值,则无法使用索引,其中函数的使用不是说使用了MySQL函数就不可使用索引了,而是索引列参与到函数计算时不能再使用索引,如下:

  • (1)简单加减计算:order_id为t_order表的主键,由type可知使用了全表扫描,正常是PRIMARY使用主键索引的。

    mysql> explain select * from t_order where order_id + 1 = 2;
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    4 |   100.00 | Using where |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.00 sec)
    
  • (2)索引列参与到MySQL函数的使用,如下:第一个SQL中索引列buy_date参与到了DATE_ADD函数中,故type显示ALL,全表扫描;第二个SQL只是使用MySQL的函数可以继续使用索引,type显示ref,key为idx_buy_date。

    mysql> explain select * from t_order where buy_date = DATE_ADD(buy_date,INTERVAL 1 DAY) ;
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ALL  | NULL          | NULL | NULL    | NULL |    4 |    25.00 | Using where |
    +----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.01 sec)
    
    mysql> explain select * from t_order where buy_date = DATE_ADD(CURDATE(),INTERVAL 1 DAY) ;
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    | id | select_type | table   | partitions | type | possible_keys | key          | key_len | ref   | rows | filtered | Extra |
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    |  1 | SIMPLE      | t_order | NULL       | ref  | idx_buy_date  | idx_buy_date | 3       | const |    1 |   100.00 | NULL  |
    +----+-------------+---------+------------+------+---------------+--------------+---------+-------+------+----------+-------+
    1 row in set, 1 warning (0.00 sec)
    

七. 查询返回的数据超过了表的30%

  • 如果查询需要返回的数据超过了表所有的30%,则MySQL优化器会自动放弃使用索引而进行全表扫描,注意如果可以使用覆盖索引而不需要回表查询则MySQL会继续使用索引,如下t_order表只包含5条记录,并且4条属于用户1,1条属于用户2:

    mysql> select * from t_order;
    +----------+---------+------+------------+
    | order_id | user_id | cost | buy_date   |
    +----------+---------+------+------------+
    |        1 |       1 |  100 | 2019-04-10 |
    |        2 |       1 |  102 | 2019-04-10 |
    |        3 |       1 |  103 | 2019-04-10 |
    |        4 |       1 |  104 | 2019-04-10 |
    |        5 |       2 | 1000 | 2019-04-14 |
    +----------+---------+------+------------+
    5 rows in set (0.00 sec)
    
  • 获取所有数据列:user_id列包含索引,如果查询用户1的order则放弃使用索引,type显示ALL;如果查询用户2的order则继续使用索引,type显示ref:

    mysql> explain select * from t_order where user_id=1;
    +----+-------------+---------+------------+------+----------------------------------+------+---------+------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys                    | key  | key_len | ref  | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+----------------------------------+------+---------+------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ALL  | idx_user_id,idx_user_id_buy_date | NULL | NULL    | NULL |    4 |   100.00 | Using where |
    +----+-------------+---------+------------+------+----------------------------------+------+---------+------+------+----------+-------------+
    1 row in set, 1 warning (0.00 sec)
    
    mysql> explain select * from t_order where user_id=2;
    +----+-------------+---------+------------+------+----------------------------------+-------------+---------+-------+------+----------+-------+
    | id | select_type | table   | partitions | type | possible_keys                    | key         | key_len | ref   | rows | filtered | Extra |
    +----+-------------+---------+------------+------+----------------------------------+-------------+---------+-------+------+----------+-------+
    |  1 | SIMPLE      | t_order | NULL       | ref  | idx_user_id,idx_user_id_buy_date | idx_user_id | 4       | const |    1 |   100.00 | NULL  |
    +----+-------------+---------+------------+------+----------------------------------+-------------+---------+-------+------+----------+-------+
    1 row in set, 1 warning (0.00 sec)
    
  • 同样是查询用户1的所有订单,如果只需要返回user_id和购买日期buy_date,由于存在联合索引idx_user_id_buy_date,故可以进行索引覆盖,extra显示Using index,故使用该联合索引:

    mysql> explain select user_id,buy_date from t_order where user_id=1;
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------+------+----------+-------------+
    | id | select_type | table   | partitions | type | possible_keys                    | key                  | key_len | ref   | rows | filtered | Extra       |
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------+------+----------+-------------+
    |  1 | SIMPLE      | t_order | NULL       | ref  | idx_user_id,idx_user_id_buy_date | idx_user_id_buy_date | 4       | const |    4 |   100.00 | Using index |
    +----+-------------+---------+------------+------+----------------------------------+----------------------+---------+-------+------+----------+-------------+
    1 row in set, 1 warning (0.00 sec)
    

你可能感兴趣的:(MySQL)