MyCat 学习笔记 第十七篇 . mysql 5.5 、mysql 5.6 、mysql 5.7 简单比较 上

前言
从mycat官方网站的介绍来看，作为数据库前置中间件，mycat后端可对接 mysql 、oralce 、 mongodb 等。但目前实际工作中，相信大部分的应用还是基于mysql在跑，今天就是做一个简单的比对，在相同数据表、相同数据量、相同操作的时候 mysql 几个主流版本的性能情况。

环境说明
mac 192.168.13.1 8066/9066 mycat
vm1 192.168.13.186 5503 mysql 5.5
vm1 192.168.13.186 3307 mysql 5.6
vm2 192.168.13.190 5703 mysql 5.7

* 应用场景*

1. 数据通过 mycat 批量新增至 mysql 5.5 与 直接批量新增至 mysql 5.5
2. 数据通过 mycat 批量新增至 mysql 5.6 与 直接批量新增至 mysql 5.6
3. 数据通过 mycat 批量新增至 mysql 5.7 与 直接批量新增至 mysql 5.7
4. 数据表分片 通过mycat 批量新增至 mysql 5.5 \ mysql 5.6\ mysql 5.7

mycat 配置环境：rule.xml ，用来作mycat数据分片使用

<tableRule name="ruleDateStr">
        <rule>
            <columns>date_strcolumns>
            <algorithm>partbymonthalgorithm>
        rule>
    tableRule>

mycat 配置环境：schema.xml

<mycat:schema xmlns:mycat="http://org.opencloudb/">

    <schema name="TESTDB" checkSQLschema="false" sqlMaxLimit="100">

        <table name="new_table" dataNode="dn1,dn2,dn3" rule="ruleDateStr" />

        <table name="t_3307" dataNode="dn1" ruleRequired="false" primaryKey="id"/>
        <table name="t_5503" dataNode="dn2" ruleRequired="false" primaryKey="id"/>
        <table name="t_5703" dataNode="dn3" ruleRequired="false" primaryKey="id"/>

    schema>

    <dataNode name="dn1" dataHost="h186_3307" database="mysql56" />
    <dataNode name="dn2" dataHost="h186_5503" database="mysql55" />
    <dataNode name="dn3" dataHost="h190_5703" database="mysql57" />

    <dataHost name="h186_3307" maxCon="1000" minCon="10" balance="0"
        writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
        <heartbeat>select user()heartbeat>
        
        <writeHost host="hostM1" url="192.168.13.186:3307" user="root"
            password="root123">
        writeHost>
    dataHost>

    <dataHost name="h186_5503" maxCon="1000" minCon="10" balance="0"
        writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
        <heartbeat>select user()heartbeat>
        
        <writeHost host="hostM2" url="192.168.13.186:5503" user="root"
            password="root123">
        writeHost>
    dataHost>

    <dataHost name="h190_5703" maxCon="1000" minCon="10" balance="0"
        writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
        <heartbeat>select user()heartbeat>
        
        <writeHost host="hostM2" url="192.168.13.190:5703" user="root"
            password="root123">
        writeHost>
    dataHost>

mycat:schema>

mycat 数据批量插入脚本 ， T_3307 表新增脚本
( T_5503 \ T_5703 由于表结构一样)

total=1000000
sql=insert into t_5503 (data_str,time_str) values ('${char([a-f,0-9]8:45)}','201${char([1-9]1:1)}-1${char([1-2]1:1)}-01')

数据批量压入脚本使用命令，使用mycat自带的test工具
JDBC数据库链接 jdbc:mysql://127.0.0.1:8066/TESTDB
用户名与密码 test
批量运行线程数 50
批量插入脚本的目录 “file=。。。。。”

./test_stand_insert_perf.sh jdbc:mysql://127.0.0.1:8066/TESTDB test test  50  "file=/Users/houkai/Desktop/mycat/mycat-testtool/sql/T_TEST_DB_20160331.sql"

OK，我们先来看下第一个场景

数据通过 mycat 批量新增至 mysql 5.5 与 直接批量新增至 mysql 5.5

通过 mycat 压入数据 600万条，用时 1953 sec，直联 mysql 压入数据相同量数据，用时 1067 sec，差不多可以省下近一半时间。

by mycat
31 19:06:10 finished records :5986300 failed:0 speed:3073.331687049272
31 19:06:11 finished records :5988900 failed:0 speed:3073.085652211617
31 19:06:12 finished records :5991600 failed:0 speed:3072.8927328825575
31 19:06:13 finished records :5993400 failed:0 speed:3072.238668255738
31 19:06:14 finished records :5997000 failed:0 speed:3072.5059136900963
31 19:06:15 finished records :5999900 failed:0 speed:3072.416003867212
finishend:6000000 failed:0
used time total:1953seconds
tps:3072.0393221033232

by mysql
31 14:33:02 finished records :5970900 failed:0 speed:5618.597105682396
31 14:33:03 finished records :5975800 failed:0 speed:5617.895136635242
31 14:33:04 finished records :5983600 failed:0 speed:5619.923509831767
31 14:33:05 finished records :5990100 failed:0 speed:5620.74404647405
31 14:33:06 finished records :5996800 failed:0 speed:5621.745264667695
finishend:6000000 failed:0
used time total:1067seconds
tps:5622.71577171774

看一下期间 mycat 的运行情况

mycat服务，java内部占用变化明显（或者用 jconsole 查看）

mysql> show @@server;
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
| UPTIME      | USED_MEMORY | TOTAL_MEMORY | MAX_MEMORY | RELOAD_TIME   | ROLLBACK_TIME | CHARSET | STATUS | AVG_BUFPOOL_ITEM_SIZE |
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
| 1h 24m 39s  |   351360824 |   1073741824 | 3817865216 | 1459401301333 |            -1 | utf8    | ON     |                    54 |
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
1 row in set (0.00 sec)

mysql> show @@server;
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
| UPTIME      | USED_MEMORY | TOTAL_MEMORY | MAX_MEMORY | RELOAD_TIME   | ROLLBACK_TIME | CHARSET | STATUS | AVG_BUFPOOL_ITEM_SIZE |
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
| 1h 24m 41s  |    71015576 |   1073741824 | 3817865216 | 1459401301333 |            -1 | utf8    | ON     |                    54 |
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
1 row in set (0.01 sec)

mysql> show @@server;
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
| UPTIME      | USED_MEMORY | TOTAL_MEMORY | MAX_MEMORY | RELOAD_TIME   | ROLLBACK_TIME | CHARSET | STATUS | AVG_BUFPOOL_ITEM_SIZE |
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
| 1h 24m 43s  |   286897416 |   1073741824 | 3817865216 | 1459401301333 |            -1 | utf8    | ON     |                    54 |
+-------------+-------------+--------------+------------+---------------+---------------+---------+--------+-----------------------+
1 row in set (0.00 sec)

单条数据结点

mysql> show @@datanode;
+------+-------------------+-------+-------+--------+------+------+---------+------------+----------+---------+---------------+
| NAME | DATHOST           | INDEX | TYPE  | ACTIVE | IDLE | SIZE | EXECUTE | TOTAL_TIME | MAX_TIME | MAX_SQL | RECOVERY_TIME |
+------+-------------------+-------+-------+--------+------+------+---------+------------+----------+---------+---------------+
| dn1  | h186_3307/mysql56 |     0 | mysql |      0 |   10 | 1000 |   10551 |          0 |        0 |       0 |            -1 |
| dn2  | h186_5503/mysql55 |     0 | mysql |     51 |    6 | 1000 |   13489 |          0 |        0 |       0 |            -1 |
| dn3  | h190_5703/mysql57 |     0 | mysql |      0 |   10 | 1000 |   10548 |          0 |        0 |       0 |            -1 |
+------+-------------------+-------+-------+--------+------+------+---------+------------+----------+---------+---------------+

mycat 内部处理线程

mysql> show @@processor;
+------------+----------+----------+-------------+---------+---------+-------------+--------------+------------+----------+----------+----------+
| NAME       | NET_IN   | NET_OUT  | REACT_COUNT | R_QUEUE | W_QUEUE | FREE_BUFFER | TOTAL_BUFFER | BU_PERCENT | BU_WARNS | FC_COUNT | BC_COUNT |
+------------+----------+----------+-------------+---------+---------+-------------+--------------+------------+----------+----------+----------+
| Processor0 | 53988970 | 52057208 |           0 |       0 |       0 |        3812 |         4000 |          4 |     1739 |       13 |       17 |
| Processor1 | 54594924 | 55480970 |           0 |       0 |       0 |        3812 |         4000 |          4 |     1739 |       13 |       21 |
| Processor2 | 54491098 | 56830731 |           0 |       0 |       0 |        3812 |         4000 |          4 |     1739 |       13 |       22 |
| Processor3 | 54012009 | 52710263 |           0 |       0 |       0 |        3812 |         4000 |          4 |     1739 |       13 |       20 |
+------------+----------+----------+-------------+---------+---------+-------------+--------------+------------+----------+----------+----------+
4 rows in set (0.00 sec)

mysql> show @@processor;
+------------+----------+----------+-------------+---------+---------+-------------+--------------+------------+----------+----------+----------+
| NAME       | NET_IN   | NET_OUT  | REACT_COUNT | R_QUEUE | W_QUEUE | FREE_BUFFER | TOTAL_BUFFER | BU_PERCENT | BU_WARNS | FC_COUNT | BC_COUNT |
+------------+----------+----------+-------------+---------+---------+-------------+--------------+------------+----------+----------+----------+
| Processor0 | 54041457 | 52103680 |           0 |       0 |       0 |        3812 |         4000 |          4 |     1739 |       13 |       17 |
| Processor1 | 54647052 | 55542713 |           0 |       0 |       0 |        3812 |         4000 |          4 |     1739 |       13 |       21 |
| Processor2 | 54547879 | 56886773 |           0 |       0 |       0 |        3812 |         4000 |          4 |     1739 |       13 |       22 |
| Processor3 | 54063340 | 52759263 |           0 |       0 |       0 |        3812 |         4000 |          4 |     1739 |       13 |       20 |
+------------+----------+----------+-------------+---------+---------+-------------+--------------+------------+----------+----------+----------+
4 rows in set (0.00 sec)

mysql 5.6 环境下的数据压入情况下篇继续~~