TPCC-MYSQL 应用测试

TPCC-MySQL 基于TPCC协议的MySQL 实现

TPCC简介：

TPC-C is an on-line transaction processing benchmark，TPC-C simulates a complete computing environment where a population of users executes transactions against a database. The benchmark is centered around the principal activities (transactions) of an order-entry environment. These transactions include entering and delivering orders, recording payments, checking the status of orders, and monitoring the level of stock at the warehouses. While the benchmark portrays the activity of a wholesale supplier, TPC-C is not limited to the activity of any particular business segment, but, rather represents any industry that must manage, sell, or distribute a product or service.

TPC-C involves a mix of five concurrent transactions of different types and complexity either executed on-line or queued for deferred execution. The database is comprised of nine types of tables with a wide range of record and population sizes. TPC-C is measured in transactions per minute (tpmC).

结构如下：

结果值tpmC是代表每分钟事务处理的数量，一般仓库设置为40-100个，为cpu bound,400-1000个是为了测试io bound，40以下无论事务多少，锁竞争情况也不太容易发生。

本地环境：

CPU：Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz

Mem:32G

SATA HDD：1TB

SSD:INTEL SSDSC2BP48

（这里我们分别测试MySQL Server运行在SATA硬盘和SSD上两者的差距）

1、安装：

   bzr branch lp:~percona-dev/perconatools/tpcc-mysql （需首先安装bzr)

   接下来就是编译, cd tpcc-mysql 直接make 后就OK了，会在当前目录下生成tpcc_load、tpcc_start 两个运行程序，主要也就是用这两个程序

2、使用方法简介：

1. Build binaries

   * cd scr ; make

   ( you should have mysql_config available in $PATH)



2. Load data

   * create database

     mysqladmin create tpcc

   * create tables

     mysql tpcc < create_table.sql

   * create indexes and FK ( this step can be done after loading data)

     mysql tpcc < add_fkey_idx.sql

   * populate data

     - simple step

       tpcc_load 127.0.0.1:33000 tpcc1000 root "" 1000

                 |hostname:port| |dbname| |user| |password| |WAREHOUSES|

       ref. tpcc_load --help for all options

     - load data in parallel 

       check load.sh script



3. start benchmark

   * ./tpcc_start -h127.0.0.1 -P33000 -dtpcc1000 -uroot -w1000 -c32 -r10 -l10800

                  |hostname| |port| |dbname| |user| |WAREHOUSES| |CONNECTIONS| |WARMUP TIME| |BENCHMARK TIME|

   * ref. tpcc_start --help for all options 

 

接下来，生成测试数据：

#./tpcc_load 127.0.0.1 tpcc root "XXXX" 100 （测试时视仓库数量而定，运行时间会比较久）

进行测试：

tpcc_start Usage:

Usage: tpcc_start -h server_host -P port -d database_name -u mysql_user -p mysql_password -w warehouses -c connections -r warmup_time -l running_time -i report_interval -f report_file -t trx_file

-w 仓库数量
-c 并发连接数
-r 测试前warmup的时间，测试前预热时间
-l 测试持续时间
-I 生成报告间隔时长
-f 生成的报告文件名

运行命令如下：

#./tpcc_start -h127.0.0.1 -dtpcc -uroot -pXXXX -w100 -c32 -r10 -l10800 

 

运行结果：

[root@github tpcc-mysql]# more benchmark.log

*** ###easy### TPC-C Load Generator ***
***************************************
option h with value '127.0.0.1'
option d with value 'tpcc'
option u with value 'root'
option w with value '100'
option c with value '32'
option r with value '10'
option l with value '10800'
<Parameters>
[server]: 127.0.0.1
[port]: 3306
[DBname]: tpcc
[user]: root
[pass]:
[warehouse]: 100
[connection]: 32
[rampup]: 10 (sec.)
[measure]: 10800 (sec.)

RAMP-UP TIME.(10 sec.)

MEASURING START.

10, 668(0):1.310|1.377, 657(0):0.280|0.560, 67(0):0.143|0.154, 64(0):1.663|1.688, 67(0):3.992|4.120
20, 916(0):1.232|1.271, 914(0):0.258|0.423, 91(0):0.149|0.911, 91(0):1.552|1.571, 91(0):3.930|3.940
30, 917(0):1.235|1.431, 918(0):0.255|0.275, 92(0):0.132|0.133, 92(0):1.521|1.551, 92(0):4.016|4.265
40, 965(0):1.232|1.287, 961(0):0.256|0.287, 97(0):0.132|0.133, 96(0):1.554|1.573, 96(0):4.004|4.071
50, 982(0):1.228|1.296, 981(0):0.257|0.310, 98(0):0.130|0.150, 98(0):1.502|1.503, 99(0):3.910|3.960
60, 977(0):1.229|1.302, 974(0):0.255|0.292, 98(0):0.135|0.136, 98(0):1.551|1.612, 97(0):3.987|4.224
70, 985(0):1.229|1.269, 988(0):0.261|0.372, 98(0):0.130|0.134, 99(0):1.524|1.563, 99(0):4.000|4.086
80, 939(0):1.243|1.275, 937(0):0.261|0.301, 94(0):0.139|0.146, 94(0):1.631|1.638, 95(0):4.000|4.128
90, 978(0):1.249|1.269, 979(0):0.256|0.339, 98(0):0.132|0.136, 98(0):1.728|1.750, 97(0):3.919|4.024
100, 916(0):1.211|1.300, 918(0):0.254|0.286, 92(0):0.137|0.146, 91(0):1.528|1.624, 91(0):3.963|4.005
110, 895(0):1.211|1.274, 893(0):0.256|0.306, 89(0):0.133|0.139, 90(0):1.542|1.545, 90(0):3.936|3.944
120, 953(0):1.216|1.284, 951(0):0.254|0.271, 95(0):0.132|0.136, 95(0):1.510|1.518, 95(0):3.837|3.852
130, 913(0):1.217|1.280, 907(0):0.256|0.263, 92(0):0.133|0.136, 91(0):1.506|1.525, 91(0):3.885|4.105
140, 913(0):1.226|1.277, 911(0):0.257|0.333, 91(0):0.131|0.133, 91(0):1.531|1.558, 92(0):3.922|3.940
150, 924(0):1.228|1.277, 926(0):0.256|0.296, 93(0):0.145|0.147, 93(0):1.544|1.591, 93(0):3.922|3.987
160, 924(0):1.221|1.328, 927(0):0.256|0.271, 93(0):0.132|0.141, 93(0):1.558|1.584, 93(0):3.924|3.995
170, 924(0):1.212|1.271, 921(0):0.255|0.267, 92(0):0.143|0.161, 92(0):1.524|1.536, 92(0):3.794|3.864
180, 877(0):1.216|1.273, 871(0):0.254|0.329, 88(0):0.131|0.132, 88(0):1.536|1.545, 87(0):3.866|3.938
190, 899(0):1.223|1.261, 902(0):0.254|0.330, 89(0):0.133|0.136, 90(0):1.519|1.523, 91(0):3.911|3.943
200, 892(0):1.227|1.294, 892(0):0.252|0.278, 90(0):0.134|0.137, 90(0):1.544|1.608, 89(0):3.923|4.061
210, 901(0):1.216|1.311, 899(0):0.257|0.311, 90(0):0.133|0.138, 90(0):1.539|1.553, 90(0):3.861|3.993
220, 883(0):1.220|1.265, 882(0):0.255|0.267, 88(0):0.132|0.138, 88(0):1.546|1.565, 89(0):3.869|3.923
230, 890(0):1.217|1.283, 892(0):0.253|0.284, 90(0):0.129|0.135, 89(0):1.514|1.538, 89(0):3.775|3.994
240, 928(0):1.232|1.284, 926(0):0.257|0.493, 92(0):0.133|0.181, 93(0):1.620|1.666, 92(0):3.893|3.968
250, 950(0):1.227|1.271, 949(0):0.251|0.268, 96(0):0.139|0.169, 95(0):1.507|1.517, 95(0):3.903|3.935
260, 874(0):1.221|1.287, 875(0):0.255|0.283, 87(0):0.134|0.140, 87(0):1.532|1.544, 87(0):3.818|3.932
270, 908(0):1.224|1.287, 906(0):0.263|0.402, 91(0):0.133|0.136, 91(0):1.563|1.598, 91(0):3.903|4.179
280, 929(0):1.235|1.275, 926(0):0.267|0.352, 93(0):0.131|0.191, 93(0):1.657|1.761, 93(0):3.954|3.996
290, 894(0):1.232|1.278, 893(0):0.257|0.420, 89(0):0.131|0.189, 90(0):1.603|1.612, 90(0):3.912|3.954
300, 907(0):1.229|1.272, 908(0):0.250|0.270, 90(0):0.128|0.129, 90(0):1.542|1.581, 90(0):3.779|3.911
310, 848(0):1.241|1.291, 850(0):0.258|0.416, 85(0):0.141|0.145, 85(0):1.574|1.649, 86(0):3.865|3.930
320, 842(0):1.230|1.283, 834(0):0.254|0.317, 85(0):0.135|0.158, 84(0):1.531|1.541, 83(0):3.752|3.786
330, 893(0):1.227|1.253, 897(0):0.255|0.281, 89(0):0.131|0.136, 89(0):1.532|1.534, 89(0):3.772|3.802
340, 922(0):1.215|1.275, 921(0):0.252|0.296, 93(0):0.132|0.143, 92(0):1.542|1.581, 94(0):3.749|3.764
350, 905(0):1.217|1.266, 909(0):0.252|0.312, 90(0):0.134|0.138, 91(0):1.523|1.544, 90(0):3.915|3.951

..................................................................

STOPPING THREADS................................

<Raw Results>
[0] sc:869195 lt:0 rt:0 fl:0
[1] sc:868118 lt:0 rt:1 fl:0
[2] sc:86976 lt:0 rt:0 fl:0
[3] sc:86969 lt:0 rt:14 fl:0
[4] sc:86975 lt:0 rt:0 fl:0
in 10800 sec.

<Raw Results2(sum ver.)>
[0] sc:869770 lt:0 rt:0 fl:0
[1] sc:869760 lt:0 rt:1 fl:0
[2] sc:86977 lt:0 rt:0 fl:0
[3] sc:86973 lt:0 rt:14 fl:0
[4] sc:86977 lt:0 rt:0 fl:0

<Constraint Check> (all must be [OK])
[transaction percentage]
Payment: 43.44% (>=43.0%) [OK]
Order-Status: 4.35% (>= 4.0%) [OK]
Delivery: 4.35% (>= 4.0%) [OK]
Stock-Level: 4.35% (>= 4.0%) [OK]
[response time (at least 90% passed)]
New-Order: 100.00% [OK]
Payment: 100.00% [OK]
Order-Status: 100.00% [OK]
Delivery: 100.00% [OK]
Stock-Level: 100.00% [OK]

<TpmC>
4828.861 TpmC

以上方法测试的为100个仓库，SATA硬盘的测试结果。

接下来，同样的方法测试SSD硬盘结果，同样创建100个仓库，作为测试：

1、先生成测试数据：./tpcc_load 127.0.0.1 tpcc root '' 100

2、进行测试：./tpcc_start -h127.0.0.1 -dtpcc -uroot -w100 -c32 -r10 -l10800

3、结果如下：

*** ###easy### TPC-C Load Generator ***
***************************************
option h with value '127.0.0.1'
option d with value 'tpcc'
option u with value 'root'
option w with value '100'
option c with value '32'
option r with value '10'
option l with value '10800'
<Parameters>
[server]: 127.0.0.1
[port]: 3306
[DBname]: tpcc
[user]: root
[pass]:
[warehouse]: 100
[connection]: 32
[rampup]: 10 (sec.)
[measure]: 10800 (sec.)

RAMP-UP TIME.(10 sec.)

MEASURING START.

10, 9000(0):1.266|1.390, 8993(0):0.247|0.319, 901(0):0.132|0.184, 902(0):1.460|1.530, 898(0):4.152|4.511
20, 9030(0):1.258|1.373, 9021(0):0.245|1.056, 903(0):0.131|0.188, 901(0):1.437|1.499, 903(0):4.085|4.216
30, 9404(0):1.255|1.311, 9396(0):0.245|0.285, 941(0):0.129|0.147, 943(0):1.444|2.130, 943(0):4.138|4.318
40, 9557(0):1.254|1.329, 9570(0):0.243|0.325, 957(0):0.131|0.150, 957(0):1.433|1.455, 955(0):4.180|4.380
50, 9487(0):1.257|1.316, 9476(0):0.243|0.271, 949(0):0.127|0.146, 948(0):1.435|1.484, 950(0):4.219|4.399
60, 9360(0):1.254|1.378, 9357(0):0.244|0.327, 936(0):0.129|0.156, 936(0):1.426|1.501, 937(0):4.066|4.309
70, 9491(0):1.256|1.326, 9486(0):0.243|0.270, 949(0):0.128|0.135, 949(0):1.429|1.472, 949(0):4.118|4.343
80, 9466(0):1.255|1.435, 9470(0):0.243|0.280, 948(0):0.129|0.155, 948(0):1.427|1.463, 946(0):4.111|4.215
90, 9525(0):1.259|1.309, 9532(0):0.242|0.262, 953(0):0.128|0.144, 954(0):1.428|1.467, 955(0):4.154|4.330
100, 9667(0):1.255|1.302, 9662(0):0.242|0.293, 968(0):0.126|0.132, 967(0):1.433|1.489, 967(0):4.147|4.339
110, 9534(0):1.257|1.316, 9534(0):0.243|0.270, 954(0):0.128|0.145, 955(0):1.437|1.526, 954(0):4.167|4.368
120, 9461(0):1.253|1.299, 9460(0):0.242|0.264, 947(0):0.126|0.143, 945(0):1.433|1.463, 947(0):4.137|4.432
130, 9458(0):1.257|1.312, 9465(0):0.243|0.281, 946(0):0.129|0.151, 947(0):1.433|1.483, 947(0):4.126|4.277
140, 9504(0):1.256|1.341, 9490(0):0.243|0.337, 951(0):0.127|0.136, 950(0):1.429|1.507, 949(0):4.188|4.626
150, 9508(0):1.255|1.307, 9501(0):0.242|0.273, 951(0):0.128|0.141, 951(0):1.434|1.474, 954(0):4.219|4.434
160, 6477(0):1.254|1.364, 6447(0):0.246|0.377, 647(0):0.130|0.153, 655(0):1.451|1.527, 648(0):4.113|4.302
170, 809(0):1.127|1.187, 805(0):0.329|0.390, 81(0):0.179|0.180, 82(0):1.968|2.065, 81(0):3.540|3.623
180, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000
190, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000
200, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000, 0(0):0.000|0.000
210, 1798(0):1.264|1.371, 1827(0):0.250|0.315, 181(0):0.139|0.162, 173(0):1.456|1.523, 180(0):4.089|4.109
220, 7057(0):1.253|1.336, 7063(0):0.248|0.319, 707(0):0.127|0.143, 708(0):1.461|1.590, 705(0):4.068|4.174
230, 9450(0):1.254|1.333, 9442(0):0.242|0.332, 945(0):0.127|0.137, 943(0):1.435|1.457, 947(0):4.128|4.304
240, 9359(0):1.257|1.314, 9342(0):0.243|0.274, 937(0):0.128|0.144, 938(0):1.432|1.479, 936(0):4.125|4.325
250, 9292(0):1.253|1.294, 9279(0):0.242|0.292, 929(0):0.129|0.144, 929(0):1.436|1.498, 930(0):4.168|4.402
260, 9527(0):1.255|1.337, 9528(0):0.242|0.294, 953(0):0.130|0.147, 953(0):1.428|1.449, 953(0):4.166|4.580
270, 9531(0):1.254|1.314, 9532(0):0.241|0.288, 954(0):0.126|0.130, 954(0):1.429|1.549, 954(0):4.130|4.341
280, 9619(0):1.252|1.296, 9612(0):0.242|0.300, 963(0):0.127|0.137, 962(0):1.431|1.453, 962(0):4.137|4.276
290, 9651(0):1.256|1.316, 9657(0):0.242|0.270, 967(0):0.129|0.159, 966(0):1.425|1.454, 967(0):4.175|4.408
300, 9531(0):1.254|1.345, 9513(0):0.242|0.274, 952(0):0.128|0.147, 953(0):1.426|1.476, 952(0):4.163|4.471
310, 9584(0):1.254|1.323, 9585(0):0.242|0.272, 958(0):0.128|0.140, 958(0):1.426|1.445, 960(0):4.227|4.344
320, 9621(0):1.255|1.298, 9622(0):0.242|0.285, 963(0):0.127|0.143, 964(0):1.434|1.457, 961(0):4.110|4.362
330, 9569(0):1.257|1.342, 9568(0):0.242|0.282, 958(0):0.130|0.148, 959(0):1.432|1.469, 956(0):4.111|4.409
340, 9594(0):1.256|1.321, 9584(0):0.242|0.269, 959(0):0.129|0.144, 957(0):1.429|1.473, 962(0):4.143|4.370
350, 9664(0):1.257|1.320, 9670(0):0.242|0.264, 968(0):0.127|0.143, 966(0):1.433|1.454, 968(0):4.183|4.395
360, 9621(0):1.257|1.311, 9612(0):0.241|0.284, 962(0):0.128|0.138, 963(0):1.427|1.444, 962(0):4.181|4.411

。。。。。。。。。。。。。。。。。。。

<Raw Results>
[0] sc:9076386 lt:0 rt:0 fl:0
[1] sc:9072471 lt:0 rt:0 fl:0
[2] sc:908318 lt:0 rt:0 fl:0
[3] sc:908225 lt:0 rt:0 fl:0
[4] sc:908292 lt:0 rt:0 fl:0
in 10800 sec.

<Raw Results2(sum ver.)>
[0] sc:9083287 lt:0 rt:0 fl:0
[1] sc:9083288 lt:0 rt:0 fl:0
[2] sc:908329 lt:0 rt:0 fl:0
[3] sc:908328 lt:0 rt:0 fl:0
[4] sc:908327 lt:0 rt:0 fl:0

<Constraint Check> (all must be [OK])
[transaction percentage]
Payment: 43.46% (>=43.0%) [OK]
Order-Status: 4.35% (>= 4.0%) [OK]
Delivery: 4.35% (>= 4.0%) [OK]
Stock-Level: 4.35% (>= 4.0%) [OK]
[response time (at least 90% passed)]
New-Order: 100.00% [OK]
Payment: 100.00% [OK]
Order-Status: 100.00% [OK]
Delivery: 100.00% [OK]
Stock-Level: 100.00% [OK]

<TpmC>
50424.367 TpmC

 

可见，同样的硬件配置，使用TPCC 100个仓库进行测试，MySQL 运行在SSD硬盘上的事物处理能力是SATA硬盘的10倍左右，当创建更多个仓库时，估计SSD的处理能力会更优，待后续测试。

同样的硬件配置(由于使用原因，机器内存被拔走16G，现剩16G)，当TPCC初始化500个仓库时，测试结果对比如下：

SATA <TpmC>
325.811 TpmC

SSD <Tpmc>

11829.700 TpmC

可见：SSD在仓库增加为500时，其事务处理能力为SATA的36倍左右。