TiDB 学习第 7 天：TiDB 性能测试

一、工具介绍

TiDB性能测试使用sysbench工具来进行压测，sysbench是数据库功能及性能测试工具，它的测试功能如下：

• CPU 运算性能测试
• 磁盘 IO 性能测试
• 调度程序性能测试
• 内存分配及传输速度测试
• POSIX 线程性能测试
• 数据库性能测试（OLTP 基准测试，需要通过 /usr/share/sysbench/ 目录中的 Lua 脚本执行，例如 oltp_read_only.lua 脚本执行只读测试）

二、sysbench工具安装及使用

1、 Centos的安装

 sudo yum install -y sysbench

2、查看帮助信息

   # 查看帮助信息
 [tidb@cdh01 ~]$ sudo sysbench --help
 # 查看测试帮助信息
 sudo sysbench fileio help / sudo sysbench cpu help

####　3、CPU测试信息

# 查看CPU帮助信息 sudo sysbench cpu help
[tidb@cdh01 ~]$ sudo sysbench cpu help   
sysbench 1.0.17 (using system LuaJIT 2.0.4)
cpu options:
  --cpu-max-prime=N upper limit for primes generator [10000]

4、CPU测试

sudo sysbench --cpu-max-prime=10000 --threads=2 cpu run

5、磁盘IO性能测试

# 磁盘准备阶段
sysbench --test=fileio --num-threads=16 --file-total-size=30G --file-test-mode=rndrw prepare

# 磁盘IO运行检测
sysbench --test=fileio --num-threads=16 --file-total-size=30G --file-test-mode=rndrw run

Extra file open flags: (none)
128 files, 240MiB each
30GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...

Threads started!


File operations:
    reads/s:                      757.99
    writes/s:                     504.77
    fsyncs/s:                     1811.27

Throughput:
    read, MiB/s:                  11.84    
    written, MiB/s:               7.89

General statistics:
    total time:                          10.1038s
    total number of events:              29017

Latency (ms):
         min:                                    0.00
         avg:                                    5.53
         max:                                  374.29
         95th percentile:                       33.12
         sum:                               160351.92

Threads fairness:    
    events (avg/stddev):           1813.5625/196.79       
    execution time (avg/stddev):   10.0220/0.03

6、线程测试

sysbench --test=threads --num-threads=64 --thread-yields=100 --thread-locks=2 run   
测试写入64线程请求，每个请求产生100个数据量，最后测试执行时间

WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
WARNING: --num-threads is deprecated, use --threads instead
sysbench 1.0.17 (using system LuaJIT 2.0.4)

Running the test with following options:
Number of threads: 64
Initializing random number generator from current time


Initializing worker threads...

Threads started!


General statistics:
    total time:                          10.0057s
    total number of events:              134204

Latency (ms):
         min:                                    0.03
         avg:                                    4.77
         max:                                   74.11
         95th percentile:                       16.71
         sum:                               639949.82

Threads fairness:
    events (avg/stddev):           2096.9375/94.14
    execution time (avg/stddev):   9.9992/0.00                   # 测试最后执行时间为9.9秒

7、内存测试

sysbench --test=memory --memory-block-size=8k --memory-total-size=40G run

WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options.
sysbench 1.0.17 (using system LuaJIT 2.0.4)

Running the test with following options:
Number of threads: 1
Initializing random number generator from current time

Running memory speed test with the following options:
  block size: 8KiB
  total size: 40960MiB
  operation: write
  scope: global
Initializing worker threads...
Threads started!
Total operations: 5242880 (1073492.41 per second)
40960.00 MiB transferred (8386.66 MiB/sec)
General statistics:
    total time:                          4.8814s
    total number of events:              5242880

Latency (ms):
         min:                                    0.00
         avg:                                    0.00
         max:                                    1.30
         95th percentile:                        0.00
         sum:                                 3911.72

Threads fairness:
    events (avg/stddev):           5242880.0000/0.00
    execution time (avg/stddev):   3.9117/0.00         # 测试内存总大小40G   内存缓冲区大小为8k

8、TiDB OLTP基准测试

基准测试分为3个步骤进行：基准测试脚本如下：

1）首先需要先生成测试表

运行命令查看mysql数据库测试参数标准：sudo sysbeach --help
  --mysql-host=[LIST,...]          MySQL server host [localhost]    mysql主机ip
  --mysql-port=[LIST,...]          MySQL server port [3306]         端口/mysql默认端口3306   tidb端口 4000
  --mysql-socket=[LIST,...]        MySQL socket                     mysql安装路径
  --mysql-user=STRING              MySQL user [sbtest]               数据库用户名
  --mysql-password=STRING          MySQL password []                  密码
  --mysql-db=STRING                MySQL database name [sbtest]       需要进行压测的数据库名称
  --mysql-ssl[=on|off]             use SSL connections, if available in the client library [off]
  --mysql-ssl-cipher=STRING        use specific cipher for SSL connections []
  --mysql-compression[=on|off]     use compression, if available in the client library [off]
  --mysql-debug[=on|off]           trace all client library calls [off]
  --mysql-ignore-errors=[LIST,...] list of errors to ignore, or "all" [1213,1020,1205]
  --mysql-dry-run[=on|off]         Dry run, pretend that all MySQL client API calls are successful without executing them [off]

#  运行语句生成测试表  
sudo sysbench --db-driver=mysql --mysql-host=ip --mysql-port=4000 \
				   --mysql-user=root --mysql-password= --mysql-db=BIGDATA \
				   --range_size=100 --table_size=10000 --threads=2 --events=0 --time=60 \
				   --rand-type=uniform /usr/share/sysbench/oltp_read_only.lua prepare

2）运行语句进行测试（表的读性能统计基于表的数据是10000）

运行测试.lua测试语句并指定线程数
 sudo sysbench --db-driver=mysql --mysql-host=ip --mysql-port=4000 \
				   --mysql-user=root --mysql-password= --mysql-db=BIGDATA \
				   --range_size=100 --table_size=10000 --threads=2 --events=0 --time=60 \
				   --rand-type=uniform /usr/share/sysbench/oltp_read_only.lua run

程序运行结果如下：

Threads started!
SQL statistics:
    queries performed:                    #性能统计
        read:                            63854   # 读统计 （select语句）
        write:                           0       # 写统计 （insert、delete、update语句）
        other:                           9122    # 其他语句 （如commit等）
        total:                           72976   # 总执行语句之和
    transactions:                        4561   (76.00 per sec.)   #总事务数（每秒处理的事务）
    queries:                             72976  (1216.08 per sec.)  #查询性能（每秒的查询性能）
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          60.0072s    # 总时长
    total number of events:              4561        # 事务总数
 
Latency (ms):                                   #延迟时间
         min:                                   19.30   # 最小延迟时间
         avg:                                   26.30   # 平均延迟
         max:                                 1238.63   # 最大延迟
         95th percentile:                       33.12   # 95%以上语句响应时间
         sum:                               119963.62   # 总延迟时长

Threads fairness:                      # 线程公平性
    events (avg/stddev):           2280.5000/0.50
    execution time (avg/stddev):   59.9818/0.00

3、清除测试表

# 清除测试表
 sudo sysbench --db-driver=mysql --mysql-host=ip --mysql-port=4000 \
				   --mysql-user=root --mysql-password= --mysql-db=BIGDATA \
				   --range_size=100 --table_size=10000 --threads=2 --events=0 --time=60 \
				   --rand-type=uniform /usr/share/sysbench/oltp_read_only.lua clearnup