Oracle-RAC集群网络，IO性能测试
前言:
Oracle RAC集群对于网络，IO性能有一定的要求，特别是心跳网络，需要千兆以上，Oracle官方最佳实践建议配置万兆网络，而IO性能则是直接影响数据库的用户操作执行，因此，在进行RAC集群的部署时，我们需要对集群网络，IO性能进行测试，以确保性能指标满足集群运行要求。
本文接下来的内容将讲述Oracle RAC集群网络，IO性能常用的测试工具以及测试方法，希望对大家有所帮助。
测试工具:
网络:netpref
IO:orion,DBMS_RESOURCE_MANAGER.CALIBRATE_IO,fio
网络性能压测步骤(netpref)
安装编译netpref工具
tar -zxvf netperf-2.7.0.tar.gz
cd netperf-netperf-2.7.0/
./configure
make
make install验证安装成功
[root@rac1 ~]# netperf -h
Usage: netperf [global options] -- [test options]
Global options:
-a send,recv Set the local send,recv buffer alignment
-A send,recv Set the remote send,recv buffer alignment
-B brandstr Specify a string to be emitted with brief output
-c [cpu_rate] Report local CPU usage
-C [cpu_rate] Report remote CPU usage
-d Increase debugging output
-D time,[units] * Display interim results at least every time interval
using units as the initial guess for units per second
A negative value for time will make heavy use of the
system's timestamping functionality
-f G|M|K|g|m|k Set the output units
-F lfill[,rfill]* Pre-fill buffers with data from specified file
-h Display this text
-H name|ip,fam * Specify the target machine and/or local ip and family
-i max,min Specify the max and min number of iterations (15,1)
-I lvl[,intvl] Specify confidence level (95 or 99) (99)
and confidence interval in percentage (10)
-j Keep additional timing statistics
-l testlen Specify test duration (>0 secs) (<0 bytes|trans)
-L name|ip,fam * Specify the local ip|name and address family
-o send,recv Set the local send,recv buffer offsets
-O send,recv Set the remote send,recv buffer offset
-n numcpu Set the number of processors for CPU util
-N Establish no control connection, do 'send' side only
-p port,lport* Specify netserver port number and/or local port
-P 0|1 Don't/Do display test headers
-r Allow confidence to be hit on result only
-s seconds Wait seconds between test setup and test start
-S Set SO_KEEPALIVE on the data connection
-t testname Specify test to perform
-T lcpu,rcpu Request netperf/netserver be bound to local/remote cpu
-v verbosity Specify the verbosity level
-W send,recv Set the number of send,recv buffers
-v level Set the verbosity level (default 1, min 0)
-V Display the netperf version and exit
-y local,remote Set the socket priority
-Y local,remote Set the IP_TOS. Use hexadecimal.
-Z passphrase Set and pass to netserver a passphrase
For those options taking two parms, at least one must be specified;
specifying one value without a comma will set both parms to that
value, specifying a value with a leading comma will set just the second
parm, a value with a trailing comma will set just the first. To set
each parm to unique values, specify both and separate them with a
comma.
* For these options taking two parms, specifying one value with no comma
will only set the first parms and will leave the second at the default
value. To set the second value it must be preceded with a comma or be a
comma-separated pair. This is to retain previous netperf behaviour.进行网络性能压测，netpref通过模拟服务端与客户端的TCP/UDP包网络发送，来测试主机之间的网络性能，这里测试的RAC为两节点的集群，所以节点一与节点二将分别作为服务段以及客户端进行测试
节点一RAC1作为服务端，启动netserver
[root@rac1 tmp]# netserver
Starting netserver with host 'IN(6)ADDR_ANY' port '12865' and family AF_UNSPEC
#关闭netserver,可以通过kill方式节点二RAC2作为客户端，发送TCP/UDP进行测试
进行TCP协议测试
参数说明
#TCP_RR 在一个TCP连接中进行多次TCP request和response的交易过程，这种模式常常出现在数据库应用中
#TCP_STREAM: TCP批量传输，在测试过程中，netperf向netserver发送批量的TCP数据分组，以确定数据传输过程中的吞吐量
#-H 主机名IP
#-l 时长
[root@rac2 netperf-netperf-2.7.0]# netperf -t TCP_RR -H rac1-priv -l 60
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to rac1-priv () port 0 AF_INET : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec
16384 87380 1 1 60.00 1721.92
16384 87380
#Trans Rate per sec 每秒的交易1721/s
[root@rac2 netperf-netperf-2.7.0]# netperf -t TCP_STREAM -H rac1-priv -l 60
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to rac1-priv () port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 16384 16384 60.00 2808.18
#Throughput 10^6bits/sec 网络吞吐量每秒2808Mbps/s进行UDP协议测试
参数说明
#UDP_RR 进行多次UDP request和response的交易过程，这种模式常常出现在数据库应用中
#UDP_STREAM: UDP批量传输，在测试过程中，netperf向netserver发送批量的UDP数据分组，以确定数据传输过程中的吞吐量
#-H 主机名IP
#-l 时长
[root@rac2 netperf-netperf-2.7.0]# netperf -t UDP_RR -H rac1-priv -l 60
MIGRATED UDP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to rac1-priv () port 0 AF_INET : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec
262144 262144 1 1 60.00 1727.69
262144 262144
#Trans Rate per sec 每秒的交易1727/s
[root@rac2 netperf-netperf-2.7.0]# netperf -t UDP_STREAM -H rac1-priv -l 60
MIGRATED UDP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to rac1-priv () port 0 AF_INET
Socket Message Elapsed Messages
Size Size Time Okay Errors Throughput
bytes bytes secs # # 10^6bits/sec
262144 65507 60.00 44194 0 386.00
262144 60.00 44188 385.95
#Throughput 10^6bits/sec 网络吞吐量每秒385Mbps/s总结:通过netpref的测试，我们可以看到TCP_RR的指标为每秒1721/s,TCP_STREAM每秒的吞吐量为2808Mbps/s,UDP_RR的指标为每秒1727/s,UDP_STREAM每秒的吞吐量为385Mbps/s,这大概就是千兆网络的性能
IO性能压测步骤(orion)
ORION (Oracle I/O Calibration Tool) 是校准用于 Oracle 数据库的存储系统 I/O 性能的独立工具。校准结果对于了解存储系统的性能有很大帮助，不仅可以找出影响 Oracle 数据库性能的问题，还能测量新数据库安装的大小。由于 ORION 是一个Oracle软件自带的独立工具，用户不需要创建和运行 Oracle 数据库
查看工具orion
[root@rac1 tmp]# /u01/app/oracle/product/11.2.0/db_1/bin/orion -help
ORION: ORacle IO Numbers -- Version 11.2.0.4.0
ORION runs IO performance tests that model Oracle RDBMS IO workloads.
It measures the performance of small (2-32K) IOs and large (128K+) IOs
at various load levels.
Each Orion data point is a test for a specific mix of small and large
IO loads sustained for a duration. An Orion test consists of multiple
data point tests. These data point tests can be represented as a
two-dimensional matrix. Each column in the matrix represents data
point tests with the same small IO load, but varying large IO loads.
Each row represents data point tests with the same large IO load, but
varying small IO loads. An Orion test can be for a single point, a
single row, a single column, or the whole matrix.
The 'run' parameter is the only mandatory parameter. Defaults
are indicated for all other parameters. For additional information on
the user interface, see the Orion User Guide.
is the prefix used for all input and output filenames. By
default, it is 'orion'. It can be specified with the 'testname'
parameter.
.lun should contain a carriage-return-separated list of LUNs.
The output files for a test run are prefixed by _ where
date is "yyyymmdd_hhmm".
The output files are:
__summary.txt - Summary of the input parameters, along with
the minimum small IO latency (in usecs), the maximum
MBPS, and the maximum IOPS observed.
__mbps.csv - Performance results of large IOs in MBPS.
__iops.csv - Performance results of small IOs in IOPS.
__lat.csv - Latency of small IOs in microseconds.
__hist.csv - Histogram of IO latencies.
__trace.txt - Extended, unprocessed output.
WARNING: IF YOU ARE PERFORMING WRITE TESTS, BE PREPARED TO LOSE ANY DATA STORED
ON THE LUNS.
Mandatory parameters:
run Type of workload to run (simple, normal, advanced, dss, oltp).
simple - Tests random small (8K) IOs at various loads,
then random large (1M) IOs at various loads.
normal - Tests combinations of random small (8K) IOs and
random large (1M) IOs.
advanced - Tests the workload specified by the user
using optional parameters.
dss - Tests with random large (1M) IOs at increasing
loads to determine the maximum throughput.
oltp - Tests with random small (8K) IOs at increasing
loads to determine the maximum IOPS.
Optional parameters:
testname Name of the test run.
num_disks Number of disks (physical spindles). This number is
used to gauge the range of loads that Orion should test
at. Increasing this parameter results in Orion using
heavier IO loads. Default is the number of LUNs in
.lun.
size_small Size of small IOs in KB. Default is 8.
size_large Size of large IOs in KB. Default is 1024.
type Type of large IOs (rand, seq). Default is rand.
rand - Randomly distributed large IOs.
seq - Sequential streams of large IOs.
num_streamIO Number of concurrent IOs per stream. This parameter is only
used if -type is seq. Default is 4.
simulate Orion tests on a virtual LUN formed by combining the
specified LUNs in one of these ways. This parameter is
typically only used if -type is seq. Default is concat.
concat - A serial concatenation of the LUNs. Each
sequential stream issues IOs to only one LUN.
raid0 - A RAID-0 mapping across the LUNs. Each
sequential stream issues IOs across all LUNs,
using RAID-0 striping.
write Percentage of IOs that are writes (SEE WARNING ABOVE).
Default is 0.
cache_size Size in MBs of the array's cache.
Unless this option is set to 0, Orion issues a number
of unmeasured, random IOs before each large sequential
data point. These IOs fill up the storage array's cache
(if any) with random data so that IOs from one
data point do not result in cache hits for the next
data point. Read tests are preceded with junk reads
and write tests are preceded with junk writes. If
specified, this 'cache warming' is performed until
cache_size MBs of IO have been read or written.
Default behavior is to issue 2 minutes of unmeasured random
IOs before each data point.
duration Duration of each data point in seconds. Default is 60.
num_small Number of outstanding small IOs. This parameter controls the
small IO load. Only used if -matrix is point, col, or max.
No default.
num_large This parameter controls the large IO load.
For -type rand, number of outstanding large IOs.
For -type seq, number of sequential IO streams. Only used
if -matrix is point, row, or max. No default.
matrix An Orion test consists of multiple data point tests. These data
point tests can be represented as a two-dimensional matrix.
Each column in the matrix represents data point tests with
the same small IO load, but varying large IO loads. Each
row represents data point tests with the same large IO load,
but varying small IO loads. An Orion test can be for a
single point, a single row, a single column, or the whole
matrix, depending on the matrix option setting below.
Default is basic.
basic - Test small IOs only, then large IOs only.
detailed - Test entire matrix.
point - Test with num_small small IOs, num_large large
IOs.
col - Test a varying large IO load with num_small
small IOs.
row - Test a varying small IO load with num_large
large IOs.
max - Test varying loads up to the num_small and
num_large limits.
hugenotneeded Tells Orion to allocate regular pages when huge
pages are not available. With out this option, on systems
that support hugepages, Orion errors out when it cannot
allocate sufficient huge pages. -hugenotneeded is not set by
default.
verbose Prints tracing information to standard output if set.
Not set by default.
datainput Name of the input file for data to be written.
is_lgwr if set to true will run test using VIP level QOS, assuming the underlying transport supports it.
Examples:
For a preliminary set of data
-run simple
For a basic set of data
-run normal
To evaluate storage for an OLTP database
-run oltp
To evaluate storage for a data warehouse
-run dss
To generate combinations of 32KB and 1MB reads to random locations
-run advanced
-size_small 32 -size_large 1024 -type rand
-matrix detailed
To generate multiple sequential 1MB write streams, simulating RAID0 striping
-run advanced
-simulate RAID0 -write 100 -type seq
-matrix col -num_small 0
配置lun测试文件
#测试磁盘/dev/mapper/test1,/dev/mapper/test2,/dev/mapper/test3
[root@rac1 tmp]# cat tlun.lun
/dev/mapper/test1
/dev/mapper/test2
/dev/mapper/test3开始进行IO性能测试
#使用root用户
export ORACLE_HOME=/u01/app/oracle/product/11.2.0/db_1
export PATH=$ORACLE_HOME/bin:$PATH:/usr/local/bin
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/rdbms/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib
#测试参数
#模拟数据库随机小事务8K，读写占比7:3
#每个节点开启2个压测，共四个压测
/u01/app/oracle/product/11.2.0/db_1/bin/orion -run advanced -testname tlun -num_disks 3 -size_small 8 -size_large 8 -type rand -write 30 -hugenotneeded
[root@rac1 tmp]# /u01/app/oracle/product/11.2.0/db_1/bin/orion -run advanced -testname tlun -num_disks 3 -size_small 8 -size_large 8 -type rand -write 30 -hugenotneeded
ORION: ORacle IO Numbers -- Version 11.2.0.4.0
tlun_20220904_1717
Calibration will take approximately 23 minutes.
Using a large value for -cache_size may take longer.
Maximum Large MBPS=74.07 @ Small=0 and Large=6
Maximum Small IOPS=9583 @ Small=6 and Large=0
Small Read Latency: avg=623 us, min=85 us, max=10795 us, std dev=257 us @ Small=6 and Large=0
Small Write Latency: avg=627 us, min=94 us, max=10431 us, std dev=243 us @ Small=6 and Large=0
Minimum Small Latency=208 usecs @ Small=1 and Large=0
Small Read Latency: avg=232 us, min=78 us, max=6918 us, std dev=138 us @ Small=1 and Large=0
Small Write Latency: avg=151 us, min=88 us, max=4942 us, std dev=109 us @ Small=1 and Large=0
Small Read / Write Latency Histogram @ Small=1 and Large=0
Latency: # of IOs (read) # of IOs (write)
0 - 1 us: 0 0
2 - 4 us: 0 0
4 - 8 us: 0 0
8 - 16 us: 0 0
16 - 32 us: 0 0
32 - 64 us: 0 0
64 - 128 us: 42901 42021
128 - 256 us: 98451 38491
256 - 512 us: 57190 4334
512 - 1024 us: 2413 549
1024 - 2048 us: 522 173
2048 - 4096 us: 126 37
4096 - 8192 us: 15 3
8192 - 16384 us: 0 0
16384 - 32768 us: 0 0
32768 - 65536 us: 0 0
65536 - 131072 us: 0 0
131072 - 262144 us: 0 0
262144 - 524288 us: 0 0
524288 - 1048576 us: 0 0
1048576 - 2097152 us: 0 0
2097152 - 4194304 us: 0 0
4194304 - 8388608 us: 0 0
8388608 - 16777216 us: 0 0
16777216 - 33554432 us: 0 0
33554432 - 67108864 us: 0 0
67108864 - 134217728 us: 0 0
134217728 - 268435456 us: 0 0
---测试结果
Maximum Large MBPS=74.07
Maximum Small IOPS=9583会在当前目录下生成测试的mbps,iops,latency的数据
[root@rac1 tmp]# ls -lrt tlun*
-rw-r--r-- 1 root root 54 Sep 4 17:13 tlun.lun
-rw-r--r-- 1 root root 20240 Sep 4 17:38 tlun_20220904_1717_trace.txt
-rw-r--r-- 1 root root 658 Sep 4 17:38 tlun_20220904_1717_mbps.csv
-rw-r--r-- 1 root root 758 Sep 4 17:38 tlun_20220904_1717_lat.csv
-rw-r--r-- 1 root root 734 Sep 4 17:38 tlun_20220904_1717_iops.csv
-rw-r--r-- 1 root root 20587 Sep 4 17:38 tlun_20220904_1717_hist.txt
-rw-r--r-- 1 root root 2151 Sep 4 17:38 tlun_20220904_1717_summary.txt总结:测试的结果为最大IOPS:9583,最大吞吐量74Mbps/s
IO性能压测步骤(DBMS_RESOURCE_MANAGER.CALIBRATE_IO)
DBMS_RESOURCE_MANAGER.CALIBRATE_IO为Oracle 11g数据库中的PL/SQL存储过程，可以用于对当前数据库实例的I/O性能进行测试，使用简单
CALIBRATE_IO参数：
进行IO性能压测
set timing on
set time on
SET SERVEROUTPUT ON
DECLARE
lat INTEGER;
iops INTEGER;
mbps INTEGER;
BEGIN
DBMS_RESOURCE_MANAGER.CALIBRATE_IO (num_physical_disks => 2,
max_iops => iops,
max_mbps => mbps,
actual_latency => lat
);
DBMS_OUTPUT.PUT_LINE ('max_iops = ' || iops);
DBMS_OUTPUT.PUT_LINE ('latency = ' || lat);
DBMS_OUTPUT.PUT_LINE ('max_mbps = ' || mbps);
end;
/查看IO性能压测结果
SELECT TO_CHAR(start_time, 'DD-MON-YYY HH24:MI:SS') AS start_time,
TO_CHAR(end_time, 'DD-MON-YYY HH24:MI:SS') AS end_time,
max_iops,
max_mbps,
max_pmbps,
latency,
num_physical_disks AS disks
FROM dba_rsrc_io_calibrate;18:03:16 SQL> 18:03:16 SQL> 18:03:16 2 18:03:16 3 18:03:16 4 18:03:16 5 18:03:16 6 18:03:16 7 18:03:16 8
START_TIME END_TIME MAX_IOPS MAX_MBPS MAX_PMBPS LATENCY DISKS
---------------------- ---------------------- ---------- ---------- ---------- ---------- ----------
04-SEP-022 17:53:24 04-SEP-022 18:02:55 13070 1200 690 11 2总结:测试的结果为最大IOPS:13070,最大吞吐量1200Mbps/s
IO性能压测步骤(FIO)
fio软件安装
tar xvf fio-2.17.tar.gz
cd fio-3.13
./configure
make
make install 确认安装成功
[root@rac1 fio-2.17]# fio --help
fio-2.17
fio [options] [job options]
--debug=options Enable debug logging. May be one/more of:
process,file,io,mem,blktrace,verify,random,parse,
diskutil,job,mutex,profile,time,net,rate,compress,
steadystate,helperthread
--parse-only Parse options only, don't start any IO
--output Write output to file
--bandwidth-log Generate aggregate bandwidth logs
--minimal Minimal (terse) output
--output-format=type Output format (terse,json,json+,normal)
--terse-version=type Set terse version output format (default 3, or 2 or 4)
--version Print version info and exit
--help Print this page
--cpuclock-test Perform test/validation of CPU clock
--crctest=type Test speed of checksum functions
--cmdhelp=cmd Print command help, "all" for all of them
--enghelp=engine Print ioengine help, or list available ioengines
--enghelp=engine,cmd Print help for an ioengine cmd
--showcmd Turn a job file into command line options
--eta=when When ETA estimate should be printed
May be "always", "never" or "auto"
--eta-newline=time Force a new line for every 'time' period passed
--status-interval=t Force full status dump every 't' period passed
--readonly Turn on safety read-only checks, preventing writes
--section=name Only run specified section in job file, multiple sections can be specified
--alloc-size=kb Set smalloc pool to this size in kb (def 16384)
--warnings-fatal Fio parser warnings are fatal
--max-jobs=nr Maximum number of threads/processes to support
--server=args Start a backend fio server
--daemonize=pidfile Background fio server, write pid to file
--client=hostname Talk to remote backend(s) fio server at hostname
--remote-config=file Tell fio server to load this local job file
--idle-prof=option Report cpu idleness on a system or percpu basis
(option=system,percpu) or run unit work
calibration only (option=calibrate)
--inflate-log=log Inflate and output compressed log
--trigger-file=file Execute trigger cmd when file exists
--trigger-timeout=t Execute trigger af this time
--trigger=cmd Set this command as local trigger
--trigger-remote=cmd Set this command as remote trigger
--aux-path=path Use this path for fio state generated files
Fio was written by Jens Axboe
Jens Axboe
Jens Axboe 使用参数说明
filename=/dev/mapper/test1 测试文件名称
direct=1 绕过操作系统的buffer，即进行O_DIRECT操作
iodepth IO队列深度，在文件之上的保持的IO单元,默认对于每个文件来说是1，可以设置一个更大的值来提供并发度，iodepth大于1不会影响同步IO引擎(除非verify_async这个选项被设置)
thread io默认会使用fork()创建job，如果这个选项设置的话，fio将使用pthread_create来创建线程
rw=randwrite 测试随机写的I/O
rw=randrw 测试随机写和读的I/O
bs=8k 单次io的块文件大小为8k，仿照数据库的块大小
size=1g 本次的测试文件大小为1g，以每次8k的io进行测试。
numjobs=4 本次的测试线程为4.
runtime=300 测试时间为300秒，如果不写测试时间，则将1g文件按每8k写完结束
ioengine=psync io引擎使用pync方式
rwmixwrite=30 在混合读写的模式下，写占30%
rwmixread=70 在混合读写的模式下，读占70%
group_reporting 关于显示结果的，汇总每个进程的信息
lockmem=1g 只使用1g内存进行测试
zero_buffers 用0初始化系统buffer
nrfiles=8 每个进程生成文件的数量IO性能压测
---数据库场景主要是测试随机读写
---混合随机读写测试
[root@rac1 fio-2.17]# fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=randrw -rwmixread=70 -ioengine=psync -bs=8k -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest -ioscheduler=deadline
iotest: (g=0): rw=randrw, bs=8192B-8192B,8192B-8192B,8192B-8192B, ioengine=psync, iodepth=1
...
fio-2.17
Starting 4 threads
fio: os or kernel doesn't support IO scheduler switching
fio: os or kernel doesn't support IO scheduler switching
fio: os or kernel doesn't support IO scheduler switching
fio: os or kernel doesn't support IO scheduler switching
#最大读写吞吐量/IOPS
Jobs: 4 (f=4): [m(4)][100.0%][r=58.5MiB/s,w=25.4MiB/s][r=7486,w=3249 IOPS][eta 00m:00s]
iotest: (groupid=0, jobs=4): err= 0: pid=97958: Sun Sep 4 19:05:36 2022
read: IOPS=6897, BW=53.1MiB/s (56.6MB/s)(2867MiB/53205msec)
clat (usec): min=75, max=16751, avg=391.74, stdev=335.46
lat (usec): min=75, max=16751, avg=392.36, stdev=335.54
clat percentiles (usec):
| 1.00th=[ 80], 5.00th=[ 92], 10.00th=[ 108], 20.00th=[ 147],
| 30.00th=[ 205], 40.00th=[ 266], 50.00th=[ 322], 60.00th=[ 382],
| 70.00th=[ 454], 80.00th=[ 548], 90.00th=[ 724], 95.00th=[ 924],
| 99.00th=[ 1656], 99.50th=[ 2096], 99.90th=[ 3312], 99.95th=[ 3760],
| 99.99th=[ 5984]
write: IOPS=2956, BW=23.2MiB/s (24.3MB/s)(1229MiB/53205msec)
clat (usec): min=82, max=16158, avg=412.63, stdev=334.53
lat (usec): min=83, max=16158, avg=414.12, stdev=334.91
clat percentiles (usec):
| 1.00th=[ 89], 5.00th=[ 96], 10.00th=[ 110], 20.00th=[ 149],
| 30.00th=[ 219], 40.00th=[ 290], 50.00th=[ 354], 60.00th=[ 418],
| 70.00th=[ 494], 80.00th=[ 588], 90.00th=[ 756], 95.00th=[ 940],
| 99.00th=[ 1592], 99.50th=[ 2064], 99.90th=[ 3312], 99.95th=[ 3824],
| 99.99th=[ 6304]
lat (usec) : 100=6.81%, 250=29.70%, 500=37.52%, 750=16.68%, 1000=5.18%
lat (msec) : 2=3.53%, 4=0.53%, 10=0.04%, 20=0.01%
cpu : usr=1.31%, sys=19.35%, ctx=744673, majf=0, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=366973,157315,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=53.1MiB/s (56.6MB/s), 53.1MiB/s-53.1MiB/s (56.6MB/s-56.6MB/s), io=2867MiB (3006MB), run=53205-53205msec
WRITE: bw=23.2MiB/s (24.3MB/s), 23.2MiB/s-23.2MiB/s (24.3MB/s-24.3MB/s), io=1229MiB (1289MB), run=53205-53205msec
---混合顺序读写测试：
[root@rac1 fio-2.17]# fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=rw -rwmixread=70 -ioengine=psync -bs=8k -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest -ioscheduler=deadline
iotest: (g=0): rw=rw, bs=8192B-8192B,8192B-8192B,8192B-8192B, ioengine=psync, iodepth=1
...
fio-2.17
Starting 4 threads
fio: os or kernel doesn't support IO scheduler switching
fio: os or kernel doesn't support IO scheduler switching
fio: os or kernel doesn't support IO scheduler switching
fio: os or kernel doesn't support IO scheduler switching
Jobs: 4 (f=4): [M(4)][100.0%][r=48.5MiB/s,w=20.6MiB/s][r=6205,w=2627 IOPS][eta 00m:00s]
iotest: (groupid=0, jobs=4): err= 0: pid=99479: Sun Sep 4 19:09:41 2022
read: IOPS=6943, BW=54.3MiB/s (56.9MB/s)(2867MiB/52851msec)
clat (usec): min=73, max=13036, avg=392.24, stdev=346.79
lat (usec): min=73, max=13036, avg=392.79, stdev=347.15
clat percentiles (usec):
| 1.00th=[ 78], 5.00th=[ 87], 10.00th=[ 104], 20.00th=[ 147],
| 30.00th=[ 199], 40.00th=[ 262], 50.00th=[ 318], 60.00th=[ 378],
| 70.00th=[ 450], 80.00th=[ 548], 90.00th=[ 724], 95.00th=[ 948],
| 99.00th=[ 1752], 99.50th=[ 2192], 99.90th=[ 3408], 99.95th=[ 4128],
| 99.99th=[ 5984]
write: IOPS=2976, BW=23.3MiB/s (24.4MB/s)(1229MiB/52851msec)
clat (usec): min=81, max=12001, avg=413.64, stdev=360.46
lat (usec): min=81, max=12001, avg=414.78, stdev=360.97
clat percentiles (usec):
| 1.00th=[ 86], 5.00th=[ 91], 10.00th=[ 103], 20.00th=[ 143],
| 30.00th=[ 207], 40.00th=[ 278], 50.00th=[ 350], 60.00th=[ 410],
| 70.00th=[ 486], 80.00th=[ 580], 90.00th=[ 756], 95.00th=[ 980],
| 99.00th=[ 1784], 99.50th=[ 2256], 99.90th=[ 3472], 99.95th=[ 4192],
| 99.99th=[ 6496]
lat (usec) : 100=8.62%, 250=28.16%, 500=37.90%, 750=15.82%, 1000=5.00%
lat (msec) : 2=3.81%, 4=0.63%, 10=0.06%, 20=0.01%
cpu : usr=0.80%, sys=19.45%, ctx=750369, majf=0, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=366973,157315,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=54.3MiB/s (56.9MB/s), 54.3MiB/s-54.3MiB/s (56.9MB/s-56.9MB/s), io=2867MiB (3006MB), run=52851-52851msec
WRITE: bw=23.3MiB/s (24.4MB/s), 23.3MiB/s-23.3MiB/s (24.4MB/s-24.4MB/s), io=1229MiB (1289MB), run=52851-52851msec
---其他场景
---随机读测试：
fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=randread -ioengine=psync -bs=8k -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest
---随机写测试：
fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=8k -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest
---顺序读测试：
fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=read -ioengine=psync -bs=1M -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest
---顺序写测试：
fio -filename=/dev/mapper/test1 -direct=1 -iodepth 1 -thread -rw=write -ioengine=psync -bs=1M -size=1G -numjobs=4 -runtime=60 -group_reporting -name=iotest总结:
以上内容为Oracle RAC集群网络，IO性能常用的测试工具以及测试方法，大家可以根据自身的环境以及使用场景进行选择。