用几行python代码测试机器性能

用几行python代码测试机器性能

我想直观的测试对比不同机器的性能，因为我不懂什么硬件，不懂什么双精度浮点数，不懂什么tflops，不懂什么CPU或者GPU的算力，我就是想使用代码对比测试不同机器的运算性能。据我了解，我只知道 linpack测试，本质是高斯消元法求解方程组,但是我想用python代码，还要直观绘制图像，于是有了本文。
我需要这样的一个项目：能够稳定、准确、适当地测试出机器算力的python代码
稳定：多次反复运行地结果相差不能太大
准确：要和实际情况类似，能准确反映机器算力
适当：耗时适当。比如几分钟，不能太长，也不能太短，就几秒钟可能影响准确性

import datetime
import numpy as np
import time
import matplotlib.pyplot as plt
time=np.zeros(10)
for i in range(10):
    start_time = datetime.datetime.now()
    start_time_1 = datetime.datetime.strftime(start_time,'%Y-%m-%d %H:%M:%S')
    print("start time:",start_time_1)
    for j in range(10):
        a=np.ones([10000,10500],dtype=float)
        b=np.ones([10500,11000],dtype=float)
        np.matmul(a,b)
    end_time = datetime.datetime.now()
    end_time_1 = datetime.datetime.strftime(end_time,'%Y-%m-%d %H:%M:%S')
    print("end time:",end_time_1)
    print("Time:",end_time-start_time)
    time[i]=(end_time-start_time).total_seconds()
plt.title('TIME')
plt.xlabel('times')
plt.ylabel('time')
plt.plot(time)
plt.savefig('myBenchmark_hpc.png')#可以改成任意的文件名用于保存折线图
plt.show()

我的笔记本CPU为R7-5800U，运算结果如下,在114秒到119秒之间波动。

start time: 2022-05-05 16:52:53
end time: 2022-05-05 16:54:51
Time: 0:01:58.804616
start time: 2022-05-05 16:54:51
end time: 2022-05-05 16:56:46
Time: 0:01:54.866949
start time: 2022-05-05 16:56:46
end time: 2022-05-05 16:58:43
Time: 0:01:56.671524
start time: 2022-05-05 16:58:43
end time: 2022-05-05 17:00:41
Time: 0:01:58.094729
start time: 2022-05-05 17:00:41
end time: 2022-05-05 17:02:36
Time: 0:01:55.369099
start time: 2022-05-05 17:02:36
end time: 2022-05-05 17:04:31
Time: 0:01:54.412027
start time: 2022-05-05 17:04:31
end time: 2022-05-05 17:06:25
Time: 0:01:54.108223
start time: 2022-05-05 17:06:25
end time: 2022-05-05 17:08:19
Time: 0:01:54.336517
start time: 2022-05-05 17:08:19
end time: 2022-05-05 17:10:14
Time: 0:01:54.828860
start time: 2022-05-05 17:10:14
end time: 2022-05-05 17:12:10
Time: 0:01:55.496720

通过反复计算十次并绘制折线图来保证稳定，核心运算为矩阵的乘法保证准确，时间为两分钟比较适当。
下面为用了某高性能运算平台的CPU，据称其CPU计算节点为华为XH321 V5服务器，其中CPU为Intel Xeon Gold 6240R，运算结果如下,在94秒到96秒之间波动。

start time: 2022-05-05 20:53:24
end time: 2022-05-05 20:55:00
Time: 0:01:35.904996
start time: 2022-05-05 20:55:00
end time: 2022-05-05 20:56:35
Time: 0:01:34.942021
start time: 2022-05-05 20:56:35
end time: 2022-05-05 20:58:11
Time: 0:01:35.217495
start time: 2022-05-05 20:58:11
end time: 2022-05-05 20:59:45
Time: 0:01:34.585593
start time: 2022-05-05 20:59:45
end time: 2022-05-05 21:01:20
Time: 0:01:35.329601
start time: 2022-05-05 21:01:20
end time: 2022-05-05 21:02:56
Time: 0:01:35.096785
start time: 2022-05-05 21:02:56
end time: 2022-05-05 21:04:30
Time: 0:01:34.259758
start time: 2022-05-05 21:04:30
end time: 2022-05-05 21:06:04
Time: 0:01:33.945833
start time: 2022-05-05 21:06:04
end time: 2022-05-05 21:07:38
Time: 0:01:34.134881
start time: 2022-05-05 21:07:38
end time: 2022-05-05 21:09:13
Time: 0:01:35.413812

在百度飞桨平台运行该代码，选择的是脚本任务，运行环境选择P40一卡，但是我直接运行上述代码，则机器不会自动使用GPU运算，而是仍用CPU运算，如下所示.我们可以看出花费418秒-432秒之间波动，原因就是我在代码中并未指定要到GPU中运算。

0
start time: 2022-05-06 16:18:28
end time: 2022-05-06 16:25:30
Time: 0:07:01.873245
1
start time: 2022-05-06 16:25:30
end time: 2022-05-06 16:32:37
Time: 0:07:07.131342
2
start time: 2022-05-06 16:32:37
end time: 2022-05-06 16:39:37
Time: 0:07:00.062224
3
start time: 2022-05-06 16:39:37
end time: 2022-05-06 16:46:48
Time: 0:07:10.704133
4
start time: 2022-05-06 16:46:48
end time: 2022-05-06 16:53:46
Time: 0:06:57.824444
5
start time: 2022-05-06 16:53:46
end time: 2022-05-06 17:00:58
Time: 0:07:12.248488
6
start time: 2022-05-06 17:00:58
end time: 2022-05-06 17:08:01
Time: 0:07:02.951203
7
start time: 2022-05-06 17:08:01
end time: 2022-05-06 17:15:07
Time: 0:07:06.035277
8
start time: 2022-05-06 17:15:07
end time: 2022-05-06 17:22:16
Time: 0:07:09.283966
9
start time: 2022-05-06 17:22:16
end time: 2022-05-06 17:29:21
Time: 0:07:04.583901

现将该代码改为使用GPU运算，由于飞桨不支持pytorch,因此改为使用numpy_gpu模块，具体代码如下。

import datetime
import numpy as np
import numpy_gpu as gpu
import time
import matplotlib as mpl
mpl.use('Agg')
import matplotlib.pyplot as plt
print('startstartstart!')

time=np.zeros(10)
for i in range(10):
    start_time = datetime.datetime.now()
    start_time_1 = datetime.datetime.strftime(start_time,'%Y-%m-%d %H:%M:%S')
    print(i)
    print("start time:",start_time_1)
    for j in range(10):
        a=np.ones([10000,10500],dtype=float)
        b=np.ones([10500,11000],dtype=float)
        gpu.matmul(a,b)
    end_time = datetime.datetime.now()
    end_time_1 = datetime.datetime.strftime(end_time,'%Y-%m-%d %H:%M:%S')
    print("end time:",end_time_1)
    print("Time:",end_time-start_time)
    time[i]=(end_time-start_time).total_seconds()
plt.title('TIME')
plt.xlabel('times')
plt.ylabel('time')
plt.plot(time)
plt.savefig('/root/paddlejob/workspace/output/myBenchmark_paddle_GPU.png')
plt.show()

仍是使用飞桨ai stuidio ，选择脚本任务，使用P40GPU，输出如下所示，可以发现时间仍为7分钟左右。

0
start time: 2022-05-07 15:30:25
end time: 2022-05-07 15:37:31
Time: 0:07:05.700728
1
start time: 2022-05-07 15:37:31
end time: 2022-05-07 15:44:39
Time: 0:07:08.038143
2
start time: 2022-05-07 15:44:39
end time: 2022-05-07 15:51:48
Time: 0:07:09.498013
3
start time: 2022-05-07 15:51:48
end time: 2022-05-07 15:58:56
Time: 0:07:07.648908
4
start time: 2022-05-07 15:58:56
end time: 2022-05-07 16:06:05
Time: 0:07:08.982757
5
start time: 2022-05-07 16:06:05
end time: 2022-05-07 16:13:12
Time: 0:07:07.482899
6
start time: 2022-05-07 16:13:12
end time: 2022-05-07 16:20:21
Time: 0:07:08.627469
7
start time: 2022-05-07 16:20:21
end time: 2022-05-07 16:27:30
Time: 0:07:09.204312
8
start time: 2022-05-07 16:27:30
end time: 2022-05-07 16:34:38
Time: 0:07:07.801684
9
start time: 2022-05-07 16:34:38
end time: 2022-05-07 16:41:46
Time: 0:07:08.188941

接下来使用飞桨自创国产深度学习框架paddle!!!代码如下。

import datetime
import paddle
import numpy as np
import time
import matplotlib as mpl
mpl.use('Agg')
import matplotlib.pyplot as plt
use_gpu = True
paddle.set_device('gpu:0') if use_gpu else paddle.set_device('cpu')

print('startstartstart!')

time=np.zeros(shape=10)
for i in range(10):
    start_time = datetime.datetime.now()
    start_time_1 = datetime.datetime.strftime(start_time,'%Y-%m-%d %H:%M:%S')
    print(i)
    print("start time:",start_time_1)
    for j in range(10):
        a=paddle.ones([10000,10500],dtype='float64')
        b=paddle.ones([10500,11000],dtype='float64')
        paddle.matmul(a,b)
    end_time = datetime.datetime.now()
    end_time_1 = datetime.datetime.strftime(end_time,'%Y-%m-%d %H:%M:%S')
    print("end time:",end_time_1)
    print("Time:",end_time-start_time)
    time[i]=(end_time-start_time).total_seconds()
plt.title('TIME')
plt.xlabel('times')
plt.ylabel('time')
plt.plot(time)
plt.savefig('/root/paddlejob/workspace/output/myBenchmark_paddle_GPU.png')
plt.show()

速度极快！仅在第一次用到gpu时耗时7.4秒，之后九次0.002秒就完成了！可见大家还是要用GPU进行深度学习的训练。

0
start time: 2022-05-07 18:06:21
end time: 2022-05-07 18:06:28
Time: 0:00:07.402325
1
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002286
2
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002176
3
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002092
4
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002299
5
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002129
6
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002227
7
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002132
8
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002192
9
start time: 2022-05-07 18:06:28
end time: 2022-05-07 18:06:28
Time: 0:00:00.002207

但是当我把paddle.matmul(a,b)执行的循环由10改为100之后，（即将j变量的for循环次数改为100，如下所示），时间竟然发生了剧烈的变化，如下图所示,前三次耗时为10秒之内，第四次耗时6分钟，后六次耗时在11分钟左右！这说明我写的这个代码或许并不能稳定测试出机器浮点计算性能。本项目彻底破产。再见！

 for j in range(100):
        a=paddle.ones([10000,10500],dtype='float64')
        b=paddle.ones([10500,11000],dtype='float64')
        paddle.matmul(a,b)

0
start time: 2022-05-07 18:45:09
end time: 2022-05-07 18:45:16
Time: 0:00:07.521770
1
start time: 2022-05-07 18:45:16
end time: 2022-05-07 18:45:16
Time: 0:00:00.019799
2
start time: 2022-05-07 18:45:16
end time: 2022-05-07 18:45:16
Time: 0:00:00.019735
3
start time: 2022-05-07 18:45:16
end time: 2022-05-07 18:51:49
Time: 0:06:32.664528
4
start time: 2022-05-07 18:51:49
end time: 2022-05-07 19:02:55
Time: 0:11:05.898622
5
start time: 2022-05-07 19:02:55
end time: 2022-05-07 19:14:01
Time: 0:11:05.889246
6
start time: 2022-05-07 19:14:01
end time: 2022-05-07 19:25:07
Time: 0:11:05.899609
7
start time: 2022-05-07 19:25:07
end time: 2022-05-07 19:36:12
Time: 0:11:05.911267
8
start time: 2022-05-07 19:36:12
end time: 2022-05-07 19:47:18
Time: 0:11:05.911048
9
start time: 2022-05-07 19:47:18
end time: 2022-05-07 19:58:24
Time: 0:11:05.913626