RPC Benchmark Round 4

测试说明

• 仅限于 Java
• 客户端使用 JMH 进行压测, 32 线程, 3 轮预热 3 轮测试, 每轮 10s
• 每次运行前都会执行 killall java, 但没有在每轮测试时重启操作系统
• 所有类库版本在发布时都是最新的, 除非存在 bug
• 所有框架都尽量参考该项目自带的 Benchmark 实现
• 将会一直持续, 不定期发布测试结果

测试用例

1. boolean existUser(String email), 判断某个 email 是否存在
2. boolean createUser(User user), 添加一个 User
3. User getUser(long id), 根据 id 获取一个用户
4. Page listUser(int pageNo), 获取用户列表

运行结果

生成时间: 2018-08-05 10:35:11
硬件环境: 阿里云 hfc5 ecs.hfc5.xlarge 4vCPU 8GB Intel Xeon Gold 6149 3.1GHz 1.5Gbps 50万PPS
软件环境: Ubuntu x64 16.04.5, Java HotSpot(TM) 64-Bit Server VM 10.0.2+13
启动参数: java -server -Xmx1g -Xms1g -XX:+UseG1GC

existUser

framework	thrpt (ops/ms)	avgt (ms)	p90 (ms)	p99 (ms)	p999 (ms)
netty	164.148	0.193	0.263	0.349	0.471
turbo-rpc	160.824	0.198	0.268	0.348	0.476
jupiter	121.79	0.264	0.372	0.552	1.716
servicecomb	120.684	0.265	0.327	0.447	1.153
thrift	103.7	0.309	0.582	1.067	1.534
turbo-rest	95.611	0.337	0.815	1.37	2.544
undertow	92.121	0.319	0.804	1.356	2.392
undertow-async	84.401	0.383	0.743	1.319	3.527
armeria	77.714	0.419	0.73	1.214	3.874
dubbo	69.853	0.471	0.598	0.801	3.039
dubbo-kryo	69.509	0.481	0.591	0.794	3.396
motan	62.892	0.519	0.785	1.563	10.093
rapidoid	58.374	0.553	1.52	2.978	9.275
hprose	42.93	0.749	0.475	2.234	40.239
springwebflux	34.524	0.934	1.067	1.163	3.76
springboot-undertow	33.984	0.946	1.149	8.602	18.776
grpc	28.276	1.105	1.333	1.491	2.97
springboot	26.62	1.206	1.571	9.388	22.512

createUser

framework	thrpt (ops/ms)	avgt (ms)	p90 (ms)	p99 (ms)	p999 (ms)
netty	152.526	0.211	0.297	0.412	0.535
turbo-rpc	142.739	0.225	0.308	0.398	0.499
servicecomb	109.162	0.292	0.369	0.539	2.163
jupiter	108.392	0.295	0.413	0.626	1.544
thrift	96.859	0.331	0.627	1.149	1.665
undertow	91.564	0.339	0.723	1.235	3.6
turbo-rest	90.684	0.349	0.624	1.049	3.22
undertow-async	79.792	0.404	0.701	1.188	4.071
armeria	64.24	0.509	0.681	1.253	4.893
motan	56.545	0.567	0.872	1.778	10.977
dubbo-kryo	52.285	0.615	0.775	1.011	4.792
hprose	38.791	0.881	0.496	37.028	40.37
springwebflux	32.542	1.004	1.135	1.223	4.098
dubbo	32.493	0.972	1.245	1.563	4.858
grpc	29.83	1.067	1.317	1.473	2.92
springboot	25.921	1.242	1.604	9.355	22.086
springboot-undertow	24.66	1.289	2.101	6.865	14.369
rapidoid	22.559	1.436	0.956	25.756	43.516

getUser

framework	thrpt (ops/ms)	avgt (ms)	p90 (ms)	p99 (ms)	p999 (ms)
netty	150.336	0.215	0.293	0.392	0.493
turbo-rpc	142.188	0.223	0.309	0.408	0.51
servicecomb	108.511	0.296	0.377	0.565	2.245
jupiter	106.07	0.303	0.441	0.67	1.602
thrift	97.593	0.331	0.623	1.141	1.657
undertow	92.562	0.351	0.745	1.29	2.626
turbo-rest	83.16	0.381	0.739	1.3	3.105
undertow-async	82.372	0.39	0.723	1.239	2.994
armeria	71.021	0.453	0.691	1.153	3.76
motan	58.169	0.551	0.847	1.774	9.748
dubbo-kryo	55.022	0.581	0.718	0.937	4.579
rapidoid	51.649	0.62	1.87	3.76	7.897
hprose	48.095	0.667	0.692	1.198	40.042
dubbo	32.989	0.963	1.249	1.663	5.742
grpc	31.684	1.085	1.294	1.462	2.722
springboot-undertow	30.212	1.085	1.307	8.7	19.497
springwebflux	28.166	1.138	1.257	1.331	3.838
springboot	24.63	1.304	1.743	9.667	23.944

listUser

framework	thrpt (ops/ms)	avgt (ms)	p90 (ms)	p99 (ms)	p999 (ms)
turbo-rpc	56.549	0.565	0.879	1.155	8.634
netty	47.402	0.675	0.61	1.009	19.754
jupiter	47.031	0.688	1.032	1.587	16.335
servicecomb	35.554	0.897	1.163	4.481	10.076
thrift	33.46	0.95	1.835	3.715	8.405
grpc	33.428	0.945	1.2	1.487	5.538
motan	32.711	0.972	1.577	2.822	10.24
undertow-async	32.373	0.985	1.339	1.939	8.208
undertow	32.254	0.994	1.253	2.421	10.576
armeria	31.445	1.009	1.509	2.593	8.962
rapidoid	23.466	1.356	1.19	19.464	35.586
turbo-rest	21.093	1.534	1.882	3.113	9.622
dubbo-kryo	20.919	1.599	2.142	2.974	7.531
springwebflux	20.679	1.549	1.671	1.796	6.226
hprose	20.04	1.581	1.808	2.351	49.798
springboot-undertow	18.279	1.758	2.888	11.452	21.332
springboot	16.825	1.909	2.535	12.059	28.803
dubbo	4.661	6.75	9.339	12.255	16.908

更新说明

1. 硬件无变化
2. 软件变化, 升级为 jdk-10.0.2
3. 重构了 netty 实现, 手工序列化 + 合并发送请求

致谢

特别感谢下列人员对本项目的大力支持

1. turbo: hank-whu
2. jupiter: fengjiachun
3. hprose: andot
4. servicecomb: imlidian

免责声明

• 能力所限错误在所难免, 本测试用例及测试结果仅供参考
• 如果你认为 xxx 框架的代码或配置存在问题，那么欢迎发起 Pull Request
• 利益相关: 本测试用例作者同时为 turbo undertow-async 的作者

快速链接

往期评测:
RPC Benchmark Round 3
RPC Benchmark Round 2
RPC Benchmark Round 1

测试说明:
rpc-benchmark 测试说明
rpc-benchmark 项目代码