Spring Boot: 2.1.3.RELEASE
Fastjson: 1.2.60
由于要使用微信小程序官方提供的免费的图片敏感内容检查接口,本来对于使用RestTemplate进行form表单不太熟悉,在网上找了些资料后模仿着写了一段实现代码.
Resource imageResource = new FileSystemResource(tmpPath);
MultiValueMap formDataMap = new LinkedMultiValueMap<>();
formDataMap.add("media", imageResource);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity requestEntity = new HttpEntity<>(formDataMap, headers);
ResponseEntity responseEntity = restTemplate.exchange(WechatConst.URL_IMG_SEC_CHECK,
HttpMethod.POST, requestEntity, WechatCommonResponseEntity.class,
Objects.isNull(accessToken) ? getAccessToken().getAccessToken() : accessToken);
按这样的实现是可以正常跑通的,可是接下来发生了一个奇怪的bug.这处图片检查的工具类代码我用在了一个批处理的场景,对系统上的图片进行批量化检查,在测试的时候发现,在跑了一定量数据之后,程序就报错了,发现是文件描述符占用达到上限了(测试环境没优化linux内核参数).用lsof一查,程序占用了大量的临时文件的描述符.
我就纳闷了,虽然前面有做了生成临时文件的操作,可是IO流都是有做关闭的,到底是什么地方导致了资源泄露.通过本地IDEA一行行代码debug进去,一开始还没发现有什么问题.后面仔细的观察构建request报文部分的代码,我发现不对劲的地方!
//HttpEntityRequestCallback.doWithRequest()
//...
Class requestBodyClass = requestBody.getClass();
Type requestBodyType = (this.requestEntity instanceof RequestEntity ?
((RequestEntity)this.requestEntity).getType() : requestBodyClass);
HttpHeaders httpHeaders = httpRequest.getHeaders();
HttpHeaders requestHeaders = this.requestEntity.getHeaders();
MediaType requestContentType = requestHeaders.getContentType();
for (HttpMessageConverter messageConverter : getMessageConverters()) {
if (messageConverter instanceof GenericHttpMessageConverter) {
GenericHttpMessageConverter这里是没有问题的,问题出现在AllEncompassingFormHttpMessageConverter里面的writeParts()和writePart(),这两个方式实际上继承于父类FormHttpMessageConverter.
private void writeParts(OutputStream os, MultiValueMap parts, byte[] boundary) throws IOException {
for (Map.Entry> entry : parts.entrySet()) {
String name = entry.getKey();
for (Object part : entry.getValue()) {
if (part != null) {
writeBoundary(os, boundary);
//这里坑点是getHttpEntity(part)
writePart(name, getHttpEntity(part), os);
writeNewLine(os);
}
}
}
}
protected HttpEntity getHttpEntity(Object part) {
//由于我传入的并不是HttpEntity,这里会把我的Resource做个封装
return (part instanceof HttpEntity ? (HttpEntity) part : new HttpEntity<>(part));
}
@SuppressWarnings("unchecked")
private void writePart(String name, HttpEntity partEntity, OutputStream os) throws IOException {
Object partBody = partEntity.getBody();
if (partBody == null) {
throw new IllegalStateException("Empty body for part '" + name + "': " + partEntity);
}
Class partType = partBody.getClass();
HttpHeaders partHeaders = partEntity.getHeaders();
MediaType partContentType = partHeaders.getContentType();
//到了这里,我发现了不对劲的地方,下面的代码最终选择的converter是FastJsonHttpMessageConverter
for (HttpMessageConverter messageConverter : this.partConverters) {
if (messageConverter.canWrite(partType, partContentType)) {
Charset charset = isFilenameCharsetSet() ? StandardCharsets.US_ASCII : this.charset;
HttpOutputMessage multipartMessage = new MultipartHttpOutputMessage(os, charset);
multipartMessage.getHeaders().setContentDispositionFormData(name, getFilename(partBody));
if (!partHeaders.isEmpty()) {
multipartMessage.getHeaders().putAll(partHeaders);
}
((HttpMessageConverter 我传入的是一个FileSystemResource,这里竟然给我选择了FastJsonHttpMessageConverter做序列化,虽然最终序列化后的数据是正常的,不然图片检测的请求肯定会有问题,但是直觉告诉我问题就出在这里,通过检查我也知道了为什么我目前的写法会导致这里选择了fastjson作为conventer.
HttpEntity默认构造出来时,里面的HttpHeaders是空的.由于在前面我的FileSystemResource被自动封装在了一个HttpEntity里,自然也没有赋值HttpHeaders.而在messageConverter.canWrite(partType, partContentType)里,FastJsonHttpMessageConverter的处理是partType的判断直接返回true,而partContentType的判断则根据他的supportedMediaTypes属性,但是如果传入的是null,则直接通过,这里FastJsonHttpMessageConverter是直接使用了继承父类AbstractHttpMessageConverter的代码.
protected boolean canWrite(@Nullable MediaType mediaType) {
if (mediaType == null || MediaType.ALL.equalsTypeAndSubtype(mediaType)) {
return true;
}
for (MediaType supportedMediaType : getSupportedMediaTypes()) {
if (supportedMediaType.isCompatibleWith(mediaType)) {
return true;
}
}
return false;
}
找到原因后,我稍微调整了下我的实现.
Resource imageResource = new FileSystemResource(tmpPath);
MultiValueMap formDataMap = new LinkedMultiValueMap<>();
//这里自己对FileSystemResource做了HttpEntity的封装,并且加上了Content-Type
HttpHeaders imgHeader = new HttpHeaders();
imgHeader.setContentType(MediaType.APPLICATION_OCTET_STREAM);
formDataMap.add("media", new HttpEntity<>(imageResource, imgHeader));
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity requestEntity = new HttpEntity<>(formDataMap, headers);
ResponseEntity responseEntity = restTemplate.exchange(WechatConst.URL_IMG_SEC_CHECK,
HttpMethod.POST, requestEntity, WechatCommonResponseEntity.class,
Objects.isNull(accessToken) ? getAccessToken().getAccessToken() : accessToken);
改完后,conventer最后选择了ResourceHttpMessageConverter,我也稍微看了下这个类的writeContent()方法,里面是有对Resource的IO流做了close()的,然后文件描述符的泄露也消失了,最终确认罪魁祸首就是FastJsonHttpMessageConverter.
最后总结一下:
-
FastJsonHttpMessageConverter不能用于对FileSystemResource做序列化,会导致文件描述符泄露. - 对于
HttpMessageConverter还是有不少不太清楚的地方,以后还是要多看源码认真分析.