Jackson+Feign反序列化问题排查

概述

本文记录在使用Spring Cloud微服务开发时遇到的一个反序列化问题,RPC/HTTP框架使用的是Feign,JSON序列化反序列化工具是Jackson。

问题

测试环境的ELK告警日志如下:

- [43f42bf7] 500 Server Error for HTTP POST "/api/open/dialog/nextQuestion"
feign.codec.DecodeException: Error while extracting response for type [AbaResponse] 
and content type [application/json;charset=UTF-8]; 
nested exception is org.springframework.http.converter.HttpMessageNotReadableException:
JSON parse error: Expected array or string.; 
nested exception is com.fasterxml.jackson.databind.exc.MismatchedInputException: Expected array or string.
at [Source: (ByteArrayInputStream); line: 1, column: 295] (through reference chain: com.aba.common.utils.context.AbaResponse["data"]->com.aba.enduser.common.vo.UserAccountVO["privacySettings"]->java.util.LinkedHashMap["MINIMUM_LEGAL_AGE"]->com.aba.enduser.common.dto.account.PrivacySettings["timestamp"])
at feign.SynchronousMethodHandler.decode(SynchronousMethodHandler.java:180)
at feign.SynchronousMethodHandler.executeAndDecode(SynchronousMethodHandler.java:140)
at feign.SynchronousMethodHandler.invoke(SynchronousMethodHandler.java:78)
at feign.ReflectiveFeign$FeignInvocationHandler.invoke(ReflectiveFeign.java:103)

报错产生自gateway-open服务,gateway-open服务把接口请求/api/open/dialog/nextQuestion转发到dialog服务,dialog服务在Feign调用另外一个enduser服务时发生。很熟悉的报错,Feign反序列化问题。

排查

no Creators, like default construct, exist: cannot deserialize from Object value no delegate- or property-based Creator

为了排查问题,首先想到本地复现问题。本地启动dialog和enduser服务,postman请求dialog服务的接口/dialog/nextQuestion。却出现另一个问题,且这个报错发生在解析requestBody时。在Controller层方法里第一行加断点,程序都没在断点处停止,直接报错:

Caught unhandled generic exception in com.aba.dialog.controller.DialogController
org.springframework.http.converter.HttpMessageConversionException: Type definition error: [simple type, class com.aba.dialog.service.domain.assessment.dialog.answer.DialogAnswerItem]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `com.aba.dialog.service.domain.assessment.dialog.answer.DialogAnswerItem` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (PushbackInputStream); line: 1, column: 2]
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:242)
at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:227)
at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodArgumentResolver.readWithMessageConverters(AbstractMessageConverterMethodArgumentResolver.java:204)

dialog服务最近没有任何改动啊。enduser服务有改动,也和dialog服务无关;毕竟dialog服务断点没进去。

报错代码:

@PostMapping(value = "/nextQuestion")
public DialogDTO handleDialog(@RequestBody DialogAnswerItem item) {
	// 断点行
    String platform = httpServletRequest.getHeader("dialogPlatform");
}

@RequestBody注解的POJO类:

data class DialogAnswerItem(val stateId: StateId,
                            var answer: GivenAnswer,
                            val progress: Double = 0.0,
                            val entryPoint: String? = null)

不甚熟悉的kotlin语言。

看起来一时半会搞不定。

Expected array or string

既然上面的问题没搞定,先解决测试环境的问题。本地启动第三个应用gateway服务,postman模拟调用gateway服务,由gateway负责转发。问题重现:
Jackson+Feign反序列化问题排查_第1张图片
诸多分析,Google搜到一个靠谱的stackoverflow答案:feign-client-decodeexception-error-while-extracting-response。

修改enduser服务代码:

@Data
@AllArgsConstructor
@NoArgsConstructor
public class PrivacySettings implements Serializable {

    private Boolean value;

    @JsonSerialize(using = LocalDateTimeSerializer.class)
    @JsonDeserialize(using = LocalDateTimeDeserializer.class)
    private LocalDateTime timestamp;
}

本地调试,问题解决。

wait but why。

上面也提到【enduser服务有改动,也和dialog服务无关】,现在为了解决Feign + Jackson远程调用反序列化失败问题,去修改enduser代码,增加2个Jackson提供的注解@JsonSerialize@JsonDeserialize

问题虽然解决,总感觉哪里不对劲。但是测试环境里,前端等着使用相关接口,没成多想,发布测试环境。

Feign

结果发布到测试环境后,测试环境里ELK也记录到我一开始在本地调试重现问题时遇到的另外一个问题:
no Creators, like default construct, exist: cannot deserialize from Object value no delegate- or property-based Creator

看来这个问题是绕不过去的坎。诸般Google/百度搜索与尝试,始终没解决问题。

最后还是仔仔细细看Google给出的第一篇stackoverflow文章no-creators-like-default-construct-exist-cannot-deserialize-from-object-valu,看到:

register jackson module kotlin to ObjectMapper.

才突然意识到,最近对一个common-web组件库做了mvn clean deploy操作。deploy包括install,所以本地环境和测试环境都有相同问题。

再检查common-web下面的配置类:

@Component
public class JsonConfig {
    /**
     * 解决JSON parse error: Unrecognized field "xxx"异常问题
     */
    @Bean
    public MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter() {
        MappingJackson2HttpMessageConverter converter = new MappingJackson2HttpMessageConverter();
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        converter.setObjectMapper(objectMapper);
        return converter;
    }
}

如上述代码里注释所述,增加此配置是为了解决JSON: Unrecognized field, not marked as ignorable问题,参考stackoverflow的问答jackson-with-json-unrecognized-field-not-marked-as-ignorable。

之前在另外2个服务都出现过此问题,出现此问题的场景都是A服务调用B服务,B服务在业务开发时增加字段(杜绝修改字段和删除字段的开发bad practice)。A服务在微服务体系里还是在使用旧版本的B-api.jar,也就是说A服务的镜像里的jar里还是使用旧的版本,但是在Feign调用B服务时,B服务返回一个新版本的B-api.jar,多了一个字段。于是报错??

A服务重新编译新版本,则会把新版本的B-api.jar纳入到镜像里,也就是说发布新版本即可解决问题。

想要一劳永逸解决此类问题,在A服务里新增上述配置类就可以了吗?待验证。

考虑到Spring Cloud微服务体系,加字段是很常见的事情,那是不是可以把配置类放在common-web组件库,让所有服务都有此配置类。待验证。

正是因为上述猜想待验证,代码一直在本地。common-web组件库里其他类加以调整时,把JsonConfig配置类编译到dialog服务。

最后,两个问题的解决方法都是移除JsonConfig配置类,并且enduser服务的两个Jackson注解都可以revert。

问题是得以"解决",但是为啥呢?

后面仔细看dialog服务代码,好几个Jackson配置:

@Configuration
@EnableAsync
open class ApplicationConfig {
    private val log = LoggerFactory.getLogger(this.javaClass)

    @Bean
    open fun restTemplateCommon(): RestTemplate {
        val restTemplate = RestTemplate()
        addOwnMappingJackson2HttpMessageConverter(restTemplate)
        val interceptors = listOf(
            ClientHttpRequestInterceptor { request, body, execution ->
                val headers = request.headers
                headers.add("Accept", MediaType.APPLICATION_JSON_VALUE)
                headers.add("Content-Type", MediaType.APPLICATION_JSON_VALUE)
                execution.execute(request, body)
            }
        )
        restTemplate.interceptors = interceptors
        return restTemplate
    }

    private fun addOwnMappingJackson2HttpMessageConverter(restTemplate: RestTemplate) {
        val converter = MappingJackson2HttpMessageConverter()
        val objectMapper = ObjectMapper()
            .findAndRegisterModules()
            // needed that the LocalDate is not serialized to [2000,1,1] but to "2000-01-01"
            .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false)
            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        converter.objectMapper = objectMapper

        val jacksonMappers = restTemplate.messageConverters
            .filter { httpMessageConverter -> httpMessageConverter is MappingJackson2HttpMessageConverter }

        if (jacksonMappers.isNotEmpty()) {
            restTemplate.messageConverters.remove(jacksonMappers.first())
        }
        restTemplate.messageConverters.add(1, converter)
    }

}

上面这个是kotlin语言。以及

@Configuration
public class HttpConverterConfig implements WebMvcConfigurer {

    @Bean
    public MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter() {
        AdaJackson2ObjectMapperBuilder adaJackson2ObjectMapperBuilder = new AdaJackson2ObjectMapperBuilder();
        return new MappingJackson2HttpMessageConverter(adaJackson2ObjectMapperBuilder.build()) {

            @Override
            protected void writeInternal(@NotNull Object object, Type type, @NotNull HttpOutputMessage outputMessage) throws IOException, HttpMessageNotWritableException {
                if (object instanceof String) {
                    Charset charset = this.getDefaultCharset();
                    StreamUtils.copy((String) object, charset, outputMessage.getBody());
                } else {
                    super.writeInternal(object, type, outputMessage);
                }
            }
        };
    }
}

以及:

@Component
public class AdaJackson2ObjectMapperBuilder extends Jackson2ObjectMapperBuilder {

    public AdaJackson2ObjectMapperBuilder() {
        serializationInclusion(JsonInclude.Include.NON_NULL);
        serializationInclusion(JsonInclude.Include.NON_ABSENT);

        featuresToDisable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS,
                SerializationFeature.WRITE_DATE_TIMESTAMPS_AS_NANOSECONDS);
        modules(new AdaModule(), new GuavaModule(), new JavaTimeModule(), new Jdk8Module(), new ParameterNamesModule());
    }

    @Override
    public void configure(@NotNull ObjectMapper objectMapper) {
        super.configure(objectMapper);
        // disable constructor, getter and setter detection
        objectMapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.NONE);
        objectMapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);
        objectMapper.registerModule(new KotlinModule());
    }

	private static class AdaModule extends SimpleModule {
	    public AdaModule() {
	        addSerializer(JSONError.class, new JSONErrorSerializer());
	    }
	}
}

以及:

public class JSONErrorSerializer extends JsonSerializer<JSONError> {

    private static final String KEY_STATUS_CODE = "statusCode";
    private static final String KEY_ERROR = "error";
    private static final String KEY_MESSAGE = "message";

    @Override
    public void serialize(JSONError jsonError, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
        jsonGenerator.writeStartObject();
        jsonGenerator.writeStringField(KEY_STATUS_CODE, String.valueOf(jsonError.getStatusCode()));
        jsonGenerator.writeStringField(KEY_ERROR, jsonError.getError());
        if (jsonError.getMessage() != null && !jsonError.getMessage().isEmpty()) {
            jsonGenerator.writeStringField(KEY_MESSAGE, jsonError.getMessage());
        }
        jsonGenerator.writeEndObject();
    }
}

参考

你可能感兴趣的:(Spring,Cloud,Java,kotlin)