request.log日志异常排查

问题背景

线上HTTP请求日志按照Common Log Format打印。目前线上打印出的日志没有响应内容的大小。线上日志如下：
203.91.15.232 - - [2018-07-16 00:00:00.133] “GET /api/product/list?cat1Id=123&cat2Id=456&cityId=789 HTTP/1.1” 200 - 17
问题：倒数第二列应该为响应体大小，目前为’-‘。

问题定位

响应体大小为何打印为’-‘？

通过调试定位到问题所在：由于response的contentLength为-1，调用AbstractNCSARequestLog的log方法打印log日志时，直接将响应体大小打印为’-‘，9.2.14版本打印request log日志的jetty关键业务逻辑如下：

        long responseLength = response.getLongContentLength();  //responseLength为-1
        if (responseLength >= 0)
        {
            buf.append(' ');
            if (responseLength > 99999)
                buf.append(responseLength);
            else
            {
                if (responseLength > 9999)
                    buf.append((char)('0' + ((responseLength / 10000) % 10)));
                if (responseLength > 999)
                    buf.append((char)('0' + ((responseLength / 1000) % 10)));
                if (responseLength > 99)
                    buf.append((char)('0' + ((responseLength / 100) % 10)));
                if (responseLength > 9)
                    buf.append((char)('0' + ((responseLength / 10) % 10)));
                buf.append((char)('0' + (responseLength) % 10));
            }
            buf.append(' ');
        }
        else   //直接将响应体大小打印为'-'
            buf.append(" - ");

升级jetty版本后，9.4.10版本打印request log日志的jetty关键业务逻辑如下：

        long written = response.getHttpChannel().getBytesWritten();
        if (written >= 0)
        {
            buf.append(' ');
            if (written > 99999)
                buf.append(written);
            else
            {
                if (written > 9999)
                    buf.append((char)('0' + ((written / 10000) % 10)));
                if (written > 999)
                    buf.append((char)('0' + ((written / 1000) % 10)));
                if (written > 99)
                    buf.append((char)('0' + ((written / 100) % 10)));
                if (written > 9)
                    buf.append((char)('0' + ((written / 10) % 10)));
                buf.append((char)('0' + (written) % 10));
            }
            buf.append(' ');
        }
        else
            buf.append(" - ");

从如下调试信息可以看出，9.4.10获取响应体大小的逻辑修改了，修改后才能正常打印响应体大小

response的contentLength为何为-1？

By default this implementation of HttpURLConnection requests that servers use gzip compression. Since getContentLength() returns the number of bytes transmitted, you cannot use that method to predict how many bytes can be read from getInputStream(). Instead, read that stream until it is exhausted: when read() returns -1. Gzip compression can be disabled by setting the acceptable encodings in the request header。

在默认情况下，HttpURLConnection 使用 gzip方式获取，文件 getContentLength() 这个方法，每次read完成后可以获得，当前已经传送了多少数据，而不能用这个方法获取 需要传送多少字节的内容，当read() 返回 -1时，读取完成。

问题解决

方案一：http请求不再设置gzip压缩

HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setRequestProperty(“Accept-Encoding”, “identity”);
conn.connect();

方案二：返回结果前设置响应体大小

通过过滤器设置响应体大小，实现逻辑如下：

@Bean
public FilterRegistrationBean filterRegistrationBean() {
    FilterRegistrationBean filterBean = new FilterRegistrationBean();
    filterBean.setFilter(new ShallowEtagHeaderFilter());
    filterBean.setUrlPatterns(Arrays.asList("*"));
    return filterBean;
}

ShallowEtagHeaderFilter核心逻辑如下：

private void updateResponse(HttpServletRequest request, HttpServletResponse response) throws IOException {
    ContentCachingResponseWrapper responseWrapper =
            WebUtils.getNativeResponse(response, ContentCachingResponseWrapper.class);
    Assert.notNull(responseWrapper, "ContentCachingResponseWrapper not found");

    HttpServletResponse rawResponse = (HttpServletResponse) responseWrapper.getResponse();
    int statusCode = responseWrapper.getStatusCode();
    byte[] body = responseWrapper.getContentAsByteArray();

    if (rawResponse.isCommitted()) {
        if (body.length > 0) {
            StreamUtils.copy(body, rawResponse.getOutputStream());
        }
    }
    else if (isEligibleForEtag(request, responseWrapper, statusCode, body)) {
        String responseETag = generateETagHeaderValue(body);
        rawResponse.setHeader(HEADER_ETAG, responseETag);
        String requestETag = request.getHeader(HEADER_IF_NONE_MATCH);
        if (responseETag.equals(requestETag)) {
            if (logger.isTraceEnabled()) {
                logger.trace("ETag [" + responseETag + "] equal to If-None-Match, sending 304");
            }
            rawResponse.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
        }
        else {
            if (logger.isTraceEnabled()) {
                logger.trace("ETag [" + responseETag + "] not equal to If-None-Match [" + requestETag +
                        "], sending normal response");
            }
            if (body.length > 0) {
                rawResponse.setContentLength(body.length);   //设置响应体大小
                StreamUtils.copy(body, rawResponse.getOutputStream());
            }
        }
    }
    else {
        if (logger.isTraceEnabled()) {
            logger.trace("Response with status code [" + statusCode + "] not eligible for ETag");
        }
        if (body.length > 0) {
            rawResponse.setContentLength(body.length);
            StreamUtils.copy(body, rawResponse.getOutputStream());
        }
    }
}

方案三：升级jetty版本【最终方案】

将Spring boot版本由1.2.8升级为1.5.13后，jetty版本由9.2.14升级为9.4.10，jetty 9.4.10的请求日志打印逻辑可以实现正常打印响应体大小。

参考文档：
https://stackoverflow.com/questions/5428639/urlconnection-getcontentlength-is-returning-a-negative-value
https://stackoverflow.com/questions/5428639/urlconnection-getcontentlength-is-returning-a-negative-value