基于Tomcat 的Spring MVC 应用中响应数据压缩原理分析
在一个基于Tomcat的Spring MVC应用中，当我们配置了参数server.compression.enabled=true时，我们会发现，服务端返回给浏览器的js格式的文件被压缩了。这背后的原理是什么呢？我们做个简单的分析。
本文所分析的例子项目如下 :
Spring Boot 2.1.9.RELEASESpring MVCServlet容器使用缺省的Tomcat
文章目录
- 1. Web服务器创建时接收压缩参数
- 2. 启动过程中设置压缩参数
- 3. 请求到达时添加压缩处理器
- 4. 响应返回时应用压缩逻辑到响应数据
1. Web服务器创建时接收压缩参数
参考位置 :
SpringApplication#refresh
ServletWebServerApplicationContext#onRefresh
ServletWebServerApplicationContext#createWebServer
ServletWebServerApplicationContext#getWebServerFactory
方法ServletWebServerApplicationContext#getWebServerFactory会触发从容器获取类型为ServletWebServerFactory的bean,缺省情况下，它会是一个TomcatServletWebServerFactory。并且在该bean创建后，初始化前，容器会对其应用合适的BeanPostProcessor,缺省情况下，会有一个WebServerFactoryCustomizerBeanPostProcessor被应用。而这个WebServerFactoryCustomizerBeanPostProcessor会从容器中获取所有的WebServerFactoryCustomizer bean应用于TomcatServletWebServerFactory bean。缺省情况下，会有这几个WebServerFactoryCustomizer:
TomcatWebSocketServletWebServerCustomizer增加
WsContextListenerServletWebServerFactoryCustomizer设置
server.*属性TomcatServletWebServerFactoryCustomizer设置
server.tomcat.*属性,仅仅是部分针对servlet的属性 :
additionalTldSkipPatterns,redirectContextRoot,useRelativeRedirects,mbeanregistry.enabled等。TomcatWebServerFactoryCustomizer设置
server.tomcat.*属性,servlet和reactive通用部分HttpEncodingAutoConfiguration$LocaleCharsetMappingsCustomizer设置
spring.http.encoding.mapping属性
具体将配置server.compression传递给TomcatServletWebServerFactory的动作由ServletWebServerFactoryCustomizer完成,如下所示 :
public class ServletWebServerFactoryCustomizer
implements WebServerFactoryCustomizer<ConfigurableServletWebServerFactory>, Ordered {
private final ServerProperties serverProperties;
public ServletWebServerFactoryCustomizer(ServerProperties serverProperties) {
this.serverProperties = serverProperties;
}
@Override
public int getOrder() {
return 0;
}
// 将来自配置文件的参数信息对象 serverProperties 传递到 factory
@Override
public void customize(ConfigurableServletWebServerFactory factory) {
PropertyMapper map = PropertyMapper.get().alwaysApplyingWhenNonNull();
map.from(this.serverProperties::getPort).to(factory::setPort);
map.from(this.serverProperties::getAddress).to(factory::setAddress);
map.from(this.serverProperties.getServlet()::getContextPath).to(factory::setContextPath);
map.from(this.serverProperties.getServlet()::getApplicationDisplayName).to(factory::setDisplayName);
map.from(this.serverProperties.getServlet()::getSession).to(factory::setSession);
map.from(this.serverProperties::getSsl).to(factory::setSsl);
map.from(this.serverProperties.getServlet()::getJsp).to(factory::setJsp);
map.from(this.serverProperties::getCompression).to(factory::setCompression); // <== server.compression 相关
map.from(this.serverProperties::getHttp2).to(factory::setHttp2);
map.from(this.serverProperties::getServerHeader).to(factory::setServerHeader);
map.from(this.serverProperties.getServlet()::getContextParameters).to(factory::setInitParameters);
}
}
2. 启动过程中设置压缩参数
启动过程中，在上面的步骤获的经过配置参数设置的ServletWebServerFactory bean之后，就开始从ServletWebServerFactory bean创建Web服务器了,还是在ServletWebServerApplicationContext#createWebServer中 :
// ServletWebServerApplicationContext 代码片段
private void createWebServer() {
WebServer webServer = this.webServer;
ServletContext servletContext = getServletContext();
if (webServer == null && servletContext == null) {
ServletWebServerFactory factory = getWebServerFactory();
this.webServer = factory.getWebServer(getSelfInitializer()); <== 这里创建 Web 服务器
}
else if (servletContext != null) {
try {
getSelfInitializer().onStartup(servletContext);
}
catch (ServletException ex) {
throw new ApplicationContextException("Cannot initialize servlet context", ex);
}
}
initPropertySources();
}
上面代码中调用了factory.getWebServer用于创建Web服务器：
// TomcatServletWebServerFactory 代码片段
@Override
public WebServer getWebServer(ServletContextInitializer... initializers) {
if (this.disableMBeanRegistry) {
Registry.disableRegistry();
}
Tomcat tomcat = new Tomcat();
File baseDir = (this.baseDirectory != null) ? this.baseDirectory : createTempDir("tomcat");
tomcat.setBaseDir(baseDir.getAbsolutePath());
Connector connector = new Connector(this.protocol);
connector.setThrowOnFailure(true);
tomcat.getService().addConnector(connector);
customizeConnector(connector); <== 这里定制 connector
tomcat.setConnector(connector);
tomcat.getHost().setAutoDeploy(false);
configureEngine(tomcat.getEngine());
for (Connector additionalConnector : this.additionalTomcatConnectors) {
tomcat.getService().addConnector(additionalConnector);
}
prepareContext(tomcat.getHost(), initializers);
return getTomcatWebServer(tomcat);
}
上面代码是针对Tomcat创建Web服务器的情况，中间有一步是定制连接器customizeConnector,其实现如下 :
// TomcatServletWebServerFactory 代码片段
protected void customizeConnector(Connector connector) {
int port = Math.max(getPort(), 0);
connector.setPort(port);
if (StringUtils.hasText(this.getServerHeader())) {
connector.setAttribute("server", this.getServerHeader());
}
if (connector.getProtocolHandler() instanceof AbstractProtocol) {
customizeProtocol((AbstractProtocol<?>) connector.getProtocolHandler());
}
invokeProtocolHandlerCustomizers(connector.getProtocolHandler());
if (getUriEncoding() != null) {
connector.setURIEncoding(getUriEncoding().name());
}
// Don't bind to the socket prematurely if ApplicationContext is slow to start
connector.setProperty("bindOnInit", "false");
if (getSsl() != null && getSsl().isEnabled()) {
customizeSsl(connector);
}
// ==> 这里根据 compression 配置创建 TomcatConnectorCustomizer 对象并应用到 connector 上
TomcatConnectorCustomizer compression = new CompressionConnectorCustomizer(getCompression());
compression.customize(connector);
for (TomcatConnectorCustomizer customizer : this.tomcatConnectorCustomizers) {
customizer.customize(connector);
}
}
而CompressionConnectorCustomizer实现如下，从其方法customize可以看出，它将compression参数设置到了connector相应的ProtocolHandler和UpgradeProtocol上。
class CompressionConnectorCustomizer implements TomcatConnectorCustomizer {
private final Compression compression;
CompressionConnectorCustomizer(Compression compression) {
this.compression = compression;
}
@Override
public void customize(Connector connector) {
// 如果 compression 参数不为空，并且 enabled 为 true 时才应用
if (this.compression != null && this.compression.getEnabled()) {
ProtocolHandler handler = connector.getProtocolHandler();
// 仅在 ProtocolHandler 是 AbstractHttp11Protocol 时才应用压缩设置
// 缺省情况下，这里是 Http11NioProtocol
if (handler instanceof AbstractHttp11Protocol) {
customize((AbstractHttp11Protocol<?>) handler);
}
// 如果连接器上存在 Http2Protocol 类型的升级协议 UpgradeProtocol 组件， 则尝试将
// 压缩参数应用到上面
for (UpgradeProtocol upgradeProtocol : connector.findUpgradeProtocols()) {
if (upgradeProtocol instanceof Http2Protocol) {
customize((Http2Protocol) upgradeProtocol);
}
}
}
}
private void customize(Http2Protocol protocol) {
Compression compression = this.compression;
protocol.setCompression("on");
protocol.setCompressionMinSize(getMinResponseSize(compression));
protocol.setCompressibleMimeType(getMimeTypes(compression));
if (this.compression.getExcludedUserAgents() != null) {
protocol.setNoCompressionUserAgents(getExcludedUserAgents());
}
}
private void customize(AbstractHttp11Protocol<?> protocol) {
Compression compression = this.compression;
protocol.setCompression("on");
protocol.setCompressionMinSize(getMinResponseSize(compression));
protocol.setCompressibleMimeType(getMimeTypes(compression));
if (this.compression.getExcludedUserAgents() != null) {
protocol.setNoCompressionUserAgents(getExcludedUserAgents());
}
}
private int getMinResponseSize(Compression compression) {
return (int) compression.getMinResponseSize().toBytes();
}
private String getMimeTypes(Compression compression) {
return StringUtils.arrayToCommaDelimitedString(compression.getMimeTypes());
}
private String getExcludedUserAgents() {
return StringUtils.arrayToCommaDelimitedString(this.compression.getExcludedUserAgents());
}
}
3. 请求到达时添加压缩处理器
当请求到达时，Tomcat Http11NioProtocol会创建Http11Processor用于处理请求。创建Http11Processor位置如下 :
ThreadPoolExecutor$Worker#run // 在tomcat的工作者线程中， 线程名称类似 : http-nio-8080-exec-1
SocketProcessorBase#run
NioEndpoint$SocketProcessor#doRun
AbstractProtocol$ConnectionHandler#process
AbstractHttp11Protocol#createProcessor // 这里对象实例是 Http11NioProtocol
而Http11Processor创建时被调用的构造函数逻辑如下 :
public Http11Processor(AbstractHttp11Protocol<?> protocol, Adapter adapter) {
super(adapter);
this.protocol = protocol;
httpParser = new HttpParser(protocol.getRelaxedPathChars(),
protocol.getRelaxedQueryChars());
inputBuffer = new Http11InputBuffer(request, protocol.getMaxHttpHeaderSize(),
protocol.getRejectIllegalHeaderName(), httpParser);
request.setInputBuffer(inputBuffer);
outputBuffer = new Http11OutputBuffer(response, protocol.getMaxHttpHeaderSize());
response.setOutputBuffer(outputBuffer);
// Create and add the identity filters.
inputBuffer.addFilter(new IdentityInputFilter(protocol.getMaxSwallowSize()));
outputBuffer.addFilter(new IdentityOutputFilter());
// Create and add the chunked filters.
inputBuffer.addFilter(new ChunkedInputFilter(protocol.getMaxTrailerSize(),
protocol.getAllowedTrailerHeadersInternal(), protocol.getMaxExtensionSize(),
protocol.getMaxSwallowSize()));
outputBuffer.addFilter(new ChunkedOutputFilter());
// Create and add the void filters.
inputBuffer.addFilter(new VoidInputFilter());
outputBuffer.addFilter(new VoidOutputFilter());
// Create and add buffered input filter
inputBuffer.addFilter(new BufferedInputFilter());
// Create and add the gzip filters.
//inputBuffer.addFilter(new GzipInputFilter());
outputBuffer.addFilter(new GzipOutputFilter()); <== 注意这里:添加了响应数据压缩过滤器
pluggableFilterIndex = inputBuffer.getFilters().length;
}
4. 响应返回时应用压缩逻辑到响应数据
当Spring MVC处理完一个请求将响应数据冲刷flush到请求端时，Tomcat根据压缩设置决定要不要对响应数据做压缩处理。Spring MVC处理完一个请求冲刷flush响应数据的动作通常位于如下位置 :
// 调用栈（从栈顶到栈底）
copy, StreamUtils (org.springframework.util)
writeContent, ResourceHttpMessageConverter (org.springframework.http.converter)
writeInternal, ResourceHttpMessageConverter (org.springframework.http.converter)
writeInternal, ResourceHttpMessageConverter (org.springframework.http.converter)
write, AbstractHttpMessageConverter (org.springframework.http.converter)
handleRequest, ResourceHttpRequestHandler (org.springframework.web.servlet.resource)
handle, HttpRequestHandlerAdapter (org.springframework.web.servlet.mvc)
doDispatch, DispatcherServlet (org.springframework.web.servlet)
doService, DispatcherServlet (org.springframework.web.servlet)
processRequest, FrameworkServlet (org.springframework.web.servlet)
doGet, FrameworkServlet (org.springframework.web.servlet)
service, HttpServlet (javax.servlet.http)
而StreamUtils#copy实际上又会调用Tomcat的CoyoteOutputStream#flush触发写出响应数据，而写出响应数据之前，Tomcat先准备响应对象，这个动作位于Http11Processor#prepareResponse,相应的调用堆栈如下 :
// 调用栈（从栈顶到栈底）
prepareResponse, Http11Processor (org.apache.coyote.http11)
action, AbstractProcessor (org.apache.coyote)
action, Response (org.apache.coyote)
sendHeaders, Response (org.apache.coyote)
doFlush, OutputBuffer (org.apache.catalina.connector)
flush, OutputBuffer (org.apache.catalina.connector)
flush, CoyoteOutputStream (org.apache.catalina.connector)
flush, OnCommittedResponseWrapper$SaveContextServletOutputStream (org.springframework.session.web.http)
copy, StreamUtils (org.springframework.util)
我们来看看Http11Processor#prepareResponse的逻辑 :
// Http11Processor 代码片段
@Override
protected final void prepareResponse() throws IOException {
// ...
boolean useCompression = false;
if (entityBody && sendfileData == null) {
// 使用 protocol.useCompression 结合配置检测针对当前请求
// 响应决定是否使用压缩
useCompression = protocol.useCompression(request, response);
}
// ...
if (useCompression) {
// 如果要使用压缩，则将压缩过滤器添加为 outputBuffer 的一个活跃过滤器,
// 也就是随后写响应数据要使用的一个过滤器
outputBuffer.addActiveFilter(outputFilters[Constants.GZIP_FILTER]);
}
// ...
}
Http11Processor#prepareResponse的逻辑相对较多，我忽略了其中跟压缩无关的部分，仅仅保留了上面跟压缩相关的部分，由这部分代码可见，Http11Processor会根据配置和当前请求响应上下文决定是否要使用压缩，如果要使用的话，它会将压缩过滤器，实际上也就是GzipOutputFilter，应用到随后的响应数据输出上。