在《Tomcat7.0源码分析——请求原理分析(上)》一文中已经介绍了关于Tomcat7.0处理请求前作的初始化和准备工作,请读者在阅读本文前确保掌握《Tomcat7.0源码分析——请求原理分析(上)》一文中的相关知识以及HTTP协议和TCP协议的一些内容。本文重点讲解Tomcat7.0在准备好接受请求后,请求过程的原理分析。
在正式开始之前,我们先来看看图1中的Tomcat请求处理架构。
图1 Tomcat请求处理架构
图1列出了Tomcat请求处理架构中的主要组件,这里对它们做个简单介绍:
在《Tomcat7.0源码分析——请求原理分析(上)》一文中我们曾经介绍过JIoEndpoint的内部类Acceptor,Acceptor实现了Runnable接口。Acceptor作为后台线程不断循环,每次循环都会接收来自浏览器的Socket连接(用户在浏览器输入HTTP请求地址后,浏览器底层实际使用Socket通信的),最后将Socket交给外部类JIoEndpoint的processSocket方法(见代码清单1)处理。
代码清单1
/**
* Process given socket.
*/
protected boolean processSocket(Socket socket) {
try {
SocketWrapper wrapper = new SocketWrapper(socket);
wrapper.setKeepAliveLeft(getMaxKeepAliveRequests());
getExecutor().execute(new SocketProcessor(wrapper));
} catch (RejectedExecutionException x) {
log.warn("Socket processing request was rejected for:"+socket,x);
return false;
} catch (Throwable t) {
// This means we got an OOM or similar creating a thread, or that
// the pool and its queue are full
log.error(sm.getString("endpoint.process.fail"), t);
return false;
}
return true;
}
根据代码清单1,JIoEndpoint的processSocket方法的处理步骤如下:
/**
* Max keep alive requests
*/
private int maxKeepAliveRequests=100; // as in Apache HTTPD server
public int getMaxKeepAliveRequests() {
return maxKeepAliveRequests;
}
代码清单3
/**
* This class is the equivalent of the Worker, but will simply use in an
* external Executor thread pool.
*/
protected class SocketProcessor implements Runnable {
protected SocketWrapper socket = null;
protected SocketStatus status = null;
public SocketProcessor(SocketWrapper socket) {
if (socket==null) throw new NullPointerException();
this.socket = socket;
}
public SocketProcessor(SocketWrapper socket, SocketStatus status) {
this(socket);
this.status = status;
}
public void run() {
boolean launch = false;
try {
if (!socket.processing.compareAndSet(false, true)) {
log.error("Unable to process socket. Invalid state.");
return;
}
SocketState state = SocketState.OPEN;
// Process the request from this socket
if ( (!socket.isInitialized()) && (!setSocketOptions(socket.getSocket())) ) {
state = SocketState.CLOSED;
}
socket.setInitialized(true);
if ( (state != SocketState.CLOSED) ) {
state = (status==null)?handler.process(socket):handler.process(socket,status);
}
if (state == SocketState.CLOSED) {
// Close socket
if (log.isTraceEnabled()) {
log.trace("Closing socket:"+socket);
}
try {
socket.getSocket().close();
} catch (IOException e) {
// Ignore
}
} else if (state == SocketState.OPEN){
socket.setKeptAlive(true);
socket.access();
//keepalive connection
//TODO - servlet3 check async status, we may just be in a hold pattern
launch = true;
} else if (state == SocketState.LONG) {
socket.access();
waitingRequests.add(socket);
}
} finally {
socket.processing.set(false);
if (launch) getExecutor().execute(new SocketProcessor(socket));
socket = null;
}
// Finish up this request
}
}
SocketProcessor线程专门用于处理Acceptor转交的Socket,其执行步骤如下:
/**
* Set the options for the current socket.
*/
protected boolean setSocketOptions(Socket socket) {
// Process the connection
try {
// 1: Set socket options: timeout, linger, etc
socketProperties.setProperties(socket);
} catch (SocketException s) {
//error here is common if the client has reset the connection
if (log.isDebugEnabled()) {
log.debug(sm.getString("endpoint.err.unexpected"), s);
}
// Close the socket
return false;
} catch (Throwable t) {
log.error(sm.getString("endpoint.err.unexpected"), t);
// Close the socket
return false;
}
try {
// 2: SSL handshake
serverSocketFactory.handshake(socket);
} catch (Throwable t) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("endpoint.err.handshake"), t);
}
// Tell to close the socket
return false;
}
return true;
}
代码清单5
public void setProperties(Socket socket) throws SocketException{
if (rxBufSize != null)
socket.setReceiveBufferSize(rxBufSize.intValue());
if (txBufSize != null)
socket.setSendBufferSize(txBufSize.intValue());
if (ooBInline !=null)
socket.setOOBInline(ooBInline.booleanValue());
if (soKeepAlive != null)
socket.setKeepAlive(soKeepAlive.booleanValue());
if (performanceConnectionTime != null && performanceLatency != null &&
performanceBandwidth != null)
socket.setPerformancePreferences(
performanceConnectionTime.intValue(),
performanceLatency.intValue(),
performanceBandwidth.intValue());
if (soReuseAddress != null)
socket.setReuseAddress(soReuseAddress.booleanValue());
if (soLingerOn != null && soLingerTime != null)
socket.setSoLinger(soLingerOn.booleanValue(),
soLingerTime.intValue());
if (soTimeout != null && soTimeout.intValue() >= 0)
socket.setSoTimeout(soTimeout.intValue());
if (tcpNoDelay != null)
socket.setTcpNoDelay(tcpNoDelay.booleanValue());
if (soTrafficClass != null)
socket.setTrafficClass(soTrafficClass.intValue());
}
以Http11ConnectionHandler为例,我们重点分析它是如何进一步处理Socket的。Http11ConnectionHandler的process方法,见代码清单6。
代码清单6
public SocketState process(SocketWrapper socket) {
return process(socket,SocketStatus.OPEN);
}
public SocketState process(SocketWrapper socket, SocketStatus status) {
Http11Processor processor = connections.remove(socket);
boolean recycle = true;
try {
if (processor == null) {
processor = recycledProcessors.poll();
}
if (processor == null) {
processor = createProcessor();
}
processor.action(ActionCode.ACTION_START, null);
if (proto.isSSLEnabled() && (proto.sslImplementation != null)) {
processor.setSSLSupport
(proto.sslImplementation.getSSLSupport(socket.getSocket()));
} else {
processor.setSSLSupport(null);
}
SocketState state = socket.isAsync()?processor.asyncDispatch(status):processor.process(socket);
if (state == SocketState.LONG) {
connections.put(socket, processor);
socket.setAsync(true);
recycle = false;
} else {
connections.remove(socket);
socket.setAsync(false);
}
return state;
} catch(java.net.SocketException e) {
// SocketExceptions are normal
Http11Protocol.log.debug
(sm.getString
("http11protocol.proto.socketexception.debug"), e);
} catch (java.io.IOException e) {
// IOExceptions are normal
Http11Protocol.log.debug
(sm.getString
("http11protocol.proto.ioexception.debug"), e);
}
// Future developers: if you discover any other
// rare-but-nonfatal exceptions, catch them here, and log as
// above.
catch (Throwable e) {
// any other exception or error is odd. Here we log it
// with "ERROR" level, so it will show up even on
// less-than-verbose logs.
Http11Protocol.log.error
(sm.getString("http11protocol.proto.error"), e);
} finally {
// if(proto.adapter != null) proto.adapter.recycle();
// processor.recycle();
if (recycle) {
processor.action(ActionCode.ACTION_STOP, null);
recycledProcessors.offer(processor);
}
}
return SocketState.CLOSED;
}
根据代码清单6,可见Http11ConnectionHandler的process方法的处理步骤如下:
protected Http11Processor createProcessor() {
Http11Processor processor =
new Http11Processor(proto.getMaxHttpHeaderSize(), (JIoEndpoint)proto.endpoint);
processor.setAdapter(proto.adapter);
processor.setMaxKeepAliveRequests(proto.getMaxKeepAliveRequests());
processor.setKeepAliveTimeout(proto.getKeepAliveTimeout());
processor.setTimeout(proto.getTimeout());
processor.setDisableUploadTimeout(proto.getDisableUploadTimeout());
processor.setCompressionMinSize(proto.getCompressionMinSize());
processor.setCompression(proto.getCompression());
processor.setNoCompressionUserAgents(proto.getNoCompressionUserAgents());
processor.setCompressableMimeTypes(proto.getCompressableMimeTypes());
processor.setRestrictedUserAgents(proto.getRestrictedUserAgents());
processor.setSocketBuffer(proto.getSocketBuffer());
processor.setMaxSavePostSize(proto.getMaxSavePostSize());
processor.setServer(proto.getServer());
register(processor);
return processor;
}
根据之前的分析,我们知道Socket的处理方式有异步和同步两种,分别调用Http11Processor的asyncDispatch方法或process方法,我们以同步处理为例,来看看接下来的处理逻辑。
Http11Processor的process方法(见代码清单8)用于同步处理,由于其代码很多,所以此处在代码后面追加一些注释,便于读者理解。这里面有一些关键方法重点拿出来解释下:
RequestInfo rp = request.getRequestProcessor();
rp.setStage(org.apache.coyote.Constants.STAGE_PARSE);
this.socket = socketWrapper;
inputBuffer.setInputStream(socket.getSocket().getInputStream());//设置输入流
outputBuffer.setOutputStream(socket.getSocket().getOutputStream());//设置输出流
int keepAliveLeft = maxKeepAliveRequests>0?socketWrapper.decrementKeepAlive():-1;//保持连接递减
int soTimeout = endpoint.getSoTimeout();//socket超时时间
socket.getSocket().setSoTimeout(soTimeout);//设置超时时间
boolean keptAlive = socketWrapper.isKeptAlive();//是否保持连接
while (started && !error && keepAlive) {
// Parsing the request header
try {
//TODO - calculate timeout based on length in queue (System.currentTimeMills() - wrapper.getLastAccess() is the time in queue)
if (keptAlive) {//是否保持连接
if (keepAliveTimeout > 0) {
socket.getSocket().setSoTimeout(keepAliveTimeout);
}
else if (soTimeout > 0) {
socket.getSocket().setSoTimeout(soTimeout);
}
}
inputBuffer.parseRequestLine(false);//读取请求行
request.setStartTime(System.currentTimeMillis());
keptAlive = true;
if (disableUploadTimeout) {
socket.getSocket().setSoTimeout(soTimeout);
} else {
socket.getSocket().setSoTimeout(timeout);
}
inputBuffer.parseHeaders();//解析请求头
} catch (IOException e) {
error = true;
break;
} catch (Throwable t) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("http11processor.header.parse"), t);
}
// 400 - Bad Request
response.setStatus(400);
adapter.log(request, response, 0);
error = true;
}
if (!error) {
// Setting up filters, and parse some request headers
rp.setStage(org.apache.coyote.Constants.STAGE_ENDED);
try {
prepareRequest();//对请求内容增加过滤器——协议、方法、请求头、host等
} catch (Throwable t) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("http11processor.request.prepare"), t);
}
// 400 - Internal Server Error
response.setStatus(400);
adapter.log(request, response, 0);
error = true;
}
}
if (maxKeepAliveRequests > 0 && keepAliveLeft == 0)
keepAlive = false;
// Process the request in the adapter
if (!error) {
try {
rp.setStage(org.apache.coyote.Constants.STAGE_SERVICE);
adapter.service(request, response); //将进一步处理交给CoyoteAdapter
// Handle when the response was committed before a serious
// error occurred. Throwing a ServletException should both
// set the status to 500 and set the errorException.
// If we fail here, then the response is likely already
// committed, so we can't try and set headers.
if(keepAlive && !error) { // Avoid checking twice.
error = response.getErrorException() != null ||
statusDropsConnection(response.getStatus());
}
} catch (InterruptedIOException e) {
error = true;
} catch (Throwable t) {
log.error(sm.getString("http11processor.request.process"), t);
// 500 - Internal Server Error
response.setStatus(500);
adapter.log(request, response, 0);
error = true;
}
}
// Finish the handling of the request
try {
rp.setStage(org.apache.coyote.Constants.STAGE_ENDINPUT);
// If we know we are closing the connection, don't drain input.
// This way uploading a 100GB file doesn't tie up the thread
// if the servlet has rejected it.
if(error && !async)
inputBuffer.setSwallowInput(false);
if (!async)
endRequest();
} catch (Throwable t) {
log.error(sm.getString("http11processor.request.finish"), t);
// 500 - Internal Server Error
response.setStatus(500);
adapter.log(request, response, 0);
error = true;
}
try {
rp.setStage(org.apache.coyote.Constants.STAGE_ENDOUTPUT);
} catch (Throwable t) {
log.error(sm.getString("http11processor.response.finish"), t);
error = true;
}
// If there was an error, make sure the request is counted as
// and error, and update the statistics counter
if (error) {
response.setStatus(500);
}
request.updateCounters();
rp.setStage(org.apache.coyote.Constants.STAGE_KEEPALIVE);
// Don't reset the param - we'll see it as ended. Next request
// will reset it
// thrA.setParam(null);
// Next request
if (!async || error) {
inputBuffer.nextRequest();
outputBuffer.nextRequest();
}
//hack keep alive behavior
break;
}
rp.setStage(org.apache.coyote.Constants.STAGE_ENDED);
if (error) {
recycle();
return SocketState.CLOSED;
} else if (async) {
return SocketState.LONG;
} else {
if (!keepAlive) {
recycle();
return SocketState.CLOSED;
} else {
return SocketState.OPEN;
}
}
从代码清单8可以看出,最后的请求处理交给了CoyoteAdapter,CoyoteAdapter的service方法(见代码清单9)用于真正处理请求。
代码清单9
/**
* Service method.
*/
public void service(org.apache.coyote.Request req,
org.apache.coyote.Response res)
throws Exception {
Request request = (Request) req.getNote(ADAPTER_NOTES);
Response response = (Response) res.getNote(ADAPTER_NOTES);
if (request == null) {
// Create objects
request = connector.createRequest();
request.setCoyoteRequest(req);
response = connector.createResponse();
response.setCoyoteResponse(res);
// Link objects
request.setResponse(response);
response.setRequest(request);
// Set as notes
req.setNote(ADAPTER_NOTES, request);
res.setNote(ADAPTER_NOTES, response);
// Set query string encoding
req.getParameters().setQueryStringEncoding
(connector.getURIEncoding());
}
if (connector.getXpoweredBy()) {
response.addHeader("X-Powered-By", POWERED_BY);
}
boolean comet = false;
boolean async = false;
try {
// Parse and set Catalina and configuration specific
// request parameters
req.getRequestProcessor().setWorkerThreadName(Thread.currentThread().getName());
if (postParseRequest(req, request, res, response)) {
//check valves if we support async
request.setAsyncSupported(connector.getService().getContainer().getPipeline().isAsyncSupported());
// Calling the container
connector.getService().getContainer().getPipeline().getFirst().invoke(request, response);
if (request.isComet()) {
if (!response.isClosed() && !response.isError()) {
if (request.getAvailable() || (request.getContentLength() > 0 && (!request.isParametersParsed()))) {
// Invoke a read event right away if there are available bytes
if (event(req, res, SocketStatus.OPEN)) {
comet = true;
res.action(ActionCode.ACTION_COMET_BEGIN, null);
}
} else {
comet = true;
res.action(ActionCode.ACTION_COMET_BEGIN, null);
}
} else {
// Clear the filter chain, as otherwise it will not be reset elsewhere
// since this is a Comet request
request.setFilterChain(null);
}
}
}
AsyncContextImpl asyncConImpl = (AsyncContextImpl)request.getAsyncContext();
if (asyncConImpl!=null && asyncConImpl.getState()==AsyncContextImpl.AsyncState.STARTED) {
res.action(ActionCode.ACTION_ASYNC_START, request.getAsyncContext());
async = true;
} else if (request.isAsyncDispatching()) {
asyncDispatch(req, res, SocketStatus.OPEN);
if (request.isAsyncStarted()) {
async = true;
res.action(ActionCode.ACTION_ASYNC_START, request.getAsyncContext());
}
} else if (!comet) {
response.finishResponse();
req.action(ActionCode.ACTION_POST_REQUEST , null);
}
} catch (IOException e) {
// Ignore
} catch (Throwable t) {
log.error(sm.getString("coyoteAdapter.service"), t);
} finally {
req.getRequestProcessor().setWorkerThreadName(null);
// Recycle the wrapper request and response
if (!comet && !async) {
request.recycle();
response.recycle();
} else {
// Clear converters so that the minimum amount of memory
// is used by this processor
request.clearEncoders();
response.clearEncoders();
}
}
}
从代码清单9可以看出,CoyoteAdapter的service方法的执行步骤如下:
/**
* Parse additional request parameters.
*/
protected boolean postParseRequest(org.apache.coyote.Request req,
Request request,
org.apache.coyote.Response res,
Response response)
throws Exception {
// 省略前边的次要代码
parsePathParameters(req, request);
// URI decoding
// %xx decoding of the URL
try {
req.getURLDecoder().convert(decodedURI, false);
} catch (IOException ioe) {
res.setStatus(400);
res.setMessage("Invalid URI: " + ioe.getMessage());
connector.getService().getContainer().logAccess(
request, response, 0, true);
return false;
}
// Normalization
if (!normalize(req.decodedURI())) {
res.setStatus(400);
res.setMessage("Invalid URI");
connector.getService().getContainer().logAccess(
request, response, 0, true);
return false;
}
// Character decoding
convertURI(decodedURI, request);
// Check that the URI is still normalized
if (!checkNormalize(req.decodedURI())) {
res.setStatus(400);
res.setMessage("Invalid URI character encoding");
connector.getService().getContainer().logAccess(
request, response, 0, true);
return false;
}
// Set the remote principal
String principal = req.getRemoteUser().toString();
if (principal != null) {
request.setUserPrincipal(new CoyotePrincipal(principal));
}
// Set the authorization type
String authtype = req.getAuthType().toString();
if (authtype != null) {
request.setAuthType(authtype);
}
// Request mapping.
MessageBytes serverName;
if (connector.getUseIPVHosts()) {
serverName = req.localName();
if (serverName.isNull()) {
// well, they did ask for it
res.action(ActionCode.ACTION_REQ_LOCAL_NAME_ATTRIBUTE, null);
}
} else {
serverName = req.serverName();
}
if (request.isAsyncStarted()) {
//TODO SERVLET3 - async
//reset mapping data, should prolly be done elsewhere
request.getMappingData().recycle();
}
connector.getMapper().map(serverName, decodedURI,
request.getMappingData());
request.setContext((Context) request.getMappingData().context);
request.setWrapper((Wrapper) request.getMappingData().wrapper);
// Filter trace method
if (!connector.getAllowTrace()
&& req.method().equalsIgnoreCase("TRACE")) {
Wrapper wrapper = request.getWrapper();
String header = null;
if (wrapper != null) {
String[] methods = wrapper.getServletMethods();
if (methods != null) {
for (int i=0; i
从代码清单10可以看出,postParseRequest方法的执行步骤如下:
public void map(MessageBytes host, MessageBytes uri,
MappingData mappingData)
throws Exception {
if (host.isNull()) {
host.getCharChunk().append(defaultHostName);
}
host.toChars();
uri.toChars();
internalMap(host.getCharChunk(), uri.getCharChunk(), mappingData);
}
CoyoteAdapter的service方法最后会将请求交给Engine的Pipeline去处理,我将在《Tomcat7.0源码分析——请求原理分析(下)》一文中具体讲解。
后记:个人总结整理的《深入理解Spark:核心思想与源码分析》一书现在已经正式出版上市,目前京东、当当、天猫等网站均有销售,欢迎感兴趣的同学购买。
京东:http://item.jd.com/11846120.html
当当:http://product.dangdang.com/23838168.html