tomcat原理解析(五)：http请求处理
一 概述
好久没坐地铁上班了，地铁人真多。最近公司搬到虹口足球场了，上班得花70分钟，早7点40就要起床还真有点不适应。不过公司还挺人性化，上班时间从9点半延迟到了10点，只要一天能工作8个小时就可以，每天可以不用在路上赶时间害怕迟到啦。开始进入主题啦！前面章节已经讲到了tomcat服务开启了sokcet监听，现在只要浏览器发起http请求，就能响应服务了。
二http请求处理
1.在浏览器中输入ip和端口开始请求服务，相当于在浏览器中输入ip和端口发起对tomcat的socket的请求。服务端接受客户端socket的请求，查看入口代码如下：protected class Acceptor implements Runnable {
/**
* The background thread that listens for incoming TCP/IP connections and
* hands them off to an appropriate processor.
*/
public void run() {
// Loop until we receive a shutdown command
while (running) {
// Loop if endpoint is paused
while (paused) {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// Ignore
}
}
// Accept the next incoming connection from the server socket
try {
Socket socket = serverSocketFactory.acceptSocket(serverSocket); //服务器接受请求生成socket对象
serverSocketFactory.initSocket(socket);
// Hand this socket off to an appropriate processor
if (!processSocket(socket)) {
// Close socket right away
try {
socket.close();
} catch (IOException e) {
// Ignore
}
}
}catch ( IOException x ) {
if ( running ) log.error(sm.getString("endpoint.accept.fail"), x);
} catch (Throwable t) {
log.error(sm.getString("endpoint.accept.fail"), t);
}
// The processor will recycle itself when it finishes
}
}
}
查看里面的Socket socket = serverSocketFactory.acceptSocket(serverSocket); serverSocketFactory对象是DefaultServerSocketFactory类，跟进到DefaultServerSocketFactory类中查看代码中的acceptSocket方法实现，貌似很熟悉啊！就是java的socket对象生成代码！没什么神秘感。看看类中的其它方法都是比较熟悉的实现，一看即懂。
class DefaultServerSocketFactory extends ServerSocketFactory {
DefaultServerSocketFactory () {
/* NOTHING */
}
@Override
public ServerSocket createSocket (int port)
throws IOException {
return new ServerSocket (port);
}
@Override
public ServerSocket createSocket (int port, int backlog)
throws IOException {
return new ServerSocket (port, backlog);
}
@Override
public ServerSocket createSocket (int port, int backlog,
InetAddress ifAddress)
throws IOException {
return new ServerSocket (port, backlog, ifAddress);
}
@Override
public Socket acceptSocket(ServerSocket socket)
throws IOException {
return socket.accept();
}
@Override
public void handshake(Socket sock)
throws IOException {
// NOOP
}
} protected boolean processSocket(Socket socket) {
try {
SocketWrapper wrapper = new SocketWrapper(socket);
wrapper.setKeepAliveLeft(getMaxKeepAliveRequests());
getExecutor().execute(new SocketProcessor(wrapper));
} catch (RejectedExecutionException x) {
log.warn("Socket processing request was rejected for:"+socket,x);
return false;
} catch (Throwable t) {
// This means we got an OOM or similar creating a thread, or that
// the pool and its queue are full
log.error(sm.getString("endpoint.process.fail"), t);
return false;
}
return true;
}
将传入的socket对象SocketWrapper类中并保存了起来，类中还存在其它一些属性，估计是socket链接操作的一些属性值.比如保持长连接的最大时长(keepAliveLeft),同步异步(async)，(keptAlive)是否保存长链。
继续跟进代码getExecutor().execute(new SocketProcessor(wrapper)); 获取一个线程池，然后新建一个SocketProcessor线程，并传入包装好的SocketWrapper对象。看到这里，也就解释了http的请求是基于socket的，而且所有的链接处理是在SocketProcessor线程中完成。SocketProcessor类代码如下：
protected class SocketProcessor implements Runnable {
protected SocketWrapper socket = null;
protected SocketStatus status = null;
public SocketProcessor(SocketWrapper socket) {
if (socket==null) throw new NullPointerException();
this.socket = socket; //构造函数传入socket对象
}
public SocketProcessor(SocketWrapper socket, SocketStatus status) {
this(socket);
this.status = status;
}
public void run() { //线程处理入口
boolean launch = false;
try {
if (!socket.processing.compareAndSet(false, true)) {
log.error("Unable to process socket. Invalid state.");
return;
}
SocketState state = SocketState.OPEN;
// Process the request from this socket
if ( (!socket.isInitialized()) && (!setSocketOptions(socket.getSocket())) ) {
state = SocketState.CLOSED;
}
socket.setInitialized(true);
//前面做了一系列的socket状态判断，
if ( (state != SocketState.CLOSED) ) {
state = (status==null)?handler.process(socket):handler.process(socket,status);//socket链接处理的真正入口处
}
if (state == SocketState.CLOSED) {
// Close socket
if (log.isTraceEnabled()) {
log.trace("Closing socket:"+socket);
}
try {
socket.getSocket().close();
} catch (IOException e) {
// Ignore
}
} else if (state == SocketState.OPEN){
socket.setKeptAlive(true);
socket.access();
//keepalive connection
//TODO - servlet3 check async status, we may just be in a hold pattern
launch = true;
} else if (state == SocketState.LONG) {
socket.access();
waitingRequests.add(socket);
}
} finally {
socket.processing.set(false);
if (launch) getExecutor().execute(new SocketProcessor(socket));
socket = null;
}
// Finish up this request
}
}
链接处理的代码入口如：state = (status==null)?handler.process(socket):handler.process(socket,status); 这里的handler就是链接处理对象
Http11ConnectionHandler类。调用了该类的process方法，并传入socket链接对象。Http11ConnectionHandler类代码如下：
protected static class Http11ConnectionHandler implements Handler {
protected Http11Protocol proto;
protected AtomicLong registerCount = new AtomicLong(0);
protected RequestGroupInfo global = new RequestGroupInfo();
protected ConcurrentHashMap, Http11Processor> connections = new ConcurrentHashMap, Http11Processor>();
protected ConcurrentLinkedQueue recycledProcessors = new ConcurrentLinkedQueue() {
private static final long serialVersionUID = 1L;
protected AtomicInteger size = new AtomicInteger(0);
@Override
public boolean offer(Http11Processor processor) {
boolean offer = (proto.getProcessorCache() == -1) ? true : (size.get() < proto.getProcessorCache());
//avoid over growing our cache or add after we have stopped
boolean result = false;
if ( offer ) {
result = super.offer(processor);
if ( result ) {
size.incrementAndGet();
}
}
if (!result) unregister(processor);
return result;
}
@Override
public Http11Processor poll() {
Http11Processor result = super.poll();
if ( result != null ) {
size.decrementAndGet();
}
return result;
}
@Override
public void clear() {
Http11Processor next = poll();
while ( next != null ) {
unregister(next);
next = poll();
}
super.clear();
size.set(0);
}
};
Http11ConnectionHandler(Http11Protocol proto) {
this.proto = proto;
}
public SocketState process(SocketWrapper socket) {
return process(socket,SocketStatus.OPEN);
}
public SocketState process(SocketWrapper socket, SocketStatus status) {
Http11Processor processor = connections.remove(socket);//connections一个高并发的map对象，里面以socket为key，Http11Processor为对象。说明connections中保存了所有的链接socket对象和socket的处理器对象。
boolean recycle = true;
try {
if (processor == null) {
processor = recycledProcessors.poll();//高并发队列中取出一个Http11Processor对象
}
if (processor == null) {
processor = createProcessor();//为空就创建一个Http11Processor对象
}
processor.action(ActionCode.ACTION_START, null);
//没看懂这里是做什么
if (proto.isSSLEnabled() && (proto.sslImplementation != null)) {
processor.setSSLSupport
(proto.sslImplementation.getSSLSupport(socket.getSocket()));
} else {
processor.setSSLSupport(null);
}
SocketState state = socket.isAsync()?processor.asyncDispatch(status):processor.process(socket);//判断socket是异步的请求还是同步的请求，这里我只看了同步的请求所以调用的是processor.process(socket)方法
if (state == SocketState.LONG) {
connections.put(socket, processor);
socket.setAsync(true);
recycle = false;
} else {
connections.remove(socket);
socket.setAsync(false);
}
return state;
} catch(java.net.SocketException e) {
// SocketExceptions are normal
Http11Protocol.log.debug
(sm.getString
("http11protocol.proto.socketexception.debug"), e);
} catch (java.io.IOException e) {
// IOExceptions are normal
Http11Protocol.log.debug
(sm.getString
("http11protocol.proto.ioexception.debug"), e);
}
// Future developers: if you discover any other
// rare-but-nonfatal exceptions, catch them here, and log as
// above.
catch (Throwable e) {
// any other exception or error is odd. Here we log it
// with "ERROR" level, so it will show up even on
// less-than-verbose logs.
Http11Protocol.log.error
(sm.getString("http11protocol.proto.error"), e);
} finally {
// if(proto.adapter != null) proto.adapter.recycle();
// processor.recycle();
if (recycle) {
processor.action(ActionCode.ACTION_STOP, null);
recycledProcessors.offer(processor);
}
}
return SocketState.CLOSED;
}
protected Http11Processor createProcessor() {
Http11Processor processor =
new Http11Processor(proto.getMaxHttpHeaderSize(), (JIoEndpoint)proto.endpoint);
processor.setAdapter(proto.adapter);
processor.setMaxKeepAliveRequests(proto.getMaxKeepAliveRequests());
processor.setKeepAliveTimeout(proto.getKeepAliveTimeout());
processor.setTimeout(proto.getTimeout());
processor.setDisableUploadTimeout(proto.getDisableUploadTimeout());
processor.setCompressionMinSize(proto.getCompressionMinSize());
processor.setCompression(proto.getCompression());
processor.setNoCompressionUserAgents(proto.getNoCompressionUserAgents());
processor.setCompressableMimeTypes(proto.getCompressableMimeTypes());
processor.setRestrictedUserAgents(proto.getRestrictedUserAgents());
processor.setSocketBuffer(proto.getSocketBuffer());
processor.setMaxSavePostSize(proto.getMaxSavePostSize());
processor.setServer(proto.getServer());
register(processor);
return processor;
}
protected void register(Http11Processor processor) {
if (proto.getDomain() != null) {
synchronized (this) {
try {
long count = registerCount.incrementAndGet();
RequestInfo rp = processor.getRequest().getRequestProcessor();
rp.setGlobalProcessor(global);
ObjectName rpName = new ObjectName
(proto.getDomain() + ":type=RequestProcessor,worker="
+ proto.getName() + ",name=HttpRequest" + count);
if (log.isDebugEnabled()) {
log.debug("Register " + rpName);
}
Registry.getRegistry(null, null).registerComponent(rp, rpName, null);
rp.setRpName(rpName);
} catch (Exception e) {
log.warn("Error registering request");
}
}
}
}
protected void unregister(Http11Processor processor) {
if (proto.getDomain() != null) {
synchronized (this) {
try {
RequestInfo rp = processor.getRequest().getRequestProcessor();
rp.setGlobalProcessor(null);
ObjectName rpName = rp.getRpName();
if (log.isDebugEnabled()) {
log.debug("Unregister " + rpName);
}
Registry.getRegistry(null, null).unregisterComponent(rpName);
rp.setRpName(null);
} catch (Exception e) {
log.warn("Error unregistering request", e);
}
}
}
}
}
Http11Processor类中的process方法如下代码：
这里主要是拿到socket对象的输入和输出流，并设置到InternalInputBuffer对象中。我们知道socket通信都是byte[]类型的数据。我们现在拿到了socekt链接对象，要获取http请求时传递给服务器的消息数据，比如请求的方法名称，请求的参数，请求的项目名等这些重要数据。这些请求都是遵守http协议的。所以需要将浏览器发过来的byte数据分析处理。InternalInputBuffer类已经实现了这些功能，它将输入的二进制流数据按http协议规则，将传入的参数数据解析出来并存在request对象中，根据这些参数就可以很好的定位请求资源了。
public SocketState process(SocketWrapper socketWrapper)
throws IOException {
RequestInfo rp = request.getRequestProcessor();
rp.setStage(org.apache.coyote.Constants.STAGE_PARSE);
// Set the remote address
remoteAddr = null;
remoteHost = null;
localAddr = null;
localName = null;
remotePort = -1;
localPort = -1;
// Setting up the I/O
this.socket = socketWrapper;
inputBuffer.setInputStream(socket.getSocket().getInputStream());//使用InternalInputBuffer类对socket的输入流做了一次封装
outputBuffer.setOutputStream(socket.getSocket().getOutputStream());//使用InternalOutputBuffer类对socket的输出流做了一次封装
// Error flag
error = false;
keepAlive = true;
int keepAliveLeft = maxKeepAliveRequests>0?socketWrapper.decrementKeepAlive():-1;
int soTimeout = endpoint.getSoTimeout();//socket超时时间
try {
socket.getSocket().setSoTimeout(soTimeout);//设置超时时间
} catch (Throwable t) {
log.debug(sm.getString("http11processor.socket.timeout"), t);
error = true;
}
boolean keptAlive = socketWrapper.isKeptAlive();//是否保持连接
while (started && !error && keepAlive) {
// Parsing the request header
try {
//TODO - calculate timeout based on length in queue (System.currentTimeMills() - wrapper.getLastAccess() is the time in queue)
if (keptAlive) {//是否保持连接
if (keepAliveTimeout > 0) {
socket.getSocket().setSoTimeout(keepAliveTimeout);
}
else if (soTimeout > 0) {
socket.getSocket().setSoTimeout(soTimeout);
}
}
inputBuffer.parseRequestLine(false);//根据http协议规则，读取输入流中的请求字节码，并分析出http的请求方法名，请求的资源地址，请求的协议类型。所有分析出的数据都存储在request对象中。查看InternalInputBuffer中的代码将解析的http协议的数据解析功能代码封装在类中。
request.setStartTime(System.currentTimeMillis());
keptAlive = true;
if (disableUploadTimeout) {
socket.getSocket().setSoTimeout(soTimeout);
} else {
socket.getSocket().setSoTimeout(timeout);
}
inputBuffer.parseHeaders();//解析请求头
} catch (IOException e) {
error = true;
break;
} catch (Throwable t) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("http11processor.header.parse"), t);
}
// 400 - Bad Request
response.setStatus(400);
adapter.log(request, response, 0);
error = true;
}
if (!error) {
// Setting up filters, and parse some request headers
rp.setStage(org.apache.coyote.Constants.STAGE_PREPARE);
try {
prepareRequest();//对请求内容增加过滤器——协议、方法、请求头、host等
} catch (Throwable t) {
if (log.isDebugEnabled()) {
log.debug(sm.getString("http11processor.request.prepare"), t);
}
// 400 - Internal Server Error
response.setStatus(400);
adapter.log(request, response, 0);
error = true;
}
}
if (maxKeepAliveRequests > 0 && keepAliveLeft == 0)
keepAlive = false;
// Process the request in the adapter
if (!error) {
try {
rp.setStage(org.apache.coyote.Constants.STAGE_SERVICE);
adapter.service(request, response);//将进一步处理交给CoyoteAdapter。这里的request对象中包含了http请求资源所有相关的数据，该数据都是通过InternalInputBuffer分析计算出来的。也就是说这里包含了一个http请求的方法名，方法参数，请求协议等重要参数，tomcat服务器就是根据这些参数来定位客户端请求的资源
// Handle when the response was committed before a serious
// error occurred. Throwing a ServletException should both
// set the status to 500 and set the errorException.
// If we fail here, then the response is likely already
// committed, so we can't try and set headers.
if(keepAlive && !error) { // Avoid checking twice.
error = response.getErrorException() != null ||
statusDropsConnection(response.getStatus());
}
} catch (InterruptedIOException e) {
error = true;
} catch (Throwable t) {
log.error(sm.getString("http11processor.request.process"), t);
// 500 - Internal Server Error
response.setStatus(500);
adapter.log(request, response, 0);
error = true;
}
}
// Finish the handling of the request
try {
rp.setStage(org.apache.coyote.Constants.STAGE_ENDINPUT);
// If we know we are closing the connection, don't drain input.
// This way uploading a 100GB file doesn't tie up the thread
// if the servlet has rejected it.
if(error && !async)
inputBuffer.setSwallowInput(false);
if (!async)
endRequest();
} catch (Throwable t) {
log.error(sm.getString("http11processor.request.finish"), t);
// 500 - Internal Server Error
response.setStatus(500);
adapter.log(request, response, 0);
error = true;
}
try {
rp.setStage(org.apache.coyote.Constants.STAGE_ENDOUTPUT);
} catch (Throwable t) {
log.error(sm.getString("http11processor.response.finish"), t);
error = true;
}
// If there was an error, make sure the request is counted as
// and error, and update the statistics counter
if (error) {
response.setStatus(500);
}
request.updateCounters();
rp.setStage(org.apache.coyote.Constants.STAGE_KEEPALIVE);
// Don't reset the param - we'll see it as ended. Next request
// will reset it
// thrA.setParam(null);
// Next request
if (!async || error) {
inputBuffer.nextRequest();
outputBuffer.nextRequest();
}
//hack keep alive behavior
break;
}
rp.setStage(org.apache.coyote.Constants.STAGE_ENDED);
if (error) {
recycle();
return SocketState.CLOSED;
} else if (async) {
return SocketState.LONG;
} else {
if (!keepAlive) {
recycle();
return SocketState.CLOSED;
} else {
return SocketState.OPEN;
}
}
}
继续跟进CoyoteAdapter类的service方法
/**
* Service method.
*/
public void service(org.apache.coyote.Request req,
org.apache.coyote.Response res)
throws Exception {
Request request = (Request) req.getNote(ADAPTER_NOTES);//判断是否绑定了HttpServletRequest
Response response = (Response) res.getNote(ADAPTER_NOTES);//判断是否绑定了HttpServletResponse
if (request == null) {
// Create objects
request = connector.createRequest();//连接器创建一个http请求对象request
request.setCoyoteRequest(req);//设置该http请求对象的真正req对象
response = connector.createResponse();
response.setCoyoteResponse(res);
// Link objects
request.setResponse(response);//将请求对象与响应对象绑定相关联
response.setRequest(request);
// Set as notes
req.setNote(ADAPTER_NOTES, request);
res.setNote(ADAPTER_NOTES, response);
// Set query string encoding
req.getParameters().setQueryStringEncoding(connector.getURIEncoding());
}
if (connector.getXpoweredBy()) {
response.addHeader("X-Powered-By", POWERED_BY);
}
boolean comet = false;
boolean async = false;
try {
// Parse and set Catalina and configuration specific
// request parameters
req.getRequestProcessor().setWorkerThreadName(Thread.currentThread().getName());
if (postParseRequest(req, request, res, response)) {//将当前http请求对象request与相关的host,Context、Wrapper对象引用相绑定。
//check valves if we support async
request.setAsyncSupported(connector.getService().getContainer().getPipeline().isAsyncSupported());
// Calling the container
connector.getService().getContainer().getPipeline().getFirst().invoke(request, response);//将真正的请求处理交给Engine的Pipeline去处理
if (request.isComet()) {
if (!response.isClosed() && !response.isError()) {
if (request.getAvailable() || (request.getContentLength() > 0 && (!request.isParametersParsed()))) {
// Invoke a read event right away if there are available bytes
if (event(req, res, SocketStatus.OPEN)) {
comet = true;
res.action(ActionCode.ACTION_COMET_BEGIN, null);
}
} else {
comet = true;
res.action(ActionCode.ACTION_COMET_BEGIN, null);
}
} else {
// Clear the filter chain, as otherwise it will not be reset elsewhere
// since this is a Comet request
request.setFilterChain(null);
}
}
}
AsyncContextImpl asyncConImpl = (AsyncContextImpl)request.getAsyncContext();
if (asyncConImpl!=null && asyncConImpl.getState()==AsyncContextImpl.AsyncState.STARTED) {
res.action(ActionCode.ACTION_ASYNC_START, request.getAsyncContext());
async = true;
} else if (request.isAsyncDispatching()) {
asyncDispatch(req, res, SocketStatus.OPEN);
if (request.isAsyncStarted()) {
async = true;
res.action(ActionCode.ACTION_ASYNC_START, request.getAsyncContext());
}
} else if (!comet) {
response.finishResponse();
req.action(ActionCode.ACTION_POST_REQUEST , null);
}
} catch (IOException e) {
// Ignore
} catch (Throwable t) {
log.error(sm.getString("coyoteAdapter.service"), t);
} finally {
req.getRequestProcessor().setWorkerThreadName(null);
// Recycle the wrapper request and response
if (!comet && !async) {
request.recycle();
response.recycle();
} else {
// Clear converters so that the minimum amount of memory
// is used by this processor
request.clearEncoders();
response.clearEncoders();
}
}
}
到这里tomcat已经完全的将一次tcp的请求转换成了我们常见HttpServletRequest对象，request对象中包含了所有与本次请求相关的Host、Context、Wrapper对象。封装完HttpServletRequest对象后，就开始了走管道处理如以下代码：connector.getService().getContainer().getPipeline().getFirst().invoke(request, response);
三总结
从浏览器请求到后台tomcat的连接器的(connector)处理，基本可以理解为connector是如何管理tcp链接请求，然后又是如何将二进制的http协议请求数据解析，然后将相关联的数据和请求资源包装到我们常见的HttpServletRequest对象中，再根据请求资源标记映射到相关资源