现在作为一个开发人员,http server相关的内容已经是无论如何都要了解的知识了。用curl发一个请求,配置一下apache,部署一个web server对我们来说都不是很难,但要想搞清楚这些背后都发生了什么技术细节还真不是很简单的。所以新的系列将是分享我学习Http Server的过程。
NanoHttpd是Github上的一个开源项目,号称只用一个java文件就能创建一个http server,我将通过分析NanoHttpd的源码解析如何开发自己的HttpServer。Github 地址:https://github.com/NanoHttpd/nanohttpd
在开始前首先简单说明HttpServer的基本要素:
1.能接受HttpRequest并返回HttpResponse
2.满足一个Server的基本特征,能够长时间运行
关于Http协议一般HttpServer都会声明支持Http协议的哪些特性,nanohttpd作为一个轻量级的httpserver只实现了最简单、最常用的功能,不过我们依然可以从中学习很多。
首先看下NanoHttpd类的start函数
public void start() throws IOException { myServerSocket = new ServerSocket(); myServerSocket.bind((hostname != null) ? new InetSocketAddress(hostname, myPort) : new InetSocketAddress(myPort)); myThread = new Thread(new Runnable() { @Override public void run() { do { try { final Socket finalAccept = myServerSocket.accept(); registerConnection(finalAccept); finalAccept.setSoTimeout(SOCKET_READ_TIMEOUT); final InputStream inputStream = finalAccept.getInputStream(); asyncRunner.exec(new Runnable() { @Override public void run() { OutputStream outputStream = null; try { outputStream = finalAccept.getOutputStream(); TempFileManager tempFileManager = tempFileManagerFactory.create(); HTTPSession session = new HTTPSession(tempFileManager, inputStream, outputStream, finalAccept.getInetAddress()); while (!finalAccept.isClosed()) { session.execute(); } } catch (Exception e) { // When the socket is closed by the client, we throw our own SocketException // to break the "keep alive" loop above. if (!(e instanceof SocketException && "NanoHttpd Shutdown".equals(e.getMessage()))) { e.printStackTrace(); } } finally { safeClose(outputStream); safeClose(inputStream); safeClose(finalAccept); unRegisterConnection(finalAccept); } } }); } catch (IOException e) { } } while (!myServerSocket.isClosed()); } }); myThread.setDaemon(true); myThread.setName("NanoHttpd Main Listener"); myThread.start(); }1.创建ServerSocket,bind制定端口
2.创建主线程,主线程负责和client建立连接
3.建立连接后会生成一个runnable对象放入asyncRunner中,asyncRunner.exec会创建一个线程来处理新生成的连接。
4.新线程首先创建了一个HttpSession,然后while(true)的执行httpSession.exec。
这里介绍下HttpSession的概念,HttpSession是java里Session概念的实现,简单来说一个Session就是一次httpClient->httpServer的连接,当连接close后session就结束了,如果没结束则session会一直存在。这点从这里的代码也能看到:如果socket不close或者exec没有抛出异常(异常有可能是client段断开连接)session会一直执行exec方法。
一个HttpSession中存储了一次网络连接中server应该保存的信息,比如:URI,METHOD,PARAMS,HEADERS,COOKIES等。
5.这里accept一个client的socket就创建一个独立线程的server模型是ThreadServer模型,特点是一个connection就会创建一个thread,是比较简单、常见的socket server实现。缺点是在同时处理大量连接时线程切换需要消耗大量的资源,如果有兴趣可以了解更加高效的NIO实现方式。
当获得client的socket后自然要开始处理client发送的httprequest。
Http Request Header的parse:
// Read the first 8192 bytes. // The full header should fit in here. // Apache's default header limit is 8KB. // Do NOT assume that a single read will get the entire header at once! byte[] buf = new byte[BUFSIZE]; splitbyte = 0; rlen = 0; { int read = -1; try { read = inputStream.read(buf, 0, BUFSIZE); } catch (Exception e) { safeClose(inputStream); safeClose(outputStream); throw new SocketException("NanoHttpd Shutdown"); } if (read == -1) { // socket was been closed safeClose(inputStream); safeClose(outputStream); throw new SocketException("NanoHttpd Shutdown"); } while (read > 0) { rlen += read; splitbyte = findHeaderEnd(buf, rlen); if (splitbyte > 0) break; read = inputStream.read(buf, rlen, BUFSIZE - rlen); } }1.读取socket数据流的前8192个字节,因为http协议中头部最长为8192
2.通过findHeaderEnd函数找到header数据的截止位置,并把位置保存到splitbyte内。
if (splitbyte < rlen) { inputStream.unread(buf, splitbyte, rlen - splitbyte); } parms = new HashMap<String, String>(); if(null == headers) { headers = new HashMap<String, String>(); } // Create a BufferedReader for parsing the header. BufferedReader hin = new BufferedReader(new InputStreamReader(new ByteArrayInputStream(buf, 0, rlen))); // Decode the header into parms and header java properties Map<String, String> pre = new HashMap<String, String>(); decodeHeader(hin, pre, parms, headers);
1.使用unread函数将之前读出来的body pushback回去,这里使用了pushbackstream,用法比较巧妙,因为一旦读到了header的尾部就需要进入下面的逻辑来判断是否需要再读下去了,而不应该一直读,读到没有数据为止
2.decodeHeader,将byte的header转换为java对象
private int findHeaderEnd(final byte[] buf, int rlen) { int splitbyte = 0; while (splitbyte + 3 < rlen) { if (buf[splitbyte] == '\r' && buf[splitbyte + 1] == '\n' && buf[splitbyte + 2] == '\r' && buf[splitbyte + 3] == '\n') { return splitbyte + 4; } splitbyte++; } return 0; }1.http协议规定header和body之间使用两个回车换行分割
private void decodeHeader(BufferedReader in, Map<String, String> pre, Map<String, String> parms, Map<String, String> headers) throws ResponseException { try { // Read the request line String inLine = in.readLine(); if (inLine == null) { return; } StringTokenizer st = new StringTokenizer(inLine); if (!st.hasMoreTokens()) { throw new ResponseException(Response.Status.BAD_REQUEST, "BAD REQUEST: Syntax error. Usage: GET /example/file.html"); } pre.put("method", st.nextToken()); if (!st.hasMoreTokens()) { throw new ResponseException(Response.Status.BAD_REQUEST, "BAD REQUEST: Missing URI. Usage: GET /example/file.html"); } String uri = st.nextToken(); // Decode parameters from the URI int qmi = uri.indexOf('?'); if (qmi >= 0) { decodeParms(uri.substring(qmi + 1), parms); uri = decodePercent(uri.substring(0, qmi)); } else { uri = decodePercent(uri); } // If there's another token, it's protocol version, // followed by HTTP headers. Ignore version but parse headers. // NOTE: this now forces header names lowercase since they are // case insensitive and vary by client. if (st.hasMoreTokens()) { String line = in.readLine(); while (line != null && line.trim().length() > 0) { int p = line.indexOf(':'); if (p >= 0) headers.put(line.substring(0, p).trim().toLowerCase(Locale.US), line.substring(p + 1).trim()); line = in.readLine(); } } pre.put("uri", uri); } catch (IOException ioe) { throw new ResponseException(Response.Status.INTERNAL_ERROR, "SERVER INTERNAL ERROR: IOException: " + ioe.getMessage(), ioe); } }1.Http协议第一行是Method URI HTTP_VERSION
2.后面每行都是KEY:VALUE格式的header
3.uri需要经过URIDecode处理后才能使用
4.uri中如果包含?则表示有param,httprequest的param一般表现为:/index.jsp?username=xiaoming&id=2
下面是处理cookie,不过这里cookie的实现较为简单,所以跳过。之后是serve方法,serve方法提供了用户自己实现httpserver具体逻辑的很好接口。在NanoHttpd中的serve方法实现了一个默认的简单处理功能。
/** * Override this to customize the server. * <p/> * <p/> * (By default, this delegates to serveFile() and allows directory listing.) * * @param session The HTTP session * @return HTTP response, see class Response for details */ public Response serve(IHTTPSession session) { Map<String, String> files = new HashMap<String, String>(); Method method = session.getMethod(); if (Method.PUT.equals(method) || Method.POST.equals(method)) { try { session.parseBody(files); } catch (IOException ioe) { return new Response(Response.Status.INTERNAL_ERROR, MIME_PLAINTEXT, "SERVER INTERNAL ERROR: IOException: " + ioe.getMessage()); } catch (ResponseException re) { return new Response(re.getStatus(), MIME_PLAINTEXT, re.getMessage()); } } Map<String, String> parms = session.getParms(); parms.put(QUERY_STRING_PARAMETER, session.getQueryParameterString()); return serve(session.getUri(), method, session.getHeaders(), parms, files); }这个默认的方法处理了PUT和POST方法,如果不是就返回默认的返回值。
parseBody方法中使用了tmpFile的方法保存httpRequest的content信息,然后处理,具体逻辑就不细说了,不是一个典型的实现。
最后看一下发response的逻辑:
/** * Sends given response to the socket. */ protected void send(OutputStream outputStream) { String mime = mimeType; SimpleDateFormat gmtFrmt = new SimpleDateFormat("E, d MMM yyyy HH:mm:ss 'GMT'", Locale.US); gmtFrmt.setTimeZone(TimeZone.getTimeZone("GMT")); try { if (status == null) { throw new Error("sendResponse(): Status can't be null."); } PrintWriter pw = new PrintWriter(outputStream); pw.print("HTTP/1.1 " + status.getDescription() + " \r\n"); if (mime != null) { pw.print("Content-Type: " + mime + "\r\n"); } if (header == null || header.get("Date") == null) { pw.print("Date: " + gmtFrmt.format(new Date()) + "\r\n"); } if (header != null) { for (String key : header.keySet()) { String value = header.get(key); pw.print(key + ": " + value + "\r\n"); } } sendConnectionHeaderIfNotAlreadyPresent(pw, header); if (requestMethod != Method.HEAD && chunkedTransfer) { sendAsChunked(outputStream, pw); } else { int pending = data != null ? data.available() : 0; sendContentLengthHeaderIfNotAlreadyPresent(pw, header, pending); pw.print("\r\n"); pw.flush(); sendAsFixedLength(outputStream, pending); } outputStream.flush(); safeClose(data); } catch (IOException ioe) { // Couldn't write? No can do. } }发送response的步骤如下:
1.设置mimeType和Time等内容。
2.创建一个PrintWriter,按照HTTP协议依次开始写入内容
3.第一行是HTTP的返回码
4.然后是content-Type
5.然后是Date时间
6.之后是其他的HTTP Header
7.设置Keep-Alive的Header,Keep-Alive是Http1.1的新特性,作用是让客户端和服务器端之间保持一个长链接。
8.如果客户端指定了ChunkedEncoding则分块发送response,Chunked Encoding是Http1.1的又一新特性。一般在response的body比较大的时候使用,server端会首先发送response的HEADER,然后分块发送response的body,每个分块都由chunk length\r\n和chunk data\r\n组成,最后由一个0\r\n结束。
private void sendAsChunked(OutputStream outputStream, PrintWriter pw) throws IOException { pw.print("Transfer-Encoding: chunked\r\n"); pw.print("\r\n"); pw.flush(); int BUFFER_SIZE = 16 * 1024; byte[] CRLF = "\r\n".getBytes(); byte[] buff = new byte[BUFFER_SIZE]; int read; while ((read = data.read(buff)) > 0) { outputStream.write(String.format("%x\r\n", read).getBytes()); outputStream.write(buff, 0, read); outputStream.write(CRLF); } outputStream.write(String.format("0\r\n\r\n").getBytes()); }
9.如果没指定ChunkedEncoding则需要指定Content-Length来让客户端指定response的body的size,然后再一直写body直到写完为止。
private void sendAsFixedLength(OutputStream outputStream, int pending) throws IOException { if (requestMethod != Method.HEAD && data != null) { int BUFFER_SIZE = 16 * 1024; byte[] buff = new byte[BUFFER_SIZE]; while (pending > 0) { int read = data.read(buff, 0, ((pending > BUFFER_SIZE) ? BUFFER_SIZE : pending)); if (read <= 0) { break; } outputStream.write(buff, 0, read); pending -= read; } } }
最后总结下实现HttpServer最重要的几个部分:
1.能够accept tcp连接并从socket中读取request数据
2.把request的比特流转换成request对象中的对象数据
3.根据http协议的规范处理http request
4.产生http response再写回到socket中传给client。