Servlet - Upload、Download、Async

Upload、Download、Async

标签 ： Java与Web

Upload-上传

随着3.0版本的发布,文件上传终于成为Servlet规范的一项内置特性,不再依赖于像Commons FileUpload之类组件,因此在服务端进行文件上传编程变得不费吹灰之力.

客户端

要上传文件, 必须利用multipart/form-data设置HTML表单的enctype属性,且method必须为POST:

<form action="simple_file_upload_servlet.do" method="POST" enctype="multipart/form-data">
    <table align="center" border="1" width="50%">
        <tr>
            <td>Author:</td>
            <td><input type="text" name="author"></td>
        </tr>
        <tr>
            <td>Select file to Upload:</td>
            <td><input type="file" name="file"></td>
        </tr>
        <tr>
            <td><input type="submit" value="上传"></td>
        </tr>
    </table>
</form>

服务端

服务端Servlet主要围绕着@MultipartConfig注解和Part接口:

处理上传文件的Servlet必须用@MultipartConfig注解标注:

@MultipartConfig属性	描述
fileSizeThreshold	The size threshold after which the file will be written to disk
location	The directory location where files will be stored
maxFileSize	The maximum size allowed for uploaded files.
maxRequestSize	The maximum size allowed for multipart/form-data requests

在一个由多部件组成的请求中, 每一个表单域(包括非文件域), 都会被封装成一个Part,HttpServletRequest中提供如下两个方法获取封装好的Part:

HttpServletRequest	描述
Part getPart(String name)	Gets the Part with the given name.
Collection<Part> getParts()	Gets all the Part components of this request, provided that it is of type multipart/form-data.

Part中提供了如下常用方法来获取/操作上传的文件/数据:

Part	描述
InputStream getInputStream()	Gets the content of this part as an InputStream
void write(String fileName)	A convenience method to write this uploaded item to disk.
String getSubmittedFileName()	Gets the file name specified by the client(需要有Tomcat 8.x 及以上版本支持)
long getSize()	Returns the size of this fille.
void delete()	Deletes the underlying storage for a file item, including deleting any associated temporary disk file.
String getName()	Gets the name of this part
String getContentType()	Gets the content type of this part.
Collection<String> getHeaderNames()	Gets the header names of this Part.
String getHeader(String name)	Returns the value of the specified mime header as a String.

文件流解析

通过抓包获取到客户端上传文件的数据格式:

------WebKitFormBoundaryXJ6TxfJ9PX5hJHGh
Content-Disposition: form-data; name="author"

feiqing
------WebKitFormBoundaryXJ6TxfJ9PX5hJHGh
Content-Disposition: form-data; name="file"; filename="memcached.txt"
Content-Type: text/plain


------WebKitFormBoundaryXJ6TxfJ9PX5hJHGh--

可以看到:

A. 如果HTML表单输入项为文本( <input type="text"/>),将只包含一个请求头 Content-Disposition.

B. 如果HTML表单输入项为文件( <input type="file"/>), 则包含两个头:
Content-Disposition与 Content-Type.

在Servlet中处理上传文件时, 需要:

- 通过查看是否存在`Content-Type`标头, 检验一个Part是封装的普通表单域,还是文件域.
- 若有`Content-Type`存在, 但文件名为空, 则表示没有选择要上传的文件.
- 如果有文件存在, 则可以调用`write()`方法来写入磁盘, 调用同时传递一个绝对路径, 或是相对于`@MultipartConfig`注解的`location`属性的相对路径.

• SimpleFileUploadServlet

/** * @author jifang. * @since 2016/5/8 16:27. */
@MultipartConfig
@WebServlet(name = "SimpleFileUploadServlet", urlPatterns = "/simple_file_upload_servlet.do")
public class SimpleFileUploadServlet extends HttpServlet {

    protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        response.setContentType("text/html;charset=UTF-8");
        PrintWriter writer = response.getWriter();
        Part file = request.getPart("file");
        if (!isFileValid(file)) {
            writer.print("<h1>请确认上传文件是否正确!");
        } else {
            String fileName = file.getSubmittedFileName();
            String saveDir = getServletContext().getRealPath("/WEB-INF/files/");
            mkdirs(saveDir);
            file.write(saveDir + fileName);

            writer.print("<h3>Uploaded file name: " + fileName);
            writer.print("<h3>Size: " + file.getSize());
            writer.print("<h3>Author: " + request.getParameter("author"));
        }
    }

    private void mkdirs(String saveDir) {
        File dir = new File(saveDir);
        if (!dir.exists()) {
            dir.mkdirs();
        }
    }

    private boolean isFileValid(Part file) {
        // 上传的并非文件
        if (file.getContentType() == null) {
            return false;
        }
        // 没有选择任何文件
        else if (Strings.isNullOrEmpty(file.getSubmittedFileName())) {
            return false;
        }
        return true;
    }
}

优化

• 善用WEB-INF
  存放在/WEB-INF/目录下的资源无法在浏览器地址栏直接访问, 利用这一特点可将某些受保护资源存放在WEB-INF目录下, 禁止用户直接访问(如用户上传的可执行文件,如JSP等),以防被恶意执行, 造成服务器信息泄露等危险.

getServletContext().getRealPath("/WEB-INF/")

• 文件名乱码
  当文件名包含中文时,可能会出现乱码,其解决方案与POST相同:

request.setCharacterEncoding("UTF-8");

• 避免文件同名
  如果上传同名文件,会造成文件覆盖.因此可以为每份文件生成一个唯一ID,然后连接原始文件名:

private String generateUUID() {
    return UUID.randomUUID().toString().replace("-", "_");
}

• 目录打散
  如果一个目录下存放的文件过多, 会导致文件检索速度下降,因此需要将文件打散存放到不同目录中, 在此我们采用Hash打散法(根据文件名生成Hash值, 取Hash值的前两个字符作为二级目录名), 将文件分布到一个二级目录中:

private String generateTwoLevelDir(String destFileName) {
    String hash = Integer.toHexString(destFileName.hashCode());
    return String.format("%s/%s", hash.charAt(0), hash.charAt(1));
}

采用Hash打散的好处是:在根目录下最多生成16个目录,而每个子目录下最多再生成16个子子目录,即一共256个目录,且分布较为均匀.

示例-简易存储图片服务器

需求: 提供上传图片功能, 为其生成外链, 并提供下载功能(见下)

• 客户端

<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <title>IFS</title>
</head>
<body>
<form action="ifs_upload.action" method="POST" enctype="multipart/form-data">
    <table align="center" border="1" width="50%">
        <tr>
            <td>Select A Image to Upload:</td>
            <td><input type="file" name="image"></td>
        </tr>
        <tr>
            <td>&nbsp;</td>
            <td><input type="submit" value="上传"></td>
        </tr>
    </table>
</form>
</body>
</html>

• 服务端

@MultipartConfig
@WebServlet(name = "ImageFileUploadServlet", urlPatterns = "/ifs_upload.action")
public class ImageFileUploadServlet extends HttpServlet {

    private Set<String> imageSuffix = new HashSet<>();

    private static final String SAVE_ROOT_DIR = "/images";

    {
        imageSuffix.add(".jpg");
        imageSuffix.add(".png");
        imageSuffix.add(".jpeg");
    }

    @Override
    protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        request.setCharacterEncoding("UTF-8");
        response.setContentType("text/html;charset=UTF-8");
        PrintWriter writer = response.getWriter();
        Part image = request.getPart("image");
        String fileName = getFileName(image);
        if (isFileValid(image, fileName) && isImageValid(fileName)) {
            String destFileName = generateDestFileName(fileName);
            String twoLevelDir = generateTwoLevelDir(destFileName);

            // 保存文件
            String saveDir = String.format("%s/%s/", getServletContext().getRealPath(SAVE_ROOT_DIR), twoLevelDir);
            makeDirs(saveDir);
            image.write(saveDir + destFileName);

            // 生成外链
            String ip = request.getLocalAddr();
            int port = request.getLocalPort();
            String path = request.getContextPath();
            String urlPrefix = String.format("http://%s:%s%s", ip, port, path);
            String urlSuffix = String.format("%s/%s/%s", SAVE_ROOT_DIR, twoLevelDir, destFileName);
            String url = urlPrefix + urlSuffix;
            String result = String.format("<a href=%s>%s</a><hr/><a href=ifs_download.action?location=%s>下载</a>",
                    url,
                    url,
                    saveDir + destFileName);
            writer.print(result);
        } else {
            writer.print("Error : Image Type Error");
        }
    }

    /** * 校验文件表单域有效 * * @param file * @param fileName * @return */
    private boolean isFileValid(Part file, String fileName) {
        // 上传的并非文件
        if (file.getContentType() == null) {
            return false;
        }
        // 没有选择任何文件
        else if (Strings.isNullOrEmpty(fileName)) {
            return false;
        }

        return true;
    }

    /** * 校验文件后缀有效 * * @param fileName * @return */
    private boolean isImageValid(String fileName) {
        for (String suffix : imageSuffix) {
            if (fileName.endsWith(suffix)) {
                return true;
            }
        }
        return false;
    }

    /** * 加速图片访问速度, 生成两级存放目录 * * @param destFileName * @return */
    private String generateTwoLevelDir(String destFileName) {
        String hash = Integer.toHexString(destFileName.hashCode());
        return String.format("%s/%s", hash.charAt(0), hash.charAt(1));
    }

    private String generateUUID() {
        return UUID.randomUUID().toString().replace("-", "_");
    }

    private String generateDestFileName(String fileName) {
        String destFileName = generateUUID();
        int index = fileName.lastIndexOf(".");
        if (index != -1) {
            destFileName += fileName.substring(index);
        }
        return destFileName;
    }

    private String getFileName(Part part) {
        String[] elements = part.getHeader("content-disposition").split(";");
        for (String element : elements) {
            if (element.trim().startsWith("filename")) {
                return element.substring(element.indexOf("=") + 1).trim().replace("\"", "");
            }
        }
        return null;
    }

    private void makeDirs(String saveDir) {
        File dir = new File(saveDir);
        if (!dir.exists()) {
            dir.mkdirs();
        }
    }
}

由于getSubmittedFileName()方法需要有Tomcat 8.X以上版本的支持, 因此为了通用期间, 我们自己解析content-disposition请求头, 获取filename.

Download-下载

文件下载是向客户端响应二进制数据(而非字符),浏览器不会直接显示这些内容,而是会弹出一个下载框, 提示下载信息.

为了将资源发送给浏览器, 需要在Servlet中完成以下工作:

• 使用Content-Type响应头来规定响应体的MIME类型, 如image/pjpeg、application/octet-stream;
• 添加Content-Disposition响应头,赋值为attachment;filename=xxx.yyy, 设置文件名;
• 使用response.getOutputStream()给浏览器发送二进制数据;

文件名中文乱码

当文件名包含中文时(attachment;filename=文件名.后缀名),在下载框中会出现乱码, 需要对文件名编码后在发送, 但不同的浏览器接收的编码方式不同:

    * FireFox: Base64编码
    * 其他大部分Browser: URL编码

因此最好将其封装成一个通用方法:

private String filenameEncoding(String filename, HttpServletRequest request) throws IOException {
    // 根据浏览器信息判断
    if (request.getHeader("User-Agent").contains("Firefox")) {
        filename = String.format("=?utf-8?B?%s?=", BaseEncoding.base64().encode(filename.getBytes("UTF-8")));
    } else {
        filename = URLEncoder.encode(filename, "utf-8");
    }
    return filename;
}

示例-IFS下载功能

/** * @author jifang. * @since 2016/5/9 17:50. */
@WebServlet(name = "ImageFileDownloadServlet", urlPatterns = "/ifs_download.action")
public class ImageFileDownloadServlet extends HttpServlet {

    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        response.setContentType("application/octet-stream");
        String fileLocation = request.getParameter("location");
        String fileName = fileLocation.substring(fileLocation.lastIndexOf("/") + 1);
        response.setHeader("Content-Disposition", "attachment;filename=" + filenameEncoding(fileName, request));

        ByteStreams.copy(new FileInputStream(fileLocation), response.getOutputStream());
    }

    @Override
    protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        doGet(req, resp);
    }
}

Async-异步处理

Servlet/Filter默认会一直占用请求处理线程, 直到它完成任务.如果任务耗时长久, 且并发用户请求量大, Servlet容器将会遇到超出线程数的风险.

Servlet 3.0 中新增了一项特性, 用来处理异步操作. 当Servlet/Filter应用程序中有一个/多个长时间运行的任务时, 你可以选择将任务分配给一个新的线程, 从而将当前请求处理线程返回到线程池中,释放线程资源,准备为下一个请求服务.

异步Servlet/Filter

• 异步支持
  @WebServlet/@WebFilter注解提供了新的asyncSupport属性:

@WebFilter(asyncSupported = true)
@WebServlet(asyncSupported = true)

同样部署描述符中也添加了<async-supportted/>标签:

<servlet>
    <servlet-name>HelloServlet</servlet-name>
    <servlet-class>com.fq.web.servlet.HelloServlet</servlet-class>
    <async-supported>true</async-supported>
</servlet>

• Servlet/Filter
  支持异步处理的Servlet/Filter可以通过在ServletRequest中调用startAsync()方法来启动新线程:

ServletRequest	描述
AsyncContext startAsync()	Puts this request into asynchronous mode, and initializes its AsyncContext with the original (unwrapped) ServletRequest and ServletResponse objects.
AsyncContext startAsync(ServletRequest servletRequest, ServletResponse servletResponse)	Puts this request into asynchronous mode, and initializes its AsyncContext with the given request and response objects.

注意:
1. 只能将原始的ServletRequest/ServletResponse或其包装器(Wrapper/Decorator,详见Servlet - Listener、Filter、Decorator)传递给第二个startAsync()方法.
2. 重复调用startAsync()方法会返回相同的AsyncContext实例, 如果在不支持异步处理的Servlet/Filter中调用, 会抛出java.lang.IllegalStateException异常.
3. AsyncContext的start()方法不会造成方法阻塞.

这两个方法都返回AsyncContext实例, AsyncContext中提供了如下常用方法:

AsyncContext	描述
void start(Runnable run)	Causes the container to dispatch a thread, possibly from a managed thread pool, to run the specified Runnable.
void dispatch(String path)	Dispatches the request and response objects of this AsyncContext to the given path.
void dispatch(ServletContext context, String path)	Dispatches the request and response objects of this AsyncContext to the given path scoped to the given context.
void addListener(AsyncListener listener)	Registers the given AsyncListener with the most recent asynchronous cycle that was started by a call to one of the ServletRequest.startAsync() methods.
ServletRequest getRequest()	Gets the request that was used to initialize this AsyncContext by calling ServletRequest.startAsync() or ServletRequest.startAsync(ServletRequest, ServletResponse).
ServletResponse getResponse()	Gets the response that was used to initialize this AsyncContext by calling ServletRequest.startAsync() or ServletRequest.startAsync(ServletRequest, ServletResponse).
boolean hasOriginalRequestAndResponse()	Checks if this AsyncContext was initialized with the original or application-wrapped request and response objects.
void setTimeout(long timeout)	Sets the timeout (in milliseconds) for this AsyncContext.

在异步Servlet/Filter中需要完成以下工作, 才能真正达到异步的目的:

• 调用AsyncContext的start()方法, 传递一个执行长时间任务的Runnable;
• 任务完成时, 在Runnable内调用AsyncContext的complete()方法或dispatch()方法

示例-改造文件上传

在前面的图片存储服务器中, 如果上传图片过大, 可能会耗时长久,为了提升服务器性能, 可将其改造为异步上传(其改造成本较小):

@Override
protected void doPost(final HttpServletRequest request, final HttpServletResponse response) throws ServletException, IOException {
    final AsyncContext asyncContext = request.startAsync();
    asyncContext.start(new Runnable() {
        @Override
        public void run() {
            try {
                request.setCharacterEncoding("UTF-8");
                response.setContentType("text/html;charset=UTF-8");
                PrintWriter writer = response.getWriter();
                Part image = request.getPart("image");
                final String fileName = getFileName(image);
                if (isFileValid(image, fileName) && isImageValid(fileName)) {
                    String destFileName = generateDestFileName(fileName);
                    String twoLevelDir = generateTwoLevelDir(destFileName);

                    // 保存文件
                    String saveDir = String.format("%s/%s/", getServletContext().getRealPath(SAVE_ROOT_DIR), twoLevelDir);
                    makeDirs(saveDir);
                    image.write(saveDir + destFileName);
                    // 生成外链
                    String ip = request.getLocalAddr();
                    int port = request.getLocalPort();
                    String path = request.getContextPath();
                    String urlPrefix = String.format("http://%s:%s%s", ip, port, path);
                    String urlSuffix = String.format("%s/%s/%s", SAVE_ROOT_DIR, twoLevelDir, destFileName);
                    String url = urlPrefix + urlSuffix;
                    String result = String.format("<a href=%s>%s</a><hr/><a href=ifs_download.action?location=%s>下载</a>",
                            url,
                            url,
                            saveDir + destFileName);
                    writer.print(result);
                } else {
                    writer.print("Error : Image Type Error");
                }
                asyncContext.complete();
            } catch (ServletException | IOException e) {
                LOGGER.error("error: ", e);
            }
        }
    });
}

注意: Servlet异步支持只适用于长时间运行,且想让用户知道执行结果的任务. 如果只有长时间, 但用户不需要知道处理结果,那么只需提供一个Runnable提交给Executor, 并立即返回即可.

AsyncListener

Servlet 3.0 还新增了一个AsyncListener接口, 以便通知用户在异步处理期间发生的事件, 该接口会在异步操作的启动/完成/失败/超时情况下调用其对应方法:

• ImageUploadListener

/** * @author jifang. * @since 2016/5/10 17:33. */
public class ImageUploadListener implements AsyncListener {

    @Override
    public void onComplete(AsyncEvent event) throws IOException {
        System.out.println("onComplete...");
    }

    @Override
    public void onTimeout(AsyncEvent event) throws IOException {
        System.out.println("onTimeout...");
    }

    @Override
    public void onError(AsyncEvent event) throws IOException {
        System.out.println("onError...");
    }

    @Override
    public void onStartAsync(AsyncEvent event) throws IOException {
        System.out.println("onStartAsync...");
    }
}

与其他监听器不同, 他没有@WebListener标注AsyncListener的实现, 因此必须对有兴趣收到通知的每个AsyncContext都手动注册一个AsyncListener:

asyncContext.addListener(new ImageUploadListener());