Liferay MinifierFilter的研究

 大家都知道，在Web应用程序中，为了节省网络开销，往往吧多个小的js文件整合成一个大的js文件，吧多个小的css文件整合成一个大的js文件,这样原本N次小文件的请求就可以合并成单次的网络请求。最典型的做这件事情的工具是大名鼎鼎的yui-compressor.

 

其实在Liferay中，我们为了达到合并css,js的目的，用了不同于yui-compressor的方法，这就是我们的主角 MinifierFilter.

 

既然是Filter，那么它肯定有filter-mapping，我们轻易的在liferay-web.xml中找到了Filter的定义和Filter的mapping.

...
<filter>
        <filter-name>Minifier Filter</filter-name>
        <filter-class>com.liferay.portal.servlet.filters.minifier.MinifierFilter</filter-class>
    </filter>
<filter> <filter-name>Minifier Filter - JSP</filter-name> <filter-class>com.liferay.portal.servlet.filters.minifier.MinifierFilter</filter-class> <init-param> <param-name>url-regex-pattern</param-name> <param-value>.+/(aui_lang|barebone|css|everything|main)\.jsp</param-value> </init-param> </filter>

...
<filter-mapping>
        <filter-name>Minifier Filter</filter-name>
        <url-pattern>*.css</url-pattern>
    </filter-mapping>
    <filter-mapping>
        <filter-name>Minifier Filter</filter-name>
        <url-pattern>*.js</url-pattern>
    </filter-mapping>
<filter-mapping> <filter-name>Minifier Filter - JSP</filter-name> <url-pattern>*.jsp</url-pattern> </filter-mapping>
...

所以，我们可以看到，当客户端对Liferay服务器上请求任意css或者javascript资源时候，都会被这个MinifierFilter所过滤，我们现在就来看下庐山真面目。

 

因为MinifierFilter最终实现了Filter接口，而doFilter方法在父类的父类BaseFilter中定义，这个doFilter仅仅是调用processFilter()方法，

public void doFilter(
            ServletRequest servletRequest, ServletResponse servletResponse,
            FilterChain filterChain)
        throws IOException, ServletException {

        try {
            HttpServletRequest request = (HttpServletRequest)servletRequest;
            HttpServletResponse response = (HttpServletResponse)servletResponse;

            processFilter(request, response, filterChain);
        }
        catch (IOException ioe) {
            throw ioe;
        }
        catch (ServletException se) {
            throw se;
        }

所以这就是我们的入口：

protected void processFilter(
            HttpServletRequest request, HttpServletResponse response,
            FilterChain filterChain)
        throws Exception {

        Object minifiedContent = getMinifiedContent(
            request, response, filterChain);

        if (minifiedContent == null) {
            minifiedContent = getMinifiedBundleContent(request, response);
        }

        if (minifiedContent == null) {
            processFilter(MinifierFilter.class, request, response, filterChain);
        }
        else {
            if (minifiedContent instanceof File) {
                ServletResponseUtil.write(response, (File)minifiedContent);
            }
            else if (minifiedContent instanceof String) {
                ServletResponseUtil.write(response, (String)minifiedContent);
            }
        }
    }

首先，它会去执行06-07行的getMinifiedContent()方法,它会调用以下代码：

在getMinifiedContent()方法中，它会调用2个方法来分别最小化css和最小化js.

如下：

protected Object getMinifiedContent(
            HttpServletRequest request, HttpServletResponse response,
            FilterChain filterChain)
        throws Exception {

        ..

       ..

        String minifiedContent = null;

        if (realPath.endsWith(_CSS_EXTENSION)) {
            if (_log.isInfoEnabled()) {
                _log.info("Minifying CSS " + file);
            }

            minifiedContent = minifyCss(request, response, file);

            response.setContentType(ContentTypes.TEXT_CSS);

            FileUtil.write(cacheContentTypeFile, ContentTypes.TEXT_CSS);
        }
        else if (realPath.endsWith(_JAVASCRIPT_EXTENSION)) {
            if (_log.isInfoEnabled()) {
                _log.info("Minifying JavaScript " + file);
            }

            minifiedContent = minifyJavaScript(file);

            response.setContentType(ContentTypes.TEXT_JAVASCRIPT);

            FileUtil.write(cacheContentTypeFile, ContentTypes.TEXT_JAVASCRIPT);
        }
        else if (realPath.endsWith(_JSP_EXTENSION)) {
            if (_log.isInfoEnabled()) {
                _log.info("Minifying JSP " + file);
            }

            StringServletResponse stringResponse = new StringServletResponse(
                response);

            processFilter(
                MinifierFilter.class, request, stringResponse, filterChain);

            CacheResponseUtil.setHeaders(response, stringResponse.getHeaders());

            response.setContentType(stringResponse.getContentType());

            minifiedContent = stringResponse.getString();

            if (minifierType.equals("css")) {
                minifiedContent = minifyCss(
                    request, response, realPath, minifiedContent);
            }
            else if (minifierType.equals("js")) {
                minifiedContent = minifyJavaScript(minifiedContent);
            }

            FileUtil.write(
                cacheContentTypeFile, stringResponse.getContentType());
        }
        else {
            return null;
        }

        FileUtil.write(cacheDataFile, minifiedContent);

        return minifiedContent;
    }

 

minifyCSS:

从第12行可以看出，如果判断扩展名是.css，那么需要吧文件minify一下，并且设置content-type为text/css,最后把这个文件放入cacheContentTypeFile中。

我们来看下minifyCSS到底做了什么事情：

protected String minifyCss(
            HttpServletRequest request, HttpServletResponse response, File file)
        throws IOException {

        String content = FileUtil.read(file);

        content = aggregateCss(file.getParent(), content);

        return minifyCss(request, response, file.getAbsolutePath(), content);
    }

从这里可以清楚的看出，

 

05行是先吧这个css文件的内容通过FileUtil读出来，其实这个FileUtil的读的方式会去除所有的换行，参见最终调用的FileImpl的read方法：

public String read(File file, boolean raw) throws IOException {
        byte[] bytes = getBytes(file);

        if (bytes == null) {
            return null;
        }

        String s = new String(bytes, StringPool.UTF8);

        if (raw) {
            return s;
        }
        else {
            return StringUtil.replace(
                s, StringPool.RETURN_NEW_LINE, StringPool.NEW_LINE);
        }
    }

然后把去除了所有换行的css文件的内容存入到变量content中。

 

07行会调用aggregateCSS来对这个css文件的内容做进一步处理，如何处理呢，我们看代码：

public static String aggregateCss(String dir, String content)
        throws IOException {

        StringBuilder sb = new StringBuilder(content.length());

        int pos = 0;

        while (true) {
            int commentX = content.indexOf(_CSS_COMMENT_BEGIN, pos);
            int commentY = content.indexOf(
                _CSS_COMMENT_END, commentX + _CSS_COMMENT_BEGIN.length());

            int importX = content.indexOf(_CSS_IMPORT_BEGIN, pos);
            int importY = content.indexOf(
                _CSS_IMPORT_END, importX + _CSS_IMPORT_BEGIN.length());

            if ((importX == -1) || (importY == -1)) {
                sb.append(content.substring(pos, content.length()));

                break;
            }
            else if ((commentX != -1) && (commentY != -1) &&
                     (commentX < importX) && (commentY > importX)) {

                commentY += _CSS_COMMENT_END.length();

                sb.append(content.substring(pos, commentY));

                pos = commentY;
            }
            else {
                sb.append(content.substring(pos, importX));

                String importFileName = content.substring(
                    importX + _CSS_IMPORT_BEGIN.length(), importY);

                String importFullFileName = dir.concat(StringPool.SLASH).concat(
                    importFileName);

                String importContent = FileUtil.read(importFullFileName);

                if (importContent == null) {
                    if (_log.isWarnEnabled()) {
                        _log.warn(
                            "File " + importFullFileName + " does not exist");
                    }

                    importContent = StringPool.BLANK;
                }

                String importDir = StringPool.BLANK;

                int slashPos = importFileName.lastIndexOf(CharPool.SLASH);

                if (slashPos != -1) {
                    importDir = StringPool.SLASH.concat(
                        importFileName.substring(0, slashPos + 1));
                }

                importContent = aggregateCss(dir + importDir, importContent);

                int importDepth = StringUtil.count(
                    importFileName, StringPool.SLASH);

                // LEP-7540

                String relativePath = StringPool.BLANK;

                for (int i = 0; i < importDepth; i++) {
                    relativePath += "../";
                }

                importContent = StringUtil.replace(
                    importContent,
                    new String[] {
                        "url('" + relativePath,
                        "url(\"" + relativePath,
                        "url(" + relativePath
                    },
                    new String[] {
                        "url('[$TEMP_RELATIVE_PATH$]",
                        "url(\"[$TEMP_RELATIVE_PATH$]",
                        "url([$TEMP_RELATIVE_PATH$]"
                    });

                importContent = StringUtil.replace(
                    importContent, "[$TEMP_RELATIVE_PATH$]", StringPool.BLANK);

                sb.append(importContent);

                pos = importY + _CSS_IMPORT_END.length();
            }
        }

        return sb.toString();
    }

其实这段代码非常简单，它就是找出页面上所有的css注释 /* */，然后把这些注释去除，然后找出页面上@import(url=)的这种外部css文件，递归的调用aggregateCSS直到他们不含有外部引入标记，然后把这些文件的内容（已经被去除了注释，换行符等）插入到引入它们的css文件中。

 

minifyJavaScript:

从第23行可以看出，当遇到文件扩展名是.js时，它就会调用minifyJavaScript方法来最小化这个js文件，并且设置content-type为text/javascript,最后把minify之后的文件存入cacheContentTypeFile。

我们来看下minifyJavaScript到底做了什么事情：

protected String minifyJavaScript(File file) throws IOException {
        String content = FileUtil.read(file);

        return minifyJavaScript(content);
    }

它首先还是利用FileUtil来去除换行（见minifyCSS部分对这个方法的讲解），然后对于已经没有换行符的js文件继续调用minifyJavaScript()：

protected String minifyJavaScript(String content) {
        return MinifierUtil.minifyJavaScript(content);
    }

它又去调用MinifierUtil工具类方法来完成任务，最终执行任务的是MinifierUtil的_minifyJavaScript方法：

private String _minifyJavaScript(String content) {
        UnsyncStringWriter unsyncStringWriter = new UnsyncStringWriter();

        try {
            JavaScriptCompressor javaScriptCompressor =
                new JavaScriptCompressor(
                    new UnsyncStringReader(content),
                    new JavaScriptErrorReporter());

            javaScriptCompressor.compress(
                    unsyncStringWriter, _JS_LINE_BREAK, _JS_MUNGE, _JS_VERBOSE,
                    _JS_PRESERVE_ALL_SEMICOLONS, _JS_DISABLE_OPTIMIZATIONS);
        }
        catch (Exception e) {
            _log.error("JavaScript Minifier failed for\n" + content);

            unsyncStringWriter.append(content);
        }

        return unsyncStringWriter.toString();
    }

它会先创建一个JavaScriptCompressor对象，然后用它来压缩没有换行符的js文件，采用的方式是和yahoo的yui-compressor一样的方式,算法很复杂，没必要一行行看了。

 

minifyJSP:

从34行可以看到，它会先判断是jsp扩展名，当然了， 它也不会对所有的jsp都生效，它生效的jsp文件都在liferay-web.xml中的这个filter的<init-param>中，具体的就是

barebone.jsp,everything.jsp等因为满足init-param的正则表达式的pattern,所以会通过这个过滤器。从第47-56行可以看出他会吧原来cache的所有被minify处理过的css或者js内容再minify一下然后写入String变量，然后第59-60行利用FileUtil进一步去除换行符，然后把cacheContentFile的内容以最终请求的MIME格式来复写一遍。

 

当我们在文章一开始的MinifierFilter的processFilter()方法中执行了所有的getMinifiedContent调用后：

Object minifiedContent = getMinifiedContent(
           request, response, filterChain);

此时，这个Object minifiedContent的内容就不是null了。

 

现在我们来执行MinifierFilter的最后2个语句，它可以判断这个minifiedContent是个文件还是字符串，从而让其写入ServletResponse对象中并且返回给客户端，写入的方式是调用ServletResponseUtil工具类：

else {
          if (minifiedContent instanceof File) {
              ServletResponseUtil.write(response, (File)minifiedContent);
          }
          else if (minifiedContent instanceof String) {
              ServletResponseUtil.write(response, (String)minifiedContent);
          }
      }

 

由此大功告成，我们所有的css,js资源文件都得到了最小化，然后整合成单个文件.

 

 

高级话题：barebone.jsp和everything.jsp

事实上，Liferay启用了2个配置，一个只引入最少最需要的js文件，最终组合为barebone.jsp，一个是引入所有的js文件，最终组合为everything.jsp,他们可以自由切换，切换代码在top_js.jspf中：

<c:choose>
    <c:when test="<%= themeDisplay.isThemeJsFastLoad() %>">
        <c:choose>
            <c:when test="<%= themeDisplay.isThemeJsBarebone() %>">
                <script src="<%= HtmlUtil.escape(PortalUtil.getStaticResourceURL(request, themeDisplay.getPathJavaScript() + "/barebone.jsp", "minifierBundleId=javascript.barebone.files", javaScriptLastModified)) %>" type="text/javascript"></script>
            </c:when>
            <c:otherwise>
                <script src="<%= HtmlUtil.escape(PortalUtil.getStaticResourceURL(request, themeDisplay.getPathJavaScript() + "/everything.jsp", "minifierBundleId=javascript.everything.files", javaScriptLastModified)) %>" type="text/javascript"></script>
            </c:otherwise>
        </c:choose>
    </c:when>
    <c:otherwise>

这里可以看出来，切换主要去判断themeDisplay.isThemeJSBarebone，而这个配置在portal.properties中，比如我们服务器设置了javascript.barebone.enabled=true，则开启了barebone,则最后看情况可以有barebone.jsp可以有everything.jsp:

#
    # Set this property to false to always load JavaScript files listed in the
    # property "javascript.everything.files". Set this to true to sometimes
    # load "javascript.barebone.files" and sometimes load
    # "javascript.everything.files".
    #
    # The default logic is coded in com.liferay.portal.events.ServicePreAction
    # in such a way that unauthenticated users get the list of barebone
    # JavaScript files whereas authenticated users get both the list of barebone
    # JavaScript files and the list of everything JavaScript files.
    #
    javascript.barebone.enabled=true

 

无论是barebone.jsp还是everything.jsp，他们的bundleId和读取目录都是预先是定好的：

#
  # Input a list of comma delimited properties that are valid bundle ids for
  # the JavaScript minifier.
  #
  javascript.bundle.ids=\
      javascript.barebone.files,\
      javascript.everything.files

  #
  # Define a bundle directory for each property listed in
  # "javascript.bundle.ids".
  #
  javascript.bundle.dir[javascript.barebone.files]=/html/js
  javascript.bundle.dir[javascript.everything.files]=/html/js

  #

 

只不过barebone.jsp合并的js文件少，而everything.jsp文件合并全部的js文件：

barebone.jsp合并并且最小化哪些js文件呢?这也可以从portal.properties文件中找到答案：

javascript.barebone.files=\
       \
       #
       # YUI core
       #
       \
       aui/yui/yui.js,\
       \
       #
       # YUI modules
       #
       \
       aui/anim-base/anim-base.js,\
       aui/anim-color/anim-color.js,\
       aui/anim-curve/anim-curve.js,\
...

 

而everything.jsp合并并且最小化哪些js文件呢？它是由javascript.barebone.files 包含的所有js文件，外加如下列表的不在barebone中的文件：

#
    # Specify the list of everything files (everything else not already in the
    # list of barebone files).
    #
    javascript.everything.files=\
        \
        #
        # YUI modules
        #
        \
        aui/async-queue/async-queue.js,\
        aui/cookie/cookie.js,\
        aui/event-touch/event-touch.js,\
        aui/querystring-stringify/querystring-stringify.js,\
        \
        #
        # Alloy modules
        #
        \
        aui/aui-io/aui-io-plugin.js,\
        aui/aui-io/aui-io-request.js,\
        aui/aui-loading-mask/aui-loading-mask.js,\
        aui/aui-parse-content/aui-parse-content.js,\
        \
        #
        # Liferay modules
        #
        \
        liferay/address.js,\
        liferay/dockbar.js,\
        liferay/layout_configuration.js,\
        liferay/layout_exporter.js,\
        liferay/session.js,\
        \
        #
        # Deprecated JS
        #
        \
        liferay/deprecated.js

 

 

 

这样一分析下来，整个Liferay框架的静态资源加载文件就非常清晰了。