XslCompiledTransform Slower than XslTransform?

XslCompiledTransform Slower than XslTransform?

This post discusses:

• Why XslCompiledTransform may be slower than XslTransform
• How to reduce start-up time if you use one of the managed XSLT processors
• Why it is important to cache loaded XslCompiledTransform instances

The .NET Framework 2.0 provides a new System.Xml.Xsl.XslCompiledTransform XSLT processor class, which is intended to replace the obsoleted XslTransform class. One of the major differences between the two is that while the latter is an XSLT interpreter, the former is a real XSLT compiler, allowing significantly faster execution times. Does it mean XslCompiledTransform is always faster? Surprisingly, the answer is not that simple.

Let's write a simple test application that measures Load and Transform times for both XslTransform and XslCompiledTransform processors. Here is the most interesting part of the code, and the full source code is available in the attached file.

private void TestXslCompiledTransform() {

    XslCompiledTransform xslt = null;



    for (int i = 0; i < numberOfIterations; i++) {

        Stopwatch stopwatch = Stopwatch.StartNew();

        xslt = new XslCompiledTransform();

        xslt.Load(xslFile);

        stopwatch.Stop();

        Console.WriteLine("Load time: {0} ms", FormatTime(stopwatch));

    }



    Console.WriteLine("------------------------");

    XPathDocument doc = new XPathDocument(xmlFile);



    for (int i = 0; i < numberOfIterations; i++) {

        Stopwatch stopwatch = Stopwatch.StartNew();

        xslt.Transform(doc, (XsltArgumentList)null, XmlWriter.Create(TextWriter.Null, xslt.OutputSettings));

        stopwatch.Stop();

        Console.WriteLine("Transform time: {0} ms", FormatTime(stopwatch));

    }

}

Note that both Load and Transform are executed multiple times in a loop, and their times are measured separately. Also we pre-load the input document and output results of the transformation to TextWriter.Null, so that file input/output operations are not taken into account. (I accidentally forgot to pre-load the stylesheet in memory, however that did not make a noticeable difference on the results obtained below, because I ran XsltPerf with the same stylesheet multiple times, and the stylesheet file was sitting in the disk cache after the first run.) If you are new to the XslCompiledTransform class and wondering what xslt.OutputSettings is doing in this snippet, you may find the answer in Erik Saltwell's post "What the heck is OutputSettings".

The application allows you to specify filenames of the input document and the XSLT stylesheet, the number of iterations, and which XSLT processor to use:

C:\XsltPerf>XsltPerf.exe /?

XSLT Load & Transform Performance Test Utility

for Microsoft (R) Windows (R) 2005 Framework version 2.0.50727



XsltPerf [/xt | /xct] [/i:<n>] <xml-file> <xsl-file>



Options:



/xt         Use XslTransform

/xct        Use XslCompiledTransform (default)

/i:<n>      Iterate n times (default is 5)

For testing purposes, we take one of XSLTMark benchmark stylesheets, namely queens.xsl, which finds all the possible solutions to the problem of placing N queens on an N×N chess board without any queen attacking another. XSLTMark uses N = 6, and so will we. Let's run the XsltPerf utility several times before taking readings to ensure the measurements take place on a "warm" machine:

C:\XsltPerf>XsltPerf.exe /xt queens.xml queens.xsl

The results for my Intel^® Xeon^® 3GHz box shows that the very first Load in a process is very slow for XslTransform and incredibly slow in case of XslCompiledTransform. The very first Transform is also significantly slower than subsequent ones. If you sum up the time for the first Load and the time for the first Transform, you'll get 239 ms for XslTransform versus 928 ms for XslCompiledTransform. In this particular scenario the new XslCompiledTransform class is almost 4 times slower!

XslTransform	XslCompiledTransform
Load time: 90.69 ms Load time: 2.847 ms Load time: 1.841 ms Load time: 2.681 ms Load time: 1.891 ms Transform time: 148.0 ms Transform time: 76.57 ms Transform time: 73.82 ms Transform time: 74.49 ms Transform time: 74.22 ms	Load time: 882.2 ms Load time: 4.582 ms Load time: 3.060 ms Load time: 3.119 ms Load time: 3.072 ms Transform time: 45.31 ms Transform time: 2.027 ms Transform time: 2.027 ms Transform time: 1.962 ms Transform time: 1.977 ms

So what the hell happens on the first call? In the .NET Framework 2.0 implementations of XslTransform and XslCompiledTransform were moved to a helper System.Data.SqlXml assembly in order to reduce the size of System.Xml. And apparently that helper assembly is not NGen'd by default, so its methods are JIT-compiled on first use:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727>ngen display System.Data.SqlXml

Microsoft (R) CLR Native Image Generator - Version 2.0.50727.42

Copyright (C) Microsoft Corporation 1998-2002. All rights reserved.

Error: The specified assembly is not installed.

(If you don't know what NGen or JIT is, I highly recommend reading the "NGen Revs Up Your Performance with Powerful New Features" article.) JIT-compilation affects the first XslCompiledTransform.Load call much more significantly comparing to XslTransform.Load, because the compiler uses substantially more complex code than the interpreter.

Considering that other key .NET Framework assemblies are NGen'd, missing of System.Data.SqlXml in the native image cache may be a simple oversight on the part of .NET Framework installer. Let's generate a native image for the System.Data.SqlXml assembly:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727>ngen install "System.Data.SqlXml, Version=2.0.0.0,

Culture=neutral, PublicKeyToken=b77a5c561934e089" /nologo

Installing assembly System.Data.SqlXml, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

Compiling 1 assembly:

Compiling assembly System.Data.SqlXml, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 ...

System.Data.SqlXml, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

and run our test application again. I have copied the previous results to make it's easier for you to compare:

XslTransform	XslCompiledTransform
System.Data.SqlXml is not NGen'd
Load time: 90.69 ms Load time: 2.847 ms Load time: 1.841 ms Load time: 2.681 ms Load time: 1.891 ms Transform time: 148.0 ms Transform time: 76.57 ms Transform time: 73.82 ms Transform time: 74.49 ms Transform time: 74.22 ms	Load time: 882.2 ms Load time: 4.582 ms Load time: 3.060 ms Load time: 3.119 ms Load time: 3.072 ms Transform time: 45.31 ms Transform time: 2.027 ms Transform time: 2.027 ms Transform time: 1.962 ms Transform time: 1.977 ms
System.Data.SqlXml is NGen'd
Load time: 19.27 ms Load time: 2.894 ms Load time: 1.895 ms Load time: 2.753 ms Load time: 1.911 ms Transform time: 77.22 ms Transform time: 75.93 ms Transform time: 73.56 ms Transform time: 74.49 ms Transform time: 73.78 ms	Load time: 58.57 ms Load time: 4.862 ms Load time: 3.238 ms Load time: 3.280 ms Load time: 3.272 ms Transform time: 15.23 ms Transform time: 1.918 ms Transform time: 1.921 ms Transform time: 1.949 ms Transform time: 1.926 ms

As you can see, the first Load call became much faster, though it still consumes some extra time to load the helper assembly into process memory and initialize all needed classes. Now XslCompiledTransform.Load is "only" 3 times slower than XslTransform.Load; however, this is expected — compiling, in general, is more expensive than the interpreter preparation work. It is the price you have to pay for faster execution: XslCompiledTransform.Transform, except the first call, is about 40 times faster!

If you look at new Transform times, you may note that the difference between the first and subsequent calls to XslTransform.Transform, thanks to NGen'ing, almost disappeared. But why is the first call to XslCompiledTransform.Transform still 8 times slower?! Here we need to recall that XslCompiledTransform compiles the stylesheet to MSIL methods. All those generated methods are subject to JIT-compiling on first use. While JIT-compilation is relatively expensive, its cost is usually amortized over several executions of compiled methods. For example, in our scenario the first XslCompiledTransform.Transform call is still faster than XslTransform.Transform one. However, in case of very simple stylesheets like <xsl:template match="/"><foo/></xsl:template> the first XslCompiledTransform.Transform call may perform several times worse.

I'd like to emphasize the principal difference between the "first Load" issue (relates to both XslTransform and XslCompiledTransform) and the "first Transform" issue (relates to XslCompiledTransform only). In the former case, the first Load per process¹ is affected. In the latter one, the first Transform per loaded stylesheet is affected. If you load a different stylesheet or even reload the same stylesheet into the same XslCompiledTransform instance, a new bunch of MSIL methods will be generated and JIT-compiled on their first use.

¹ Or per AppDomain in case of multi-AppDomain applications. For more information on sharing native images across AppDomains, you may read this and this.

Let's make some conclusions of this little experiment:

• If you are using one of the .NET Framework 2.0 XSLT processors (XslTransform or XslCompiledTransform), NGen'ing the System.Data.SqlXml assembly may improve the application start-up time. This does not relate to the .NET Framework 1.1, which has the XslTransform implementation NGen'd by default.
• XslCompiledTransform.Load is slower than XslTransform.Load, though it's unlikely to become a concern.
• XslCompiledTransform.Transform performance may degrade by several times due to JIT-compilation. JIT-compilation happens on first use of a stylesheet template, and in many cases it would be the very first XslCompiledTransform.Transform call for the given stylesheet loaded into an XslCompiledTransform instance that is affected the most.
• While XslCompiledTransform is a best choice for the "one Load, many Transforms" scenario, it may be slow for the "one Load, one Transform" scenario, especially for very simple XML/XSLT files and when stylesheet templates are executed only few times. In those cases XslTransform may be faster.
• It is important, especially for server applications, to cache an XslCompiledTransform instance if the same stylesheet is likely to be executed again.

post:http://blogs.msdn.com/antosha/archive/2006/07/16/667221.aspx