Generating Java classes dynamically through Java compiler API

Posted on March 6, 2011 by Upendra

Some of the java coders around the world might have thought of an option to be able to compile a java source file dynamically.

To my surprise, almost at the end of Java 6 (I am expecting Java 7 to be out soon…), I noticed this feature underjavax.tools package. May be I am the last one to notice this!! .

This dynamic compiler API is included with Java 6 under javax.tools package.

How does it work?

javax.tools package has all the required interfaces and classes. Here, we will see how to compile a simple “HelloWorld” program source code stored inan in-memory String variable.

Able to compile a piece of source code stored in a string variable, WOW! this is interesting! isn’t it?

Follow the sequence of steps mentioned below. I explained these steps with the required code-snippets at that point. The full version of source code is available at the end of the article.

The most important classes in this API are,

• JavaCompiler - This is used to create a compilation task
• JavaCompiler.CompilationTask – The compilation task, on which we execute compile operation using it’scall method
• JavaFileManager:Manages how the compiler read and writes to the files
• JavaFileObject: The file object that abstracts the java source and class files
• DiagnosticListener: This listens to the compilation diagnostic events
• ToolProvider: Which is used to get the compiler object from the underlying platform.

We will discuss these classes further in the example below. Let’s start…

1. Build the source code to compile; we can read it from file system, retrieve from database, or generate it dynamically in memory!!

/**Java source code to be compiled dynamically*/
static String sourceCode = "package com.accordess.ca;" +
    "class DynamicCompilationHelloWorld{" +
        "public static void main (String args[]){" +
            "System.out.println (\"Hello, dynamic compilation world!\");" +
        "}" +
    "}" ;

2. Create a JavaFileObject instance for each of the compilation unit.

• a) If the source is not from file system, then we need to write a class implementing fromJavaFileObject interface. Java 6 provides a sample implementation of this in the form ofSimpleJavaFileObject. We can extend from this and customize it as per our needs.

/**
 * Creates a dynamic source code file object
 *
 * This is an example of how we can prepare a dynamic java source code for compilation.
 * This class reads the java code from a string and prepares a JavaFileObject
 *
 */
class DynamicJavaSourceCodeObject extends SimpleJavaFileObject{
    private String qualifiedName ;
    private String sourceCode ;

    /**
     * Converts the name to an URI, as that is the format expected by JavaFileObject
     *
     *
     * @param fully qualified name given to the class file
     * @param code the source code string
     */
    protected DynamicJavaSourceCodeObject(String name, String code) {
        super (URI.create( "string:///" +name.replaceAll( "\\." , "/" ) + Kind.SOURCE.extension), Kind.SOURCE);
        this .qualifiedName = name ;
        this .sourceCode = code ;
    }

    @Override
    public CharSequence getCharContent( boolean ignoreEncodingErrors)
            throws IOException {
        return sourceCode ;
    }

    public String getQualifiedName() {
        return qualifiedName;
    }

    public void setQualifiedName(String qualifiedName) {
        this .qualifiedName = qualifiedName;
    }

    public String getSourceCode() {
        return sourceCode;
    }

    public void setSourceCode(String sourceCode) {
        this .sourceCode = sourceCode;
    }
}

Once you have this customized JavaFileObject implemented, make sure you create objects of this for each of your dynamic java source code entity.

/*Creating dynamic java source code file object*/
SimpleJavaFileObject fileObject = new DynamicJavaSourceCodeObject ( "com.accordess.ca.DynamicCompilationHelloWorld" , sourceCode) ;
JavaFileObject javaFileObjects[] = new JavaFileObject[]{fileObject} ;

• b) If the source code is from file system, then create JavaFileObjectinstances from the File objects read from the file system.

/*Java source files read from file system*/
File []files = new File[]{file1, file2} ;
Iterable<? extends JavaFileObject> compilationUnits1 =
           fileManager.getJavaFileObjectsFromFiles(Arrays.asList(files1));

3. Build a list of all your compilation units

/* Prepare a list of compilation units (java source code file objects) to input to compilation task*/
Iterable<? extends JavaFileObject> compilationUnits = Arrays.asList(javaFileObjects);

4. If you need to provide any compilation options that you use in command line ‘javac’, such as ‘-d’, ‘-classpath’ and etc. Create a list of these options as a list of string objects.

/*Prepare any compilation options to be used during compilation*/
//In this example, we are asking the compiler to place the output files under bin folder.
String[] compileOptions = new String[]{ "-d" , "bin" } ;
Iterable<String> compilationOptionss = Arrays.asList(compileOptions);

5. Now, retrieve the compiler from ToolProvider.

/*Instantiating the java compiler*/
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

This gets the compiler from the current platform.

6. As a next step, get a standard file manager from compiler, this file manager helps us to customize how a compiler reads and writes to files.

/**
 * Retrieving the standard file manager from compiler object, which is used to provide
 * basic building block for customizing how a compiler reads and writes to files.
 *
 * The same file manager can be reopened for another compiler task.
 * Thus we reduce the overhead of scanning through file system and jar files each time
 */
StandardJavaFileManager stdFileManager = compiler.getStandardFileManager( null , Locale.getDefault(), null );

7. Create a diagnostic collector object, which collects the compilation problems.

/*Create a diagnostic controller, which holds the compilation problems*/
DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();

8. Create a compilation task from compiler object by passing all of the above building blocks as input as required.

/*Create a compilation task from compiler by passing in the required input objects prepared above*/
CompilationTask compilerTask = compiler.getTask( null , stdFileManager, diagnostics, compilationOptionss, null , compilationUnits) ;

Let us spend some time understanding this line.

Here is the declaration of the above method from JavaCompiler class

CompilationTask getTask(Writer out,
                           JavaFileManager fileManager,
                           DiagnosticListener<? super JavaFileObject> diagnosticListener,
                           Iterable<String> options,
                           Iterable<String> classes,
                           Iterable<? extends JavaFileObject> compilationUnits);

out: is a writer object, if not null, this would be used to write all the compilation errors. If null, the errors are written to the standard error console, i.e., System.err.

fileManager: is an instance of the JavaFileManager, which abstracts programming language source and class files. In this context,file means an abstraction of regular files and other sources of data.

diagnosticListener: which acts as a listener to the compilation events happening and logs any issues found. TheDiagnosticCollector, we have created here is impleting this listener and that stores all the compilation diagnostic messages. If this is not passed, the compilation issues will be logged on standard error console.

options: the compilation options to be passed during compilation, this can be null if there are no options to be used.

classes: class names (for annotation processing), null means no class names

compilationUnits: these are the list of JavaFileObject instances, that need to be compiled.

9. Finally, call the method ‘call’ on compilation task, which does the actual job, and returns ‘true’ on success or ‘false’ otherwise.

//Perform the compilation by calling the call method on compilerTask object.
boolean status = compilerTask.call();

10. On compilation failure, we can use the diagnostic collector to read the error messages and log them in specific format.

if (!status){ //If compilation error occurs
    /*Iterate through each compilation problem and print it*/
    for (Diagnostic diagnostic : diagnostics.getDiagnostics()){
        System.out.format( "Error on line %d in %s" , diagnostic.getLineNumber(), diagnostic);
    }
}

If we need to compile another set of compilation units, just create another compilation task by passing the new set of compilation units and execute the call method on it.

Finally close the fileManager instance to flush out anything that is there in the buffer.

try {
            stdFileManager.close() ; //Close the file manager
    } catch (IOException e) {
            e.printStackTrace();
    }

that’s it. We are pretty much done with the example.  The piece of code mentioned over here doesn’t result into any errors. Introduce some error into the code string and play with it. By the way, after successful compilation the class file would be generated under the current folder if you don’t pass in the java options I mentioned over here. If you are passing the same java options I mentioned here, make sure you create a folder by the name ‘bin’ under your current folder. Otherwise this will result into an error.

Who will benefit from this feature?

Application server developers: Application server need to generate java files from JSP code and compile them dynamically, thus reducing the application hot deployment time.

IDEs and Developer Tools like Ant: This API helps them to load the compiler once and perform compilation as and when needed instead of loading an external compiler each time the code changes.

Here is the full-version of the example source code.

package com.accordess.ca;

import java.io.IOException;
import java.net.URI;
import java.util.Arrays;
import java.util.Locale;
import java.util.logging.Logger;

import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaCompiler.CompilationTask;
import javax.tools.JavaFileObject;
import javax.tools.SimpleJavaFileObject;
import javax.tools.StandardJavaFileManager;
import javax.tools.ToolProvider;

/**
 * A test class to test dynamic compilation API.
 *
 */
public class CompilerAPITest {
    final Logger logger = Logger.getLogger(CompilerAPITest. class .getName()) ;

    /**Java source code to be compiled dynamically*/
    static String sourceCode = "package com.accordess.ca;" +
        "class DynamicCompilationHelloWorld{" +
            "public static void main (String args[]){" +
                "System.out.println (\"Hello, dynamic compilation world!\");" +
            "}" +
        "}" ;

    /**
     * Does the required object initialization and compilation.
     */
    public void doCompilation (){
        /*Creating dynamic java source code file object*/
        SimpleJavaFileObject fileObject = new DynamicJavaSourceCodeObject ("com.accordess.ca.DynamicCompilationHelloWorld", sourceCode) ;
        JavaFileObject javaFileObjects[] = new JavaFileObject[]{fileObject} ;

        /*Instantiating the java compiler*/
        JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

        /**
         * Retrieving the standard file manager from compiler object, which is used to provide
         * basic building block for customizing how a compiler reads and writes to files.
         *
         * The same file manager can be reopened for another compiler task.
         * Thus we reduce the overhead of scanning through file system and jar files each time
         */
        StandardJavaFileManager stdFileManager = compiler.getStandardFileManager(null, Locale.getDefault(), null);

        /* Prepare a list of compilation units (java source code file objects) to input to compilation task*/
        Iterable<? extends JavaFileObject> compilationUnits = Arrays.asList(javaFileObjects);

        /*Prepare any compilation options to be used during compilation*/
        //In this example, we are asking the compiler to place the output files under bin folder.
        String[] compileOptions = new String[]{"-d", "bin"} ;
        Iterable<String> compilationOptionss = Arrays.asList(compileOptions);

        /*Create a diagnostic controller, which holds the compilation problems*/
        DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();

        /*Create a compilation task from compiler by passing in the required input objects prepared above*/
        CompilationTask compilerTask = compiler.getTask(null, stdFileManager, diagnostics, compilationOptionss, null, compilationUnits) ;

        //Perform the compilation by calling the call method on compilerTask object.
        boolean status = compilerTask.call();

        if (!status){//If compilation error occurs
            /*Iterate through each compilation problem and print it*/
            for (Diagnostic diagnostic : diagnostics.getDiagnostics()){
                System.out.format("Error on line %d in %s", diagnostic.getLineNumber(), diagnostic);
            }
        }
        try {
            stdFileManager.close() ;//Close the file manager
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static void main(String args[]){
        new CompilerAPITest ().doCompilation() ;
    }

}

/**
 * Creates a dynamic source code file object
 *
 * This is an example of how we can prepare a dynamic java source code for compilation.
 * This class reads the java code from a string and prepares a JavaFileObject
 *
 */
class DynamicJavaSourceCodeObject extends SimpleJavaFileObject{
    private String qualifiedName ;
    private String sourceCode ;

    /**
     * Converts the name to an URI, as that is the format expected by JavaFileObject
     *
     *
     * @param fully qualified name given to the class file
     * @param code the source code string
     */
    protected DynamicJavaSourceCodeObject(String name, String code) {
        super (URI.create( "string:///" +name.replaceAll( "\\." , "/" ) + Kind.SOURCE.extension), Kind.SOURCE);
        this .qualifiedName = name ;
        this .sourceCode = code ;
    }

    @Override
    public CharSequence getCharContent( boolean ignoreEncodingErrors)
            throws IOException {
        return sourceCode ;
    }

    public String getQualifiedName() {
        return qualifiedName;
    }

    public void setQualifiedName(String qualifiedName) {
        this .qualifiedName = qualifiedName;
    }

    public String getSourceCode() {
        return sourceCode;
    }

    public void setSourceCode(String sourceCode) {
        this .sourceCode = sourceCode;
    }
}

I have also attached the eclipse project to this post. You can download and import this project intoEclipse IDE. You will be ready to play with it…!

Download the source code eclipse project here