Extend Gson to support field level custom conversion

Extend Gson to support field level custom conversion

Gson is library created by google guys, it is used for java bean to json serialization/deserialization.  Gson can serialize any java bean, collection, map to json, and don't need to care about the actual type of the object being processed. It also supports type level custom conversion by register custom type adapters or serializer/deserializer for a certain type. However, sometimes we just want to customize the transformation of only one filed of a java bean, but let other fields be processed as normal. consider this example below:

public  class Person {
     private String name;
    
     private  int age;
 
     private  boolean married;
    
     private String partner;

    ...getters and setters
}

If we want the "married" field to be translated to "Married" when it's value is true, otherwise "Not Married". With Gson, we can define a custom serializer for Person class to achieve this:

public  class PersonSerializer  implements JsonSerializer<Person>{

     public JsonElement serialize(Person person, Type typeOfSrc,
            JsonSerializationContext context) {
        JsonObject jPerson =  new JsonObject();
        jPerson.addProperty("name", person.getName());
        jPerson.addProperty("age", person.getAge());
        jPerson.addProperty("married", person.isMarried() ? "Married" : "Not Married");
        jPerson.addProperty("partner", person.getPartner());
         return jPerson;
    }

}

As you can see, we need to write a Serializer class for a type, even we just want to apply some simple conversion rule to a single field. Can we make this in a simpler way ? Let's dig into the source code of Gson!

Gson( final Excluder excluder,  final FieldNamingStrategy fieldNamingPolicy,
       final Map<Type, InstanceCreator<?>> instanceCreators,  boolean serializeNulls,
       boolean complexMapKeySerialization,  boolean generateNonExecutableGson,  boolean htmlSafe,
       boolean prettyPrinting,  boolean serializeSpecialFloatingPointValues,
      LongSerializationPolicy longSerializationPolicy,
      List<TypeAdapterFactory> typeAdapterFactories) {
     this.constructorConstructor =  new ConstructorConstructor(instanceCreators);
     this.serializeNulls = serializeNulls;
     this.generateNonExecutableJson = generateNonExecutableGson;
     this.htmlSafe = htmlSafe;
     this.prettyPrinting = prettyPrinting;

    List<TypeAdapterFactory> factories =  new ArrayList<TypeAdapterFactory>();

     //  built-in type adapters that cannot be overridden
    factories.add(TypeAdapters.JSON_ELEMENT_FACTORY);
    factories.add(ObjectTypeAdapter.FACTORY);

     //  the excluder must precede all adapters that handle user-defined types
    factories.add(excluder);

     //  user's type adapters
    factories.addAll(typeAdapterFactories);

     //  type adapters for basic platform types
    factories.add(TypeAdapters.STRING_FACTORY);
    factories.add(TypeAdapters.INTEGER_FACTORY);
    factories.add(TypeAdapters.BOOLEAN_FACTORY);
    factories.add(TypeAdapters.BYTE_FACTORY);
    factories.add(TypeAdapters.SHORT_FACTORY);
    factories.add(TypeAdapters.newFactory( long. class, Long. class, longAdapter(longSerializationPolicy)));
    factories.add(TypeAdapters.newFactory( double. class, Double. class, doubleAdapter(serializeSpecialFloatingPointValues)));
    factories.add(TypeAdapters.newFactory( float. class, Float. class, floatAdapter(serializeSpecialFloatingPointValues)));
    factories.add(TypeAdapters.NUMBER_FACTORY);
    factories.add(TypeAdapters.CHARACTER_FACTORY);
    factories.add(TypeAdapters.STRING_BUILDER_FACTORY);
    factories.add(TypeAdapters.STRING_BUFFER_FACTORY);
    factories.add(TypeAdapters.newFactory(BigDecimal. class, TypeAdapters.BIG_DECIMAL));
    factories.add(TypeAdapters.newFactory(BigInteger. class, TypeAdapters.BIG_INTEGER));
    factories.add(TypeAdapters.URL_FACTORY);
    factories.add(TypeAdapters.URI_FACTORY);
    factories.add(TypeAdapters.UUID_FACTORY);
    factories.add(TypeAdapters.LOCALE_FACTORY);
    factories.add(TypeAdapters.INET_ADDRESS_FACTORY);
    factories.add(TypeAdapters.BIT_SET_FACTORY);
    factories.add(DateTypeAdapter.FACTORY);
    factories.add(TypeAdapters.CALENDAR_FACTORY);
    factories.add(TimeTypeAdapter.FACTORY);
    factories.add(SqlDateTypeAdapter.FACTORY);
    factories.add(TypeAdapters.TIMESTAMP_FACTORY);
    factories.add(ArrayTypeAdapter.FACTORY);
    factories.add(TypeAdapters.ENUM_FACTORY);
    factories.add(TypeAdapters.CLASS_FACTORY);

     //  type adapters for composite and user-defined types
    factories.add( new CollectionTypeAdapterFactory(constructorConstructor));
    factories.add( new MapTypeAdapterFactory(constructorConstructor, complexMapKeySerialization));
    factories.add( new ReflectiveTypeAdapterFactory(constructorConstructor, fieldNamingPolicy, excluder));

     this.factories = Collections.unmodifiableList(factories);
  }

code above is the invisible constructor of Gson class, this constructor always get called, no matter how you create Gson instance. It accept some parameters that control how Gson will perform serialization/deserialization and a list of user defined type adapter factories. The most important thing here
is the list of user defined type adapter factories, it can holds instances of TypeAdapter, JsonSerializer or JsonDeserializer. For example, the PersonSerializer in previous example will be passed in, and Gson instance will use it to serialize instances of Person class. As there are so many adapter factories, how does Gson know which one to use for a certain type? Let's take a look at the code of the method Gson#getAdapter:

public <T> TypeAdapter<T> getAdapter(TypeToken<T> type) {
    TypeAdapter<?> cached = typeTokenCache.get(type);
     if (cached !=  null) {
       return (TypeAdapter<T>) cached;
    }

    Map<TypeToken<?>, FutureTypeAdapter<?>> threadCalls = calls.get();
     boolean requiresThreadLocalCleanup =  false;
     if (threadCalls ==  null) {
      threadCalls =  new HashMap<TypeToken<?>, FutureTypeAdapter<?>>();
      calls.set(threadCalls);
      requiresThreadLocalCleanup =  true;
    }

     //  the key and value type parameters always agree
    FutureTypeAdapter<T> ongoingCall = (FutureTypeAdapter<T>) threadCalls.get(type);
     if (ongoingCall !=  null) {
       return ongoingCall;
    }

     try {
      FutureTypeAdapter<T> call =  new FutureTypeAdapter<T>();
      threadCalls.put(type, call);

       for (TypeAdapterFactory factory : factories) {
        TypeAdapter<T> candidate = factory.create( this, type);
         if (candidate !=  null) {
          call.setDelegate(candidate);
          typeTokenCache.put(type, candidate);
           return candidate;
        }
      }
       throw  new IllegalArgumentException("GSON cannot handle " + type);
    }  finally {
      threadCalls.remove(type);

       if (requiresThreadLocalCleanup) {
        calls.remove();
      }
    }
  }

Gson will choose the factory which don't return null for method call on TypeAdapter#create(Gson gson, TypeToken<T> type), also notice that all factories are contained in a ArrayList which is ordered, that means the first factory added in will be checked firstly. According to the order factories added in, we can find that, strings and wrapper types of primitive types and some other types will be handled before  collections and maps, and there is
a   ReflectiveTypeAdapterFactory at last, that's something we are looking for! The adapter factory for user defined types, let's dig into it:

/*
 * Copyright (C) 2011 Google Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *       http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
  */

package com.google.gson.internal.bind;

import com.google.gson.FieldNamingStrategy;
import com.google.gson.Gson;
import com.google.gson.JsonSyntaxException;
import com.google.gson.TypeAdapter;
import com.google.gson.TypeAdapterFactory;
import com.google.gson.annotations.SerializedName;
import com.google.gson.extend.Convertor;
import com.google.gson.extend.annotations.Convert;
import com.google.gson.internal.$Gson$Types;
import com.google.gson.internal.ConstructorConstructor;
import com.google.gson.internal.Excluder;
import com.google.gson.internal.ObjectConstructor;
import com.google.gson.internal.Primitives;
import com.google.gson.reflect.TypeToken;
import com.google.gson.stream.JsonReader;
import com.google.gson.stream.JsonToken;
import com.google.gson.stream.JsonWriter;
import java.io.IOException;
import java.lang.reflect.Field;
import java.lang.reflect.Type;
import java.util.LinkedHashMap;
import java.util.Map;

/**
 * Type adapter that reflects over the fields and methods of a class.
  */
public  final  class ReflectiveTypeAdapterFactory  implements TypeAdapterFactory {
   private  final ConstructorConstructor constructorConstructor;
   private  final FieldNamingStrategy fieldNamingPolicy;
   private  final Excluder excluder;

   public ReflectiveTypeAdapterFactory(ConstructorConstructor constructorConstructor,
      FieldNamingStrategy fieldNamingPolicy, Excluder excluder) {
     this.constructorConstructor = constructorConstructor;
     this.fieldNamingPolicy = fieldNamingPolicy;
     this.excluder = excluder;
  }

   public  boolean excludeField(Field f,  boolean serialize) {
     return !excluder.excludeClass(f.getType(), serialize) && !excluder.excludeField(f, serialize);
  }

   private String getFieldName(Field f) {
    SerializedName serializedName = f.getAnnotation(SerializedName. class);
     return serializedName ==  null ? fieldNamingPolicy.translateName(f) : serializedName.value();
  }

   public <T> TypeAdapter<T> create(Gson gson,  final TypeToken<T> type) {
    Class<?  super T> raw = type.getRawType();

     if (!Object. class.isAssignableFrom(raw)) {
       return  null;  //  it's a primitive!
    }

    ObjectConstructor<T> constructor = constructorConstructor.get(type);
     return  new Adapter<T>(constructor, getBoundFields(gson, type, raw));
  }

   private ReflectiveTypeAdapterFactory.BoundField createBoundField(
       final Gson context,  final Field field,  final String name,
       final TypeToken<?> fieldType,  boolean serialize,  boolean deserialize) {
     final  boolean isPrimitive = Primitives.isPrimitive(fieldType.getRawType());

     //  special casing primitives here saves ~5% on Android
     return  new ReflectiveTypeAdapterFactory.BoundField(name, serialize, deserialize) {
       final TypeAdapter<?> typeAdapter = context.getAdapter(fieldType);
      @SuppressWarnings({"unchecked", "rawtypes"})  //  the type adapter and field type always agree
      @Override  void write(JsonWriter writer, Object value)
           throws IOException, IllegalAccessException {
        Object fieldValue = field.get(value);
        TypeAdapter t =  new TypeAdapterRuntimeTypeWrapper(context,  this.typeAdapter, fieldType.getType());
        t.write(writer, fieldValue);
      }
      @Override  void read(JsonReader reader, Object value)
           throws IOException, IllegalAccessException {
        Object fieldValue = typeAdapter.read(reader);
         if (fieldValue !=  null || !isPrimitive) {
          field.set(value, fieldValue);
        }
      }
    };
  }

   private Map<String, BoundField> getBoundFields(Gson context, TypeToken<?> type, Class<?> raw) {
    Map<String, BoundField> result =  new LinkedHashMap<String, BoundField>();
     if (raw.isInterface()) {
       return result;
    }

    Type declaredType = type.getType();
     while (raw != Object. class) {
      Field[] fields = raw.getDeclaredFields();
       for (Field field : fields) {
         boolean serialize = excludeField(field,  true);
         boolean deserialize = excludeField(field,  false);
         if (!serialize && !deserialize) {
           continue;
        }
        field.setAccessible( true);
        Type fieldType = $Gson$Types.resolve(type.getType(), raw, field.getGenericType());
        BoundField boundField = createBoundField(context, field, getFieldName(field),
            TypeToken.get(fieldType), serialize, deserialize);
        BoundField previous = result.put(boundField.name, boundField);
         if (previous !=  null) {
           throw  new IllegalArgumentException(declaredType
              + " declares multiple JSON fields named " + previous.name);
        }
      }
      type = TypeToken.get($Gson$Types.resolve(type.getType(), raw, raw.getGenericSuperclass()));
      raw = type.getRawType();
    }
     return result;
  }

   static  abstract  class BoundField {
     final String name;
     final  boolean serialized;
     final  boolean deserialized;

     protected BoundField(String name,  boolean serialized,  boolean deserialized) {
       this.name = name;
       this.serialized = serialized;
       this.deserialized = deserialized;
    }

     abstract  void write(JsonWriter writer, Object value)  throws IOException, IllegalAccessException;
     abstract  void read(JsonReader reader, Object value)  throws IOException, IllegalAccessException;
  }

   public  static  final  class Adapter<T>  extends TypeAdapter<T> {
     private  final ObjectConstructor<T> constructor;
     private  final Map<String, BoundField> boundFields;

     private Adapter(ObjectConstructor<T> constructor, Map<String, BoundField> boundFields) {
       this.constructor = constructor;
       this.boundFields = boundFields;
    }

    @Override  public T read(JsonReader in)  throws IOException {
       if (in.peek() == JsonToken.NULL) {
        in.nextNull();
         return  null;
      }

      T instance = constructor.construct();

       try {
        in.beginObject();
         while (in.hasNext()) {
          String name = in.nextName();
          BoundField field = boundFields.get(name);
           if (field ==  null || !field.deserialized) {
            in.skipValue();
          }  else {
            field.read(in, instance);
          }
        }
      }  catch (IllegalStateException e) {
         throw  new JsonSyntaxException(e);
      }  catch (IllegalAccessException e) {
         throw  new AssertionError(e);
      }
      in.endObject();
       return instance;
    }

    @Override  public  void write(JsonWriter out, T value)  throws IOException {
       if (value ==  null) {
        out.nullValue();
         return;
      }

      out.beginObject();
       try {
         for (BoundField boundField : boundFields.values()) {
           if (boundField.serialized) {
            out.name(boundField.name);
            boundField.write(out, value);
          }
        }
      }  catch (IllegalAccessException e) {
         throw  new AssertionError();
      }
      out.endObject();
    }
  }
}

It's just as you ever imagined, Gson use java reflection API to serialize/deserialize java beans. Please notice the method ReflectiveTypeAdapterFactory.BoundField#write(JsonWriter writer, Object value), this is where a field of a java bean being translated into json, and we can extend Gson library from here, let's add some code into it:

void write(JsonWriter writer, Object value)  throws IOException,
                    IllegalAccessException, GsonExtensionException {
                Object fieldValue = field.get(value);
                TypeAdapter t =  new TypeAdapterRuntimeTypeWrapper(context,  this.typeAdapter, fieldType.getType());
                
                 if (field.isAnnotationPresent(Convert. class)) {
                    Convert anno = field.getAnnotation(Convert. class);

                    String exp = anno.toJsonExpression();
                     if(exp !=  null && exp.length() > 0){
                        Map<String, Object> vars =  new HashMap<String, Object>();
                        vars.put("object", value);
                        vars.put("field", fieldValue);
                        fieldValue = MVEL.eval(exp, vars);
                    } else{
                        String convertorClass = anno.toJsonConvertor();
                         if(!convertorClass.equals("")){
                             try {
                                Object obj = Class.forName(convertorClass).newInstance();
                                 if(obj  instanceof Convertor){
                                    Convertor convertor = (Convertor)obj;
                                    fieldValue = convertor.Convert(fieldValue);
                                } else{
                                     throw  new GsonExtensionException("the convertor class [" + convertorClass + "] does't implement " + "the cn.ggfan.gson.ex.Convertor interface.");
                                }
                            }  catch (InstantiationException e) {
                                e.printStackTrace();
                                 throw  new GsonExtensionException("the convertor class [" + convertorClass + "] can't be instantiated: " + e.getMessage());
                            }  catch (ClassNotFoundException e) {
                                e.printStackTrace();
                                 throw  new GsonExtensionException("the convertor class [" + convertorClass + "] can't be found on classpath: " + e.getMessage());
                            }
                        }
                    }
                }
                
                t.write(writer, fieldValue);
            }

the annotation:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @ interface Convert {
    
     public String toJsonConvertor()  default "";
    
     public String toJsonExpression()  default "";
    
     public String fromJsonConvertor()  default "";
    
     public String fromJsonExpression()  default "";

}

now annotated Person class:

package cn.ggfan.gson.ex.test.model;

import cn.ggfan.gson.ex.annotations.Convert;

public  class Person {
     private String name;
    
     private  int age;
    
    @Convert(
        toJsonExpression = "field ? '已婚' : '未婚'", 
        fromJsonExpression = "field.equals(\"已婚\") ? true : false"
    )
     private  boolean married;
    
     private String partner;

     public String getName() {
         return name;
    }

     public  void setName(String name) {
         this.name = name;
    }

     public  int getAge() {
         return age;
    }

     public  void setAge( int age) {
         this.age = age;
    }

     public  boolean isMarried() {
         return married;
    }

     public  void setMarried( boolean married) {
         this.married = married;
    }

     public String getPartner() {
         return partner;
    }

     public  void setPartner(String partner) {
         this.partner = partner;
    }

    @Override
     public String toString() {
         return "Person [name=" + name + ", age=" + age + ", married=" + married
                + ", partner=" + partner + "]";
    }

}

the expression used in our annotation is MVEL expressions, I will not explain it here. It just interpret some expression to a value. 
Finally, let's see some example:

        Person girl =  new Person();
        girl.setName("Li Ying");
        girl.setAge(24);
        girl.setMarried( false);
        girl.setPartner("Liu Dehua");
        System.out.println(new Gson().toJson(girl));

output :

{
  "name": "Li Ying",
  "age": 24,
  "married": "未婚",
  "partner": "Liu Dehua"
}

don't be confused, the Chinese word "未婚" means "Not Married", and "已婚" means "Married" :D