对象序列化技术比较,及其使用场景
什么是对象序列化?
序列化(Serialization)是将对象的状态信息转换为可以存储或传输的形式的过程.
这是百度百科的说明,也很浅显易懂,比如说,我有一个java对象,我想传输给远端的程序,如何传输尼?
java提供了序列化接口,只要实现了Serializable接口,就可将java对象序列化成字节,然后以流的形式传输,然后在远端再反序列化成对象,这就达到了传输消息的目的.
但是这种序列化存在一些缺陷,1不能跨语言,2,序列化体积大
所以为了支持跨语言,并提高序列效率,减少传输时的码流大小,好多公司或者大神开发了各种序列化库,hession,protostuff,kryo,jackson,xml等,比如json也算是一种序列化,方式,也是目前使用的最为广泛的一种,但是json严格来说并不是一种序列化,它只是一种通用的信息描述方式,一组json信息,可能对应不同的对象模式.比如你可以将一组json转换成java bean对象也可以转换成map,形式具有不确定性.所以一般的远程调用不会采用json来作为序列化实现.
| 序列类型 | 是否跨语言 | 优缺点 |
|---|---|---|
| hession | 支持 | 跨语言,序列化后体积小,速度较快 |
| protostuff | 支持 | 跨语言,序列化后体积小,速度快,但是需要Schema,可以动态生成 |
| jackson | 支持 | 跨语言,序列化后体积小,速度较快,且具有不确定性 |
| fastjson | 支持 | 跨语言支持较困难,序列化后体积小,速度较快,只支持java,c# |
| kryo | 支持 | 跨语言支持较困难,序列化后体积小,速度较快 |
| fst | 不支持 | 跨语言支持较困难,序列化后体积小,速度较快，兼容jdk |
|jdk|不支持|序列化后体积很大,速度快|
序列化可以做什么?
如第一节提的一样,对象序列化可以用来传输消息,并且以流的形式传输,提高传输效率,使用场景如远程服务调用rpc
各个序列化库比较
0.准备依赖
<dependency>
<groupId>com.dyuproject.protostuffgroupId>
<artifactId>protostuff-coreartifactId>
<version>1.0.8version>
dependency>
<dependency>
<groupId>com.dyuproject.protostuffgroupId>
<artifactId>protostuff-runtimeartifactId>
<version>1.0.8version>
dependency>
<dependency>
<groupId>org.objenesisgroupId>
<artifactId>objenesisartifactId>
<version>2.1version>
dependency>
<dependency>
<groupId>com.cauchogroupId>
<artifactId>hessianartifactId>
<version>4.0.38version>
dependency>
<dependency>
<groupId>com.fasterxml.jackson.coregroupId>
<artifactId>jackson-databindartifactId>
<version>2.9.0version>
dependency>
<dependency>
<groupId>com.alibabagroupId>
<artifactId>fastjsonartifactId>
<version>1.2.47version>
dependency>
<dependency>
<groupId>com.esotericsoftwaregroupId>
<artifactId>kryoartifactId>
<version>4.0.0version>
dependency>
<dependency>
<groupId>de.ruedigermoellergroupId>
<artifactId>fstartifactId>
<version>2.57version>
dependency>1.编写序列化接口
public interface Serializer {
byte[] serialize(T obj);
Object deserialize(byte[] bytes, Class clazz);
} 2.实现接口
hession实现
public class HessianSerializer implements Serializer {
@Override
public byte[] serialize(T obj){
ByteArrayOutputStream os = new ByteArrayOutputStream();
HessianOutput ho = new HessianOutput(os);
try {
ho.writeObject(obj);
} catch (IOException e) {
throw new IllegalStateException(e.getMessage(), e);
}
return os.toByteArray();
}
@Override
public Object deserialize(byte[] bytes, Class clazz) {
ByteArrayInputStream is = new ByteArrayInputStream(bytes);
HessianInput hi = new HessianInput(is);
try {
return hi.readObject();
} catch (IOException e) {
throw new IllegalStateException(e.getMessage(), e);
}
}
} jackson实现
public class JacksonSerializer implements Serializer {
private final static ObjectMapper objectMapper = new ObjectMapper();
@Override
public byte[] serialize(T obj) {
try {
return objectMapper.writeValueAsBytes(obj);
} catch (JsonProcessingException e) {
throw new IllegalStateException(e.getMessage(), e);
}
}
@Override
public Object deserialize(byte[] bytes, Class clazz) {
try {
return objectMapper.readValue(bytes, clazz);
} catch (JsonParseException e) {
throw new IllegalStateException(e.getMessage(), e);
} catch (JsonMappingException e) {
throw new IllegalStateException(e.getMessage(), e);
} catch (IOException e) {
throw new IllegalStateException(e.getMessage(), e);
}
}
} fastjson实现
import com.alibaba.fastjson.JSON;
public class FastJsonSerializer implements Serializer {
@Override
public byte[] serialize(T obj) {
try {
return JSON.toJSONBytes(obj);
} catch (Exception e) {
throw new IllegalStateException(e.getMessage(), e);
}
}
@Override
public Object deserialize(byte[] bytes, Class clazz) {
try {
return JSON.parseObject(bytes, clazz);
} catch (Exception e) {
throw new IllegalStateException(e.getMessage(), e);
}
}
} prostfuff实现
public class ProtostuffSerializer implements Serializer {
private static Objenesis objenesis = new ObjenesisStd(true);
private static Map, Schema> cachedSchema = new ConcurrentHashMap, Schema>();
private static Schema getSchema(Class cls) {
@SuppressWarnings("unchecked")
Schema schema = (Schema) cachedSchema.get(cls);
if (schema == null) {
schema = RuntimeSchema.createFrom(cls);
if (schema != null) {
cachedSchema.put(cls, schema);
}
}
return schema;
}
@Override
public byte[] serialize(T obj) {
@SuppressWarnings("unchecked")
Class cls = (Class) obj.getClass();
LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
try {
Schema schema = getSchema(cls);
return ProtostuffIOUtil.toByteArray(obj, schema, buffer);
} catch (Exception e) {
throw new IllegalStateException(e.getMessage(), e);
} finally {
buffer.clear();
}
}
@Override
public Object deserialize(byte[] bytes, Class clazz) {
try {
T message = (T) objenesis.newInstance(clazz);
Schema schema = getSchema(clazz);
ProtostuffIOUtil.mergeFrom(bytes, message, schema);
return message;
} catch (Exception e) {
throw new IllegalStateException(e.getMessage(), e);
}
}
}
kryo实现
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import com.esotericsoftware.kryo.pool.KryoFactory;
import com.esotericsoftware.kryo.pool.KryoPool;
/**
* @author 都市桃源
*
*/
public class KryoSerializer implements Serializer {
/**
* kryo 不是线程安全的，所以使用池控制
*/
private static final KryoPool kryoPool = new KryoPool.Builder(
new KryoFactory() {
public Kryo create() {
Kryo kryo = new Kryo();
return kryo;
}
}).build();
@Override
public byte[] serialize(T obj) throws Exception {
try (
Output output = new Output(new ByteArrayOutputStream())) {
Kryo kryo = kryoPool.borrow();
kryo.writeObject(output, obj);
kryoPool.release(kryo);
output.flush();
return ((ByteArrayOutputStream) output.getOutputStream()).toByteArray();
}
}
@Override
public Object deserialize(byte[] bytes, Class clazz)
throws Exception {
try (Input input = new Input(new ByteArrayInputStream(bytes))) {
Kryo kryo = kryoPool.borrow();
T res = kryo.readObject(input, clazz);
kryoPool.release(kryo);
return res;
}
}
}
jdk实现
public class JdkSerializer implements Serializer {
@Override
public byte[] serialize(T obj) {
try {
ByteArrayOutputStream byteArr = new ByteArrayOutputStream();
ObjectOutputStream out=new ObjectOutputStream(byteArr);
out.writeObject(obj);
out.flush();
return byteArr.toByteArray();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
@Override
public Object deserialize(byte[] bytes, Class clazz) {
try {
ObjectInputStream input=new ObjectInputStream(new ByteArrayInputStream(bytes));
return input.readObject();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
}
fst实现
import org.nustaq.serialization.FSTConfiguration;
public class FstSerializer implements Serializer {
private static final FSTConfiguration configuration = FSTConfiguration
.createStructConfiguration();
@Override
public byte[] serialize(T obj) {
return configuration.asByteArray(obj);
}
@Override
public Object deserialize(byte[] bytes, Class clazz) {
return configuration.asObject(bytes);
}
} 3,准备序列对象
序列化对象需要实现Serializable,有的需要有的不需要,hession需要,而且还需要有默认的无参构造方法jackson需要
public class User implements Serializable{
private String name;
private String address;
private Integer age;
public User() {
super();
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getAddress() {
return address;
}
public void setAddress(String address) {
this.address = address;
}
public Integer getAge() {
return age;
}
public void setAge(Integer age) {
this.age = age;
}
public User(String name, String address, Integer age) {
super();
this.name = name;
this.address = address;
this.age = age;
}
}
4.序列化速度比较
public class SerTest {
public static void main(String[] args) {
int len=10;
test(new HessianSerializer(), "hession",len);
test(new JacksonSerializer(), "jackson",len);
test(new ProtostuffSerializer(), "protostu",len);
test(new JdkSerializer(),"jdk",len);
test(new KryoSerializer(),"kryo",len);
test(new FstSerializer(),"fst",len);
test(new FastJsonSerializer(),"fastjson",len);
}
public static void test(Serializer s,String tag,int len){
long cap=0;
long sumTime=0;
for(int i=0;inew User("taoyuan"+i, i+"addr", i);
long start=System.currentTimeMillis();
byte[] serialize = s.serialize(u);
cap+=serialize.length;
s.deserialize(serialize, User.class);
long cal= System.currentTimeMillis()-start;
sumTime+=cal;
}
System.out.println(tag+"-->["+len+"]对象序列总体积大小:"+cap);
System.out.println(tag+"->["+len+"]对象总消耗时间为:"+sumTime);
}
}
结果:
hession–>[100]对象序列总体积大小:7780
hession->[100]对象总消耗时间为:197
jackson–>[100]对象序列总体积大小:4770
jackson->[100]对象总消耗时间为:122
protostu–>[100]对象序列总体积大小:2080
protostu->[100]对象总消耗时间为:102
jdk–>[100]对象序列总体积大小:21380
jdk->[100]对象总消耗时间为:122
kryo–>[100]对象序列总体积大小:2016
kryo->[100]对象总消耗时间为:47
fst–>[100]对象序列总体积大小:4880
fst->[100]对象总消耗时间为:35
fastjson–>[100]对象序列总体积大小:4770
fastjson->[100]对象总消耗时间为:206
5结论
从以上结果来看,序列化综合性能比较好的是protostuff和kryo
jdk的序列化速度比较快,就是序列体积较大.
所以当我们的需要开发一套rpc服务时,可以采用kryo来序列化对象,提高传输效率
rpc服务简单实现原理简介
个人理解
要想实现一套rpc服务,首先我们得将我们得从本质理解rpc
rpc简单来说就是序列化+传输协议+调用实现+服务管理
序列化是基础也决定了rpc是否支持跨语言
传输协议可以选择tcp,http,udp等协议
调用实现即rpc服务的业务实现
服务管理即是在以上基础之上的优化管理层次,负载均衡,调用追踪等
一旦我们从本质上理解了rpc之后,写一套简单的rpc服务也是很简单的事情,具体业务再去分析优化,每一层隔离开,优化时,改动也不会太大,,有机会,我们写一篇,手把手教你实现rpc的博文