Java相关的序列化与反序列化
一、序列化的目的
- 存储
- 传输
- 深拷贝
- 扩大权限
定义
- 将对象的状态信息组织为字节流（二进制流）的形式过程。
实质
- 一种用来处理对象流的机制
- 结构化数据的编码
二、序列化的技术分类
-
Java语言专用
- Java内置序列化
- kryo
- FST
-
IDL
-
Thrift
-
Avro
-
protocol buffer
-
-
非IDL技术
- XML
- JSON
三、Java序列化核心技术
-
实现方式
-
自定义实现Serializable（readObject和writeObject方法 ）
- 可申明readObject和writeObject自定义序列化和反序列化方式，JDK内置彩蛋
-
自定义类实现Externalizable，覆盖readExternal和writeExternal方法
-
-
Java内置序列化方式一
import lombok.AllArgsConstructor;
import lombok.Data;
import java.io.Serializable;
@AllArgsConstructor
@Data
public class Person implements Serializable {
private static final long serialVersionUID = 1L;
private String name;
private int age;
// private void writeObject(ObjectOutputStream outputStream) throws Exception{
// //实现
// }
//
// private void readObject(ObjectInputStream inputStream) throws Exception{
// //实现
// }
}
- 序列化内置方式二
import lombok.AllArgsConstructor;
import lombok.Data;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
@AllArgsConstructor
@Data
public class Person2 implements Externalizable {
private static final long serialVersionUID = 1L;
private String name;
private int age;
// 必不可少，否则会抛异常
public Person2(){}
@Override
public void writeExternal(ObjectOutput out) throws IOException {
}
@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
}
}
- 测试代码
import java.io.*;
public class Main {
public static void main(String[] args) {
serialize();
deserialize();
}
private static void serialize(){
Person person = new Person("test1", 4);
try(ObjectOutputStream outputStream =
new ObjectOutputStream(new FileOutputStream(new File("out1")))){
outputStream.writeObject(person);
}catch (IOException e){
e.printStackTrace();
}
}
private static void deserialize(){
try(ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream("out1"))){
Person person = (Person)inputStream.readObject();
System.out.println(person.toString());
}catch (IOException | ClassNotFoundException e){
e.printStackTrace();
}
}
}
- 写入磁盘的内容为明文（左侧16进制，右侧解码内容）
- notepad++插件hex editor
-
代码分析
- ObjectOutputStream outputStream =
new ObjectOutputStream(new FileOutputStream(new File(“out1”)))- new FileOutputStream(new File(“out1”)
- 采用适配器模式将File类适配为OutputStream类
- 适配器模式
- 不同类型适配
- 适配器模式
- 采用适配器模式将File类适配为OutputStream类
- new ObjectOutputStream(new FileOutputStream(new File(“out1”)))
- 采用装饰器模式将FileOutputStream扩展为ObjectOutputStream
- 装饰器模式
- 同类型功能扩展
- new FileOutputStream(new File(“out1”)
- ObjectOutputStream outputStream =
-
核心类
-
序列化
- ObjectOutputStream
-
反序列化
- ObjectInputStream
-
JDK通过ObjectOutputStream实现序列化，基本所有序列化的功能都在ObjectOutputStream内部类中实现，封装在此对象中，降低继承带来的高复杂度
其内部类结构如下：
-
Java序列化基本类型处理
- •Java序列化对基本类型的处理，严格按照其内存占用大小进行
- 采用大端的方式写入
- 内部写入调用此类java.io.Bits（大端写入）
- 内部写入调用此类java.io.Bits（大端写入）
-
使用序列化做深拷贝
- 实现原型模式（又称克隆模式）
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.io.*;
import java.util.Date;
@AllArgsConstructor
@Data
@NoArgsConstructor
public class Person implements Serializable, Cloneable {
private static final long serialVersionUID = 1L;
private String name;
private Date birthday;
@Override
protected Object clone() throws CloneNotSupportedException {
return super.clone();
}
protected Person deepCloneBySerialize(){
try(ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(bos)){
oos.writeObject(this);
try(ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bos.toByteArray()))){
return (Person)ois.readObject();
}
} catch (IOException | ClassNotFoundException e) {
e.printStackTrace();
}
throw new RuntimeException();
}
}
- 测试代码
import com.github.xiaour.learning.serial.performance.StandardEntity;
import org.springframework.util.StopWatch;
import java.util.Date;
public class Main {
public static void main(String[] args) throws CloneNotSupportedException {
testPerformance();
}
private static void testPerformance() throws CloneNotSupportedException {
StopWatch stopWatch = new StopWatch("new Object performance");
stopWatch.start("new Object");
for (int i = 0; i < 10000; i++) {
StandardEntity standardEntity = new StandardEntity();
}
stopWatch.stop();
stopWatch.start("copy Object");
Person person = new Person("test", new Date());
for (int i = 0; i < 10000; i++) {
Person person1 = person.deepCloneBySerialize();
}
stopWatch.stop();
System.out.println(stopWatch.prettyPrint());
}
private static void test() throws CloneNotSupportedException {
Person person = new Person("test", new Date());
Person personClone = (Person)person.clone();
Person deepCloneBySerialize = person.deepCloneBySerialize();
System.out.println("origin Person: " + person);
System.out.println("clone Person: " + personClone);
System.out.println("deepCloneBySerialize Person: " + deepCloneBySerialize);
person.getBirthday().setTime(System.currentTimeMillis() / 10);
System.out.println("-----------------after set-----------------------");
System.out.println("origin Person: " + person);
System.out.println("clone Person: " + personClone);
System.out.println("deepCloneBySerialize Person: " + deepCloneBySerialize);
}
}
- 破坏单例
import java.io.*;
public class Singleton implements Serializable, Cloneable {
private static final long serialVersionUID = 1L;
private volatile static Singleton singleton;
private Singleton (){}
public static Singleton getSingleton() {
if (singleton == null) {
synchronized (Singleton.class) {
if (singleton == null) {
singleton = new Singleton();
}
}
}
return singleton;
}
}
- 测试代码
import com.alibaba.fastjson.JSON;
import java.io.*;
public class Main {
public static void main(String[] args) {
Singleton singleton = Singleton.getSingleton();
Singleton singleton1 = Singleton.getSingleton();
Singleton singleton2 = deepCloneBySerialize(singleton);
System.out.println(singleton == singleton1);
System.out.println(singleton == singleton2);
}
protected static Singleton deepCloneBySerialize(Singleton singleton){
try(ByteArrayOutputStream bos = new ByteArrayOutputStream();
ObjectOutputStream oos = new ObjectOutputStream(bos)){
oos.writeObject(singleton);
try(ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bos.toByteArray()))){
return (Singleton)ois.readObject();
}
} catch (IOException | ClassNotFoundException e) {
e.printStackTrace();
}
throw new RuntimeException();
}
protected static Singleton deeepCopyByJSON(Singleton singleton){
return JSON.parseObject(JSON.toJSONString(singleton), Singleton.class);
}
}
原型模式与单例模式本身是互斥的，所以序列化能实现原型模式，则必定会破坏单例
- Java序列化特性
- transient关键字和static关键字申明字段不会被序列化
- JDK源码ObjectStreamClass 中的getDefaultSerialFields
- serialVersionUID实现版本兼容
- 当类字段发生变化， serialVersionUID值会重新计算，导致反序列化失败
- 安全性问题（Java序列化后的数据是明文）
- 数据校验
- 实现ObjectInputValidation
- 加密
- SealedObject
- 对称加密
- SignedObject
- 非对称加密
- SealedObject
- 数据校验
- transient关键字和static关键字申明字段不会被序列化
- 加密一
import lombok.AllArgsConstructor;
import lombok.Data;
import javax.crypto.*;
import java.io.*;
import java.security.InvalidKeyException;
import java.security.Key;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
@AllArgsConstructor
@Data
public class Person implements Serializable {
private static final long serialVersionUID = 1L;
static Main instance;
static Key key;
static Cipher encryptCipher;
static Cipher decryptCipher;
private String name;
private int age;
public Person(String name, int age, String strKey) {
this.name = name;
this.age = age;
key = setKey(strKey);
try {
encryptCipher = Cipher.getInstance("DES");
encryptCipher.init(Cipher.ENCRYPT_MODE, key);
decryptCipher = Cipher.getInstance("DES");
decryptCipher.init(Cipher.DECRYPT_MODE, key);
} catch (NoSuchAlgorithmException | NoSuchPaddingException | InvalidKeyException e) {
e.printStackTrace();
}
}
private Key setKey(String strKey) {
try {
KeyGenerator _generator = KeyGenerator.getInstance("DES");
_generator.init(new SecureRandom(strKey.getBytes()));
return _generator.generateKey();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
void serialize(){
try(ObjectOutputStream outputStream =
new ObjectOutputStream(new FileOutputStream("out1"))){
SealedObject so = new SealedObject(this, encryptCipher);
outputStream.writeObject(so);
}catch (IOException | IllegalBlockSizeException e){
e.printStackTrace();
}
}
void deserialize(){
try(ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream("out1"))){
SealedObject sealedObject = (SealedObject)inputStream.readObject();
Person person = (Person)sealedObject.getObject(key);
System.out.println(person.toString());
}catch (IOException | ClassNotFoundException | NoSuchAlgorithmException | InvalidKeyException e){
e.printStackTrace();
}
}
}
- 测试
import java.security.NoSuchAlgorithmException;
public class Main {
public static void main(String[] args) throws NoSuchAlgorithmException {
Person person = new Person("test1",22,"11");
person.serialize();
person.deserialize();
}
}
- 加密二
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.io.*;
import java.security.*;
@AllArgsConstructor
@Data
@NoArgsConstructor
@Builder
public class Person implements Serializable {
private static final long serialVersionUID = 1L;
static Main instance;
static KeyPair keyPair;
static Signature signature;
private String name;
private int age;
private KeyPair setKey() {
try {
signature = Signature.getInstance("SHA1withRSA");
return KeyPairGenerator.getInstance("RSA").generateKeyPair();
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
void serialize(){
keyPair = setKey();
try(ObjectOutputStream outputStream =
new ObjectOutputStream(new FileOutputStream("out1"))){
SignedObject so = new SignedObject(this, keyPair.getPrivate(),signature);
outputStream.writeObject(so);
}catch (IOException | SignatureException | InvalidKeyException e){
e.printStackTrace();
}
}
void deserialize(){
try(ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream("out1"))){
SignedObject sealedObject = (SignedObject)inputStream.readObject();
Person person = (Person)sealedObject.getObject();
System.out.println(person.toString());
}catch (IOException | ClassNotFoundException e){
e.printStackTrace();
}
}
}
- 测试
import java.security.NoSuchAlgorithmException;
public class Main {
public static void main(String[] args) throws NoSuchAlgorithmException {
Person person = new Person("test1",22);
person.serialize();
person.deserialize();
}
}
Java第三方框架序列化
- kryo（推荐使用）
- https://github.com/EsotericSoftware/kryo/blob/master/README.md
- 高效率
- 低存储
- 便捷的API
| 类型 | 方案 | 补充 |
|---|---|---|
| Int、long | Varint + ZigZag | 主流序列化标配 |
| String | 若可用ascii码，则采用一个字节 | char转byte（JDK9的String类） |
| bool、byte、char、short | 和java自带序列化方案一致 | 不做优化处理 |
| float | IEEE 754编码标准，转换为int类型（Float .floatToIntBits） | 然后按大端序列，使用固定长度4字节来存储float, |
| double | Double遵循IEEE 754编码标准转换为Long | 然后才去固定8字节存储 |
- FST
- https://www.pudn.com/news/628f8373bf399b7f351e909d.html
- 比JDK序列化快十倍
- 100%兼容JDK序列化
- 堆外内存
- JSON（不推荐，有许多更好的JSON框架）
Interface Description Language
技术实现：Thrift(facebook), protocol Buffer(Google), Avro(Apache)
- 有比较完整的规约和框架实现
- 不同语言（组件）间的通信
- 在RPC中使用
Thrift
- https://github.com/apache/thrift
- 支持28种语言
- 点对点RPC实现
非IDL
JavaScript Object Notation
JSON语法
- 两种结构
- 键值结构
- 数组结构
- WS
- 若干个( 空字符串、空格、\r、\n、\t )
GSON
- Maven
<dependency>
<groupId>com.google.code.gsongroupId>
<artifactId>gsonartifactId>
<version>2.9.1version>
dependency>
- fastJson
<dependency>
<groupId>com.alibabagroupId>
<artifactId>fastjsonartifactId>
<version>1.2.76version>
dependency>
Java第三方框架
- kryo
<dependency>
<groupId>com.esotericsoftwaregroupId>
<artifactId>kryoartifactId>
<version>5.3.0version>
dependency>
- fst
<dependency>
<groupId>de.ruedigermoellergroupId>
<artifactId>fstartifactId>
<version>2.56version>
dependency>
- kryo代码实例
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import java.io.*;
import java.util.Date;
@AllArgsConstructor
@Data
@NoArgsConstructor
public class Person {
private String name;
private Date birthday;
}
public class HelloKryo {
static public void main (String[] args) throws Exception {
Kryo kryo = new Kryo();
kryo.register(SomeClass.class);
SomeClass object = new SomeClass();
object.value = "Hello Kryo!";
try(Output output = new Output(new FileOutputStream("file.bin"))){
kryo.writeObject(output, object);
}
Input input = new Input(new FileInputStream("file.bin"));
SomeClass object2 = kryo.readObject(input, SomeClass.class);
// 必须执行close()或者flush()方法确保buffer数据写入OutputStream
input.close();
}
static public class SomeClass {
String value;
}
}
import com.esotericsoftware.kryo.Kryo;
import org.objenesis.strategy.StdInstantiatorStrategy;
import java.util.Date;
public class DeepAndShallowCopy {
public static void main(String[] args) {
Kryo kryo = new Kryo();
// 最重要的三个配置
kryo.setReferences(false);
kryo.setRegistrationRequired(false);
kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
Person person = new Person("test", new Date());
Person personCopy = kryo.copy(person);
Person personCopyShallow = kryo.copyShallow(person);
person.getBirthday().setTime(System.currentTimeMillis() / 10);
System.out.println(personCopy.getBirthday());
System.out.println(personCopyShallow.getBirthday());
}
}
- fst
import java.io.Serializable;
public class User implements Serializable {
private String username;
private int age;
private String password;
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
public String getPassword() {
return password;
}
public void setPassword(String password) {
this.password = password;
}
}
public class FSTSeriazle {
public static void main(String[] args) {
User bean = new User();
bean.setUsername("xxxxx");
bean.setPassword("123456");
bean.setAge(1000000);
System.out.println("序列化 ， 反序列化 对比测试：");
long size = 0;
long time1 = System.currentTimeMillis();
for (int i = 0; i < 10000; i++) {
byte[] jdkserialize = JRedisSerializationUtils.jdkserialize(bean);
size += jdkserialize.length;
JRedisSerializationUtils.jdkdeserialize(jdkserialize);
}
System.out.println("原生序列化方案[序列化10000次]耗时："
+ (System.currentTimeMillis() - time1) + "ms size:=" + size);
size = 0;
long time2 = System.currentTimeMillis();
for (int i = 0; i < 10000; i++) {
byte[] serialize = JRedisSerializationUtils.serialize(bean);
size += serialize.length;
User u = (User) JRedisSerializationUtils.unserialize(serialize);
}
System.out.println("fst序列化方案[序列化10000次]耗时："
+ (System.currentTimeMillis() - time2) + "ms size:=" + size);
size = 0;
long time3 = System.currentTimeMillis();
for (int i = 0; i < 10000; i++) {
byte[] serialize = JRedisSerializationUtils.kryoSerizlize(bean);
size += serialize.length;
User u = (User) JRedisSerializationUtils.kryoUnSerizlize(serialize);
}
System.out.println("kryo序列化方案[序列化10000次]耗时："
+ (System.currentTimeMillis() - time3) + "ms size:=" + size);
}
}
- Java堆外内存(非JVM堆中分配)
import java.nio.ByteBuffer;
public class JavaOffHeap {
static ByteBuffer byteBuffer;
public static void main(String[] args) {
byte[] bytes = "hello world".getBytes();
allocateDirect(bytes);
byte[] bytes1 = getBytes(bytes.length);
System.out.println(new String(bytes1));
}
public static void allocateDirect(byte[] bytes){
// 定义好要申请的堆外内存的大小，这里是1GB
int memorySize = 1024 * 1024 * 1024;
// 用Java里的ByteBuffer.allocateDirect方法就可以申请一块堆外内存
byteBuffer = ByteBuffer.allocateDirect(memorySize);
// 把数据写入到堆外内存里去
byteBuffer.put(bytes);
}
public static byte[] getBytes(int length){
// 从堆外内存里读取数据
byteBuffer.flip();
byte[] readBytes = new byte[length];
byteBuffer.get(readBytes, 0, length);
return readBytes;
}
}
四、性能对比
