Java相关的序列化与反序列化

一、序列化的目的

  • 存储
  • 传输
  • 深拷贝
  • 扩大权限

定义

  • 将对象的状态信息组织为字节流(二进制流)的形式过程。

实质

  • 一种用来处理对象流的机制
  • 结构化数据的编码

二、序列化的技术分类

  • Java语言专用

    • Java内置序列化
    • kryo
    • FST
  • IDL

    • Thrift

    • Avro

    • protocol buffer

  • 非IDL技术

    • XML
    • JSON

三、Java序列化核心技术

  • 实现方式

    • 自定义实现Serializable(readObject和writeObject方法 )

      • 可申明readObject和writeObject自定义序列化和反序列化方式,JDK内置彩蛋
    • 自定义类实现Externalizable,覆盖readExternal和writeExternal方法

  • Java内置序列化方式一

import lombok.AllArgsConstructor;
import lombok.Data;
import java.io.Serializable;

@AllArgsConstructor
@Data
public class Person implements Serializable {

    private static final long serialVersionUID = 1L;

    private String name;

    private int age;

//    private void writeObject(ObjectOutputStream outputStream) throws Exception{
//        //实现
//    }
//
//    private void readObject(ObjectInputStream inputStream) throws Exception{
//        //实现
//    }
}
  • 序列化内置方式二

import lombok.AllArgsConstructor;
import lombok.Data;

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
@AllArgsConstructor
@Data
public class Person2 implements Externalizable {

    private static final long serialVersionUID = 1L;

    private String name;

    private int age;

    // 必不可少,否则会抛异常
    public Person2(){}

    @Override
    public void writeExternal(ObjectOutput out) throws IOException {

    }

    @Override
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {

    }
}
  • 测试代码

import java.io.*;

public class Main {
    public static void main(String[] args) {
        serialize();
        deserialize();
    }

    private static void serialize(){
        Person person = new Person("test1", 4);
        try(ObjectOutputStream outputStream =
                    new ObjectOutputStream(new FileOutputStream(new File("out1")))){
            outputStream.writeObject(person);
        }catch (IOException e){
            e.printStackTrace();
        }
    }

    private static void deserialize(){
        try(ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream("out1"))){
            Person person = (Person)inputStream.readObject();
            System.out.println(person.toString());
        }catch (IOException | ClassNotFoundException e){
            e.printStackTrace();
        }
    }
}
  • 写入磁盘的内容为明文(左侧16进制,右侧解码内容)
    • notepad++插件hex editor

Java相关的序列化与反序列化_第1张图片

  • 代码分析

    • ObjectOutputStream outputStream =
      new ObjectOutputStream(new FileOutputStream(new File(“out1”)))
      • new FileOutputStream(new File(“out1”)
        • 采用适配器模式将File类适配为OutputStream类
          • 适配器模式
            • 不同类型适配
      • new ObjectOutputStream(new FileOutputStream(new File(“out1”)))
        • 采用装饰器模式将FileOutputStream扩展为ObjectOutputStream
        • 装饰器模式
          • 同类型功能扩展
  • 核心类

    • 序列化

      • ObjectOutputStream
    • 反序列化

      • ObjectInputStream

JDK通过ObjectOutputStream实现序列化,基本所有序列化的功能都在ObjectOutputStream内部类中实现,封装在此对象中,降低继承带来的高复杂度

其内部类结构如下:
Java相关的序列化与反序列化_第2张图片

  • Java序列化基本类型处理

    • •Java序列化对基本类型的处理,严格按照其内存占用大小进行
    • 采用大端的方式写入
      • 内部写入调用此类java.io.Bits(大端写入)
        Java相关的序列化与反序列化_第3张图片
        Java相关的序列化与反序列化_第4张图片
  • 使用序列化做深拷贝

    • 实现原型模式(又称克隆模式)

import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.io.*;
import java.util.Date;

@AllArgsConstructor
@Data
@NoArgsConstructor
public class Person implements Serializable, Cloneable {

    private static final long serialVersionUID = 1L;

    private String name;

    private Date birthday;

    @Override
    protected Object clone() throws CloneNotSupportedException {
        return super.clone();
    }

    protected Person deepCloneBySerialize(){
        try(ByteArrayOutputStream bos = new ByteArrayOutputStream();
                ObjectOutputStream oos = new ObjectOutputStream(bos)){
            oos.writeObject(this);
            try(ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bos.toByteArray()))){
                return (Person)ois.readObject();
            }
        } catch (IOException | ClassNotFoundException e) {
            e.printStackTrace();
        }
        throw new RuntimeException();
    }

}
  • 测试代码

import com.github.xiaour.learning.serial.performance.StandardEntity;
import org.springframework.util.StopWatch;

import java.util.Date;

public class Main {

    public static void main(String[] args) throws CloneNotSupportedException {
        testPerformance();
    }
    private static void testPerformance() throws CloneNotSupportedException {
        StopWatch stopWatch = new StopWatch("new Object performance");
        stopWatch.start("new Object");
        for (int i = 0; i < 10000; i++) {
            StandardEntity standardEntity = new StandardEntity();
        }
        stopWatch.stop();
        stopWatch.start("copy Object");
        Person person = new Person("test", new Date());
        for (int i = 0; i < 10000; i++) {
            Person person1 = person.deepCloneBySerialize();
        }
        stopWatch.stop();
        System.out.println(stopWatch.prettyPrint());
    }

    private static void test() throws CloneNotSupportedException {
        Person person = new Person("test", new Date());
        Person personClone = (Person)person.clone();
        Person deepCloneBySerialize = person.deepCloneBySerialize();

        System.out.println("origin Person: " + person);
        System.out.println("clone Person: " + personClone);
        System.out.println("deepCloneBySerialize Person: " + deepCloneBySerialize);

        person.getBirthday().setTime(System.currentTimeMillis() / 10);
        System.out.println("-----------------after set-----------------------");

        System.out.println("origin Person: " + person);
        System.out.println("clone Person: " + personClone);
        System.out.println("deepCloneBySerialize Person: " + deepCloneBySerialize);
    }
}
  • 破坏单例

import java.io.*;


public class Singleton implements Serializable, Cloneable {

    private static final long serialVersionUID = 1L;

    private volatile static Singleton singleton;
    private Singleton (){}
    public static Singleton getSingleton() {
        if (singleton == null) {
            synchronized (Singleton.class) {
                if (singleton == null) {
                    singleton = new Singleton();
                }
            }
        }
        return singleton;
    }


}
  • 测试代码
import com.alibaba.fastjson.JSON;

import java.io.*;

public class Main {

    public static void main(String[] args) {
        Singleton singleton = Singleton.getSingleton();
        Singleton singleton1 = Singleton.getSingleton();
        Singleton singleton2 = deepCloneBySerialize(singleton);

        System.out.println(singleton == singleton1);
        System.out.println(singleton == singleton2);


    }

    protected static Singleton deepCloneBySerialize(Singleton singleton){
        try(ByteArrayOutputStream bos = new ByteArrayOutputStream();
            ObjectOutputStream oos = new ObjectOutputStream(bos)){
            oos.writeObject(singleton);
            try(ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(bos.toByteArray()))){
                return (Singleton)ois.readObject();
            }
        } catch (IOException | ClassNotFoundException e) {
            e.printStackTrace();
        }
        throw new RuntimeException();
    }

    protected static Singleton deeepCopyByJSON(Singleton singleton){
        return JSON.parseObject(JSON.toJSONString(singleton), Singleton.class);
    }
}

原型模式与单例模式本身是互斥的,所以序列化能实现原型模式,则必定会破坏单例

  • Java序列化特性
    • transient关键字和static关键字申明字段不会被序列化
      • JDK源码ObjectStreamClass 中的getDefaultSerialFields
    • serialVersionUID实现版本兼容
      • 当类字段发生变化, serialVersionUID值会重新计算,导致反序列化失败
    • 安全性问题(Java序列化后的数据是明文)
      • 数据校验
        • 实现ObjectInputValidation
      • 加密
        • SealedObject
          • 对称加密
        • SignedObject
          • 非对称加密
  • 加密一

import lombok.AllArgsConstructor;
import lombok.Data;

import javax.crypto.*;
import java.io.*;
import java.security.InvalidKeyException;
import java.security.Key;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;

@AllArgsConstructor
@Data
public class Person implements Serializable {

    private static final long serialVersionUID = 1L;

    static Main instance;
    static Key key;
    static Cipher encryptCipher;
    static Cipher decryptCipher;

    private String name;

    private int age;

    public Person(String name, int age, String strKey) {
        this.name = name;
        this.age = age;
        key = setKey(strKey);
        try {
            encryptCipher = Cipher.getInstance("DES");
            encryptCipher.init(Cipher.ENCRYPT_MODE, key);
            decryptCipher = Cipher.getInstance("DES");
            decryptCipher.init(Cipher.DECRYPT_MODE, key);
        } catch (NoSuchAlgorithmException | NoSuchPaddingException | InvalidKeyException e) {
            e.printStackTrace();
        }
    }

    private Key setKey(String strKey) {
        try {
            KeyGenerator _generator = KeyGenerator.getInstance("DES");
            _generator.init(new SecureRandom(strKey.getBytes()));
            return _generator.generateKey();

        } catch (Exception e) {
            e.printStackTrace();
        }

        return null;
    }

    void serialize(){
        try(ObjectOutputStream outputStream =
                    new ObjectOutputStream(new FileOutputStream("out1"))){
            SealedObject so = new SealedObject(this, encryptCipher);
            outputStream.writeObject(so);
        }catch (IOException | IllegalBlockSizeException e){
            e.printStackTrace();
        }
    }

    void deserialize(){
        try(ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream("out1"))){
            SealedObject sealedObject = (SealedObject)inputStream.readObject();
            Person person = (Person)sealedObject.getObject(key);
            System.out.println(person.toString());
        }catch (IOException | ClassNotFoundException | NoSuchAlgorithmException | InvalidKeyException e){
            e.printStackTrace();
        }
    }

}
  • 测试
import java.security.NoSuchAlgorithmException;

public class Main {

    public static void main(String[] args) throws NoSuchAlgorithmException {
        Person person = new Person("test1",22,"11");
        person.serialize();
        person.deserialize();
    }

}
  • 加密二

import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.io.*;
import java.security.*;

@AllArgsConstructor
@Data
@NoArgsConstructor
@Builder
public class Person implements Serializable {

    private static final long serialVersionUID = 1L;

    static Main instance;
    static KeyPair keyPair;
    static Signature signature;

    private String name;

    private int age;

    private KeyPair setKey() {
        try {
            signature = Signature.getInstance("SHA1withRSA");
            return KeyPairGenerator.getInstance("RSA").generateKeyPair();
        } catch (Exception e) {
            e.printStackTrace();
        }

        return null;
    }

    void serialize(){
        keyPair = setKey();
        try(ObjectOutputStream outputStream =
                    new ObjectOutputStream(new FileOutputStream("out1"))){
            SignedObject so = new SignedObject(this, keyPair.getPrivate(),signature);
            outputStream.writeObject(so);
        }catch (IOException | SignatureException | InvalidKeyException e){
            e.printStackTrace();
        }
    }

    void deserialize(){
        try(ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream("out1"))){
            SignedObject sealedObject = (SignedObject)inputStream.readObject();
            Person person = (Person)sealedObject.getObject();
            System.out.println(person.toString());
        }catch (IOException | ClassNotFoundException e){
            e.printStackTrace();
        }
    }

}
  • 测试

import java.security.NoSuchAlgorithmException;

public class Main {

    public static void main(String[] args) throws NoSuchAlgorithmException {
        Person person = new Person("test1",22);
        person.serialize();
        person.deserialize();
    }

}

Java第三方框架序列化

  • kryo(推荐使用)
    • https://github.com/EsotericSoftware/kryo/blob/master/README.md
    • 高效率
    • 低存储
    • 便捷的API
类型 方案 补充
Int、long Varint + ZigZag 主流序列化标配
String 若可用ascii码,则采用一个字节 char转byte(JDK9的String类)
bool、byte、char、short 和java自带序列化方案一致 不做优化处理
float IEEE 754编码标准,转换为int类型(Float .floatToIntBits) 然后按大端序列,使用固定长度4字节来存储float,
double Double遵循IEEE 754编码标准转换为Long 然后才去固定8字节存储
  • FST
    • https://www.pudn.com/news/628f8373bf399b7f351e909d.html
    • 比JDK序列化快十倍
    • 100%兼容JDK序列化
    • 堆外内存
    • JSON(不推荐,有许多更好的JSON框架)

Interface Description Language

Java相关的序列化与反序列化_第5张图片

技术实现:Thrift(facebook), protocol Buffer(Google), Avro(Apache)

  • 有比较完整的规约和框架实现
  • 不同语言(组件)间的通信
  • 在RPC中使用

Thrift

  • https://github.com/apache/thrift
  • 支持28种语言
  • 点对点RPC实现

Java相关的序列化与反序列化_第6张图片

非IDL

JavaScript Object Notation

Java相关的序列化与反序列化_第7张图片

JSON语法

  • 两种结构
    • 键值结构
    • 数组结构
  • WS
    • 若干个( 空字符串、空格、\r、\n、\t )

Java相关的序列化与反序列化_第8张图片
Java相关的序列化与反序列化_第9张图片

GSON

  • Maven
		<dependency>
			<groupId>com.google.code.gsongroupId>
			<artifactId>gsonartifactId>
			<version>2.9.1version>
		dependency>
  • fastJson
<dependency>
			<groupId>com.alibabagroupId>
			<artifactId>fastjsonartifactId>
			<version>1.2.76version>
		dependency>

Java第三方框架

  • kryo
<dependency>
			<groupId>com.esotericsoftwaregroupId>
			<artifactId>kryoartifactId>
			<version>5.3.0version>
		dependency>
  • fst
<dependency>
			<groupId>de.ruedigermoellergroupId>
			<artifactId>fstartifactId>
			<version>2.56version>
		dependency>
  • kryo代码实例
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.io.*;
import java.util.Date;

@AllArgsConstructor
@Data
@NoArgsConstructor
public class Person {

    private String name;

    private Date birthday;

}

public class HelloKryo {
    static public void main (String[] args) throws Exception {
        Kryo kryo = new Kryo();
        kryo.register(SomeClass.class);

        SomeClass object = new SomeClass();
        object.value = "Hello Kryo!";

        try(Output output = new Output(new FileOutputStream("file.bin"))){
            kryo.writeObject(output, object);
        }

        Input input = new Input(new FileInputStream("file.bin"));
        SomeClass object2 = kryo.readObject(input, SomeClass.class);
        // 必须执行close()或者flush()方法确保buffer数据写入OutputStream
        input.close();
    }
    static public class SomeClass {
        String value;
    }
}

import com.esotericsoftware.kryo.Kryo;
import org.objenesis.strategy.StdInstantiatorStrategy;

import java.util.Date;

public class DeepAndShallowCopy {
    public static void main(String[] args) {
        Kryo kryo = new Kryo();
        // 最重要的三个配置
        kryo.setReferences(false);
        kryo.setRegistrationRequired(false);
        kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
        Person person = new Person("test", new Date());
        Person personCopy = kryo.copy(person);
        Person personCopyShallow = kryo.copyShallow(person);

        person.getBirthday().setTime(System.currentTimeMillis() / 10);

        System.out.println(personCopy.getBirthday());
        System.out.println(personCopyShallow.getBirthday());
    }
}

  • fst

import java.io.Serializable;

public class User implements Serializable {
    private String username;
    private int age;
    private String password;

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

}

public class FSTSeriazle {

    public static void main(String[] args) {
        User bean = new User();
        bean.setUsername("xxxxx");
        bean.setPassword("123456");
        bean.setAge(1000000);
        System.out.println("序列化 , 反序列化 对比测试:");
        long size = 0;
        long time1 = System.currentTimeMillis();
        for (int i = 0; i < 10000; i++) {
            byte[] jdkserialize = JRedisSerializationUtils.jdkserialize(bean);
            size += jdkserialize.length;
            JRedisSerializationUtils.jdkdeserialize(jdkserialize);
        }
        System.out.println("原生序列化方案[序列化10000次]耗时:"
                + (System.currentTimeMillis() - time1) + "ms size:=" + size);

        size = 0;
        long time2 = System.currentTimeMillis();
        for (int i = 0; i < 10000; i++) {
            byte[] serialize = JRedisSerializationUtils.serialize(bean);
            size += serialize.length;
            User u = (User) JRedisSerializationUtils.unserialize(serialize);
        }
        System.out.println("fst序列化方案[序列化10000次]耗时:"
                + (System.currentTimeMillis() - time2) + "ms size:=" + size);
        size = 0;
        long time3 = System.currentTimeMillis();
        for (int i = 0; i < 10000; i++) {
            byte[] serialize = JRedisSerializationUtils.kryoSerizlize(bean);
            size += serialize.length;
            User u = (User) JRedisSerializationUtils.kryoUnSerizlize(serialize);
        }
        System.out.println("kryo序列化方案[序列化10000次]耗时:"
                + (System.currentTimeMillis() - time3) + "ms size:=" + size);

    }

}
  • Java堆外内存(非JVM堆中分配)

import java.nio.ByteBuffer;

public class JavaOffHeap {

    static ByteBuffer byteBuffer;

    public static void main(String[] args) {
        byte[] bytes = "hello world".getBytes();
        allocateDirect(bytes);
        byte[] bytes1 = getBytes(bytes.length);
        System.out.println(new String(bytes1));
    }

    public static void allocateDirect(byte[] bytes){
        // 定义好要申请的堆外内存的大小,这里是1GB
        int memorySize = 1024 * 1024 * 1024;
        // 用Java里的ByteBuffer.allocateDirect方法就可以申请一块堆外内存
        byteBuffer = ByteBuffer.allocateDirect(memorySize);
        // 把数据写入到堆外内存里去
        byteBuffer.put(bytes);

    }

    public static byte[] getBytes(int length){
        // 从堆外内存里读取数据
        byteBuffer.flip();
        byte[] readBytes = new byte[length];
        byteBuffer.get(readBytes, 0, length);
        return  readBytes;
    }
}

四、性能对比

Java相关的序列化与反序列化_第10张图片
Java相关的序列化与反序列化_第11张图片

你可能感兴趣的:(Java,java,序列化,反序列化)