对象序列化 avro的使用

1. 官网

https://avro.apache.org/

2.Avro简介

Apache Avro(以下简称 Avro)是一个数据序列化系统,是一种与编程语言无关的序列化格式,是提供一种共享数据文件的方式。Avro是Hadoop中的一个子项目,Avro是一个基于二进制数据传输高性能的中间件。Avro可以做到将数据进行序列化,适用于远程或本地大批量数据交互。在传输的过程中Avro对数据二进制序列化后节约数据存储空间和网络传输带宽。
序列化就是将对象转换成二进制流,相应的反序列化就是将二进制流再转换成对应的对象。因此,Avro就是用来在传输数据之前,将对象转换成二进制流,然后此二进制流达到目标地址后,Avro再将二进制流转换成对象

3.Avro 的特点

1.丰富的数据结构
2.一个紧凑的,快速的,二进制的数据格式
3.一个容器文件,来存储持久化数据
4.远程过程调用(RPC)
5.简单的动态语言集成。
6.Avro模式是使用JSON定义的 。这有助于以已经具有JSON库的语言实现。
JSON是一种轻量级的数据传输格式,对于大数据集,JSON数据会显示力不从心,因为JSON的格式是key:value型,每条记录都要附上key的名字,
有的时候,光key消耗的空间甚至会超过value所占空间,这对空间的浪费十分严重,尤其是对大型数据集来说,因为它不仅不够紧凑,还要重复
地加上key信息,不仅会造成存储空间上的浪费,更会增加了数据传输的压力,从而给集群增加负担,进而影响整个集群的吞吐量。而采用Avro
数据序列化系统可以比较好的解决此问题,因为用Avro序列化后的文件由schema和真实内容组成,schema只是数据的元数据,相当于JSON数据
的key信息,schema单独存放在一个JSON文件中,这样一来,数据的元数据只存了一次,相比JSON数据格式的文件,大大缩小了存储容量。从而
使得Avro文件可以更加紧凑地组织数据

4.原生类型

原生类型如下所示:
null: 表示没有值
boolean: 表示一个二进制布尔值
int: 表示32位有符号整数
long: 表示64位有符号整数
float: 表示32位的单精度浮点数
double: 表示64位双精度浮点数
bytes: 表示8位的无符号字节序列
string: Unicode 编码的字符序列

5.复杂类型

AVRO支持6种复杂类型,分别是:records, enums, arrays, maps, unions,fixed,常用的 Recoeds类型。
1.Records
Records使用类型名称“record”,并且支持三个必选属性。
type: 必有属性。
name: 必有属性,是一个JSON string,提供了记录的名字。
namespace,也是一个JSON string,用来限定和修饰name属性。
doc: 可选属性,是一个JSON string,为使用这个Schema的用户提供文档。
aliases: 可选属性,是JSON的一个string数组,为这条记录提供别名。
fields: 必选属性,是一个JSON数组,数组中列举了所有的field。
每一个field都是一个JSON对象,并且具有如下属性:
(1)name: 必选属性,field的名字,是一个JSON string
(2)doc: 可选属性,为使用此Schema的用户提供了描述此field的文档。
(3)type: 必选属性,定义Schema的一个JSON对象,或者是命名一条记录定义的JSON string。
(4default: 可选属性,即field的默认值,当读到缺少这个field的实例时用到。默认值的允许的范围由这个field的Schama的类型决定。
order: 可选属性,指定这个field如何影响record的排序。有效的可选值为“ascending”(默认),"descending""ignore"
alias: JSON的string数组,为这个field提供别名。

5.代码实战

对象序列化 avro的使用_第1张图片

5.1 添加依赖 和插件

最新的版本:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>wudl-bigdata</artifactId>
        <groupId>com-wudl-bigdata</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>kafka-avro</artifactId>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro</artifactId>
            <version>1.11.0</version>
        </dependency>


    </dependencies>


    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.avro</groupId>
                <artifactId>avro-maven-plugin</artifactId>
                <version>1.11.0</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>schema</goal>
                        </goals>
                        <configuration>
                            <sourceDirectory>${project.basedir}/src/main/avro/</sourceDirectory>
                            <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>

        </plugins>
    </build>

</project>

5.2 定义定义schema

{"namespace": "com.wudl.avro.bean",
 "type": "record",
 "name": "User",
 "fields": [
     {"name": "id", "type": "int"},
     {"name": "name", "type": "string"},
     {"name": "adress",  "type":"string"},
     {"name": "age",  "type":"int"}

 ]
}

5.3 用maven 插件 package 打包 代码自动生产的类

`用maven  clean package  就能生产相应的序列化类`
内容如下: 
/**
 * Autogenerated by Avro
 *
 * DO NOT EDIT DIRECTLY
 */
package com.wudl.avro.bean;

import org.apache.avro.generic.GenericArray;
import org.apache.avro.specific.SpecificData;
import org.apache.avro.util.Utf8;
import org.apache.avro.message.BinaryMessageEncoder;
import org.apache.avro.message.BinaryMessageDecoder;
import org.apache.avro.message.SchemaStore;

@org.apache.avro.specific.AvroGenerated
public class User extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
  private static final long serialVersionUID = 8427336651465463617L;


  public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"com.wudl.avro.bean\",\"fields\":[{\"name\":\"id\",\"type\":\"int\"},{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"adress\",\"type\":\"string\"},{\"name\":\"age\",\"type\":\"int\"}]}");
  public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }

  private static final SpecificData MODEL$ = new SpecificData();

  private static final BinaryMessageEncoder<User> ENCODER =
      new BinaryMessageEncoder<User>(MODEL$, SCHEMA$);

  private static final BinaryMessageDecoder<User> DECODER =
      new BinaryMessageDecoder<User>(MODEL$, SCHEMA$);

  /**
   * Return the BinaryMessageEncoder instance used by this class.
   * @return the message encoder used by this class
   */
  public static BinaryMessageEncoder<User> getEncoder() {
    return ENCODER;
  }

  /**
   * Return the BinaryMessageDecoder instance used by this class.
   * @return the message decoder used by this class
   */
  public static BinaryMessageDecoder<User> getDecoder() {
    return DECODER;
  }

  /**
   * Create a new BinaryMessageDecoder instance for this class that uses the specified {@link SchemaStore}.
   * @param resolver a {@link SchemaStore} used to find schemas by fingerprint
   * @return a BinaryMessageDecoder instance for this class backed by the given SchemaStore
   */
  public static BinaryMessageDecoder<User> createDecoder(SchemaStore resolver) {
    return new BinaryMessageDecoder<User>(MODEL$, SCHEMA$, resolver);
  }

  /**
   * Serializes this User to a ByteBuffer.
   * @return a buffer holding the serialized data for this instance
   * @throws java.io.IOException if this instance could not be serialized
   */
  public java.nio.ByteBuffer toByteBuffer() throws java.io.IOException {
    return ENCODER.encode(this);
  }

  /**
   * Deserializes a User from a ByteBuffer.
   * @param b a byte buffer holding serialized data for an instance of this class
   * @return a User instance decoded from the given buffer
   * @throws java.io.IOException if the given bytes could not be deserialized into an instance of this class
   */
  public static User fromByteBuffer(
      java.nio.ByteBuffer b) throws java.io.IOException {
    return DECODER.decode(b);
  }

  private int id;
  private java.lang.CharSequence name;
  private java.lang.CharSequence adress;
  private int age;

  /**
   * Default constructor.  Note that this does not initialize fields
   * to their default values from the schema.  If that is desired then
   * one should use newBuilder().
   */
  public User() {}

  /**
   * All-args constructor.
   * @param id The new value for id
   * @param name The new value for name
   * @param adress The new value for adress
   * @param age The new value for age
   */
  public User(java.lang.Integer id, java.lang.CharSequence name, java.lang.CharSequence adress, java.lang.Integer age) {
    this.id = id;
    this.name = name;
    this.adress = adress;
    this.age = age;
  }

  public org.apache.avro.specific.SpecificData getSpecificData() { return MODEL$; }
  public org.apache.avro.Schema getSchema() { return SCHEMA$; }
  // Used by DatumWriter.  Applications should not call.
  public java.lang.Object get(int field$) {
    switch (field$) {
    case 0: return id;
    case 1: return name;
    case 2: return adress;
    case 3: return age;
    default: throw new IndexOutOfBoundsException("Invalid index: " + field$);
    }
  }

  // Used by DatumReader.  Applications should not call.
  @SuppressWarnings(value="unchecked")
  public void put(int field$, java.lang.Object value$) {
    switch (field$) {
    case 0: id = (java.lang.Integer)value$; break;
    case 1: name = (java.lang.CharSequence)value$; break;
    case 2: adress = (java.lang.CharSequence)value$; break;
    case 3: age = (java.lang.Integer)value$; break;
    default: throw new IndexOutOfBoundsException("Invalid index: " + field$);
    }
  }

  /**
   * Gets the value of the 'id' field.
   * @return The value of the 'id' field.
   */
  public int getId() {
    return id;
  }


  /**
   * Sets the value of the 'id' field.
   * @param value the value to set.
   */
  public void setId(int value) {
    this.id = value;
  }

  /**
   * Gets the value of the 'name' field.
   * @return The value of the 'name' field.
   */
  public java.lang.CharSequence getName() {
    return name;
  }


  /**
   * Sets the value of the 'name' field.
   * @param value the value to set.
   */
  public void setName(java.lang.CharSequence value) {
    this.name = value;
  }

  /**
   * Gets the value of the 'adress' field.
   * @return The value of the 'adress' field.
   */
  public java.lang.CharSequence getAdress() {
    return adress;
  }


  /**
   * Sets the value of the 'adress' field.
   * @param value the value to set.
   */
  public void setAdress(java.lang.CharSequence value) {
    this.adress = value;
  }

  /**
   * Gets the value of the 'age' field.
   * @return The value of the 'age' field.
   */
  public int getAge() {
    return age;
  }


  /**
   * Sets the value of the 'age' field.
   * @param value the value to set.
   */
  public void setAge(int value) {
    this.age = value;
  }

  /**
   * Creates a new User RecordBuilder.
   * @return A new User RecordBuilder
   */
  public static com.wudl.avro.bean.User.Builder newBuilder() {
    return new com.wudl.avro.bean.User.Builder();
  }

  /**
   * Creates a new User RecordBuilder by copying an existing Builder.
   * @param other The existing builder to copy.
   * @return A new User RecordBuilder
   */
  public static com.wudl.avro.bean.User.Builder newBuilder(com.wudl.avro.bean.User.Builder other) {
    if (other == null) {
      return new com.wudl.avro.bean.User.Builder();
    } else {
      return new com.wudl.avro.bean.User.Builder(other);
    }
  }

  /**
   * Creates a new User RecordBuilder by copying an existing User instance.
   * @param other The existing instance to copy.
   * @return A new User RecordBuilder
   */
  public static com.wudl.avro.bean.User.Builder newBuilder(com.wudl.avro.bean.User other) {
    if (other == null) {
      return new com.wudl.avro.bean.User.Builder();
    } else {
      return new com.wudl.avro.bean.User.Builder(other);
    }
  }

  /**
   * RecordBuilder for User instances.
   */
  @org.apache.avro.specific.AvroGenerated
  public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase<User>
    implements org.apache.avro.data.RecordBuilder<User> {

    private int id;
    private java.lang.CharSequence name;
    private java.lang.CharSequence adress;
    private int age;

    /** Creates a new Builder */
    private Builder() {
      super(SCHEMA$, MODEL$);
    }

    /**
     * Creates a Builder by copying an existing Builder.
     * @param other The existing Builder to copy.
     */
    private Builder(com.wudl.avro.bean.User.Builder other) {
      super(other);
      if (isValidValue(fields()[0], other.id)) {
        this.id = data().deepCopy(fields()[0].schema(), other.id);
        fieldSetFlags()[0] = other.fieldSetFlags()[0];
      }
      if (isValidValue(fields()[1], other.name)) {
        this.name = data().deepCopy(fields()[1].schema(), other.name);
        fieldSetFlags()[1] = other.fieldSetFlags()[1];
      }
      if (isValidValue(fields()[2], other.adress)) {
        this.adress = data().deepCopy(fields()[2].schema(), other.adress);
        fieldSetFlags()[2] = other.fieldSetFlags()[2];
      }
      if (isValidValue(fields()[3], other.age)) {
        this.age = data().deepCopy(fields()[3].schema(), other.age);
        fieldSetFlags()[3] = other.fieldSetFlags()[3];
      }
    }

    /**
     * Creates a Builder by copying an existing User instance
     * @param other The existing instance to copy.
     */
    private Builder(com.wudl.avro.bean.User other) {
      super(SCHEMA$, MODEL$);
      if (isValidValue(fields()[0], other.id)) {
        this.id = data().deepCopy(fields()[0].schema(), other.id);
        fieldSetFlags()[0] = true;
      }
      if (isValidValue(fields()[1], other.name)) {
        this.name = data().deepCopy(fields()[1].schema(), other.name);
        fieldSetFlags()[1] = true;
      }
      if (isValidValue(fields()[2], other.adress)) {
        this.adress = data().deepCopy(fields()[2].schema(), other.adress);
        fieldSetFlags()[2] = true;
      }
      if (isValidValue(fields()[3], other.age)) {
        this.age = data().deepCopy(fields()[3].schema(), other.age);
        fieldSetFlags()[3] = true;
      }
    }

    /**
      * Gets the value of the 'id' field.
      * @return The value.
      */
    public int getId() {
      return id;
    }


    /**
      * Sets the value of the 'id' field.
      * @param value The value of 'id'.
      * @return This builder.
      */
    public com.wudl.avro.bean.User.Builder setId(int value) {
      validate(fields()[0], value);
      this.id = value;
      fieldSetFlags()[0] = true;
      return this;
    }

    /**
      * Checks whether the 'id' field has been set.
      * @return True if the 'id' field has been set, false otherwise.
      */
    public boolean hasId() {
      return fieldSetFlags()[0];
    }


    /**
      * Clears the value of the 'id' field.
      * @return This builder.
      */
    public com.wudl.avro.bean.User.Builder clearId() {
      fieldSetFlags()[0] = false;
      return this;
    }

    /**
      * Gets the value of the 'name' field.
      * @return The value.
      */
    public java.lang.CharSequence getName() {
      return name;
    }


    /**
      * Sets the value of the 'name' field.
      * @param value The value of 'name'.
      * @return This builder.
      */
    public com.wudl.avro.bean.User.Builder setName(java.lang.CharSequence value) {
      validate(fields()[1], value);
      this.name = value;
      fieldSetFlags()[1] = true;
      return this;
    }

    /**
      * Checks whether the 'name' field has been set.
      * @return True if the 'name' field has been set, false otherwise.
      */
    public boolean hasName() {
      return fieldSetFlags()[1];
    }


    /**
      * Clears the value of the 'name' field.
      * @return This builder.
      */
    public com.wudl.avro.bean.User.Builder clearName() {
      name = null;
      fieldSetFlags()[1] = false;
      return this;
    }

    /**
      * Gets the value of the 'adress' field.
      * @return The value.
      */
    public java.lang.CharSequence getAdress() {
      return adress;
    }


    /**
      * Sets the value of the 'adress' field.
      * @param value The value of 'adress'.
      * @return This builder.
      */
    public com.wudl.avro.bean.User.Builder setAdress(java.lang.CharSequence value) {
      validate(fields()[2], value);
      this.adress = value;
      fieldSetFlags()[2] = true;
      return this;
    }

    /**
      * Checks whether the 'adress' field has been set.
      * @return True if the 'adress' field has been set, false otherwise.
      */
    public boolean hasAdress() {
      return fieldSetFlags()[2];
    }


    /**
      * Clears the value of the 'adress' field.
      * @return This builder.
      */
    public com.wudl.avro.bean.User.Builder clearAdress() {
      adress = null;
      fieldSetFlags()[2] = false;
      return this;
    }

    /**
      * Gets the value of the 'age' field.
      * @return The value.
      */
    public int getAge() {
      return age;
    }


    /**
      * Sets the value of the 'age' field.
      * @param value The value of 'age'.
      * @return This builder.
      */
    public com.wudl.avro.bean.User.Builder setAge(int value) {
      validate(fields()[3], value);
      this.age = value;
      fieldSetFlags()[3] = true;
      return this;
    }

    /**
      * Checks whether the 'age' field has been set.
      * @return True if the 'age' field has been set, false otherwise.
      */
    public boolean hasAge() {
      return fieldSetFlags()[3];
    }


    /**
      * Clears the value of the 'age' field.
      * @return This builder.
      */
    public com.wudl.avro.bean.User.Builder clearAge() {
      fieldSetFlags()[3] = false;
      return this;
    }

    @Override
    @SuppressWarnings("unchecked")
    public User build() {
      try {
        User record = new User();
        record.id = fieldSetFlags()[0] ? this.id : (java.lang.Integer) defaultValue(fields()[0]);
        record.name = fieldSetFlags()[1] ? this.name : (java.lang.CharSequence) defaultValue(fields()[1]);
        record.adress = fieldSetFlags()[2] ? this.adress : (java.lang.CharSequence) defaultValue(fields()[2]);
        record.age = fieldSetFlags()[3] ? this.age : (java.lang.Integer) defaultValue(fields()[3]);
        return record;
      } catch (org.apache.avro.AvroMissingFieldException e) {
        throw e;
      } catch (java.lang.Exception e) {
        throw new org.apache.avro.AvroRuntimeException(e);
      }
    }
  }

  @SuppressWarnings("unchecked")
  private static final org.apache.avro.io.DatumWriter<User>
    WRITER$ = (org.apache.avro.io.DatumWriter<User>)MODEL$.createDatumWriter(SCHEMA$);

  @Override public void writeExternal(java.io.ObjectOutput out)
    throws java.io.IOException {
    WRITER$.write(this, SpecificData.getEncoder(out));
  }

  @SuppressWarnings("unchecked")
  private static final org.apache.avro.io.DatumReader<User>
    READER$ = (org.apache.avro.io.DatumReader<User>)MODEL$.createDatumReader(SCHEMA$);

  @Override public void readExternal(java.io.ObjectInput in)
    throws java.io.IOException {
    READER$.read(this, SpecificData.getDecoder(in));
  }

  @Override protected boolean hasCustomCoders() { return true; }

  @Override public void customEncode(org.apache.avro.io.Encoder out)
    throws java.io.IOException
  {
    out.writeInt(this.id);

    out.writeString(this.name);

    out.writeString(this.adress);

    out.writeInt(this.age);

  }

  @Override public void customDecode(org.apache.avro.io.ResolvingDecoder in)
    throws java.io.IOException
  {
    org.apache.avro.Schema.Field[] fieldOrder = in.readFieldOrderIfDiff();
    if (fieldOrder == null) {
      this.id = in.readInt();

      this.name = in.readString(this.name instanceof Utf8 ? (Utf8)this.name : null);

      this.adress = in.readString(this.adress instanceof Utf8 ? (Utf8)this.adress : null);

      this.age = in.readInt();

    } else {
      for (int i = 0; i < 4; i++) {
        switch (fieldOrder[i].pos()) {
        case 0:
          this.id = in.readInt();
          break;

        case 1:
          this.name = in.readString(this.name instanceof Utf8 ? (Utf8)this.name : null);
          break;

        case 2:
          this.adress = in.readString(this.adress instanceof Utf8 ? (Utf8)this.adress : null);
          break;

        case 3:
          this.age = in.readInt();
          break;

        default:
          throw new java.io.IOException("Corrupt ResolvingDecoder.");
        }
      }
    }
  }
}

5.4 序列化和反序列的使用

package com.wudl.avro.testtool;

import com.wudl.avro.bean.User;
import org.apache.avro.file.DataFileReader;
import org.apache.avro.file.DataFileWriter;
import org.apache.avro.file.SeekableByteArrayInput;
import org.apache.avro.io.DatumReader;
import org.apache.avro.io.DatumWriter;
import org.apache.avro.specific.SpecificDatumReader;
import org.apache.avro.specific.SpecificDatumWriter;

import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

/**
 * @author :wudl
 * @date :Created in 2022-01-11 23:40
 * @description:
 * @modified By:
 * @version: 1.0
 */

public class UserTest {

    public static void main(String[] args) throws IOException {
        // 序列化
        // SerializingUser();
        // 反序列化
        // DeserializingUser();
        // Byte 序列化
        byte[] bytes = byteSerializingUser();
        // 将数据反序列化
        byteDeserializingUser(bytes);

    }

    /**
     * User 类的序列化
     */
    private static void SerializingUser() throws IOException {
        List<User> list = new ArrayList<>();
        User u = new User();
        u.setId(1);
        u.setName("flink");
        u.setAdress("深圳");
        u.setAge(21);
        list.add(u);

        User u01 = new User();
        u01.setId(2);
        u01.setName("flink02");
        u01.setAdress("深圳-02");
        u01.setAge(20);
        list.add(u01);

//        DatumWriter userDatumWriter = new SpecificDatumWriter(User.class);
//        DataFileWriter dataFileWriter = new DataFileWriter(userDatumWriter);
//        dataFileWriter.create(u.getSchema(), new File("D:\\tmp\\users-QQ.avro"));
//        dataFileWriter.append(u);
//        dataFileWriter.append(u01);
//        System.out.println(dataFileWriter);
//        dataFileWriter.close();

        DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
        DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
        dataFileWriter.create(u.getSchema(), new File("D:\\tmp\\users-QQ.avro"));
        // List 数组序列化
        for (User user : list) {
            dataFileWriter.append(user);
        }
        System.out.println(dataFileWriter);
        dataFileWriter.close();
    }

    /**
     * 反序列化方式
     */
    private static void DeserializingUser() throws IOException {
        DatumReader<User> userDatumReader = new SpecificDatumReader<User>(User.class);
        DataFileReader<User> dataFileReader = new DataFileReader<User>(new File("D:\\tmp\\users-QQ.avro"), userDatumReader);
        User user = null;
        while (dataFileReader.hasNext()) {
            user = dataFileReader.next(user);
            System.out.println(user);
        }
    }


    /**
     * 序列化Byte 数组类型
     *
     * @return
     * @throws IOException
     */
    private static byte[] byteSerializingUser() throws IOException {

        List<User> list = new ArrayList<>();
        User u = new User();
        u.setId(1);
        u.setName("flink");
        u.setAdress("深圳");
        u.setAge(21);
        list.add(u);

        User u01 = new User();
        u01.setId(2);
        u01.setName("flink02");
        u01.setAdress("深圳-02");
        u01.setAge(20);
        list.add(u01);
        ByteArrayOutputStream byteUser = new ByteArrayOutputStream();
        DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
        DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
        dataFileWriter.create(list.get(0).getSchema(), byteUser);
        for (User user : list) {
            dataFileWriter.append(user);
        }
        dataFileWriter.close();
        System.out.println(byteUser.toString());
        return byteUser.toByteArray();
    }


    /**
     * 反序列化
     *
     * @param usersByteArray
     * @throws IOException
     */
    public static void byteDeserializingUser(byte[] usersByteArray) throws IOException {
        SeekableByteArrayInput sbai = new SeekableByteArrayInput(usersByteArray);
        DatumReader<User> userDatumReader = new SpecificDatumReader<User>(User.class);
        DataFileReader<User> dataFileReader = new DataFileReader<User>(sbai, userDatumReader);
        System.out.println("----------------deserialzeAvroFromByteArray-------------------");
        User readUser = null;
        while (dataFileReader.hasNext()) {
            readUser = dataFileReader.next(readUser);
            System.out.println(readUser);
        }
    }

}

6. 效果

"C:\Program Files\Java\jdk1.8.0_241\bin\java.exe" -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:56135,suspend=y,server=n -Dvisualvm.id=6229052673800 -javaagent:C:\Users\Administrator\AppData\Local\JetBrains\IntelliJIdea2020.3\captureAgent\debugger-agent.jar -Dfile.encoding=UTF-8 -classpath "C:\Program Files\Java\jdk1.8.0_241\jre\lib\charsets.jar;C:\Program 2020.3.4\lib\idea_rt.jar" com.wudl.avro.testtool.UserTest
Connected to the target VM, address: '127.0.0.1:56135', transport: 'socket'
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Objavro.schema�{"type":"record","name":"User","namespace":"com.wudl.avro.bean","fields":[{"name":"id","type":"int"},{"name":"name","type":"string"},{"name":"adress","type":"string"},{"name":"age","type":"int"}]} �Y%4��lV�c�F
flink深圳*flink02深圳-02(�Y%4��lV�c�
----------------deserialzeAvroFromByteArray-------------------
{"id": 1, "name": "flink", "adress": "深圳", "age": 21}
{"id": 2, "name": "flink02", "adress": "深圳-02", "age": 20}
Disconnected from the target VM, address: '127.0.0.1:56135', transport: 'socket'

Process finished with exit code 0

你可能感兴趣的:(BigData-tools,java,java,rpc,开发语言,big,data)