Protocol Buffer(Protobuf) For Java
为什么用到了Protocol Buffer
后端服务语言:Java
前端JavaScript
交互协议 全双工 WebSocket
数据交互:数据序列化后的Protocol Buffer 二进制数据
采用Protocol Buffer 序列化后体积小,传输速度快。
Protocol Buffer Basics: Java: https://developers.google.com/protocol-buffers/docs/javatutorial
Netty 实战P159
Protocol Buffers 以一种紧凑而高效的方式对结构化的数据进行编码以及解码。他具有许多编程语言的绑定,使他很适合跨语言项目。
github:https://github.com/protocolbuffers/protobuf/releases/tag/v3.6.1
protostuff
maven:
io.protostuff
protostuff-core
1.5.9
io.protostuff
protostuff-runtime
1.5.9
简版序列化和反序列化:ProtoBufUtil
public class ProtoBufUtil {
public static byte[] serializer(T o) {
Schema schema = RuntimeSchema.getSchema(o.getClass());
return ProtobufIOUtil.toByteArray(o, schema, LinkedBuffer.allocate(LinkedBuffer.MIN_BUFFER_SIZE));
}
public static T deserializer(byte[] bytes, Class clazz) {
Schema schema = RuntimeSchema.getSchema(clazz);
T obj = schema.newMessage();
ProtostuffIOUtil.mergeFrom(bytes, obj, schema);
return obj;
}
}
Exception in thread "main" java.lang.RuntimeException: Reading from a byte array threw an IOException (should never happen).
at io.protostuff.IOUtil.mergeFrom(IOUtil.java:54)
at io.protostuff.ProtostuffIOUtil.mergeFrom(ProtostuffIOUtil.java:104)
at com.lsj.sanguo.common.util.ProtoBufUtil.deserializer(ProtoBufUtil.java:41)
at com.lsj.sanguo.common.util.ProtoBufUtil.main(ProtoBufUtil.java:63)
Caused by: io.protostuff.ProtobufException: Protocol message contained an invalid tag (zero).
at io.protostuff.ProtobufException.invalidTag(ProtobufException.java:106)
at io.protostuff.ByteArrayInput.readFieldNumber(ByteArrayInput.java:253)
at io.protostuff.runtime.RuntimeSchema.mergeFrom(RuntimeSchema.java:457)
at io.protostuff.ByteArrayInput.mergeObjectEncodedAsGroup(ByteArrayInput.java:518)
at io.protostuff.ByteArrayInput.mergeObject(ByteArrayInput.java:490)
at io.protostuff.runtime.RuntimeRepeatedFieldFactory$3.mergeFrom(RuntimeRepeatedFieldFactory.java:157)
at io.protostuff.runtime.RuntimeSchema.mergeFrom(RuntimeSchema.java:466)
at io.protostuff.IOUtil.mergeFrom(IOUtil.java:45)
... 3 more
###这个出现的原因可能是:
1.你在序列化数组,map等对象
2.字节数组在传输过程中丢失了
****问题困扰了我许久
3.用了我上面的工具类:错误原因是你用ProtobufIOUtil进行序列化，而用ProtostuffIOUtil进行反序列化
注意这两个Util不一致哦~~~ 我被折磨了30+天
但是序列化和反序列化普通对象的时候没有问题，且序列化给前端也没有问题，我猜想可能前端接收做了适配，但序列化并没有做适配工作。
又因为protostuff的效率比ProtobufIOUtil效率高，因此我现在采用的是：
/**
* 描述
*
* @author BoomMan
*/
@Log4j2
@NoArgsConstructor
public class ProtoBufUtil {
public static byte[] serializer(T o) {
Schema schema = RuntimeSchema.getSchema(o.getClass());
return ProtostuffIOUtil.toByteArray(o, schema, LinkedBuffer.allocate(LinkedBuffer.MIN_BUFFER_SIZE));
}
public static T deserializer(byte[] bytes, Class clazz) {
Schema schema = RuntimeSchema.getSchema(clazz);
T obj = schema.newMessage();
ProtobufIOUtil.mergeFrom(bytes, obj, schema);
return obj;
}
}
但是为了稳定使用如下配置较好
/**
* 描述
*
* @author BoomMan
*/
@Log4j2
@NoArgsConstructor
public class ProtoBufUtil {
public static byte[] serializer(T o) {
Schema schema = RuntimeSchema.getSchema(o.getClass());
return ProtobufIOUtil.toByteArray(o, schema, LinkedBuffer.allocate(LinkedBuffer.MIN_BUFFER_SIZE));
}
public static T deserializer(byte[] bytes, Class clazz) {
Schema schema = RuntimeSchema.getSchema(clazz);
T obj = schema.newMessage();
ProtobufIOUtil.mergeFrom(bytes, obj, schema);
return obj;
}
}
2018.11月16最新
支持List 和Map
import io.protostuff.LinkedBuffer;
import io.protostuff.ProtobufIOUtil;
import io.protostuff.Schema;
import io.protostuff.runtime.RuntimeSchema;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.extern.log4j.Log4j2;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
/**
* 描述 Google Protocol Buffer Utils
* BoomMan [email protected]
* 对 List 和Map 进行简单封装 使用时用接口不要用实现类
*/
@Log4j2
@NoArgsConstructor
public class ProtoBufUtil {
public static byte[] serializer(T o) {
if (o instanceof List || o instanceof Map) {
@SuppressWarnings("unchecked")
Schema schema = RuntimeSchema.getSchema(SerializerWrapper.class);
return ProtobufIOUtil.toByteArray(new SerializerWrapper(o), schema, LinkedBuffer.allocate(LinkedBuffer.MIN_BUFFER_SIZE));
}
@SuppressWarnings("unchecked")
Schema schema = (Schema) RuntimeSchema.getSchema(o.getClass());
return ProtobufIOUtil.toByteArray(o, schema, LinkedBuffer.allocate(LinkedBuffer.MIN_BUFFER_SIZE));
}
public static T deserializer(byte[] bytes, Class clazz) {
if (clazz==List.class || clazz == Map.class) {
Schema wrapperSchema = RuntimeSchema.getSchema(SerializerWrapper.class);
SerializerWrapper wrapper = wrapperSchema.newMessage();
ProtobufIOUtil.mergeFrom(bytes, wrapper, wrapperSchema);
return (T) wrapper.getData();
}
Schema schema = RuntimeSchema.getSchema(clazz);
T obj = schema.newMessage();
ProtobufIOUtil.mergeFrom(bytes, obj, schema);
return obj;
}
@Data
@NoArgsConstructor
@AllArgsConstructor
private static class SerializerWrapper {
private T data;
}
public static void main(String[] args) {
ArrayList testList = new ArrayList<>();
testList.add("123");
testList.add("456");
testList.add("789");
testList.add("测试");
byte[] serializer = ProtoBufUtil.serializer(testList);
log.error("序列化结果: {}", serializer);
List deserializer = ProtoBufUtil.deserializer(serializer, List.class);
log.error("反序列化结果: {}", deserializer);
}
}