Kryo 学习总结
kryo是一个序列化框架
InstantiatorStrategy即初始化策略，默认kryo在反序列化对象时需要对象的类有一个零参数构造器，该构造器可以是private的，kryo通过反射调用该构造器来实例化对象。如果没有这样一个构造器，就需要使用kryo.setInstantiatorStrategy(new StdInstantiatorStrategy())了，该策略通过jvm api创建对象，这会创建一个完全空的对象（即不执行任何代码中的初始化工作），如果对象在实例化时需要一些初始化操作（比如在构造代码块中执行一些计算逻辑），这种策略就不可行了。
比较好的策略是kryo.setInstantiatorStrategy(new Kryo.DefaultInstantiatorStrategy(new StdInstantiatorStrategy()));，即先尝试通过零参数构造实例化对象，如果类中没有零参数构造器，则会使用StdInstantiatorStrategy策略。
References即引用，对A对象序列化时，默认情况下kryo会在每个成员对象第一次序列化时写入一个数字，该数字逻辑上就代表了对该成员对象的引用，如果后续有引用指向该成员对象，则直接序列化之前存入的数字即可，而不需要再次序列化对象本身。这种默认策略对于成员存在互相引用的情况较有利，否则就会造成空间浪费（因为没序列化一个成员对象，都多序列化一个数字），通常情况下可以将该策略关闭，kryo.setReferences(false);
Registration即注册，kryo在序列化对象时，首先会序列化其类的全限定名，由于我们通常序列化的对象都是有限范围内的类的实例，这样重复序列化同样的类的全限定名是低效的。通过注册kryo可以将类的全限定名抽象为一个数字，即用一个数字代表全限定名，这样就要高效一些。kryo.register(SomeClass.class);，注册方法的完整签名为public Registration register (Class type, Serializer serializer, int id)，我们通常只需要使用其重载方法即可public Registration register (Class type)，serializer和id在kryo内部会指定。
PS：使用kryo序列化时，可以使用transient关键字忽略某字段。
序列化方法
kryo大体有三种序列化方法，每种方式都有其优势和劣势。
1，kryo.writeObject，这种方法只会序列化对象实例，而不会记录对象所属类的任何信息。优势是最节省空间，劣势是在反序列化时，需要提供类作为模板才能顺利反序列。反序列化时使用readObject。
2，直接kryo.writeClassAndObject，这种方法会先将对象所属类的全限定名序列化，然后再依次序列化对象实例的成员。优势是完全动态序列化，整个kryo周期都不需要提供类信息。反序列化时使用readClassAndObject
3，先注册，再kryo.writeClassAndObject，这种方式时最理想的，其结合了前两种优势，又有效规避了劣势，事先将需要序列化的类注册给kryo（此时类和唯一id绑定），之后使用writeClassAndObject序列化时，只会序列化注册id，而不会序列化类的全限定名了，这样大大节省了空间（通常只比writeObject多一个字节）。反序列化时使用readClassAndObject。注意序列化和反序列的kryo的注册信息应当保持一致。
实例：
KryoUtils.java
package com.eastcom.second.stream.util;
import java.util.HashMap;
import java.util.LinkedHashMap;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.PooledObjectFactory;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.apache.commons.pool2.impl.SoftReferenceObjectPool;
import com.eastcom.second.stream.data.Radius;
import com.eastcom.second.stream.data.RadiusSerializer;
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import com.esotericsoftware.kryo.serializers.MapSerializer;
/**
*
* KryoUtils为序列化类，使用了了对象池的“软引入对象池工厂”。
*
* 用于序列化对象。
*
* @author Administrator
*
*
*/
public class KryoUtils implements PooledObjectFactory {
public static class KryoResigerIds {
public static final int HASH_MAP = 1001, LINKED_HASH_MAP = 1002, IMMUTABLE_PAIR = 1004, DATE = 1005,
HASH_BASED_TABLE = 1006, RADIUS = 1101;
}
private final static SoftReferenceObjectPool kryoPool;
static {
PooledObjectFactory factory = new KryoUtils();
kryoPool = new SoftReferenceObjectPool(factory);
}
public static Kryo create(boolean registrationRequired) {
MapSerializer mapSerializer = new MapSerializer();
Kryo kryo = new Kryo();
kryo.register(HashMap.class, mapSerializer, KryoResigerIds.HASH_MAP);
kryo.register(LinkedHashMap.class, mapSerializer, KryoResigerIds.LINKED_HASH_MAP);
kryo.register(Radius.class, new RadiusSerializer(), KryoResigerIds.RADIUS);
kryo.setReferences(false);
kryo.setRegistrationRequired(registrationRequired);
return kryo;
}
public static Kryo create() {
return create(false);
}
public static void serialize(Kryo kryo, Output output, Object obj) {
kryo.writeClassAndObject(output, obj);
}
public static byte[] serialize(Object obj) {
try {
Kryo kryo = kryoPool.borrowObject();
byte[] bs = serialize(kryo, obj);
kryoPool.returnObject(kryo);
return bs;
} catch (Exception e) {
// TODO Auto-generated catch block
throw new RuntimeException(e);
}
}
public static byte[] serialize(Kryo kryo, Object obj) {
Output output = new Output(50, Integer.MAX_VALUE);
kryo.writeClassAndObject(output, obj);
output.close();
return output.toBytes();
}
public static T deserialize(byte[] data) {
try {
Kryo kryo = kryoPool.borrowObject();
T res = deserialize(kryo, data);
kryoPool.returnObject(kryo);
return res;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public static T deserialize(Kryo kryo, byte[] bs) {
T obj = deserialize(kryo, new Input(bs));
return obj;
}
public static T deserialize(Kryo kryo, Input input) {
@SuppressWarnings("unchecked")
T obj = (T) kryo.readClassAndObject(input);
return obj;
}
public void activateObject(PooledObject arg0) throws Exception {
// TODO Auto-generated method stub
}
public void destroyObject(PooledObject arg0) throws Exception {
// TODO Auto-generated method stub
}
public PooledObject makeObject() throws Exception {
// TODO Auto-generated method stub
Kryo kryo = create();
return new DefaultPooledObject(kryo);
}
public void passivateObject(PooledObject arg0) throws Exception {
// TODO Auto-generated method stub
}
public boolean validateObject(PooledObject arg0) {
// TODO Auto-generated method stub
return true;
}
}
其中用到commons-pool2创建kryo对象池。采用了类注册false，但是已经有些采用了注册，并且为其设置了特定的数据。如果注册设置为true，则需要序列化的对象如果没有注册则会报错。已知的设置采用注册并且为设置数组编号，这样就不用全报名类名为标识此对象，提高 了序列化效率。
Radius.java
package com.eastcom.second.stream.data;
import java.io.Serializable;
import com.eastcom.second.stream.enums.RadiusType;
/**
* radius 日志话单
*
* @author Louyj
*
*/
public class Radius implements Serializable {
private static final long serialVersionUID = 1L;
/*
* 1 报文时间
*/
private long datetime; // 时间戳类型
/*
* 2 BRAS设备IP地址
*/
private String srcIp;
/*
* 3 BRAS设备端口号
*/
private int srcPort;
/*
* 4 日志提供厂家
*/
private String factory;
/*
* 5 BRAS归属地市
*/
private String city;
/*
* 6 报文类别
*/
private String code;
/*
* 7 宽带账号
*/
private String userName;
/*
* 8 NAS IP地址
*/
private String nasIpAddress;
/*
* 9 用户终端MAC
*/
private String callingStationId;
/*
* 10 BRAS设备名称
*/
private String nasIdentifier;
/*
* 11 CVLAN
*/
private String vlanid;
/*
* 12 SVLAN
*/
private String vlanid2;
/*
* 13 NAS 端口号
*/
private int nasPortId;
/*
* 14 上行带宽
*/
private long inputBasicRate;
/*
* 15 下行带宽
*/
private long outputBasicRate;
/*
* 16 计费报文识别标志
*/
private String acctStatusType;
/*
* 17 上行流量
*/
private long acctInputOctets;
/*
* 18 下行流量
*/
private long acctOutputOctets;
/*
* 19 用户私网IP地址
*/
private String framedIpAddressl;
/*
* 20 用户公网IP地址
*/
private String natIpAddress;
/*
* 21 公网起始端口号
*/
private int natStartPort;
/*
* 22 公网结束端口号
*/
private int natEndPort;
private RadiusType type;
public RadiusType getType() {
return type;
}
public void setType(RadiusType type) {
this.type = type;
}
public long getDatetime() {
return datetime;
}
public void setDatetime(long datetime) {
this.datetime = datetime;
}
public String getSrcIp() {
return srcIp;
}
public void setSrcIp(String srcIp) {
this.srcIp = srcIp;
}
public int getSrcPort() {
return srcPort;
}
public void setSrcPort(int srcPort) {
this.srcPort = srcPort;
}
public String getFactory() {
return factory;
}
public void setFactory(String factory) {
this.factory = factory;
}
public String getCity() {
return city;
}
public void setCity(String city) {
this.city = city;
}
public String getCode() {
return code;
}
public void setCode(String code) {
this.code = code;
}
public String getUserName() {
return userName;
}
public void setUserName(String userName) {
this.userName = userName;
}
public String getNasIpAddress() {
return nasIpAddress;
}
public void setNasIpAddress(String nasIpAddress) {
this.nasIpAddress = nasIpAddress;
}
public String getCallingStationId() {
return callingStationId;
}
public void setCallingStationId(String callingStationId) {
this.callingStationId = callingStationId;
}
public String getNasIdentifier() {
return nasIdentifier;
}
public void setNasIdentifier(String nasIdentifier) {
this.nasIdentifier = nasIdentifier;
}
public String getVlanid() {
return vlanid;
}
public void setVlanid(String vlanid) {
this.vlanid = vlanid;
}
public String getVlanid2() {
return vlanid2;
}
public void setVlanid2(String vlanid2) {
this.vlanid2 = vlanid2;
}
public int getNasPortId() {
return nasPortId;
}
public void setNasPortId(int nasPortId) {
this.nasPortId = nasPortId;
}
public long getInputBasicRate() {
return inputBasicRate;
}
public void setInputBasicRate(long inputBasicRate) {
this.inputBasicRate = inputBasicRate;
}
public long getOutputBasicRate() {
return outputBasicRate;
}
public void setOutputBasicRate(long outputBasicRate) {
this.outputBasicRate = outputBasicRate;
}
public String getAcctStatusType() {
return acctStatusType;
}
public void setAcctStatusType(String acctStatusType) {
this.acctStatusType = acctStatusType;
}
public long getAcctInputOctets() {
return acctInputOctets;
}
public void setAcctInputOctets(long acctInputOctets) {
this.acctInputOctets = acctInputOctets;
}
public void incrAcctInputOctets(long acctInputOctets) {
this.acctInputOctets += acctInputOctets;
}
public long getAcctOutputOctets() {
return acctOutputOctets;
}
public void setAcctOutputOctets(long acctOutputOctets) {
this.acctOutputOctets = acctOutputOctets;
}
public void incrAcctOutputOctets(long acctOutputOctets) {
this.acctOutputOctets += acctOutputOctets;
}
public String getFramedIpAddressl() {
return framedIpAddressl;
}
public void setFramedIpAddressl(String framedIpAddressl) {
this.framedIpAddressl = framedIpAddressl;
}
public String getNatIpAddress() {
return natIpAddress;
}
public void setNatIpAddress(String natIpAddress) {
this.natIpAddress = natIpAddress;
}
public int getNatStartPort() {
return natStartPort;
}
public void setNatStartPort(int natStartPort) {
this.natStartPort = natStartPort;
}
public int getNatEndPort() {
return natEndPort;
}
public void setNatEndPort(int natEndPort) {
this.natEndPort = natEndPort;
}
/**
* redis 中zset 排序的分 时间戳+过程ID
*
* @return
*/
public double getSortScore() {
return this.getDatetime() + this.getType().intValue() * 0.1;
}
@Override
public String toString() {
return "Radius [datetime=" + datetime + ", srcIp=" + srcIp + ", srcPort=" + srcPort + ", factory=" + factory
+ ", city=" + city + ", code=" + code + ", userName=" + userName + ", nasIpAddress=" + nasIpAddress
+ ", callingStationId=" + callingStationId + ", nasIdentifier=" + nasIdentifier + ", vlanid=" + vlanid
+ ", vlanid2=" + vlanid2 + ", nasPortId=" + nasPortId + ", inputBasicRate=" + inputBasicRate
+ ", outputBasicRate=" + outputBasicRate + ", acctStatusType=" + acctStatusType + ", acctInputOctets="
+ acctInputOctets + ", acctOutputOctets=" + acctOutputOctets + ", framedIpAddressl=" + framedIpAddressl
+ ", natIpAddress=" + natIpAddress + ", natStartPort=" + natStartPort + ", natEndPort=" + natEndPort
+ ", type=" + type + "]";
}
}
自定义序列化器RadiusSerializer.java
package com.eastcom.second.stream.data;
import com.eastcom.second.stream.enums.RadiusType;
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.Serializer;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
/**
*
* 自定义序列化器，没有用到writeClassAndObject(obj) ,不需对 对象进行反射，这样可以提高序列化性能。
*
* @author Administrator
*
*/
public class RadiusSerializer extends Serializer {
public RadiusSerializer() {
super(false, true);
}
@Override
public void write(Kryo kryo, Output output, Radius object) {
output.writeLong(object.getDatetime(), true);
output.writeString(object.getSrcIp());
output.writeInt(object.getSrcPort(), true);
output.writeString(object.getFactory());
output.writeString(object.getCity());
output.writeString(object.getCode());
output.writeString(object.getUserName());
output.writeString(object.getNasIpAddress());
output.writeString(object.getCallingStationId());
output.writeString(object.getNasIdentifier());
output.writeString(object.getVlanid());
output.writeString(object.getVlanid2());
output.writeInt(object.getNasPortId(), true);
output.writeLong(object.getInputBasicRate(), true);
output.writeLong(object.getOutputBasicRate(), true);
output.writeString(object.getAcctStatusType());
output.writeLong(object.getAcctInputOctets(), true);
output.writeLong(object.getAcctOutputOctets(), true);
output.writeString(object.getFramedIpAddressl());
output.writeString(object.getNatIpAddress());
output.writeInt(object.getNatStartPort(), true);
output.writeInt(object.getNatEndPort(), true);
output.writeString(object.getType().name());
// TODO Auto-generated method stub
}
@Override
public Radius read(Kryo kryo, Input input, Class type) {
Radius radius = new Radius();
radius.setDatetime(input.readLong(true));
radius.setSrcIp(input.readString());
radius.setSrcPort(input.readInt(true));
radius.setFactory(input.readString());
radius.setCity(input.readString());
radius.setCode(input.readString());
radius.setUserName(input.readString());
radius.setNasIpAddress(input.readString());
radius.setCallingStationId(input.readString());
radius.setNasIdentifier(input.readString());
radius.setVlanid(input.readString());
radius.setVlanid2(input.readString());
radius.setNasPortId(input.readInt(true));
radius.setInputBasicRate(input.readLong(true));
radius.setOutputBasicRate(input.readLong(true));
radius.setAcctStatusType(input.readString());
radius.setAcctInputOctets(input.readLong(true));
radius.setAcctOutputOctets(input.readLong(true));
radius.setFramedIpAddressl(input.readString());
radius.setNatIpAddress(input.readString());
radius.setNatStartPort(input.readInt(true));
radius.setNatEndPort(input.readInt(true));
radius.setType(RadiusType.valueOf(input.readString()));
return radius;
}
}
测试
package com.eastcom.second.stream.data.parser;
import com.eastcom.second.stream.data.Radius;
import com.eastcom.second.stream.util.KryoUtils;
public class ParserTest {
public static void main(String[] args) {
// TODO Auto-generated method stub
String line = "2017-09-25 14:15:00|221.131.204.223|1813||宁波|Accounting-Request|nbyza58239926|221.131.204.223|c8:3a:35:02:83:20|ZJNBO-MC-CMNET-BRAS66-YZZXME6|477|1024|4|12583000|20972000|Interim-Update|46655120|1127558565|10.121.105.238|39.190.249.213|30720|31743||10.212.68.16|1813|108|0";
// csvParserTest(line);
topicParserTest(line);
}
private static Object csvParserTest(String line) {
RadiusCSVParser radiusCSVParser = new RadiusCSVParser();
Radius radius = radiusCSVParser.parse(line);
System.out.println(radius.toString());
return radius;
}
private static Object topicParserTest(String line) {
Radius radius2 = (Radius) TopicCSVParsers.parse("radius", line, null);
System.out.println(radius2.toString());
byte[] data = KryoUtils.serialize(radius2);
System.out.println(data.length);
Radius radius = KryoUtils.deserialize(data);
System.out.println(radius.toString());
return radius2;
}
}
运行效果
Radius [datetime=1506320100000, srcIp=221.131.204.223, srcPort=1813, factory=, city=宁波, code=Accounting-Request, userName=nbyza58239926, nasIpAddress=221.131.204.223, callingStationId=c8:3a:35:02:83:20, nasIdentifier=ZJNBO-MC-CMNET-BRAS66-YZZXME6, vlanid=477, vlanid2=1024, nasPortId=4, inputBasicRate=12583000, outputBasicRate=20972000, acctStatusType=Interim-Update, acctInputOctets=46655120, acctOutputOctets=1127558565, framedIpAddressl=10.121.105.238, natIpAddress=39.190.249.213, natStartPort=30720, natEndPort=31743, type=ACCOUNTING]
208
Radius [datetime=1506320100000, srcIp=221.131.204.223, srcPort=1813, factory=, city=宁波, code=Accounting-Request, userName=nbyza58239926, nasIpAddress=221.131.204.223, callingStationId=c8:3a:35:02:83:20, nasIdentifier=ZJNBO-MC-CMNET-BRAS66-YZZXME6, vlanid=477, vlanid2=1024, nasPortId=4, inputBasicRate=12583000, outputBasicRate=20972000, acctStatusType=Interim-Update, acctInputOctets=46655120, acctOutputOctets=1127558565, framedIpAddressl=10.121.105.238, natIpAddress=39.190.249.213, natStartPort=30720, natEndPort=31743, type=ACCOUNTING]
参考链接： http://www.cnblogs.com/holoyong/p/7420880.html
参考链接： http://www.cnblogs.com/hntyzgn/p/7122709.html
参考链接： https://github.com/fontain1128/kryo
参考链接： https://github.com/EsotericSoftware/kryo#quickstart