MapReduce 生成Hfile 的Mrunit单元测试提示Serialization空指针

 

使用Mrunit对生成Hfile 的程序进行单元测试的时候发现提示 Serialization的  NullPointerException 跟踪代码后发现se 初始化后是null

提示信息：

1. java.lang.NullPointerException
2. at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:61)
3. at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:81)

原因无法初始化  serializer

1. try {
2. serializer = (Serializer<Object>) serializationFactory //这里为空
3. .getSerializer(clazz);
4. deserializer = (Deserializer<Object>) serializationFactory
5. .getDeserializer(clazz);
6. } catch (NullPointerException e) {
7. throw new IllegalStateException(
8. "No applicable class implementing Serialization in conf at io.serializations for "
9. + orig.getClass(), e);
10. }
11. try {
12. final DataOutputBuffer outputBuffer = new DataOutputBuffer();
13. serializer.open(outputBuffer);
14. serializer.serialize(orig);
15. final DataInputBuffer inputBuffer = new DataInputBuffer();
16. inputBuffer.reset(outputBuffer.getData(), outputBuffer.getLength());
17. deserializer.open(inputBuffer);
18. return (T) deserializer.deserialize(copy);
19. } catch (final IOException e) {
20. throw new RuntimeException(e);
21. }

需要设置：

1. Configuration conf = mapDriver.getConfiguration();
2. conf.setStrings("io.serializations", conf.get("io.serializations"),
3. MutationSerialization.class.getName(),
4. ResultSerialization.class.getName(),
5. KeyValueSerialization.class.getName());

使用Mrunit对生成Hfile 的程序进行单元测试的时候发现提示 Serialization的  NullPointerException 跟踪代码后发现se 初始化后是null

提示信息：

1. java.lang.NullPointerException
2. at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:61)
3. at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:81)

原因无法初始化  serializer

1. try {
2. serializer = (Serializer<Object>) serializationFactory //这里为空
3. .getSerializer(clazz);
4. deserializer = (Deserializer<Object>) serializationFactory
5. .getDeserializer(clazz);
6. } catch (NullPointerException e) {
7. throw new IllegalStateException(
8. "No applicable class implementing Serialization in conf at io.serializations for "
9. + orig.getClass(), e);
10. }
11. try {
12. final DataOutputBuffer outputBuffer = new DataOutputBuffer();
13. serializer.open(outputBuffer);
14. serializer.serialize(orig);
15. final DataInputBuffer inputBuffer = new DataInputBuffer();
16. inputBuffer.reset(outputBuffer.getData(), outputBuffer.getLength());
17. deserializer.open(inputBuffer);
18. return (T) deserializer.deserialize(copy);
19. } catch (final IOException e) {
20. throw new RuntimeException(e);
21. }

需要设置：

1. Configuration conf = mapDriver.getConfiguration();
2. conf.setStrings("io.serializations", conf.get("io.serializations"),
3. MutationSerialization.class.getName(),
4. ResultSerialization.class.getName(),
5. KeyValueSerialization.class.getName());