java 对象存储_在Java中将大量对象存储到磁盘的最佳方法

顺便说一句,您不需要列表包装​​器即可将许多项目写入文件,但是您的项目因此需要可序列化.

public class SObject {

private String value;

private int occurences;

private String key;

}

来写

List list = new ArrayList<>();

ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(fileName));

oos.writeObject(list);

oos.close();

读书

ObjectInputStream ois = new ObjectInputStream(new FileInputStream(fileName));

List list = (List) ois.readObject(); // cast is needed.

ois.close()

写压缩

ObjectOutputStream oos = new ObjectOutputStream(new DeflaterOutputStream(new FileOutputStream(fileName)));

oos.writeObject(list);

oos.close();

阅读压缩

ObjectInputStream ois = new ObjectInputStream(new InflaterInputStream(new FileInputStream(fileName)));

List list = (List) ois.readObject(); // cast is needed.

ois.close()

Is this an efficient way with minimal code work to write data?

您必须使SObject实现Serializable,仅此而已.您不能使其更短.

您可以通过使其实现“可外部化”来使其效率更高,但这需要更多工作.

I also find it easier to read the data from the disk by using serialization.

听起来是这样做的一个很好的理由.

Because the size of list could be very high (~10k+),

除非您的弦很长,否则10K对我来说听起来很小.我有一个针对我的序列化/持久性库的测试,它可以在几秒钟内写入和读取10,000,000个条目.

I should find a balance between ease of code and size. Is there any way to achieve this balance?

我怀疑轻松是您需要担心的.如果您想缩小文件大小,可以通过使用GZIP或压缩器/充气器流包装文件流来对其进行压缩.这将添加一行代码,并使文件缩小4倍.

Please let me know if there are any other methods to store data in a reduced size compared to serialization. I cannot use any external libraries.

它们都需要外部库.您可以使用XMLEncoder和XMLDecoder,但这会变得越来越慢.

既然我对此有疑问；如果您对我的书架感兴趣Java Chronicle 2.x这是笔记本电脑上运行的测试结果.这次,写入5亿个条目并在第二个线程中读取它们(写入时)花费了多长时间,只花了12秒钟以上.

Rate = 41.0 Mmsg/sec for 16 byte messages

测试是IndexedChronicleTest中的multiThreaded()