ActiveMQ源码分析之KahaDB索引存储
前一篇我们分析完了KahaDB消息的存储机制，接下来将分析KahaDB的索引存储机制，跟索引存储相关的文件有*.data,*.redo,*.free。当Broker接收到Producer发送的消息数据之后将会将消息存储起来，而当Producer发送提交事务命令的时候，Broker会为刚才保存的消息生成对应的索引，存储在KahaDB中，以提升消息读取的效率。
Broker接收到的事务信息如下:
TransactionInfo {commandId = 7, responseRequired = true,
type = 2, connectionId = ID:jiangzhiqiangdeMacBook-Pro.local-53092-1556977982195-1:1,
transactionId = TX:ID:jiangzhiqiangdeMacBook-Pro.local-53092-1556977982195-1:1:1}接着在KahaDBTransactionStore类的commit方法中执行事务提交操作。首先根据事务txid获取到对应的事务信息：
local_transaction_id {
connection_id: ID:jiangzhiqiangdeMacBook-Pro.local-53092-1556977982195-1:1
transaction_id: 1
}接着创建KahaCommitCommand对象，用于封装事务信息，然后调用DataFileAppender类的storeItem方法保存事务信息到磁盘，该流程跟消息存储流程一样，此处不再重述，我们重点来看MessageDatabase的updateIndex索引存储部分的逻辑。
long updateIndex(Transaction tx, KahaAddMessageCommand command, Location location) throws IOException {
StoredDestination sd = getStoredDestination(command.getDestination(), tx);
// Skip adding the message to the index if this is a topic and there are
// no subscriptions.
if (sd.subscriptions != null && sd.subscriptions.isEmpty(tx)) {
return -1;
}
// Add the message.
int priority = command.getPrioritySupported() ? command.getPriority() : javax.jms.Message.DEFAULT_PRIORITY;
long id = sd.orderIndex.getNextMessageId();
Long previous = sd.locationIndex.put(tx, location, id);
if (previous == null) {
previous = sd.messageIdIndex.put(tx, command.getMessageId(), id);
if (previous == null) {
incrementAndAddSizeToStoreStat(command.getDestination(), location.getSize());
sd.orderIndex.put(tx, priority, id, new MessageKeys(command.getMessageId(), location));
if (sd.subscriptions != null && !sd.subscriptions.isEmpty(tx)) {
addAckLocationForNewMessage(tx, command.getDestination(), sd, id);
}
metadata.lastUpdate = location;
LOG.info("metadata.lastUpdate is:" + location);
} else {
MessageKeys messageKeys = sd.orderIndex.get(tx, previous);
if (messageKeys != null && messageKeys.location.compareTo(location) < 0) {
// If the message ID is indexed, then the broker asked us to store a duplicate before the message was dispatched and acked, we ignore this add attempt
LOG.warn("Duplicate message add attempt rejected. Destination: {}://{}, Message id: {}", command.getDestination().getType(), command.getDestination().getName(), command.getMessageId());
}
sd.messageIdIndex.put(tx, command.getMessageId(), previous);
sd.locationIndex.remove(tx, location);
id = -1;
}
} else {
// restore the previous value.. Looks like this was a redo of a previously
// added message. We don't want to assign it a new id as the other indexes would
// be wrong..
sd.locationIndex.put(tx, location, previous);
// ensure sequence is not broken
sd.orderIndex.revertNextMessageId();
metadata.lastUpdate = location;
}
// record this id in any event, initial send or recovery
metadata.producerSequenceIdTracker.isDuplicate(command.getMessageId());
return id;
}updateIndex方法中首先获取StoredDestination对像信息，该对象是创建其他索引的入口。通过StoredDestination中的orderIndex创建此时保存的消息对应的顺序Id，如当前message对应的顺序Id为7。接着通过StoredDestination对象的locationIndex索引保存对应的location，也就是具体存储位置。因为KahaDB索引存储结构是一棵B+树，所以创建索引的时候需要先获取索引存储对应的节点：
synchronized public Value put(Transaction tx, Key key, Value value) throws IOException {
assertLoaded();
return getRoot(tx).put(tx, key, value);
}private BTreeNode getRoot(Transaction tx) throws IOException {
return loadNode(tx, pageId, null);
} BTreeNode loadNode(Transaction tx, long pageId, BTreeNode parent) throws IOException {
Page> page = tx.load(pageId, marshaller);
BTreeNode node = page.get();
node.setPage(page);
node.setParent(parent);
return node;
} 根据pageId获取对应的page值，不同的索引存储在不同的page中，page和Page类对应。Page类是一个存储类，对应于磁盘上的page，是数据存储的最小单元，Page的唯一标识是pageId。如我们根据location索引找到的pageId为5，说明位置索引都存储在该page中。
BTreeNode表示索引节点，有两个重要的属性，keys和values。其中keys存储的是索引对应的key值，而value则是对应的顺序Id。如location索引的keys和values内容为：
keys：[1:10779, 1:27152, 1:105640, 1:117641, 1:124686, 1:170367]
values：[1, 2, 3, 4, 5, 6]
location索引的key值，每一个对应一条消息实际存储的偏移量位置。
获取到索引需要存储的节点信息之后，需要将新保存的消息索引信息存储到对应的节点上，BTreeNode的put方法用于实现该功能。put方法中首先用二分查找确认keys中没有当前要保存的key值，接着将当前要保存的key和value添加到之前的keys和values中。如本次保存最终得到的keys和values内容为:
keys：[1:10779, 1:27152, 1:105640, 1:117641, 1:124686, 1:170367, 1:201484]
values：[1, 2, 3, 4, 5, 6, 7]
最后通过Transaction的store将索引信息BTreeNode存储到磁盘中。
同理，messageIdIndex和orderIndex存储的机制跟locationIndex一致。其中
messageIdIndex信息保存在 pageId = 6的page上，对应的keys和values为：
keys：[ID:jiangzhiqiangdeMacBook-Pro.local-49633-1556954797292-1:1:1:1:1, ID:jiangzhiqiangdeMacBook-Pro.local-49923-1556959139334-1:1:1:1:1, ID:jiangzhiqiangdeMacBook-Pro.local-49943-1556959422344-1:1:1:1:1, ID:jiangzhiqiangdeMacBook-Pro.local-51539-1556891457996-1:1:1:1:1, ID:jiangzhiqiangdeMacBook-Pro.local-51580-1556892301557-1:1:1:1:1, ID:jiangzhiqiangdeMacBook-Pro.local-52983-1556976736145-1:1:1:1:1, ID:jiangzhiqiangdeMacBook-Pro.local-53092-1556977982195-1:1:1:1:1]
values：[3, 4, 5, 1, 2, 6, 7]
orderIndex信息保存在pageId = 2的page上，对应的keys和values为：
keys：[1, 2, 3, 4, 5, 6, 7]
values：[[ID:jiangzhiqiangdeMacBook-Pro.local-51539-1556891457996-1:1:1:1:1,1:10779], [ID:jiangzhiqiangdeMacBook-Pro.local-51580-1556892301557-1:1:1:1:1,1:27152], [ID:jiangzhiqiangdeMacBook-Pro.local-49633-1556954797292-1:1:1:1:1,1:105640], [ID:jiangzhiqiangdeMacBook-Pro.local-49923-1556959139334-1:1:1:1:1,1:117641], [ID:jiangzhiqiangdeMacBook-Pro.local-49943-1556959422344-1:1:1:1:1,1:124686], [ID:jiangzhiqiangdeMacBook-Pro.local-52983-1556976736145-1:1:1:1:1,1:170367], [ID:jiangzhiqiangdeMacBook-Pro.local-53092-1556977982195-1:1:1:1:1,1:201484]]
updateIndex方法中metadata.lastUpdate = location;这一句需要格外重视，它的作用是用于保存最后一次消息存储对应的偏移量值，比如我们最后一次保存的偏移量值为1:201484，那么metadata.lastUpdate对应的值就是1:201484。这个偏移量在从KahaDB中获取消息内容的时候会用到，这块在消费者部分会详细分析。
通过对消息创建对应的索引，让我们在读取KahaDB中消息的时候能够根据索引信息快速找到对应的消息存储地址，极大的提高了消息读取速度。