翻译 Basic Operations Compaction Filter

原文地址: https://github.com/facebook/rocksdb/wiki/Compaction-Filter

(有道)

RocksDB provides a way to delete or modify key/value pairs based on custom logic in background. It is handy for implementing custom garbage collection, like removing expired keys based on TTL, or dropping a range of keys in the background. It can also update the value of an existing key.
RocksDB提供了一种基于后台自定义逻辑删除或修改键/值对的方法。它便于实现自定义垃圾收集，比如根据TTL删除过期的键，或删除背景中的键范围。它还可以更新现有键的值。

To use compaction filter, applications need to implement the CompactionFilter interface found in rocksdb/compaction_filter.h and set it to ColumnFamilyOptions. Alternatively, applications can implement the CompactionFilterFactory interface, which gives the flexibility to create different compaction filter instance per (sub)compaction. The compaction filter factory also gets to know some context from the compaction (whether it is a full compaction or whether it is a manual compaction) through the given CompactionFilter::Context param. The factory can choose to return different compaction filter based on the context.
要使用压缩过滤器，应用程序需要实现在rocksdb/compaction_filter.h中的CompactionFilter接口，并将其设置为ColumnFamilyOptions。另外，应用程序还可以实现CompactionFilterFactory接口，该接口可以灵活地为每个(子)压缩创建不同的压缩过滤器实例。压缩过滤器工厂还可以通过给定的CompactionFilter:: context参数从压缩中了解一些上下文(无论是完整的压缩还是手动压缩)。工厂可以根据上下文选择返回不同的压缩过滤器。

options.compaction_filter = new CustomCompactionFilter();
// or
options.compaction_filter_factory.reset(new CustomCompactionFilterFactory());

The two ways of providing compaction filter also come with different thread-safety requirement. If a single compaction filter is provided to RocksDB, it has to be thread-safe since multiple sub-compactions can run in parallel, and they all make use of the same compaction filter instance. If a compaction filter factory is provided, each sub-compaction will call the factory to create a compaction filter instance. It is thus guaranteed that each compaction filter instance will be access from a single thread and thread-safety of the compaction filter is not required. The compaction filter factory, though, can be accessed concurrently by sub-compactions.
提供压缩过滤器的两种方式也具有不同的线程安全需求。如果RocksDB提供了一个压缩过滤器，它必须是线程安全的，因为多个子压缩可以并行运行，而且它们都使用相同的压缩过滤器实例。如果提供了压缩过滤器工厂，则每个子压缩将调用工厂来创建一个压缩过滤器实例。这样就保证了每个压缩过滤器实例将从单个线程访问，并且不需要压缩过滤器的线程安全性。但是，可以通过子压缩并发地访问压缩过滤器工厂。

Compaction filter will not be invoked during flush, despite arguably flush is a special type of compaction.
在刷新期间将不会调用压缩过滤器，尽管刷新是一种特殊类型的压缩。

There are two sets of API that can be implement with compaction filter. The Filter/FilterMergeOperand API provide a simple callback to let compaction know whether to filter out a key/value pair. The FilterV2 API extends the basic API by allowing changing the value, or dropping a range of keys starting from the current key.
有两组API可以用压缩过滤器实现。Filter/FilterMergeOperand API提供了一个简单的回调，让压缩知道是否过滤掉一个键/值对。FilterV2 API扩展了基本API，允许更改值，或删除从当前键开始的一系列键。

Each time a (sub)compaction sees a new key from its input and when the value is a normal value, it invokes the compaction filter. Based on the result of the compaction filter:
每次(子)压缩从它的输入中看到一个新键，并且当值是一个正常值时，它就调用压缩过滤器。根据压缩过滤器的结果:

• If it decide to keep the key, nothing will change.
  如果它决定保留key，什么也不会改变。
• If it request to filter the key, the value is replaced by a deletion marker. Note that if the output level of the compaction is the bottom level, no deletion marker will need to be output.
  如果它请求过滤该键，则该值将被删除标记替换。请注意，如果压缩的输出级别是底部级别，则不需要输出删除标记。
• If it request to change the value, the value is replaced by the changed value.
  如果请求更改该值，则该值将被更改后的值替换。
• If it request to remove a range of keys by returning kRemoveAndSkipUntil, the compaction will skip over to skip_until (means skip_until will be the next possible key output by the compaction). This one is tricky because in this case the compaction does not insert deletion marker for the keys it skips. This means older version of the keys may reappears as a result. On the other hand, it is more efficient to simply dropping the keys, if the application know there aren't older versions of the keys, or reappearance of the older versions is fine.
  如果它请求通过返回kRemoveAndSkipUntil来删除一系列键，则压缩将跳过到skip_until(意味着skip_until将是压缩后的下一个可能的键输出)。这是一个棘手的问题，因为在这种情况下，压缩不会为它跳过的键插入删除标记。这意味着旧版本的键可能会重新出现。另一方面，如果应用程序知道键没有旧版本，或者可以重新出现旧版本，那么简单地删除键会更有效。

If there are multiple versions of the same key from the input of the compaction, compaction filter will only be invoked once for the newest version. If the newest version is a deletion marker, compaction filter will not be invoked. However, it is possible the compaction filter is invoked on a deleted key, if the deletion marker isn't included in the input of the compaction.
如果同一个键在压缩的输入中有多个版本，那么对于最新的版本，压缩过滤器只会被调用一次。如果最新版本是删除标记，则不会调用压缩过滤器。但是，如果删除标记没有包含在压缩的输入中，则可能对已删除的键调用压缩过滤器。

When merge is being used, compaction filter is invoked per merge operand. The result of compaction filter is applied to the merge operand before merge operator is invoked.
使用合并时，每个合并操作数都会调用压缩过滤器。在调用合并操作符之前，将压缩过滤器的结果应用于合并操作数。

Before release 6.0, if there is a snapshot taken later than the key/value pair, RocksDB always try to prevent the key/value pair from being filtered by compaction filter so that users can preserve the same view from a snapshot, unless the compaction filter returns IgnoreSnapshots() = true. However, this feature is deleted since 6.0, after realized that the feature has a bug which can't be easily fixed. Since release 6.0, with compaction filter enabled, RocksDB always invoke filtering for any key, even if it knows it will make a snapshot not repeatable.
在6.0版本之前，如果有一个比键/值对晚的快照，RocksDB总是试图阻止键/值对被压缩过滤器过滤，这样用户就可以从快照中保留相同的视图，除非压缩过滤器返回IgnoreSnapshots() = true。然而，在意识到该特性存在一个无法轻易修复的bug后，该特性从6.0开始被删除。从6.0版本开始，随着压缩过滤器的启用，RocksDB总是对任何键调用过滤，即使它知道这会使快照不可重复。