KFS 0.5发布说明

We are happy to provide a new release of KFS (kfs-0.5). This release adds new features (particularly, atomic record append) as well as stability/performance improvement over previous releases. In a bit of detail,

 

1. Add support for atomic record append. This capability is to enable multiple writes append records to a file. Writers can be writing to same chunk of a file or to different chunks of a file. The system guarantees that records will not be split across chunk boundaries. The support for atomic record append entails three parts: (1) metaserver to allocate chunks for append, (2) chunkserver to receive data from multiple clients and interleave the data, and (3) support in the client to construct records and send them to chunkserver. To limit the # of concurrent writers to a chunk, we employ a space reservation policy: clients reserve space on the chunkserver; if the reservation fails, the client will interface with the metaserver, ask for a new allocation, and retry. The atomic record append operation can be used, for instance, to do log aggregation in a cluster: Logger processes on individual nodes in a cluster open a file in KFS and atomically append log records to the file.

 

添加对原子记录追加（atomic record append）的支持。此改进可以实现对同一个文件的多次追加写操作。写入程序可以操作同一个文件的同一或不同区块（chunk），系统将保证每个记录都在相同的区块中。此改进的实现主要包括三个细节：1）metaserver为将进行追加写操作的区块分配空间；2）chunkserver从多个客户端收到数据，并将数据插入；3）客户端对构建记录并发送到chunkserver的支持。我们采用一个空间预存的策略，以限制对同一个区块的并发写数目：客户端在chunkserver上预留空间；如果预留操作失败，客户端将和metaserver交流以要求新的空间分配，并重试。举例：原子记录追加可以被用于日志合并操作，集群中的每台机器都可以运行一个日志记录器，这些记录器操作KFS中的同一个文件，日志将自动地被追加写入到这个文件中。

 

2. Add reliability support in record append. To the writer, the reliability protocol provides exactly once semantics. If the writer can't determine if the write is committed at the multiple servers, the write will fail.

 

添加记录追加写的可靠性支持。对于写入程序而言，可靠性协议将提供有且仅有一次的确认。如果写入程序无法确认是否写入操作已经被多台服务器确认，写入将失败。

 

3. Add support for doing adler-32 using Intel's IPP (Intel's performance package).

 

使用IPP(Inter's performance package)，添加对adler-32的支持。

 

4. Add support for a "chunk coalesce" operation: Data written to chunks in a different files can be coalesced into a single file. That is, a container file can be created and content from different files can be appended to the container.

 

添加区块合并操作。写入到不同区块/不同文件中的数据可以被合并到一个文件中。即，可以创建一个容器文件，来自不同文件的数据可以被追加到容器文件中。

 

5. Add support for async read/write in the KFS client. With read/write, the app can issue reads to data from multiple files/chunks concurrently; the client code multiplexes I/O from multiple chunkservers concurrently.

 

添加KFS客户端异步读写的支持。用户程序可以同时从多个文件/区块读取数据，客户端代码（kfsClient）中将I/O多重化，使其可以从多台chunkserver并发读取数据。

 

6. Add a rebalancer tool: the tool takes as input chunk sizes/locations, and then constructs a plan (which lists what chunk needs to be moved where); the plan should then be uploaded to a running metaserver, which then executes the plan.

 

（尚未实现？）添加了一个负载均衡工具。此工具读入区块的大小和位置，并由此生成一个计划（列出区块及其移动目的地）。此计划应该被上载到运行中的metaserver，并被执行。

 

7. Modifications to the write protocol for reliability. On each write sync, the client sends the adler-32 over the data that it has sent; the chunk master and the replicas should agree on the checksums; otherwise, the write is failed, and the client will retry.

 

修改写协议以增强可靠性。每次写操作同步时，客户端发送数据及adler-32校验和，区块的主/从副本需要对校验和达成一致；否则写入操作将失败，客户端将重试。

 

8. Performance tweaks to the metaserver for scale.

 

调整metaserver的性能以优化大规模环境。