Cassandra 之 commit_log

本文介绍了cassandra 之commit_log

1. Cassandra 写入数据流程如下

a) 写入commit_log 中

b) 写入内存的memtable 中

c) 最后从内存的memtable中flush 到磁盘的SStables中

注意 a) 和 b) 是并行执行的

commit_log 的作用类似于oracle的redo log 和 mongo 的 jouney log. 用于意外宕机例如断电机器或者自动重启后数据恢复

默认存在路径 /var/lib/cassandra/commitlog

可以在配置文件 cassandra.yaml 修改默认路径

# commit log.  when running on magnetic HDD, this should be a
# separate spindle than the data directories.
# If not set, the default directory is $CASSANDRA_HOME/data/commitlog.
commitlog_directory: /u01/cassandra/commitlog

commit_log 的默认大小是8G

# Total space to use for commit logs on disk.
#
# If space gets above this value, Cassandra will flush every dirty CF
# in the oldest segment and remove it.  So a small total commitlog space
# will tend to cause more flush activity on less-active columnfamilies.
#
# The default value is the smaller of 8192, and 1/4 of the total space
# of the commitlog volume.
#
# commitlog_total_space_in_mb: 8192

commit_log 设置压缩(默认不压缩，生产环境不建议压缩)

# Compression to apply to the commit log. If omitted, the commit log
# will be written uncompressed.  LZ4, Snappy, and Deflate compressors
# are supported.
#commitlog_compression:
#   - class_name: LZ4Compressor
#     parameters:
#         -

关于commit log 的归档

commit log 的归档发生在如下的几种情况:

• 当节点启动的时候
• 当commit_log 写入到磁盘的时候
• 手动执行归档命令的时候

关于commit_log 归档的设置

在配置文件 $CASSANDRA_HOME\conf\commitlog_archiving.properties 中

设置归档 参数archive_command

# commitlog archiving configuration.  Leave blank to disable.

# Command to execute to archive a commitlog segment
# Parameters: %path => Fully qualified path of the segment to archive
#             %name => Name of the commit log.
# Example: archive_command=/bin/ln %path /backup/%name
#
# Limitation: *_command= expects one command with arguments. STDOUT
# and STDIN or multiple commands cannot be executed.  You might want
# to script multiple commands and add a pointer here.
archive_command=

创建归档路径

[root@tjtestrac1 ~]# mkdir -p /u01/cassandra/arch
[root@tjtestrac1 ~]# chown -R cassandra:cassandra /u01/cassandra/arch

开发一个简单的备份脚本

-rwxrwxrwx 1 cassandra cassandra 64 Dec 19 11:02 archive_commit_log.sh
#!/bin/bash
cp /u01/cassandra/commitlog/* /u01/cassandra/arch/

设置归档参数

archive_command=/u02/cassandra/cassandra/apache-cassandra-2.2.7/script/archive_commit_log.sh  

重启Cassandra 观察归档路径

[cassandra@tjtestrac1 arch]$ ls -lhtr
total 128K
-rw-rw-r-- 1 cassandra cassandra 32M Dec 19 11:09 CommitLog-5-1545188942068.log
-rw-rw-r-- 1 cassandra cassandra 32M Dec 19 11:09 CommitLog-5-1545188702567.log
-rw-rw-r-- 1 cassandra cassandra 32M Dec 19 11:09 CommitLog-5-1545188702566.log