谈谈Redis持久化RDB和AOF

谈谈Redis持久化RDB和AOF

一、概述

• 为什么需要持久化？

  Redis是基于内存来缓存数据的，当Redis服务器宕机或者重启的时候，如果对数据没有什么措施的话，就会导致内存的数据全部丢失，从而导致缓存雪崩的发生

• 怎么做？

  为了提高Redis的可用性，能够在出现故障的时候及时恢复数据，就需要备份数据——将数据保存在本地用于数据恢复。

二、RDB(快照,snapshotting)

• 原理

  会根据你设置的保存点，对内存进行快照保存，也就是每次备份都会保存所有的数据。在进行备份的时候，主进程会fork一个子进程来完成备份的工作，因此对Redis性能的影响较小

• RDB配置

  • 保存点配置（截取自conf文件）

    ################################ SNAPSHOTTING  ################################
    #
    # Save the DB on disk:
    #
    #   save  
    #
    #   Will save the DB if both the given number of seconds and the given
    #   number of write operations against the DB occurred.
    #
    #   In the example below the behaviour will be to save:
    #   after 900 sec (15 min) if at least 1 key changed
    #	在900秒(15分钟)后，如果至少有一个键改变
    #   after 300 sec (5 min) if at least 10 keys changed 
    #	在300秒(5分钟)后，如果至少有10个键改变
    #   after 60 sec if at least 10000 keys changed	
    #	在60秒后，如果至少10000个密钥改变
    #
    #   Note: you can disable saving completely by commenting out all "save" lines.
    #	
    #   It is also possible to remove all the previously configured save
    #   points by adding a save directive with a single empty string argument
    #   like in the following example:
    #   删除所有先前配置的保存的方法是添加一个带有一个空字符串参数的保存指令，像下面的例子中:
    #
    #   save ""
    
    save 900 1
    save 300 10
    save 60 10000
    

  • RDB文件名以及保存的路径

    # The filename where to dump the DB
    # 默认的文件名
    dbfilename "dump.rdb"
    
    # The working directory.
    #
    # The DB will be written inside this directory, with the filename specified
    # above using the 'dbfilename' configuration directive.
    #
    # The Append Only File will also be created inside this directory.
    #
    # Note that you must specify a directory here, not a file name.
    # dump.rdb文件所保存的路径
    dir "/var/redis/6379"
    
    

  • RDB内容示例

    [root@eshop-cache01 6379]# cat dump.rdb 
    REDIS0007	redis-ver3.2.8
    redis-bits￀㨭e|;used-memP 
                             .masterk1m1myk2v2myk1v1ÿ>y0^Xkب[root@eshop-cache01 6379]# 
    

    因为使用了LZF压缩字符串对象，所以数据部分比较静凑，但是还是可以看到原先设置的k-v对：

    192.168.43.155:6379> get myk1
    "v1"
    192.168.43.155:6379> get myk2
    "v2"
    192.168.43.155:6379> 
    

    ​

三、AOF(只追加操作的文件，Append-only file)

• 原理

  从名字应该也能感觉出来，它是增量备份，通俗的说就是每次备份只会备份变化的部分，其实实际上AOF备份的是对数据的操作，根据配置的fsync策略（调用fsync()来将内存的数据刷入到磁盘）来进行配置。

  以set myk1 v1、set myk2 v2和set myk3 v3 操作后的aof文件为例

  $1
  0
  *3
  $3
  set
  $4
  myk1
  $2
  v1
  *3
  $3
  set
  $4
  myk2
  $2
  v2
  *3
  $3
  set
  $4
  myk3
  $2
  v3
  *2
  $6
  

  相信聪明的你肯定是可以从中看出一些规律的

  因为是追加的方式在磁盘上写aof文件，因此不需要进行寻址。****

• 重写机制(rewrite)

  随着Redis一直在运行，aof文件记录的操作会越来越多，为了防止aof文件大小不停的增大，Redis会在文件会在一定条件的时候执行重写(rewrite)操作，通俗的说就是压缩大小，除去那些多余的操作，比如说对同一个Key连续进行100次的set操作，那么我们是不是只需记录最后一次Set操作就可以了。

  在进行重写的时候，为了防止出现重写失败导致数据丢失的情况，Redis的做法是这样的

  1. 主进程fork一个子进程来进行rewrite操作
  2. 子进程开始将新 AOF 文件的内容写入到临时文件。
  3. 父进程会将新的写入操作记录到内存的一个缓冲区和追加到现有AOF文件的末尾，万一重写失败，现有的AOF还是保存着最新的数据
  4. 当子进程重写完成后会告诉父进程，然后父进程会将缓冲区中记录的新的操作追加到新的AOF文件末尾

• 配置

  • 开启AOF并配置AOF文件名

    上面的英文部分是官方的说明，建议找个翻译软件或者复制到浏览器看看中文意思

    ############################## APPEND ONLY MODE ###############################
    
    # By default Redis asynchronously dumps the dataset on disk. This mode is
    # good enough in many applications, but an issue with the Redis process or
    # a power outage may result into a few minutes of writes lost (depending on
    # the configured save points).
    #
    # The Append Only File is an alternative persistence mode that provides
    # much better durability. For instance using the default data fsync policy
    # (see later in the config file) Redis can lose just one second of writes in a
    # dramatic event like a server power outage, or a single write if something
    # wrong with the Redis process itself happens, but the operating system is
    # still running correctly.
    #
    # AOF and RDB persistence can be enabled at the same time without problems.
    # If the AOF is enabled on startup Redis will load the AOF, that is the file
    # with the better durability guarantees.
    #
    # Please check http://redis.io/topics/persistence for more information.
    
    appendonly yes
    
    # The name of the append only file (default: "appendonly.aof")
    
    appendfilename "appendonly.aof"
    

  • 配置fsync策略

    默认是使用后everysec策略，也就是每隔1s就将内存中的数据刷入磁盘一次

    # The fsync() call tells the Operating System to actually write data on disk
    # instead of waiting for more data in the output buffer. Some OS will really flush
    # data on disk, some other OS will just try to do it ASAP.
    #
    # Redis supports three different modes:
    #
    # no: don't fsync, just let the OS flush the data when it wants. Faster.
    # always: fsync after every write to the append only log. Slow, Safest.
    # everysec: fsync only one time every second. Compromise.
    #
    # The default is "everysec", as that's usually the right compromise between
    # speed and data safety. It's up to you to understand if you can relax this to
    # "no" that will let the operating system flush the output buffer when
    # it wants, for better performances (but if you can live with the idea of
    # some data loss consider the default persistence mode that's snapshotting),
    # or on the contrary, use "always" that's very slow but a bit safer than
    # everysec.
    #
    # More details please check the following article:
    # http://antirez.com/post/redis-persistence-demystified.html
    #
    # If unsure, use "everysec".
    
    # appendfsync always
    appendfsync everysec
    # appendfsync no
    

  • 配置重写策略

    # Automatic rewrite of the append only file.
    # Redis is able to automatically rewrite the log file implicitly calling
    # BGREWRITEAOF when the AOF log size grows by the specified percentage.
    #
    # This is how it works: Redis remembers the size of the AOF file after the
    # latest rewrite (if no rewrite has happened since the restart, the size of
    # the AOF at startup is used).
    #
    # This base size is compared to the current size. If the current size is
    # bigger than the specified percentage, the rewrite is triggered. Also
    # you need to specify a minimal size for the AOF file to be rewritten, this
    # is useful to avoid rewriting the AOF file even if the percentage increase
    # is reached but it is still pretty small.
    #
    # Specify a percentage of zero in order to disable the automatic AOF
    # rewrite feature.
    
    auto-aof-rewrite-percentage 100
    auto-aof-rewrite-min-size 64mb
    

四、RDB和AOF的优缺点

• RDB

  • 优点
    1. RDB所备份的数据比较紧凑并且，适合做冷备
    2. 在进行备份的时候都是交给子进程去做的，不续约主进程进行其他的IO操作，因此对性能的影响最小
    3. 数据量较大的时候，恢复速度比AOF快
  • 缺点
    1. 因为备份频率比较低，因此当发生数据丢失的时候，丢失的数据会很多
    2. 因为每次备份全部数据，因此当数据量大的时候耗时主进程fork的时候会很耗时

• AOF

  • 优点
    1. 备份频率高，当发生故障的时候，丢失的数据量比较少，比如每秒fsync一次，最多丢失1s的数据，因此数据恢复的时候，如果RDB和AOF都开启了，优先使用AOF恢复数据
    2. AOF文件中有序的保存了所有写入操作，并且以特定的格式保存着，容易读懂
  • 缺点
    1. AOF虽然有重写机制控制文件大小，对于同一份数据来说，保存操作的AOF文件的大小总是要比只保存数据的RDB文件要大的，从而导致数据恢复的时候速度比较慢
    2. 因为备份频率高，因此对性能的影响比RDB方式要大

• 补充

  针对RDB和AOF的优缺点，有一种取长补短的方法——RDB + AOF的方式来保存数据，RDB保存主要数据，用AOF来保存两次RDB备份之间的数据，在进行数据恢复的时候只需要恢复RDB和AOF两个文件的数据即可

  ​