Redis所有数据都在内存中,用户自然会想到如何有效的使用内存。Redis的作者已考虑了内存的优化,所以从用户的角度,Redis内存的优化包括两个方面,一个是Redis Server本省对内存的优化,一个是应用方面的优化。
Redis Server本身对内存的优化
1.存储编码的优化
Redis存储的数据都使用redisObject结构体来封装,包括string、hash、list、set和zset在内的所有数据类型。
redisObject结构体如下所示:
redisObject
type - 对象类型
encoding - 内部编码类型
lru - LRU计时时钟
int refcount - 引用计数器
void *ptr - 数据指针
在redisObject中有个encoding字段,表示Redis内部编码类型,同一个对象采用不同的编码实现内存占用存在明显差异。
对于编码优化的配置,可参考redis.conf:
# Lists are also encoded in a special way to save a lot of space.
# The number of entries allowed per internal list node can be specified
# as a fixed maximum size or a maximum number of elements.
# For a fixed maximum size, use -5 through -1, meaning:
# -5: max size: 64 Kb <-- not recommended for normal workloads
# -4: max size: 32 Kb <-- not recommended
# -3: max size: 16 Kb <-- probably not recommended
# -2: max size: 8 Kb <-- good
# -1: max size: 4 Kb <-- good
# Positive numbers mean store up to _exactly_ that number of elements
# per list node.
# The highest performing option is usually -2 (8 Kb size) or -1 (4 Kb size),
# but if your use case is unique, adjust the settings as necessary.
list-max-ziplist-size -2
# Lists may also be compressed.
# Compress depth is the number of quicklist ziplist nodes from *each* side of
# the list to *exclude* from compression. The head and tail of the list
# are always uncompressed for fast push/pop operations. Settings are:
# 0: disable all list compression
# 1: depth 1 means "don't start compressing until after 1 node into the list,
# going from either the head or tail"
# So: [head]->node->node->...->node->[tail]
# [head], [tail] will always be uncompressed; inner nodes will compress.
# 2: [head]->[next]->node->node->...->node->[prev]->[tail]
# 2 here means: don't compress head or head->next or tail->prev or tail,
# but compress all nodes between them.
# 3: [head]->[next]->[next]->node->node->...->node->[prev]->[prev]->[tail]
# etc.
list-compress-depth 0
# Hashes are encoded using a memory efficient data structure when they have a
# small number of entries, and the biggest entry does not exceed a given
# threshold. These thresholds can be configured using the following directives.
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
# Sets have a special encoding in just one case: when a set is composed
# of just strings that happen to be integers in radix 10 in the range
# of 64 bit signed integers.
# The following configuration setting sets the limit in the size of the
# set in order to use this special memory saving encoding.
set-max-intset-entries 512
# Similarly to hashes and lists, sorted sets are also specially encoded in
# order to save a lot of space. This encoding is only used when the length and
# elements of a sorted set are below the following limits:
zset-max-ziplist-entries 512
zset-max-ziplist-value 64
2.共享对象池(Java中也存在类似优化)
共享对象池是指Redis内部维护了[0-9999]的整数对象池,用于节约内存。除了整数值对象,其它类型如list、hash、set和zset内部元素也可以使用整数对象池。
但要注意当设置maxmemory,并启用LRU相关淘汰策略如,volatile-lru,allkeys-lru时,Redis禁止使用共享对象池。LRU算法需要获取对象最后被访问时间,以便淘汰最长未访问数据,每个对象最后访问时间存储在redisObject对象的lru字段。对象共享意味着多个引用共享同一个RedisObject,这时lru字段也会被共享,导致无法获取每个对象的最后访问时间。
3.字符串优化
Redis没有采用原生C语言的字符串类型,而是自己实现了字符串结构,简单动态字符串(simple dynamic string,SDS)。其内部实现空间预分配机制,降低内存再分配次数。但要防止预分配,带来的内存浪费。尽量减少字符串频繁修改操作append、setrange,改为直接使用set修改字符串,降低预分配带来的内存浪费和内存碎片。
应用方面的优化
1.控制键的数量,使用hash代替多个key value。
使用Redis不要进入一个误区,大量使用get/set这样的API,把其当成Memcached使用。对于存储相同的数据内容,利用Redis的数据结构降低外层键的数量,也可以节省大量内存。
2.缩减键值对象
对key长度,设计键时,在完整描述业务情况下,键值越短越好。
value长度,值对象缩减比较复杂,常见的做法是把业务对象序列化成二进制数组放入Redis,这时就要选择更高效的序列化工具。值对象除了存储二进制数据之外,通常还会使用通用格式存储数据比如json、xml等作为字符串存储在Redis中,可使用通用压缩算法压缩json、xml后再存入Redis,从而降低内存占用。
重点参考2个链接:
https://redis.io/topics/data-types-intro
https://redis.io/topics/memory-optimization