缺少set关键字
This article is part of a series, Redis in Ruby, originally published at https://redis.pjam.me/. All chapters are available on Medium as well: https://medium.com/@pierre_jambet
本文是Ruby中的Redis系列的一部分,该系列最初发布于https://redis.pjam.me/ 。 所有章节都可以在Medium上找到: https : //medium.com/@pierre_jambet
我们将介绍的内容 (What we’ll cover)
We implemented a simplified version of the SET
command in Chapter 2, in this chapter we will complete the command by implementing all its options. Note that we're still not following the Redis Protocol, we will address that in the next chapter.
我们在第2章中实现了SET
命令的简化版本,在本章中,我们将通过实现其所有选项来完成该命令。 请注意,我们仍未遵循Redis协议 ,我们将在下一章中解决。
计划我们的变更 (Planning our changes)
The commands accepts the following options:
这些命令接受以下选项:
- EX seconds — Set the specified expire time, in seconds. EX seconds —设置指定的到期时间,以秒为单位。
- PX milliseconds — Set the specified expire time, in milliseconds. PX毫秒—设置指定的终止时间(以毫秒为单位)。
- NX — Only set the key if it does not already exist. NX —仅在不存在的情况下设置密钥。
- XX — Only set the key if it already exists. XX —仅设置密钥(如果已存在)。
- KEEPTTL — Retain the Time To Live (TTL) associated with the key KEEPTTL —保留与密钥关联的生存时间(TTL)
As noted in the documentation, there is some overlap with some of the options above and the following commands: SETNX, SETEX, PSETEX. As of this writing these three commands are not officially deprecated, but the documentation mentions that it might happen soon. Given that we can access the same features through the NX
, EX
& PX
options respectively, we will not implement these three commands.
如文档中所述,上面的一些选项和以下命令有些重叠: SETNX , SETEX , PSETEX 。 撰写本文时,尚未正式弃用这三个命令,但文档中提到这可能很快就会发生。 鉴于我们可以分别通过NX
, EX
和PX
选项访问相同的功能,因此我们将不实施这三个命令。
127.0.0.1:6379> SET a-key a-value EX 10 OK 127.0.0.1:6379> TTL "a-key" (integer) 8 127.0.0.1:6379> SETEX a-key 10 a-value OK 127.0.0.1:6379> TTL "a-key" (integer) 8
TTL
returned 8 in both cases, because it took me about 2s to type the TTL
command, and by that time about 8s were left, of the initial 10.
TTL
在这两种情况下均返回8,因为键入TTL
命令花了我大约2秒钟的时间,到那时,剩下的最初10个还剩下8秒钟。
EX和PX选项 (The EX & PX options)
When a key is set with either of these two options, or through the SETEX
& PSETEX
commands, but we are ignoring these for the sake of simplicity, Redis adds the key to different dictionary, internally called in the C code. This dictionary is dedicated to storing keys with a TTL, the key is the same key and the value is the timestamp of the expiration, in milliseconds.
当使用这两个选项之一或者通过SETEX
& PSETEX
命令来设置密钥时,为简单起见,我们忽略了这些,Redis将密钥添加到C语言内部调用的不同字典中。 该字典专用于存储带有TTL的密钥,该密钥是同一密钥,并且值是到期的时间戳(以毫秒为单位)。
Redis uses two approaches to delete keys with a TTL. The first one is a lazy approach, when reading a key, it checks if it exists in the expires
dictionary, if it does and the value is lower than the current timestamp in milliseconds, it deletes the key and does not proceed with the read.
Redis使用两种方法来删除带有TTL的密钥。 第一种是惰性方法,当读取密钥时,它会检查密钥是否存在于expires
字典中,如果存在且值小于当前时间戳(以毫秒为单位),它将删除密钥并且不继续进行读取。
“Lazy” & “Eager”
“懒惰”和“渴望”
The terms lazy is often used in programming, it describes an operation that is put on the back-burner and delayed until it absolutely has to be performed.
懒惰一词经常在编程中使用,它描述了放到后燃器上并延迟到绝对必须执行的操作。
In the context of Redis, it makes sense to describe the eviction strategy described above as lazy since Redis might still store keys that are effectively expired and will only guarantee their deletion until they are accessed past their expiration timestamp.
在Redis的上下文中,将上述驱逐策略描述为懒惰是有意义的,因为Redis可能仍会存储有效过期的密钥,并且只能保证将其删除,直到在过期时间戳之前对其进行访问为止。
The opposite approach is “eager”, where an operation is performed as soon as possible, whether or not it could be postponed.
相反的方法是“急切”,即无论是否可以推迟操作,都应尽快执行。
The other one is a more proactive approach, Redis periodically scans a subset of the list of keys with a TTL value and deletes the expired one. This action, performed in the serverCron
function is part of the event loop. The event loop is defined in ae.c
and starts in the aeMain
function, it continuously executes the aeProcessEvents
function in a while loop, until the stop
flag is set to 1
, which essentially never happens when the server is running under normal circumstances. The aeStop
function is the only function doing this and it is only used in redis-benchmark.c
& example-ae.c
.
另一种是更主动的方法,Redis会定期使用TTL值扫描键列表的子集,并删除过期的键。 在serverCron
函数中执行的此操作是事件循环的一部分。 事件循环是在ae.c
定义的,并从aeMain
函数开始,它在while循环中连续执行aeProcessEvents
函数,直到stop
标志设置为1
为止,这在服务器正常运行的情况下基本上不会发生。 aeStop
函数是唯一执行此操作的函数,仅在redis-benchmark.c
和example-ae.c
。
aeProcessEvents
is a fairly big function, it would be hard to summarize it all here, but it first uses the aeApiPoll
function, which is what we covered in the previous chapter. It processes the events from the poll result, if any and then calls processTimeEvents
.
aeProcessEvents
是一个相当大的函数,在这里很难aeProcessEvents
总结,但是它首先使用aeApiPoll
函数,这是我们在上一章中介绍的功能。 它处理轮询结果中的事件(如果有),然后调用processTimeEvents
。
Redis maintains a list of time events, as described in the event loop documentation page, for periodic task. When the server is initialized, one time event is created, for the serverCron
function. This function is responsible for a lot of things, this is how it is described in the source code:
Redis维护事件事件列表,如事件循环文档页面中所述,用于定期任务。 初始化服务器后,将为serverCron
函数创建一个事件。 这个函数负责很多事情,这就是它在源代码中的描述方式:
This is our timer interrupt, called server.hz times per second.Here is where we do a number of things that need to be done asynchronously.For instance:- Active expired keys collection (it is also performed in a lazy way on lookup).- Software watchdog.- Update some statistic.- Incremental rehashing of the DBs hash tables.- Triggering BGSAVE / AOF rewrite, and handling of terminated children.- Clients timeout of different kinds.- Replication reconnection.- Many more …
这是我们的计时器中断,称为server.hz次/秒。在这里我们执行许多需要异步完成的事情,例如:-有效的过期键集合(在查找时也以惰性方式执行) .-软件看门狗。-更新一些统计信息。-DB哈希表的增量重新哈希处理。-触发BGSAVE / AOF重写,并处理终止的子项。
We’re only interested in the first item in this list for now, the active expiration of keys. Redis runs the event loop on a single thread. This means that each operation performed in the event loop effectively blocks the ones waiting to be processed. Redis tries to optimize for this by making all the operations performed in the event loop “fast”.
目前,我们仅对此列表中的第一项感兴趣,即密钥的有效期满。 Redis在单个线程上运行事件循环。 这意味着事件循环中执行的每个操作都会有效地阻止等待处理的操作。 Redis尝试通过使事件循环中执行的所有操作“快速”进行优化。
This is one of the reasons why the documentation provides the time complexity for each commands. Most commands are O(1), and commands like , with an O(n) complexity are not recommended in a production environment, it would prevent the server from processing any incoming commands while iterating through all the keys.
这是文档提供每个命令的时间复杂性的原因之一。 大多数命令都是O(1),在生产环境中不建议使用复杂度为O(n)的命令,因为这样会阻止服务器在迭代所有键时处理任何传入的命令。
If Redis were to scan all the keys in the expires dictionary on every iteration of the event loop it would be an O(n) operation, where n is the number of keys with a TTL value. Put differently, as you add keys with a TTL, you would slow down the process of active expiration. To prevent this, Redis only scans the expires up to certain amount. The activeExpireCycle
contains a lot of optimizations that we will not explore for now for the sake of simplicity.
如果Redis在事件循环的每次迭代中扫描过期字典中的所有键,则将是O(n)操作,其中n是具有TTL值的键数。 换句话说,当您使用TTL添加密钥时,会减慢活动到期的过程。 为了防止这种情况,Redis仅扫描到期的邮件,直至达到一定数量。 activeExpireCycle
包含很多优化,为简单起见,我们暂时不会进行探讨。
One of these optimizations takes care of maintaining statistics about the server, among those Redis keeps track of an estimate of the number of keys that are expired but not yet deleted. Using this it will attempt to expire more keys to try to keep this number under control and prevent it from growing too fast if keys start expiring faster than the normal rate at which they get deleted.
这些优化之一是维护有关服务器的统计信息,其中Redis跟踪已过期但尚未删除的密钥数量的估计值。 使用此方法,它将尝试使更多密钥过期,以尝试控制此数字,并防止密钥开始过期的正常速度超过其被删除的正常速度,从而防止密钥增长太快。
serverCron
is first added to the time events with a time set to 1ms in the future. The return value of functions executed as time events dictates if it is removed from the time event queue or if it is rescheduled in the future. serverCron
returns a value based on the frequency set as config. By default 100ms. That means that it won't run more than 10 times per second.
首先将serverCron
添加到时间事件中,并将时间设置为1ms。 作为时间事件执行的函数的返回值决定了是否将其从时间事件队列中删除或将来是否对其进行了重新安排。 serverCron
根据设置为config的频率返回一个值。 默认情况下为100ms。 这意味着它每秒运行不超过10次。
I think that it’s worth stopping for a second to recognize the benefits of having two eviction strategies for expired keys. The lazy approach gets the job done as it guarantees that no expired keys will ever be returned, but if a key is set with a TTL and is never read again it would unnecessarily sit in memory, using space. The incremental active approach solves this problem, while still being optimized for speed and does not pause the server to clean all the keys.
我认为值得一秒钟的时间来认识到针对过期密钥采用两种驱逐策略的好处。 惰性方法可以确保不会返回过期的密钥,因此可以完成工作,但是如果密钥设置为TTL并且再也不会被读取,它将不必要地占用内存,从而占用空间。 增量主动方法解决了此问题,同时仍针对速度进行了优化,并且不会暂停服务器以清理所有键。
Big O Notation
大O符号
The Big O Notation is used to describe the time complexity of operations. In other words, it describes how much slower, or not, an operation would be, as the size of the elements it operates on increases.
大O表示法用于描述操作的时间复杂度。 换句话说,它描述了随着操作的元素大小的增加,操作会变慢或不慢。
The way that I like to think about it is to transcribe the O notation to a function with a single parameter n, that returns the value inside the parentheses after O. So O(n) — which is the complexity of the command — would become
def fn(n); n; end;
if written in Ruby, orlet fn = (n) => n
in javascript. O(1) - which is the complexity of the command - would bedef fn(n); 1; end;
, O(logn) - which is the complexity of the command - would becomedef fn(n); Math.log(n); end;
and O(n^2) - as far as I know, no Redis command has such complexity - would becomedef fn(n); n.pow(2); end;
.我想考虑的方式是将O表示形式转换为具有单个参数n的函数,该函数在O之后返回括号内的值。因此,O(n)(这是命令的复杂性)将变为
def fn(n); n; end;
def fn(n); n; end;
如果使用Ruby编写,或者在javascript中let fn = (n) => n
。 O(1)-命令的复杂度-将为def fn(n); 1; end;
def fn(n); 1; end;
,O(logn)-这是命令的复杂性-将变为def fn(n); Math.log(n); end;
def fn(n); Math.log(n); end;
和O(n ^ 2)-据我所知,没有Redis命令具有这种复杂性-将变为def fn(n); n.pow(2); end;
def fn(n); n.pow(2); end;
。We can play with these functions to illustrate the complexity of the different commands.
SET
has a time complexity of O(1), commonly referred to as constant time. Regardless of the number of keys stored in Redis, the operations required to fulfill aSET
command are the same, so whether we are operating on an empty Redis server or one with millions of keys, it'll take a similar amount of time. With the function defined above we can see that, if n is the number of keys,fn(n)
will always return1
, regardless of n.我们可以使用这些功能来说明不同命令的复杂性。
SET
的时间复杂度为O(1),通常称为恒定时间。 无论Redis中存储的密钥数量如何,执行SET
命令所需的操作都是相同的,因此,无论我们是在空的Redis服务器上运行还是在具有数百万个密钥的服务器上运行,都将花费相似的时间。 通过上面定义的函数,我们可以看到,如果n是键的数目,则无论n是多少,fn(n)
都将始终返回1
。On the other hand
KEYS
has a complexity of O(n), where n is the number of keys stored in Redis.另一方面,
KEYS
的复杂度为O(n),其中n是Redis中存储的密钥数。It’s important to note that n is always context dependent and should therefore always be specified, which Redis does on each command page. In comparison, is also documented with having a time complexity of O(n), but, and this is the important part, where n is the number of keys given to the command. Calling
DEL a-key
has therefore a time complexity of O(1), and runs in constant time.重要的是要注意,n始终是上下文相关的,因此应始终指定它,Redis在每个命令页面上都这样做。 相比之下,也记录了时间复杂度为O(n)的情况,但这是重要的部分,其中n是赋予命令的键数 。 因此,调用
DEL a-key
的时间复杂度为O(1),并且以恒定的时间运行。
KEYS
iterates through all the items in Redis and return all the keys. With the function defined above, we can see thatfn(1)
will return1
,fn(10)
,10
, and so on. What this tells us is that the time required to executeKEYS
will grow proportionally to the value of n.
KEYS
遍历Redis中的所有项目并返回所有密钥。 使用上面定义的函数,我们可以看到fn(1)
将返回1
,fn(10)
,10
,依此类推。 这告诉我们,执行KEYS
所需的时间将与n的值成比例地增长。Lastly, it’s important to note that this does not necessarily mean that
KEYS
ran on a server with 100 items will be exactly 100 times slower than running against a server with one key. There are some operations that will have to be performed regardless, such as parsing the command and dispatching to thekeysCommand
function. These are in the category of "fixed cost", they always have to be performed. If it takes 1ms to run those and then 0.1ms per key - these are only illustrative numbers -, it would take Redis 1.1ms to runKEYS
for one key and 10.1ms with 100 keys. It's not exactly 100 times more, but it is in the order of 100 times more.最后,需要注意的重要一点是,这不一定意味着在具有100个项目的服务器上运行
KEYS
实际上比在具有一个密钥的服务器上运行的速度慢100倍。 无论如何,都必须执行一些操作,例如解析命令并将其分派到keysCommand
函数。 这些属于“固定成本”类别,必须始终执行。 如果运行这些命令需要1毫秒,然后每个密钥需要0.1毫秒-这些仅是示例性数字-那么Redis将需要1.1毫秒才能对一个密钥运行KEYS
,而使用100个密钥则需要10.1毫秒。 它不完全是100倍,但大约是100倍。
NX,XX和KEEPTTL选项 (The NX, XX & KEEPTTL options)
These options are easier to implement compared to the previous two given that they are not followed by a value. Additionally, their behavior does not require implementing more components to the server, beyond a few conditions in the method that takes care of storing the key and the value specified by the SET
command.
与前两个选项相比,这些选项不带值,因此较易于实现。 此外,它们的行为不需要在服务器中实现更多组件,而在该方法中,除了需要存储SET
和SET
命令指定的键值之外,还需要满足一些条件。
Most of the complexity resides in the validations of the command, to make sure that it has a valid format.
大多数复杂性都在于命令的验证中,以确保其具有有效的格式。
添加验证 (Adding validation)
Prior to adding these options, validating the SET
command did not require a lot of work. In its simple form, it requires a key and value. If both are present, the command is valid, if one is missing, it is a "wrong number of arguments" error.
在添加这些选项之前,验证SET
命令不需要太多工作。 它以简单的形式要求键和值。 如果两个都存在,则该命令有效;如果缺少一个,则是“参数数量错误”错误。
This rule still applies but we need to add a few more to support the different combinations of possible options. These are the rules we now need to support:
该规则仍然适用,但是我们需要添加更多一些以支持可能选项的不同组合。 这些是我们现在需要支持的规则:
You can only specify one of the PX or EX options, not both. Note that the
redis-cli
has a user friendly interface that hints at this constraint by displaying the followingkey value [EX seconds|PX milliseconds] [NX|XX] [KEEPTTL]
when you start typingSET
. The|
character betweenEX seconds
&PX milliseconds
expresses theOR
condition.您只能指定PX或EX选项之一,不能同时指定两者。 请注意,
redis-cli
具有用户友好的界面,当您开始键入SET
时,它通过显示以下key value [EX seconds|PX milliseconds] [NX|XX] [KEEPTTL]
来提示此约束。|
EX seconds
和PX milliseconds
之间的字符表示OR
条件。Following the hints from redis-cli, we can only specify
NX
orXX
, not both.根据redis-cli的提示,我们只能指定
NX
或XX
,不能同时指定两者。The redis-cli hint does not make this obvious, but you can only specify
KEEPTTL
if neitherEX
orPX
or present. The following commandSET 1 2 EX 1 KEEPTTL
is invalid and returns(error) ERR syntax error
redis-cli提示不会使此变得明显,但是如果
EX
或PX
都不存在,则只能指定KEEPTTL
。 以下命令SET 1 2 EX 1 KEEPTTL
无效,并返回(error) ERR syntax error
It’s also worth mentioning that the order of options does not matter, both commands are equivalent:
还值得一提的是,选项的顺序无关紧要,这两个命令是等效的:
SET a-key a-value NX EX 10SET a-key a-value EX 10 NX
But the following would be invalid, EX
must be followed by an integer:
但是以下内容将无效, EX
必须跟一个整数:
SET a-key a-value EX NX 10
让我们写一些代码! (Let’s write some code!)
We are making the following changes to the server:
我们正在对服务器进行以下更改:
- Add our own, simplified event loop, including support for time events 添加我们自己的简化事件循环,包括对时间事件的支持
Accepts options for the expiration related options,
EX
&PX
接受与到期相关的选项,
EX
和PX
Accepts options for the presence or absence of a key,
NX
&XX
接受存在或不存在
NX
和XX
键的选项- Delete expired keys on read 读取时删除过期的密钥
Setting a key without
KEEPTTL
removes any previously set TTL设置没有
KEEPTTL
的密钥会删除任何先前设置的TTLImplement the
TTL
&PTTL
commands as they are useful to use alongside keys with a TTL实施
TTL
和PTTL
命令,因为它们可与TTL键一起使用
I’m giving you the complete code first and we’ll look at the interesting parts one by one afterwards:
我首先给您完整的代码,然后我们将逐个介绍有趣的部分:
require 'socket'
require 'timeout'
require 'logger'
LOG_LEVEL = ENV['DEBUG'] ? Logger::DEBUG : Logger::INFOrequire_relative './expire_helper'
require_relative './get_command'
require_relative './set_command'
require_relative './ttl_command'
require_relative './pttl_command'class RedisServer COMMANDS = {
'GET' => GetCommand,
'SET' => SetCommand,
'TTL' => TtlCommand,
'PTTL' => PttlCommand,
} MAX_EXPIRE_LOOKUPS_PER_CYCLE = 20
DEFAULT_FREQUENCY = 10 # How many times server_cron runs per second TimeEvent = Struct.new(:process_at, :block) def initialize
@logger = Logger.new(STDOUT)
@logger.level = LOG_LEVEL @clients = []
@data_store = {}
@expires = {} @server = TCPServer.new 2000
@time_events = []
@logger.debug "Server started at: #{ Time.now }"
add_time_event(Time.now.to_f.truncate + 1) do
server_cron
end start_event_loop
end private def add_time_event(process_at, &block)
@time_events << TimeEvent.new(process_at, block)
end def nearest_time_event
now = (Time.now.to_f * 1000).truncate
nearest = nil
@time_events.each do |time_event|
if nearest.nil?
nearest = time_event
elsif time_event.process_at < nearest.process_at
nearest = time_event
else
next
end
end nearest
end def select_timeout
if @time_events.any?
nearest = nearest_time_event
now = (Time.now.to_f * 1000).truncate
if nearest.process_at < now
0
else
(nearest.process_at - now) / 1000.0
end
else
0
end
end def start_event_loop
loop do
timeout = select_timeout
@logger.debug "select with a timeout of #{ timeout }"
result = IO.select(@clients + [@server], [], [], timeout)
sockets = result ? result[0] : []
process_poll_events(sockets)
process_time_events
end
end def process_poll_events(sockets)
sockets.each do |socket|
begin
if socket.is_a?(TCPServer)
@clients << @server.accept
elsif socket.is_a?(TCPSocket)
client_command_with_args = socket.read_nonblock(1024, exception: false)
if client_command_with_args.nil?
@clients.delete(socket)
elsif client_command_with_args == :wait_readable
# There's nothing to read from the client, we don't have to do anything
next
elsif client_command_with_args.strip.empty?
@logger.debug "Empty request received from #{ client }"
else
commands = client_command_with_args.strip.split("\n")
commands.each do |command|
response = handle_client_command(command.strip)
@logger.debug "Response: #{ response }"
socket.puts response
end
end
else
raise "Unknown socket type: #{ socket }"
end
rescue Errno::ECONNRESET
@clients.delete(socket)
end
end
end def process_time_events
@time_events.delete_if do |time_event|
next if time_event.process_at > Time.now.to_f * 1000 return_value = time_event.block.call if return_value.nil?
true
else
time_event.process_at = (Time.now.to_f * 1000).truncate + return_value
@logger.debug "Rescheduling time event #{ Time.at(time_event.process_at / 1000.0).to_f }"
false
end
end
end def handle_client_command(client_command_with_args)
@logger.debug "Received command: #{ client_command_with_args }"
command_parts = client_command_with_args.split
command_str = command_parts[0]
args = command_parts[1..-1] command_class = COMMANDS[command_str] if command_class
command = command_class.new(@data_store, @expires, args)
command.call
else
formatted_args = args.map { |arg| "`#{ arg }`," }.join(' ')
"(error) ERR unknown command `#{ command_str }`, with args beginning with: #{ formatted_args }"
end
end def server_cron
start_timestamp = Time.now
keys_fetched = 0 @expires.each do |key, _|
if @expires[key] < Time.now.to_f * 1000
@logger.debug "Evicting #{ key }"
@expires.delete(key)
@data_store.delete(key)
end keys_fetched += 1
if keys_fetched >= MAX_EXPIRE_LOOKUPS_PER_CYCLE
break
end
end end_timestamp = Time.now
@logger.debug do
sprintf(
"Processed %i keys in %.3f ms", keys_fetched, (end_timestamp - start_timestamp) * 1000)
end 1000 / DEFAULT_FREQUENCY
end
end
listing 4.1: server.rb
清单4.1:server.rb
class SetCommand ValidationError = Class.new(StandardError) CommandOption = Struct.new(:kind)
CommandOptionWithValue = Struct.new(:kind, :validator) OPTIONS = {
'EX' => CommandOptionWithValue.new(
'expire',
->(value) { validate_integer(value) * 1000 },
),
'PX' => CommandOptionWithValue.new(
'expire',
->(value) { validate_integer(value) },
),
'KEEPTTL' => CommandOption.new('expire'),
'NX' => CommandOption.new('presence'),
'XX' => CommandOption.new('presence'),
} ERRORS = {
'expire' => '(error) ERR value is not an integer or out of range',
} def self.validate_integer(str)
Integer(str)
rescue ArgumentError, TypeError
raise ValidationError, '(error) ERR value is not an integer or out of range'
end def initialize(data_store, expires, args)
@logger = Logger.new(STDOUT)
@logger.level = LOG_LEVEL
@data_store = data_store
@expires = expires
@args = args @options = {}
end def call
key, value = @args.shift(2)
if key.nil? || value.nil?
return "(error) ERR wrong number of arguments for 'SET' command"
end parse_result = parse_options if !parse_result.nil?
return parse_result
end existing_key = @data_store[key] if @options['presence'] == 'NX' && !existing_key.nil?
'(nil)'
elsif @options['presence'] == 'XX' && existing_key.nil?
'(nil)'
else @data_store[key] = value
expire_option = @options['expire'] # The implied third branch is if expire_option == 'KEEPTTL', in which case we don't have
# to do anything
if expire_option.is_a? Integer
@expires[key] = (Time.now.to_f * 1000).to_i + expire_option
elsif expire_option.nil?
@expires.delete(key)
end 'OK'
end rescue ValidationError => e
e.message
end private def parse_options
while @args.any?
option = @args.shift
option_detail = OPTIONS[option] if option_detail
option_values = parse_option_arguments(option, option_detail)
existing_option = @options[option_detail.kind] if existing_option
return '(error) ERR syntax error'
else
@options[option_detail.kind] = option_values
end
else
return '(error) ERR syntax error'
end
end
end def parse_option_arguments(option, option_detail) case option_detail
when CommandOptionWithValue
option_value = @args.shift
option_detail.validator.call(option_value)
when CommandOption
option
else
raise "Unknown command option type: #{ option_detail }"
end
end
end
listing 4.2: set_command.rb
清单4.2:set_command.rb
class GetCommand def initialize(data_store, expires, args)
@logger = Logger.new(STDOUT)
@logger.level = LOG_LEVEL
@data_store = data_store
@expires = expires
@args = args
end def call
if @args.length != 1
"(error) ERR wrong number of arguments for 'GET' command"
else
key = @args[0]
ExpireHelper.check_if_expired(@data_store, @expires, key)
@data_store.fetch(key, '(nil)')
end
end
end
listing 4.3: get_command.rb
清单4.3:get_command.rb
class PttlCommand def initialize(data_store, expires, args)
@logger = Logger.new(STDOUT)
@logger.level = LOG_LEVEL
@data_store = data_store
@expires = expires
@args = args
end def call
if @args.length != 1
"(error) ERR wrong number of arguments for 'PTTL' command"
else
key = @args[0]
ExpireHelper.check_if_expired(@data_store, @expires, key)
key_exists = @data_store.include? key
if key_exists
ttl = @expires[key]
if ttl
(ttl - (Time.now.to_f * 1000)).round
else
-1
end
else
-2
end
end
end
end
listing 4.4: pttl_command.rb
清单4.4:pttl_command.rb
class TtlCommand def initialize(data_store, expires, args)
@data_store = data_store
@expires = expires
@args = args
end def call
if @args.length != 1
"(error) ERR wrong number of arguments for 'TTL' command"
else
pttl_command = PttlCommand.new(@data_store, @expires, @args)
result = pttl_command.call.to_i
if result > 0
(result / 1000.0).round
else
result
end
end
end
end
listing 4.5: ttl_command.rb
清单4.5:ttl_command.rb
module ExpireHelper def self.check_if_expired(data_store, expires, key)
expires_entry = expires[key]
if expires_entry && expires_entry < Time.now.to_f * 1000
logger.debug "evicting #{ key }"
expires.delete(key)
data_store.delete(key)
end
end def self.logger
@logger ||= Logger.new(STDOUT).tap do |l|
l.level = LOG_LEVEL
end
end
end
listing 4.6: expire_helper.rb
清单4.6:expire_helper.rb
变化 (The changes)
将逻辑拆分为多个文件 (Splitting it the logic in multiple files)
The server.rb
file started getting pretty big so we extracted the logic for GET
& SET
to different files, and gave them their own classes.
server.rb
文件开始变得很大,因此我们将GET
& SET
的逻辑提取到不同的文件中,并为其提供了自己的类。
时间事件 (Time events)
In order to implement the eviction logic for keys having an expiration, we refactored how we call the IO.select
method. Our implementation is loosely based on the one built in Redis, ae. The RedisServer
— renamed from BasicServer
in the previous chapters — starts the event loop in its constructor. The event loop is a never ending loop that calls select
, processes all the incoming events and then process time events, if any need to be processed.
为了实现具有过期密钥的收回逻辑,我们重构了如何调用IO.select
方法。 我们的实现大致基于Redis内建的ae 。 RedisServer
(在前几章中从BasicServer
重命名)在其构造函数中启动事件循环。 事件循环是一个永无止境的循环,它调用select
,处理所有传入事件,然后处理时间事件(如果需要处理)。
We introduced the TimeEvent
class, defined as follows:
我们介绍了TimeEvent
类,其定义如下:
TimeEvent = Struct.new(:process_at, :block)
The process_at
field is an Integer
that represents the timestamp, in milliseconds, for when the event should be processed. The block
field is the actual code that will be run. For now, there’s only one type of events, server_cron
. It is first added to the @time_events
list with a process_at
value set to 1ms in the future.
process_at
字段是一个Integer
,表示应在何时处理事件的时间戳(以毫秒为单位)。 block
字段是将要运行的实际代码。 目前,只有一种类型的事件server_cron
。 它将首先添加到@time_events
列表中,将来的process_at
值设置为1ms。
Time events can be either one-off, they’ll run only once, or repeating, they will be rescheduled at some point in the future after being processed. This behavior is driven by the return value of the block
field. If the block returns nil
, the time event is removed from the @time_events
list, if it returns an integer return_value
, the event is rescheduled for return_value
milliseconds in the future, by changing the value of process_at
. By default the server_cron
method is configured with a frequency of 10 Hertz (hz), which means it will run up to 10 times per second, or put differently, every 100ms. This is why the return value of server_cron
is 1000 / DEFAULT_FREQUENCY
— 1000 is the number of milliseconds, if frequency was 20, it would return 50, as in, it should run every 50ms.
时间事件可以是一次性事件,它们只能运行一次,也可以重复,它们将在处理后的将来某个时间重新安排。 此行为是由block
字段的返回值驱动的。 如果该块返回nil
,则将时间事件从@time_events
列表中删除,如果它返回一个整数return_value
,则通过更改process_at
的值将事件重新安排为return_value
毫秒。 默认情况下, server_cron
方法的配置频率为10赫兹(hz),这意味着它将每秒最多运行10次,或者每100ms运行一次。 这就是为什么server_cron
的返回值为1000 / DEFAULT_FREQUENCY
— 1000是毫秒数,如果频率为20,它将返回50,因为它应该每50ms运行一次。
This behavior makes sure that we don’t run the server_cron
method too often, it effectively gives a higher priority to handling client commands, and new clients connecting.
此行为可确保我们不要过于频繁地运行server_cron
方法,它实际上为处理客户端命令和新的客户端连接提供了更高的优先级。
选择超时 (Select timeout)
When we introduced IO.select
in Chapter 3, we used it without the timeout argument. This wasn’t a problem then because the server had nothing else to do. It would either need to accept a new client, or reply to a client command, and both would be handled through select
.
在第3章中介绍IO.select
时,我们使用了没有timeout参数的方法。 那时这不是问题,因为服务器没有其他事情要做。 它要么需要接受一个新客户端,要么回复一个客户端命令,并且两者都将通过select
进行处理。
The server needs to do something else beside waiting on select
now, run the time events when they need to be processed. In order to do so, Redis uses a timeout with its abstraction over select and other multiplexing libraries, aeApiPoll
. Redis can, under some conditions, use no timeout, which we’re going to ignore for now, for the sake of simplicity. When using a timeout, Redis makes sure that waiting on the timeout will not delay any future time events that should be processed instead of waiting on select
. In order to achieve this, Redis looks at all the time events and finds the nearest one, and sets a timeout equivalent to the time between now and that event. This guarantees that even if there’s no activity between now and when the next time event should be processed, redis will stop waiting on aeApiPoll
and process the time events.
除了需要等待立即select
,服务器还需要执行其他操作,在需要处理时间事件时运行它们。 为了做到这一点,Redis在选择库和其他复用库aeApiPoll
使用了一个超时及其抽象。 在某些情况下,Redis可以不使用任何超时,为简单起见,我们暂时将其忽略。 使用超时时,Redis确保等待超时不会延迟任何应该处理的未来时间事件,而不是等待select
。 为了实现这一点,Redis会查看所有时间事件并找到最接近的事件,并设置与现在到该事件之间的时间相等的超时时间。 这样可以确保即使从现在到应该处理下一个时间事件之间都没有活动,redis也会停止等待aeApiPoll
并处理时间事件。
We’re replicating this logic in the select_timeout
method. It starts by delegating the task of finding the nearest time event through the nearest_time_event
, which iterates through all the time events in the @time_events
array and find the one with the smallest value for process_at
.
我们正在select_timeout
方法中复制此逻辑。 它首先委派了一个任务,即通过nearest_time_event
查找最近的时间事件,该nearest_time_event
将遍历@time_events
数组中的所有时间事件,并为process_at
查找值最小的事件。
In concrete terms, in RedisServer
, server_cron
runs every 100ms, so when we call IO.select
, the next time event will be at most 100ms in the future. The timeout given to select
will be a value between 0 and 100ms.
RedisServer
,在RedisServer
, server_cron
每100毫秒运行一次,因此,当我们调用IO.select
,下一次事件将在将来最多100毫秒。 提供给select
的超时值为0到100ms之间的值。
解析选项 (Parsing options)
Probably one of the most complicated changes introduced in this chapter, at least for me as I was implementing it. The logic is in the SetCommand
class. We first define all the possible options in the OPTIONS
constant. Each option is a key/value pair where the key is the option as expected in the command string and the value is an instance of CommandOption
or CommandOptionWithValue
. After extracting and validating the first three elements of the string, respectively, the SET
string, followed by the key and the value, we split the rest on spaces and process them from left to right, with the shift
method. For every option we find, we look up the OPTIONS
hash to retrieve the matching CommandOption
or CommandOptionWithValue
instance. If nil
is returned, it means that the given option is invalid, this is a syntax error. Note that once again, for the sake of simplicity, we did not implement case insensitive commands the way Redis does.
可能是本章介绍的最复杂的更改之一,至少对于我在实施它时而言。 逻辑在SetCommand
类中。 我们首先在OPTIONS
常量中定义所有可能的OPTIONS
。 每个选项都是键/值对,其中键是命令字符串中期望的选项,值是CommandOption
或CommandOptionWithValue
的实例。 在分别提取并验证了字符串的前三个元素(分别为SET
字符串,键和值)并进行验证之后,我们将其余部分分开,并使用shift
方法从左到右处理它们。 对于找到的每个选项,我们都会查询OPTIONS
哈希以检索匹配的CommandOption
或CommandOptionWithValue
实例。 如果返回nil
,则意味着给定的选项无效,这是语法错误。 请再次注意,为简单起见,我们没有像Redis那样实现不区分大小写的命令。
If the an option is found, but we had already found one of the same kind, presence
or expire
, this is also a syntax error. This check allows us to consider the following commands as invalid:
如果找到了一个选项,但是我们已经找到了相同的一种( presence
或expire
,这也是语法错误。 此检查使我们认为以下命令无效:
SET key value EX 1 PX 2
SET key value EX 1 KEEPTTL
SET key value NX XX
Finally, we attempt to parse the option argument, if necessary, only EX
and PX
have an argument, the others one do not, this is why we use two different classes here. parse_option_arguments
will return the option itself if we found an option that should not be followed by an argument, that is either NX
, XX
or KEEPTTL
. If we found one of the other two options, option_detail
will be an instance of CommandOptionWithValue
, we use shift
once again to obtain the next element in the command string and feed it to the validator block.
最后,我们尝试解析选项参数,如有必要,只有EX
和PX
有一个参数,其他没有,这就是为什么我们在这里使用两个不同的类。 如果我们发现一个不应带有参数的选项NX
, XX
或KEEPTTL
,则parse_option_arguments
将返回选项本身。 如果我们发现其他两个选项之一, option_detail
会的一个实例CommandOptionWithValue
,我们用shift
再次获得命令字符串的下一个元素,并将其输送到验证块。
The validator blocks are very similar for the options, they both validate that the string is a valid integer, but the EX
validator multiplies the final result by 1000 to convert the value from seconds to milliseconds.
验证器块的选项非常相似,它们均验证字符串是有效整数,但EX
验证器将最终结果乘以1000,以将值从秒转换为毫秒。
The values are then stored in the @options
hash, with either the presence
or expire
key, based on the kind
value. This allows us to read from the @options
hash in the call method to apply the logic required to finalize the implementation of these options.
然后,根据kind
值,将这些值与presence
或expire
键一起存储在@options
哈希中。 这使我们能够从call方法中的@options
哈希中读取信息,以应用完成这些选项的实现所需的逻辑。
If @options['presence']
is set to NX
and there is already a value at the same key, we return nil
right away. Similarly if it is set to XX
and there is no key, we also return nil.
如果@options['presence']
设置为NX
并且同一键上已经有一个值,则立即返回nil
。 同样,如果将其设置为XX
并且没有密钥,我们还将返回nil。
Finally, we always set the value for the key in the @data_store
hash, but the behavior regarding the secondary hash, @expires
, is different depending on the value of @options['expire']
. If it is set to an integer, we use this integer and add it to the current time, in milliseconds, in the @expires
hash. If the value is nil, it means that KEEPTTL
was not passed, so we remove any value that may have previously been set by a previous SET
command with the same key and value for either PX
or EX
.
最后,我们总是在@data_store
哈希中设置键的值,但是关于第二个哈希@expires
的行为因@options['expire']
的值而异。 如果将其设置为整数,我们将使用该整数并将其添加到@expires
哈希中的当前时间(以毫秒为单位)。 如果值为nil,则表示未传递KEEPTTL
,因此我们将删除先前由先前SET
命令使用相同键和PX
或EX
值设置的任何值。
Why not use a regular expression?
为什么不使用正则表达式?
Good question! The short answer is that after spending some time trying to use a regular expression, it did not feel easier, as a reference this where I got, just before I gave up:
好问题! 简短的答案是,在花了一些时间尝试使用正则表达式后,在我放弃之前,将它作为我得到的参考并不难,例如:
/^SET \d+ \d+ (?:EX (?\d+)|PX (?\d+)|KEEPTTL)?(:? ?(?NX|XX))?(?: (?:EX (?\d+)|PX (?\d+)))?$/
This regexp works for some cases but incorrectly considers the following as valid, Redis cannot process a SET
command with KEEPTTL
and EX 1
:
此正则表达式在某些情况下可以使用,但错误地认为以下内容有效,Redis无法使用KEEPTTL
和EX 1
处理SET
命令:
SET 1 2 KEEPTTL XX EX 1
It might be possible to use a regular expression here, given that the grammar of the SET
command does not allow that many permutations but even if it is, I don’t think it’ll be simpler than the solution we ended up with.
这也许可以在这里使用正则表达式,考虑到的语法SET
命令不允许许多排列,但即使是这样,我不认为它会比我们结束了该解决方案更简单。
For reference, this is how Redis does it, in a way that it conceptually not that far from how we ended up doing it here. The main difference is that in the Redis source, it is one function, whereas we opted to separate the definition of the options, in the OPTIONS
constant, from the actual code that consumes the characters from the string received from the client. I find the separated option a bit more readable and easier to reason about, but the approach used by Redis is definitely more efficient as there are less “things” being allocated, no extra resources to define what the options look like, just strings.
作为参考, 这是Redis的工作方式,从概念上讲,它与我们最终在此处所做的工作相距不远。 主要区别在于,在Redis源中,它是一个函数,而我们选择将OPTIONS
常量中的OPTIONS
定义与使用从客户端接收的字符串中提取字符的实际代码分开。 我发现分隔的选项更具可读性,并且更容易推理,但是Redis所使用的方法肯定更有效,因为分配的“事物”更少,没有多余的资源来定义选项的样子,只是字符串。
懒惰驱逐 (Lazy evictions)
The server_cron
time event takes care of cleaning up expired key every 100ms, but we also want to implement the “lazy eviction”, the same way Redis does. That is, if server_cron
hasn’t had the chance to evict an expired key yet, and the server receives a GET
command for the same key, we want to return nil and evict the key instead of returning it.
server_cron
time事件负责每100毫秒清理一次过期的密钥,但是我们也想实现Redis一样的“懒惰驱逐”。 也就是说,如果server_cron
还没有机会退出过期的密钥,并且服务器收到了针对同一密钥的GET
命令,则我们想返回nil并退出该密钥而不是返回它。
This logic is implemented in the ExpireHelper
module , in the check_if_expired
method. This method checks if there is an entry in the @expires
hash, and if there is it compares its value, a timestamp in milliseconds with the current time. If the value in @expires
is smaller, the key is expired and it deletes it. This will cause the GetCommand
, TtlCommand
& PttlCommand
classes to return (nil)
even if server_cron
hasn’t had a chance to delete the expired keys.
该逻辑是在ExpireHelper
模块的check_if_expired
方法中实现的。 此方法检查@expires
哈希中是否有一个条目,如果存在,则将其值与当前时间(以毫秒为单位)进行比较。 如果@expires
的值较小,则密钥已过期并删除它。 即使server_cron
没有机会删除过期的键,这也会导致GetCommand
, TtlCommand
和PttlCommand
类返回(nil)
。
新命令:TTL和PTTL (New commands: TTL & PTTL)
We added two new commands, TTL
& PTTL
. Both return the ttl of the given key as an integer, if it exists, the difference is that TTL
returns the value in seconds, whereas PTTL
returns it in milliseconds.
我们添加了两个新命令TTL
和PTTL
。 两者都以整数形式返回给定键的ttl(如果存在),不同之处在于TTL
以秒为单位返回该值,而PTTL
以毫秒为单位返回。
Given the similarity of these two commands, we only implemented the logic in the PttlCommand
class, and reused from the TtlCommand
class where we transform the value in milliseconds to a value in seconds before returning it.
鉴于这两个命令的相似性,我们仅在PttlCommand
类中实现了该逻辑,并从TtlCommand
类中重用了该类,在该类中,我们将以毫秒为单位的值转换为以秒为单位的值,然后再返回它。
记录仪 (Logger)
As the complexity of the codebase grew, it became useful to add logging statements. Such statements could be simple calls to puts
, print
or p
, but it is useful to be able to conditionally turn them on and off based on their severity. Most of the logs we added are only useful when debugging an error and are otherwise really noisy. All these statements are logged with @logger.debug
, and the severity of the logger is set based on the DEBUG
environment variable. This allows us to enable all the debug logs by adding the DEBUG=t
statement before running the server:
随着代码库的复杂性增加,添加日志记录语句变得很有用。 这样的语句可以是对puts
, print
或p
简单调用,但是能够根据其严重性有条件地打开和关闭它们是有用的。 我们添加的大多数日志仅在调试错误时有用,否则确实很吵。 所有这些语句都是使用@logger.debug
记录的,并且记录器的严重性是根据DEBUG
环境变量设置的。 这允许我们通过在运行服务器之前添加DEBUG=t
语句来启用所有调试日志:
DEBUG=true ruby -r"./server" -e "RedisServer.new"
还有更多测试 (And a few more tests)
We changed a lot of code and added more features, this calls for more tests.
我们更改了许多代码并添加了更多功能,这需要进行更多测试。
We added a special instruction, sleep
to allow us to easily write tests for the SET
command with any of the expire based options. For instance, to test that SET key value PX 100
actually works as expected, we want to wait at least 100ms, and assert that GET key
returns (nil)
instead of value
.
我们添加了一条特殊的指令sleep
,使我们可以轻松地使用任何基于expire选项来为SET
命令编写测试。 例如,要测试SET key value PX 100
确实按预期工作,我们要等待至少100ms,然后断言GET key
返回(nil)
而不是value
。
We also added a new way to specify assertion, with the syntax [ 'PTTL key', '2000+/-20' ]
. This is useful for the PTTL
command because it would be impossible to know exactly how long it’ll take the computer running the tests to execute the PTTL
command after running the SET
command. We can however estimate a reasonable range. In this case, we are assuming that the machine running the test will take less than 20ms to run PTTL
by leveraging the minitest assertion assert_in_delta
.
我们还添加了一种使用语法[ 'PTTL key', '2000+/-20' ]
来指定断言的新方法。 这对于PTTL
命令很有用,因为不可能确切知道运行SET
命令后运行测试的计算机执行PTTL
命令需要多长时间。 但是,我们可以估计一个合理的范围。 在这种情况下,我们假设运行测试机器将大于20ms少跑PTTL
通过利用MINITEST断言assert_in_delta
。
I also added the option to set the DEBUG
environment variable, which you can use when running all the tests or an individual test:
我还添加了用于设置DEBUG
环境变量的选项,您可以在运行所有测试或单个测试时使用该变量:
// All tests:
DEBUG=t ruby test.rb // Any values will work, even "false", as long as it's not nil
// Or a specific test
DEBUG=t ruby test.rb --name "RedisServer::SET#test_0005_handles the PX option with a valid argument"
There is now a begin/rescue
for Interrupt
in the forked process. This is to prevent an annoying stacktrace from being logged when we kill the process with Process.kill('INT', child)
after sending all the commands to the server.
在分叉的过程中现在有一个begin/rescue
的Interrupt
。 这是为了防止在我们将所有命令发送到服务器后使用Process.kill('INT', child)
进程时记录令人讨厌的堆栈跟踪。
require 'minitest/autorun'
require 'timeout'
require 'stringio'
require './server'describe 'RedisServer' do # ... def with_server child = Process.fork do
unless !!ENV['DEBUG']
# We're effectively silencing the server with these two lines
# stderr would have logged something when it receives SIGINT, with a complete stacktrace
$stderr = StringIO.new
# stdout would have logged the "Server started ..." & "New client connected ..." lines
$stdout = StringIO.new
end begin
RedisServer.new
rescue Interrupt => e
# Expected code path given we call kill with 'INT' below
end
end yield ensure
if child
Process.kill('INT', child)
Process.wait(child)
end
end def assert_command_results(command_result_pairs)
with_server do
command_result_pairs.each do |command, expected_result|
if command.start_with?('sleep')
sleep command.split[1].to_f
next
end
begin
socket = connect_to_server
socket.puts command
response = socket.gets
# Matches "2000+\-10", aka 2000 plus or minus 10
regexp_match = expected_result.match /(\d+)\+\/-(\d+)/
if regexp_match
# The result is a range
assert_in_delta regexp_match[1].to_i, response.to_i, regexp_match[2].to_i
else
assert_equal expected_result + "\n", response
end
ensure
socket.close if socket
end
end
end
end
# ... describe 'TTL' do
it 'handles unexpected number of arguments' do
assert_command_results [
[ 'TTL', '(error) ERR wrong number of arguments for \'TTL\' command' ],
]
end it 'returns the TTL for a key with a TTL' do
assert_command_results [
[ 'SET key value EX 2', 'OK'],
[ 'TTL key', '2' ],
[ 'sleep 0.5' ],
[ 'TTL key', '1' ],
]
end it 'returns -1 for a key without a TTL' do
assert_command_results [
[ 'SET key value', 'OK' ],
[ 'TTL key', '-1' ],
]
end it 'returns -2 if the key does not exist' do
assert_command_results [
[ 'TTL key', '-2' ],
]
end
end describe 'PTTL' do
it 'handles unexpected number of arguments' do
assert_command_results [
[ 'PTTL', '(error) ERR wrong number of arguments for \'PTTL\' command' ],
]
end it 'returns the TTL in ms for a key with a TTL' do
assert_command_results [
[ 'SET key value EX 2', 'OK'],
[ 'PTTL key', '2000+/-20' ], # Initial 2000ms +/- 20ms
[ 'sleep 0.5' ],
[ 'PTTL key', '1500+/-20' ], # Initial 2000ms, minus ~500ms of sleep, +/- 20ms
]
end it 'returns -1 for a key without a TTL' do
assert_command_results [
[ 'SET key value', 'OK' ],
[ 'PTTL key', '-1' ],
]
end it 'returns -2 if the key does not exist' do
assert_command_results [
[ 'PTTL key', '-2' ],
]
end
end # ... describe 'SET' do # ... it 'handles the EX option with a valid argument' do
assert_command_results [
[ 'SET 1 3 EX 1', 'OK' ],
[ 'GET 1', '3' ],
[ 'sleep 1' ],
[ 'GET 1', '(nil)' ],
]
end it 'rejects the EX option with an invalid argument' do
assert_command_results [
[ 'SET 1 3 EX foo', '(error) ERR value is not an integer or out of range']
]
end it 'handles the PX option with a valid argument' do
assert_command_results [
[ 'SET 1 3 PX 100', 'OK' ],
[ 'GET 1', '3' ],
[ 'sleep 0.1' ],
[ 'GET 1', '(nil)' ],
]
end it 'rejects the PX option with an invalid argument' do
assert_command_results [
[ 'SET 1 3 PX foo', '(error) ERR value is not an integer or out of range']
]
end it 'handles the NX option' do
assert_command_results [
[ 'SET 1 2 NX', 'OK' ],
[ 'SET 1 2 NX', '(nil)' ],
]
end it 'handles the XX option' do
assert_command_results [
[ 'SET 1 2 XX', '(nil)'],
[ 'SET 1 2', 'OK'],
[ 'SET 1 2 XX', 'OK'],
]
end it 'removes ttl without KEEPTTL' do
assert_command_results [
[ 'SET 1 3 PX 100', 'OK' ],
[ 'SET 1 2', 'OK' ],
[ 'sleep 0.1' ],
[ 'GET 1', '2' ],
]
end it 'handles the KEEPTTL option' do
assert_command_results [
[ 'SET 1 3 PX 100', 'OK' ],
[ 'SET 1 2 KEEPTTL', 'OK' ],
[ 'sleep 0.1' ],
[ 'GET 1', '(nil)' ],
]
end it 'accepts multiple options' do
assert_command_results [
[ 'SET 1 3 NX EX 1', 'OK' ],
[ 'GET 1', '3' ],
[ 'SET 1 3 XX KEEPTTL', 'OK' ],
]
end it 'rejects with more than one expire related option' do
assert_command_results [
[ 'SET 1 3 PX 1 EX 2', '(error) ERR syntax error'],
[ 'SET 1 3 PX 1 KEEPTTL', '(error) ERR syntax error'],
[ 'SET 1 3 KEEPTTL EX 2', '(error) ERR syntax error'],
]
end it 'rejects with both XX & NX' do
assert_command_results [
[ 'SET 1 3 NX XX', '(error) ERR syntax error'],
]
end
end # ...
end
结论 (Conclusion)
The SET
commands implemented by RedisServer
now behaves the same way it does with Redis. Well, almost. Let’s take a look at what happens if we were to use redis-cli
against our own server. Let’s start by running our server with
现在,由RedisServer
实现的SET
命令的行为与与Redis相同。 好吧,差不多。 让我们看看如果对我们自己的服务器使用redis-cli
会发生什么。 让我们从运行服务器开始
ruby -r"./server" -e "RedisServer.new"
and in another shell open redis-cli
on port 2000:
在另一个shell中,在端口2000上打开redis-cli
:
redis-cli -p 2000
And type the following:
并输入以下内容:
SET key value EX 200
And boom! It crashes!
和繁荣! 它崩溃了!
Error: Protocol error, got "(" as reply type byte
This is because RedisServer
does not implement the Redis Protocol, RESP. This is what the next chapter is all about. At the end of chapter 5 we will be able to use redis-cli
against our own server. Exciting!
这是因为RedisServer
没有实现Redis协议RESP 。 这就是下一章的全部内容。 在第5章的结尾,我们将能够对自己的服务器使用redis-cli
。 令人兴奋!
码 (Code)
As usual, the code is available on GitHub.
和往常一样,该代码在GitHub上可用 。
附录A:链接到Redis源代码 (Appendix A: Links to the Redis source code)
If you’re interested in digging into the Redis source code but would like some pointers as to where to start, you’ve come to the right place. The Redis source code is really well architected and overall relatively easy to navigate, so you are more than welcome to start the adventure on your own. That being said, it did take me a while to find the locations of functions I was interested in, such as: “where does redis handle the eviction of expired keys”, and a few others.
如果您有兴趣深入研究Redis源代码,但想要一些有关从哪里开始的指针,那么您来对地方了。 Redis源代码的架构非常好,总体上相对易于浏览,因此非常欢迎您自己开始冒险。 话虽如此,我确实花了一些时间才找到我感兴趣的功能的位置,例如:“ redis在哪里处理过期密钥的收回”,以及其他一些功能。
Before jumping in the code, you might want to read this article that explains some of the main data structures used by Redis: http://blog.wjin.org/posts/redis-internal-data-structure-dictionary.html.
在跳入代码之前,您可能需要阅读这篇文章,其中解释了Redis使用的一些主要数据结构: http : //blog.wjin.org/posts/redis-internal-data-structure-dictionary.html 。
In no particular orders, the following is a list of links to the Redis source code on GitHub, for features related to the implementation of keys with expiration:
以下是按顺序排列的,指向GitHub上Redis源代码的链接的列表,其中包含与过期密钥实现有关的功能:
SET命令的处理: (Handling of the SET command:)
server.c
defines all the commands inredisCommand
: https://github.com/antirez/redis/blob/6.0.0/src/server.c#L182server.c
在redisCommand
定义了所有命令: https : //github.com/antirez/redis/blob/6.0.0/src/server.c#L182t_string.c
defines the handler insetCommand
: https://github.com/antirez/redis/blob/6.0.0/src/t_string.c#L97-L147t_string.c
在setCommand
定义了处理程序: https : //github.com/antirez/redis/blob/6.0.0/src/t_string.c#L97-L147t_string.c
defines a more specific handlers after options are parsed where expire values are handled: https://github.com/antirez/redis/blob/6.0.0/src/t_string.c#L71-L79 & https://github.com/antirez/redis/blob/6.0.0/src/t_string.c#L89t_string.c
在解析处理过期值的选项后定义了一个更具体的处理程序: https: //github.com/antirez/redis/blob/6.0.0/src/t_string.c#L71-L79&https :// github.com/antirez/redis/blob/6.0.0/src/t_string.c#L89db.c
defines thesetExpire
function: https://github.com/antirez/redis/blob/6.0.0/src/db.c#L1190-L1206db.c
定义了setExpire
函数: https : //github.com/antirez/redis/blob/6.0.0/src/db.c#L1190-L1206
serverCron
密钥删除 (Key deletion in serverCron
)
server.c
defines the handler forGET
: https://github.com/antirez/redis/blob/6.0.0/src/server.c#L187-L189server.c
定义GET
的处理程序: https : //github.com/antirez/redis/blob/6.0.0/src/server.c#L187-L189t_string.c
defines the handler forgetCommand
: https://github.com/antirez/redis/blob/6.0.0/src/t_string.c#L179-L181 & the generic one: https://github.com/antirez/redis/blob/6.0.0/src/t_string.c#L164-L177t_string.c
定义了getCommand
的处理程序: https : //github.com/antirez/redis/blob/6.0.0/src/t_string.c#L179-L181及通用代码: https : //github.com/antirez /redis/blob/6.0.0/src/t_string.c#L164-L177db.c
defineslookupKeyReadOrReply
: https://github.com/antirez/redis/blob/6.0.0/src/db.c#L163-L167db.c
定义lookupKeyReadOrReply
: https : //github.com/antirez/redis/blob/6.0.0/src/db.c#L163-L167db.c
defineslookupKeyRead
https://github.com/antirez/redis/blob/6.0.0/src/db.c#L143-L147 as well aslookupKeyReadWithFlags
: https://github.com/antirez/redis/blob/6.0.0/src/db.c#L149-L157db.c
定义lookupKeyRead
https://github.com/antirez/redis/blob/6.0.0/src/db.c#L143-L147以及lookupKeyReadWithFlags
: https : //github.com/antirez/redis/blob /6.0.0/src/db.c#L149-L157db.c
definesexpireIfNeeded
: https://github.com/antirez/redis/blob/6.0.0/src/db.c#L1285-L1326db.c
定义expireIfNeeded
: https : //github.com/antirez/redis/blob/6.0.0/src/db.c#L1285-L1326expire.c
definesactiveExpireCycleTryExpire
which implements the deletion of expired keys: https://github.com/antirez/redis/blob/6.0.0/src/expire.c#L35-L74expire.c
定义了activeExpireCycleTryExpire
,它实现了过期密钥的删除: https : //github.com/antirez/redis/blob/6.0.0/src/expire.c#L35-L74expire.c
definesactiveExpireCycle
which implement the sampling of keys and the logic to make sure that there are not too many expired keys in theexpires
dict: https://github.com/redis/redis/blob/6.0.0/src/expire.c#L123expire.c
定义了activeExpireCycle
,它实现了密钥采样以及确保在expires
dict中没有太多过期密钥的逻辑: https : //github.com/redis/redis/blob/6.0.0/src/ c.L123
附录B:使用nc
玩RedisServer (Appendix B: Playing with RedisServer using nc
)
If you want to manually interact with the server, an easy way is to use nc
, the same way we used in Chapter 1. nc
has no awareness of the Redis command syntax, so it will not stop you from making typos:
如果要手动与服务器交互,一种简单的方法是使用nc
,与第1章中使用的方法相同。 nc
不了解Redis命令的语法,因此不会阻止您输入错误:
❯ nc localhost 2000
GET 1
(nil)
SET 1 2
OK
GET 1
2
SET 1 2 EX 5
OK
GET 1
2
GET 1
2
GET 1
(nil)
SET 1 2 XX
(nil)
SET 1 2 NX
OK
SET 1 2 XX
OK
DEL 1
(error) ERR unknown command `DEL`, with args beginning with: `1`,
SET 1 2 PX 100
OK
SET 1 2 XX
(nil)
Originally published at https://redis.pjam.me on July 23, 2020.
最初于 2020年7月23日 发布于 https://redis.pjam.me 。
翻译自: https://medium.com/swlh/chapter-4-adding-the-missing-options-to-the-set-command-2696bc6b62ff
缺少set关键字