不知你有没有遇到APScheduler定时任务部分未执行的情况,我反正是遇到了,任务随机性的不执行,一看官网文档,才知道是自己的使用方法不对。
当前网上的大多数文章都只说了基本的使用方法,没说注意事项。我在这里整理一下:
原文:
from apscheduler.schedulers.background import BackgroundScheduler
scheduler = BackgroundScheduler()
# This will get you a BackgroundScheduler with a MemoryJobStore named “default”
# and a ThreadPoolExecutor named “default” with a default maximum thread count of 10.
如果任务是CPU密集型,可以修改默认的Executor为ProcessPoolExecutor:
executors = {
'default': ProcessPoolExecutor(5)
}
scheduler = BackgroundScheduler(executors=executors)
原因1:Executor如果是默认的10个线程池,恰好10个线程都在忙,恰好又有一个任务A该执行了,由于没有空闲线程来处理,这个任务A将被抛弃。
原文:
If the execution of a job is delayed due to no threads or processes being available in the pool, the executor may skip it due to it being run too late (compared to its originally designated run time)
原因2:如果有个任务B的执行时间是00:02,碰巧你的服务在00:00的时候宕机/重启了,等服务恢复的时候,已经过了00:02的执行时间了,那么任务B也会被抛弃。
原文:
Sometimes the scheduler may be unable to execute a scheduled job at the time it was scheduled to run. The most common case is when a job is scheduled in a persistent job store and the scheduler is shut down and restarted after the job was supposed to execute.
job_defaults = {
'coalesce': True,
'misfire_grace_time': None
}
scheduler = BackgroundScheduler(job_defaults=job_defaults) # 全局设置misfire_grace_time
# 或者
scheduler.add_job(……, misfire_grace_time=None, coalesce=True) # 对单个任务设置 misfire_grace_time
每个任务都有一个misfire_grace_time,单位:秒,默认是0秒。意思是 那些错过的任务在有条件执行时(有线程空闲出来/服务已恢复),如果还没有超过misfire_grace_time,就会被再次执行。如果misfire_grace_time=None,就是不论任务错过了多长时间,都会再次执行。
原文:
misfire_grace_time: seconds after the designated runtime that the job is still allowed to be run (or
None
to allow the job to run no matter how late it is)
如果有个任务C是每2分钟执行一次的周期性任务,且设置了较长misfire_grace_time ,结果服务停了10分钟,10分钟后服务恢复,那么错过的5个任务C都会执行。这种情况下,如果你只想执行1个任务C,可以设置coalesce = True。
原文:
When this happens, the job is considered to have “misfired”. The scheduler will then check each missed execution time against the job’s misfire_grace_time option (which can be set on per-job basis or globally in the scheduler) to see if the execution should still be triggered. This can lead into the job being executed several times in succession.
If this behavior is undesirable for your particular use case, it is possible to use coalescing to roll all these missed executions into one. In other words, if coalescing is enabled for the job and the scheduler sees one or more queued executions for the job, it will only trigger it once. No misfire events will be sent for the “bypassed” runs.