django中运行celery定时任务worker报Received unregistered task of type ‘tasks.send_sms‘

1. 问题现场

1.1 项目目录结构

项目背景：我的项目对异步没有需求，但是有很多工作依赖定时任务。

• main.py

import os
from celery import Celery

app = Celery('devops')

# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'devops.settings')
app.config_from_object('logic_celery.config')
app.autodiscover_tasks(packages=['logic_celery.sms'])

• config.py

from datetime import timedelta

broker_url = 'redis://127.0.0.1:6379/0'
result_backend = 'redis://127.0.0.1:6379/1'
beat_schedule = {
    'send-msg-per-5s': {
        'task': 'tasks.send_sms',
        'schedule': timedelta(seconds=5),
        'args': (2911, ),
    }
}

注意上面两个脚本里暗藏着两个坑。下面会讲到。

• tasks.py

from logic_celery.main import app
import time
import logging
log = logging.getLogger("django")


@app.task  # name表示设置任务的名称，如果不填写，则默认使用函数名做为任务名
def send_sms(num):
    """发送短信"""
    print("向手机号%s发送短信成功!"%num)
    time.sleep(5)

    return "send_sms OK"

1.2 启动celery beat

PS C:\work\com\code\data-analysis\devops> celery -A logic_celery.main beat -l info

输出：

celery beat v5.1.2 (sun-harmonics) is starting.
__    -    ... __   -        _
LocalTime -> 2022-05-16 14:48:14
Configuration ->
    . broker -> redis://127.0.0.1:6379/0
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> celerybeat-schedule
    . maxinterval -> 5.00 minutes (300s)
[2022-05-16 14:48:14,141: INFO/MainProcess] beat: Starting...
[2022-05-16 14:48:14,224: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
[2022-05-16 14:48:19,227: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
[2022-05-16 14:48:24,235: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
[2022-05-16 14:48:29,244: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)
[2022-05-16 14:48:34,252: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)

特别注意beat Scheduler输出：Scheduler: Sending due task send-msg-per-5s (tasks.send_sms)，这个是解决的关键之处。

1.3 启动celery worker

因为我是在windows下开发的，所以worker启动命令要加上-P eventlet参数

PS C:\work\com\code\data-analysis\devops> celery -A logic_celery.main worker -l info -P eventlet

输出：

-- ******* ---- Windows-10-10.0.19041-SP0 2022-05-16 14:49:18
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         devops:0x2a5f619f2b0
- ** ---------- .> transport:   redis://127.0.0.1:6379/0
- ** ---------- .> results:     redis://127.0.0.1:6379/1
- *** --- * --- .> concurrency: 4 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery



[tasks]
  . logic_celery.sms.tasks.send_sms
  . logic_celery.sms.tasks.send_sms2

[2022-05-16 14:49:18,244: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0
[2022-05-16 14:49:18,254: INFO/MainProcess] mingle: searching for neighbors
[2022-05-16 14:49:19,285: INFO/MainProcess] mingle: all alone
[2022-05-16 14:49:19,294: INFO/MainProcess] pidbox: Connected to redis://127.0.0.1:6379/0.
[2022-05-16 14:49:19,304: INFO/MainProcess] celery@DESKTOP-cjk ready.
[2022-05-16 14:49:19,525: ERROR/MainProcess] Received unregistered task of type 'tasks.send_sms'.
The message has been ignored and discarded.

Did you remember to import the module containing this task?
Or maybe you're using relative imports?

这里可以看到celery已经发现了我的定时任务，说明我的任务注册是没问题的。

[tasks]
  . logic_celery.sms.tasks.send_sms
  . logic_celery.sms.tasks.send_sms2

但是，为啥后面又会报这个错呢？

[2022-05-16 14:49:19,525: ERROR/MainProcess] Received unregistered task of type 'tasks.send_sms'.

答案就是task名称不匹配。观察beat的输出就能知道，beat发送的任务名称是config.py里配置的beat_schedule的task字段值（tasks.send_sms）；而worker自动发现注册的任务名称是全称（logic_celery.sms.tasks.send_sms）。这两个对不上号，所以worker就报错KeyError，其实worker没错，beat也没错，只是这俩的信息不对口，牛唇不对马嘴。

2. 解决办法

• 修改config.py中beat_schedule的task字段值，写全称。

beat_schedule = {
    'send-msg-per-5s': {
        'task': 'logic_celery.sms.tasks.send_sms',
        'schedule': timedelta(seconds=5),
        'args': (2911, ),
    }
}

• 重启beat和worker

PS C:\work\com\code\data-analysis\devops> celery -A logic_celery.main beat -l info
celery beat v5.1.2 (sun-harmonics) is starting.
__    -    ... __   -        _
LocalTime -> 2022-05-16 14:51:00
Configuration ->
    . broker -> redis://127.0.0.1:6379/0
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> celerybeat-schedule
    . logfile -> [stderr]@%INFO
    . maxinterval -> 5.00 minutes (300s)
[2022-05-16 14:51:46,035: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (logic_celery.sms.tasks.send_sms)
[2022-05-16 14:51:51,041: INFO/MainProcess] Scheduler: Sending due task send-msg-per-5s (logic_celery.sms.tasks.send_sms)

PS C:\work\com\code\data-analysis\devops> celery -A logic_celery.main worker -l info -P eventlet
--- ***** -----
-- ******* ---- Windows-10-10.0.19041-SP0 2022-05-16 14:49:18
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         devops:0x2a5f619f2b0
- ** ---------- .> transport:   redis://127.0.0.1:6379/0
- ** ---------- .> results:     redis://127.0.0.1:6379/1
- *** --- * --- .> concurrency: 4 (eventlet)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery
[tasks]
  . logic_celery.sms.tasks.send_sms
  . logic_celery.sms.tasks.send_sms2

[2022-05-16 14:49:18,244: INFO/MainProcess] Connected to redis://127.0.0.1:6379/0
[2022-05-16 14:49:18,254: INFO/MainProcess] mingle: searching for neighbors
[2022-05-16 14:49:19,285: INFO/MainProcess] mingle: all alone
[2022-05-16 14:49:19,294: INFO/MainProcess] pidbox: Connected to redis://127.0.0.1:6379/0.
[2022-05-16 14:49:19,304: INFO/MainProcess] celery@DESKTOP-Jaquan ready.
[2022-05-16 14:51:01,075: INFO/MainProcess] Task logic_celery.sms.tasks.send_sms[39970550-eb78-49a6-a9db-4bf99bfd90ff] received
[2022-05-16 14:51:01,075: WARNING/MainProcess] 向手机号2911发送短信成功!
[2022-05-16 14:51:01,075: WARNING/MainProcess]

[2022-05-16 14:51:05,997: INFO/MainProcess] Task logic_celery.sms.tasks.send_sms[707fc868-d472-49f6-b5ea-0693c0fc37b0] received
[2022-05-16 14:51:05,997: WARNING/MainProcess] 向手机号2911发送短信成功!
[2022-05-16 14:51:05,997: WARNING/MainProcess]

[2022-05-16 14:51:06,078: INFO/MainProcess] Task logic_celery.sms.tasks.send_sms[39970550-eb78-49a6-a9db-4bf99bfd90ff] succeeded in 5.0s: 'send_sms OK'
[2022-05-16 14:51:11,003: INFO/MainProcess] Task logic_celery.sms.tasks.send_sms[707fc868-d472-49f6-b5ea-0693c0fc37b0] succeeded in 5.0s: 'send_sms OK'

完美解决！