錯誤提示一：
[2019-02-16 15:47:43,063: ERROR/MainProcess] Unrecoverable error: AttributeError("'str' object has no attribute 'items'",)
Traceback (most recent call last):
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/celery/worker/__init__.py", line 206, in start
self.blueprint.start(self)
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/celery/bootsteps.py", line 123, in start
step.start(parent)
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/celery/bootsteps.py", line 374, in start
return self.obj.start()
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/celery/worker/consumer.py", line 280, in start
blueprint.start(self)
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/celery/bootsteps.py", line 123, in start
step.start(parent)
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/celery/worker/consumer.py", line 884, in start
c.loop(*c.loop_args())
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/celery/worker/loops.py", line 76, in asynloop
next(loop)
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/kombu/async/hub.py", line 340, in create_loop
cb(*cbargs)
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/kombu/transport/redis.py", line 1019, in on_readable
self._callbacks[queue](message)
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/kombu/transport/virtual/__init__.py", line 534, in _callback
self.qos.append(message, message.delivery_tag)
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/kombu/transport/redis.py", line 146, in append
pipe.zadd(self.unacked_index_key, delivery_tag, time()) \
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/redis/client.py", line 2320, in zadd
for pair in iteritems(mapping):
File "/home/damon96/.virtualenvs/oakvipENV/lib/python3.6/site-packages/redis/_compat.py", line 122, in iteritems
return iter(x.items())
AttributeError: 'str' object has no attribute 'items'
原因：項目使用的第三方redis包版本過高
解決方式：降低redis版本，pip install redis==2.10.6
錯誤提示二：
Running a worker with superuser privileges when the
worker accepts messages serialized with pickle is a very bad idea!
If you really want to continue then you have to set the C_FORCE_ROOT
environment variable (but please think about this before you do).
原因：celery需要使用root用戶執(zhí)行
解決方式：
from celery import Celery, platforms
app = Celery('tasks')
platforms.C_FORCE_ROOT = True #加上這一行
@app.task
def add(x, y):
return x + y
錯誤提示三：
Traceback (most recent call last):
File "/Users/aromanovich/Envs/celery3.1/lib/python2.7/site-packages/celery/app/trace.py", line 218, in trace_task
R = retval = fun(*args, **kwargs)
File "/Users/aromanovich/Envs/celery3.1/lib/python2.7/site-packages/celery/app/trace.py", line 398, in __protected_call__
return self.run(*args, **kwargs)
File "/Users/aromanovich/Projects/celery/app.py", line 10, in add
manager = multiprocessing.Manager()
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/__init__.py", line 99, in Manager
m.start()
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/managers.py", line 524, in start
self._process.start()
File "/usr/local/Cellar/python/2.7.6/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/process.py", line 124, in start
'daemonic processes are not allowed to have children'
原因：
解決方式：設置環(huán)境變量 export PYTHONOPTIMIZE=1
錯誤提示四
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "D:\Python27\lib\site-packages\django\core\management\__init__.py", line 354, in execute_from_command_line
utility.execute()
File "D:\Python27\lib\site-packages\django\core\management\__init__.py", line 346, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "D:\Python27\lib\site-packages\django\core\management\__init__.py", line 190, in fetch_command
klass = load_command_class(app_name, subcommand)
File "D:\Python27\lib\site-packages\django\core\management\__init__.py", line 40, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "D:\Python27\lib\importlib\__init__.py", line 37, in import_module
__import__(name)
File "D:\Python27\lib\site-packages\djcelery\management\commands\celery.py", line 11, in <module>
class Command(CeleryCommand):
File "D:\Python27\lib\site-packages\djcelery\management\commands\celery.py", line 15, in Command
base.get_options() +
TypeError: can only concatenate tuple (not "NoneType") to tuple
原因：這個問題是由于安裝包不全導致的，分別檢查是否安裝了以下安裝包
解決方式：
pip install celery
pip install celery-with-redis
pip install django-celery
錯誤提示五：
Traceback (most recent call last):
File "D:\Python27\lib\site-packages\celery\app\trace.py", line 240, in trace_task
R = retval = fun(*args, **kwargs)
File "D:\Python27\lib\site-packages\celery\app\trace.py", line 438, in __protected_call__
return self.run(*args, **kwargs)
File "F:\virtual\project\agrometeorological\agrometeorological\ganter\tasks.py", line 34, in get_access
g.account_set.all()
File "D:\Python27\lib\site-packages\django\db\models\manager.py", line 228, in all
return self.get_queryset()
File "D:\Python27\lib\site-packages\django\db\models\fields\related.py", line 712, in get_queryset
qs = qs.filter(**self.core_filters)
File "D:\Python27\lib\site-packages\django\db\models\query.py", line 679, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "D:\Python27\lib\site-packages\django\db\models\query.py", line 697, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "D:\Python27\lib\site-packages\django\db\models\sql\query.py", line 1310, in add_q
clause, require_inner = self._add_q(where_part, self.used_aliases)
File "D:\Python27\lib\site-packages\django\db\models\sql\query.py", line 1338, in _add_q
allow_joins=allow_joins, split_subq=split_subq,
File "D:\Python27\lib\site-packages\django\db\models\sql\query.py", line 1150, in build_filter
lookups, parts, reffed_expression = self.solve_lookup_type(arg)
File "D:\Python27\lib\site-packages\django\db\models\sql\query.py", line 1036, in solve_lookup_type
_, field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta())
File "D:\Python27\lib\site-packages\django\db\models\sql\query.py", line 1373, in names_to_path
if field.is_relation and not field.related_model:
File "D:\Python27\lib\site-packages\django\utils\functional.py", line 59, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "D:\Python27\lib\site-packages\django\db\models\fields\related.py", line 110, in related_model
apps.check_models_ready()
File "D:\Python27\lib\site-packages\django\apps\registry.py", line 131, in check_models_ready
raise AppRegistryNotReady("Models aren't loaded yet.")
AppRegistryNotReady: Models aren't loaded yet.
原因：我是在定時任務中使用外鍵關聯的反向查詢時報了上面的錯誤
解決方式：
# 模型加載不全，解決方式是在定時任務tasks文件的包引用最前面引用，如下：
# -*- coding:utf-8 -*-
import django
django.setup()
from celery.schedules import crontab_parser
from celery import shared_task
錯誤提示六
ERROR/MainProcess] Received unregistered task of type 'generate_static_sku_detail_html'.
The message has been ignored and discarded.
Did you remember to import the module containing this task?
Or maybe you're using relative imports?
Please see
http://docs.celeryq.org/en/latest/internals/protocol.html
for more information.
The full contents of the message body was:
b'[[2], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (78b)
Traceback (most recent call last):
File "/home/python/.virtualenvs/meiduo/lib/python3.6/site-packages/celery/worker/consumer/consumer.py", line 558, in on_task_received
strategy = strategies[type_]
KeyError: 'generate_static_sku_detail_html'
原因：項目未開啟
解決方式：先開啟項目，在開啟celery
技巧一：任務重復執(zhí)行
celery執(zhí)行定時任務的時候遇到了重復執(zhí)行的問題，當時是用redis做broker和backend。
官方文檔中有相關描述。
描述：就是說當我們設置一個ETA時間比visibility_timeout長的任務時，每過一次 visibility_timeout 時間，celery就會認為這個任務沒被worker執(zhí)行成功，重新分配給其它worker再執(zhí)行。
解決辦法就是把 visibility_timeout參數調大，比我們ETA的時間差要大。celery本身的定位就主要是實時的異步隊列，對于這種長時間定時執(zhí)行，支持不太好。
但是第二天依然重復執(zhí)行了。。。
最后我的解決方法是在每次定時任務執(zhí)行完就在redis中寫入一個唯一的key對應一個時間戳，當下次任務執(zhí)行前去獲取redis中的這個key對應的value值，和當前的時間做比較，當滿足我們的定時頻率要求時才執(zhí)行，這樣保證了同一個任務在規(guī)定的時間內只會執(zhí)行一次。
技巧二：使用多個queue來執(zhí)行任務
當你有很多任務需要執(zhí)行的時候，不要偷懶只使用默認的queue，這樣會相互影響，并且拖慢任務執(zhí)行的，導致重要的任務不能被快速的執(zhí)行。雞蛋不能放在同一個籃子里的道理大家都懂。
配置的設置：CELERY_ROUTES = {'feed.tasks.import_feed': {'queue': 'feeds'}}
使用方式：celery -A proj worker -Q feeds,celery
指定routes,就會自動生成對應的queue，然后使用-Q指定queue啟動celery就可以，默認的queue名字是celery?？梢钥?a target="_blank">官方文檔對默認queue的名字進行修改。
技巧三：啟動多個workers執(zhí)行不同的任務
在同一臺機器上，對于優(yōu)先級不同的任務最好啟動不同的worker去執(zhí)行，比如把實時任務和定時任務分開，把執(zhí)行頻率高的任務和執(zhí)行頻率低的任務分開，這樣有利于保證高優(yōu)先級的任務可以得到更多的系統(tǒng)資源，同時高頻率的實時任務日志比較多也會影響實時任務的日志查看，分開就可以記錄到不同的日志文件，方便查看。
$ celery -A proj worker --loglevel=INFO --concurrency=10 -n worker1.%h
$ celery -A proj worker --loglevel=INFO --concurrency=10 -n worker2.%h
$ celery -A proj worker --loglevel=INFO --concurrency=10 -n worker3.%h
可以像這樣啟動不同的worker，%h可以指定hostname，詳細說明可以查看官方文檔
高優(yōu)先級的任務可以分配更多的concurrency，但是并不是worker并法數越多越好，保證任務不堆積就好。
技巧四：確定是否需要關注任務執(zhí)行狀態(tài)
這個要視具體的業(yè)務場景來看，如果對結果不關心，或者任務的執(zhí)行本身會對數據產生影響，通過對數據的判斷可以知道執(zhí)行的結果那就不需要返回celery任務的退出狀態(tài)，可以設置：
進行全局設置CELERY_IGNORE_RESULT = True
或者進行單個任務設置
@app.task(ignore_result=True)
def mytask(…):
something()
但是，如果業(yè)務需要根據任務執(zhí)行的狀態(tài)進行響應的處理就不要這樣設置。
技巧五：防止內存泄漏
長時間運行Celery有可能發(fā)生內存泄露，可以像下面這樣設置
CELERYD_MAX_TASKS_PER_CHILD = 40 # 每個worker執(zhí)行了多少任務就會死掉
技巧六：遇到異常重啟之后將原來的任務重新加入到隊列中
task_reject_on_worker_lost = True
task_acks_late = True
該配置可以保證task不丟失，中斷的task在下次啟動時將會重新執(zhí)行。
task_reject_on_worker_lost作用是當worker進程意外退出時，task會被放回到隊列中
task_acks_late作用是只有當worker完成了這個task時，任務才被標記為ack狀態(tài)
需要說明的是，backend最好使用rabbitmq等支持ACK狀態(tài)的消息中間件。
