Bug #17347
[Weblate] get Celery working
0%
Description
/usr/local/weblate$ python3 manage.py check --deploy
SystemCheckError: System check identified some issues:
CRITICALS:
?: (weblate.E020) The Celery is not configured to store results, CELERY_RESULT_BACKEND is probably not set.
HINT: https://docs.weblate.org/en/weblate-3.5.1/admin/install.html#celeryWeblate itself tells us that something is not correct with Celery ;D
At least we don’t seen anything committed the last 24h, where stuff was modified! That’s why we expect that celery is not working.
But also the knowledge how to get statistics or information out of celery is missing, to check if the others tasks are done by celery.
It seems that we need to configure a CELERY_RESULT_BACKEND - Celery upstream suggests to use a proper broker
At least for me I don’t know what a broker is or why we need a result_backend. So either we find someone who can give us this information or we need to dig into that ourselves.
Subtasks
Related issues
|
Related to Tails - |
Resolved | ||
| Related to Tails - Bug #17340: Weblate: Check Celery settings for sane values | Confirmed |
History
#1 Updated by hefee 2019-12-13 22:33:07
- Priority changed from Normal to Elevated
- Target version set to Tails_4.2
#2 Updated by drebs 2019-12-23 18:37:24
Hi, my understanding is that there are 2 different configs, and the one currently preventing celery to work is CELERY_BROKER_URL rather than CELERY_RESULT_BACKEND.
Regarding the configuration of a broker:
* Celery is a distributed task queue.
* Tasks are messages, so it needs something called a message broker to transport tasks from producers to consumers.
* In the context of celery brokers can be either rabbitmq, redis or amazonsqs (there are also unmaintained experimental brokers, but i would probably rule them out unless we have more info on longer term maintainability).
* We are currently using an in-memory broker, which is only good for testing because memory of different celery worker processes is not shared.
* If we want to use celery, we need to install either redis or rabbitmq to use as a broker, and change this config appropriatelly.
Regarding the results backend, my understanding is:
* It’s possible to keep track of tasks states by using different backends, and that is optional.
* Backends that would make sense for us right now are database, filesystem or in-memory cache because our setup shouldn’t need too many resources and scalability and we wouldn’t have to install anything. But Redis can also be used as a backend and we might want to install it to get celery working any way.
* We were mislead because our version of Weblate (3.5.1) only checks for the less important variable (CELERY_RESULT_BACKEND) while the latest checks for the more important variable (CELERY_BROKER_URL)
So my conclusion is that we need to either install something like Redis to use as a broker or give up celery completelly. I don’t currently have enough info to give a strong opinion on what’s best, but I have a feeling that if Weblate is opting to use Celery as it’s scheduling mechanism, then it might be easier to just fulfil this requirement than maintaining our own sets of cron jobs in the longer term.
Anything against using Redis as a message broker to make celery work?
#3 Updated by zen 2019-12-23 19:13:19
- related to
Bug #17370: [Weblate] Get registration working added
#4 Updated by hefee 2019-12-23 21:36:19
- Assignee set to drebs
drebs wrote:
Thanks for digging into Celery.
> So my conclusion is that we need to either install something like Redis to use as a broker or give up celery completelly. I don’t currently have enough info to give a strong opinion on what’s best, but I have a feeling that if Weblate is opting to use Celery as it’s scheduling mechanism, then it might be easier to just fulfill this requirement than maintaining our own sets of cron jobs in the longer term.
IMO we don’t have the option to do all the background tasks by cronjobs, as we than would need to invest a lot of time to extract the information by Weblate, what is sent to Celery.
> Anything against using Redis as a message broker to make celery work?
I think Redis is good solution, so let’s go this way.
#5 Updated by drebs 2019-12-25 22:29:21
- Status changed from Confirmed to Resolved
Redis is now running and celery is working fine: https://git.tails.boum.org/puppet-tails/tree/manifests/weblate/redis.pp?id=580375074ace89fd8b367c0df1b8f2e8a1c26659
#6 Updated by zen 2020-01-17 15:39:24
- related to Bug #17340: Weblate: Check Celery settings for sane values added