Feature #17377
Weblate: Generate list of Python dependencies automatically
0%
Description
For maintaining the Weblate installation, we chose to:
* install Python dependencies from Debian repo as much as possible; and
* pin dependencies’ versions and install from upstream repositories whenever they are not available in Debian. (The reason for installing from repositories is that the pip repo distributes binaries which have no way to be verified.)
Currently, we’re manually maintaining the Puppet code for installing Weblate dependencies either as Debian packages or Python modules, with the aid of a script. That is hard to maintain and error prone, so we should ideally generate the list of dependencies automatically.
We need have a script that, given a Weblate version and a list of extra packages (example: redis is needed for celery which is needed — see Bug #17347), will traverse the whole dependency trees and generate whatever is needed to have a fixed list of Debian packages and Python modules installed — and nothing else.
Subtasks
History
#1 Updated by hefee 2019-12-25 14:12:30
- Affected tool set to Translation Platform
#2 Updated by zen 2019-12-25 14:19:00
- Assignee changed from Sysadmins to hefee
#3 Updated by hefee 2019-12-26 13:53:33
- Status changed from Confirmed to In Progress
- Feature Branch set to https://salsa.debian.org/hefee/puppet-tails/:hefee/update_puppet_dependecy_files
- Type of work changed from Translate to Code
version installed is: `pip3 freeze -l`
the script is: https://salsa.debian.org/hefee/puppet-tails/blob/hefee/update_puppet_dependecy_files/files/weblate/external_scripts/update_puppet_dependecy_files.py
| Dependency | Version installed | script | Status |
|---|---|---|---|
| amqp | 2.5.2 | 2.5.2 # <2.6,>=2.5.2 | ok |
| asgiref | — | 3.2.3 # ~=3.2 | dependency for django |
| billiard | 3.6.1.0 | 3.6.1.0 # <4.0,>=3.6.1 | ok |
| celery | 4.3.0 | 4.4.0 # >=4.0 | ok |
| celery-batches | 0.2 | 0.2 # >=0.2 | ok |
| defusedxml | 0.5.0 | — (installed by Debian package) | ok |
| diff-match-patch | 20121119 | — (installed by Debian package) | ok |
| django | 1.11.27 | 3.0.1 # >=1.11 | not ok |
| django-appconf | 1.0.2 | 1.0.3 # >=1.0 | ok |
| django-compressor | 2.2 | 2.3 # >=2.1.1 | ok |
| django-crispy-forms | 1.7.2 | 1.8.1 # >=1.6.1 | ok |
| djangorestframework | 3.8.2 | 3.11.0 # >=3.8 | ok |
| et-xmlfile | 1.0.1 | — | can be removed |
| filelock | 3.0.4 | 3.0.12 # >=3.0.1 | ok |
| importlib-metadata | 1.3.0 | 1.3.0 # >=0.18 | ok |
| jdcal | 1.4.1 | — | can be removed |
| jellyfish | 0.7.2 | 0.7.2 # >=0.6.1 | ok |
| kombu | 4.6.7 | 4.6.7 # <4.7,>=4.6.7 | ok |
| more-itertools | 8.0.2 | 8.0.2 | ok |
| oauthlib | 3.1.0 | 3.1.0 # >=3.0.0 | ok |
| openpyxl | 2.6.0 | 3.0.2 # >=2.5.0 | ok |
| phply | 1.2.4 | — | can be removed, we don’t handle php files in Weblate |
| ply | 3.11 | — | can be removed |
| python3-openid | 3.1.0 | — | can be removed |
| pytz | 2019.3 | — (installed by Debian package) | ok |
| rcssmin | 1.0.6 | — (installed by Debian package) | ok |
| redis | 3.3.11 | 3.3.11 | ok |
| requests-oauthlib | 1.0.0 | — | can be removed |
| rjsmin | 1.0.12 | 1.1.0 # ==1.1.0 | ok |
| ruamel.yaml | 0.16.5 | — | |
| ruamel.yaml.clib | 0.2.0 | — | |
| siphashc | 1.0 | 1.3 # >=0.8 | ok |
| social-auth-app-django | 3.1.0 | 3.1.0 # >=3.1.0 | ok |
| social-auth-core | 3.1.0 | 3.2.0 # >=3.1.0 | ok |
| translate-toolkit | 2.3.1 | 2.4.0 # >=2.3.1 | ok |
| translation-finder | 1.7 | 1.7 # >=1.0 | ok |
| ua-parser | 0.8.0 | 0.8.0 # >=0.8.0 | ok |
| user-agents | 1.1.0 | 2.0 # >=1.1.0 | ok |
| vine | 1.3.0 | 1.3.0 # ==1.3.0 | ok |
| zipp | 0.6.0 | 0.6.0 # >=0.5 | ok |
#4 Updated by hefee 2019-12-31 14:44:01
- Status changed from In Progress to Needs Validation
- Assignee changed from hefee to zen
The first version of the script is ready for review.
Please check the output of git show -w 408e4db6, that is the output of the script.
Things that need changes:
- I would only list non weblate debian dependencies in debian_dependencies.pp, so this file, won’t needs to be changed by the script and add the weblate debian depdendencies in pip_dendendencies.pp.
- rename {pip,weblate}_dendendencies.pp
- express the pip dependencies as a requirements.txt and run:
pip3 install --no-deps --ignore-installed -r weblate_requirements.txt
#maybe this works
pip3 install --no-deps --upgrade-strategy "eager" -r weblate_requirements.txt#5 Updated by zen 2020-04-29 14:14:07
- Priority changed from Normal to Elevated
Increasing priority, as discussed with Translations team.