Feature #17342

Let puppet know about python package dependencies

Added by zen 2019-12-13 21:12:34 . Updated 2019-12-16 18:36:15 .

Status:
New
Priority:
Normal
Assignee:
Sysadmins
Category:
Infrastructure
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

We’re currently installing and enforcing versions of pip packages using puppet. If we want stricter control over the packages installed, we’d have to express the whole dependency chain and make sure that nothing else than the explicitelly expressed would be installed (i am not sure what’s the current situation regarding that).

@hefee, maybe we can adapt your scripts to generate the Puppet code for this? I think it’s unfeasible to maintain this in the long run without some automation.

Does someone have more background on the decision of installing each of the Python modules from their upstreams’ repositories instead of installing from the pip repository (u, intrigeri, @hefee)? And about the current measures taken to avoid installation of dependencies from pip repo during a package installation?


Subtasks


History

#1 Updated by zen 2019-12-13 21:14:06

hefee, u, @intrigeri — ping!

#2 Updated by intrigeri 2019-12-14 08:39:50

Hi,

> Does someone have more background on the decision of installing each of the Python modules from their upstreams’ repositories instead of installing from the pip repository?

FWIW, this approach was introduced in https://git.tails.boum.org/puppet-tails/commit/?id=a8c5fb3d4244e6b18808cfe8ddd439736af24c57.

I don’t remember participating in making this decision and I’ve no idea if the current implementation was chosen for security reasons or anything else.
I don’t know what are the pros & cons of installing from the pip repo vs. from (mostly GitHub) upstream repos.
Maybe @groente would remember better?

> And about the current measures taken to avoid installation of dependencies from pip repo during a package installation?

I’m mostly clueless about pip and I don’t know if it automatically installs dependencies.
If it does, I don’t think we have any particular counter-measures in place.

#3 Updated by hefee 2019-12-16 18:27:14

intrigeri wrote:
> I don’t remember participating in making this decision and I’ve no idea if the current implementation was chosen for security reasons or anything else.
> I don’t know what are the pros & cons of installing from the pip repo vs. from (mostly GitHub) upstream repos.
> Maybe groente would remember better?

As it is unfeasible for the moment to have everything shipped within Debian. So we need to rely on pip packages. But pip packages by default are binary built packages without any signatures. That’s why we use the possibility to built our own packages via pip from the upstream repositories. We can even improve the situation, when we try to force upstream repos to sign their tags.

> > And about the current measures taken to avoid installation of dependencies from pip repo during a package installation?

We can tell pip to not resolve dependencies with: `pip install —no-deps`.

#4 Updated by hefee 2019-12-16 18:36:15

> We’re currently installing and enforcing versions of pip packages using puppet. If we want stricter control over the packages installed, we’d have to express the whole dependency chain and make sure that nothing else than the explicitelly expressed would be installed (i am not sure what’s the current situation regarding that).

The current situation is that everything, that is installed via puppet are first level dependencies of Weblate itself. With 3.5 upgrade we first hit a second level dependencies. Hopefully we have luck and don’t end up in a deep nested tree of dependencies. And we are fine with the first level and sometimes need to add a second level.

> hefee, maybe we can adapt your scripts to generate the Puppet code for this? I think it’s unfeasible to maintain this in the long run without some automation.

I think you are right, we should automate this. The question that come to my mind, is that is the preferred way of automation. We could automate this, so that a script looks into the installed Weblate source and installs the missing dependencies or do we want to keep a file in puppet with the dependencies?