Tails

#1 Updated by Dr_Whax 2014-04-04 00:03:55

The Debile link is not working, this one is: http://anonscm.debian.org/gitweb/?p=pkg-debile/debile.git

#11 Updated by intrigeri 2017-08-11 16:32:09

At this point I’m happy we did not set this up server-side, in a centralized manner, because I’ve entirely changed my mind wrt. how we should approach the problem at hand.

IMO we should build our custom packages as part of building an ISO (using good ol’ sbuild/pbuilder or something that better isolates package builds from one another, such as https://gitlab.com/uhoreg/whalebuilder or one of the other options, and track their source code as Git submodules i.e. Feature #7036 (for non-native packages, likely we want to track the branch that contains the Debian packaging).

This advantages I see with this approach, compared to what we currently do, and to extending our central infrastructure to build packages automatically:

1. We get more value out of reproducible ISO builds, by removing binary input and the required trust in the content of these packages, in the developer who built+uploaded them, and in our custom APT repo infrastructure.
2. Developers who don’t have Git commit access can work on our custom software and build an ISO with their modified version in a straightforward way (no need to have a pbuilder setup nor to copy .deb’s to config/chroot_local-packages, and the result of the work can be handled as any other Git pull request).
3. No need for all developers to set up and maintain a local pbuilder setup with N chroots.
4. We can get rid of our custom APT repository and, perhaps more importantly, of the complex & error-prone workflow that comes with it (e.g. for merging/resetting APT suites). Everything related to config/APT_overlays.d/ can go away, which simplifies our codebase and infrastructure.
5. More parameters that affect the resulting ISO are encoded in Git, which makes them easier to audit, log, and diff. The only remaining thing that’s not encoded in Git is the content of the APT snapshots we use (we already encode in Git which set of APT snapshots we use).
6. We simplify what it means to have Git commit access: developers who have Git commit access don’t also need upload rights to our custom APT repo. This simplifies sysadmin (no need to keep OpenPGP keys up-to-date) and makes it easier for developers to understand how the whole thing works.
7. The apt-cacher-ng cache can be shared between ISO build process and .deb packages build process.
8. We get finer-grained control over what exact state of the pbuilder/sbuild/whalebuilder chroot we use for building packages. E.g. we might want to use exactly the same APT snapshots that are used for building the rest of the ISO, which can matter a lot if we switch to Debian testing (Feature #12615): building against current Debian testing can create packages that are not installable on the (older) snapshot of Debian testing we use in the SquashFS.
9. More of us have the skills and access needed to work, within our Git tree, on such a Debian packages build system, than to work on a centralized package build system hosted our infrastructure. This implies that we have more options wrt. who implements, reviews & maintains the new thing, instead of increasing our reliance on a tiny sysadmin team that would use a totally different stack (Puppet, Jenkins, jenkins-debian-glue, etc.).
10. Compared to what we do today, we have the option to enforce some quality checks on the packaging and upstream code (e.g. Lintian or autopkgtests).
11. No need to design & implement yet another mapping between Git branches, Jenkins jobs and APT suites.
12. No race conditions / async design skills required: we’re guaranteed that the expected version of every custom package we want has been built when we need it.

#13 Updated by intrigeri 2017-10-02 10:53:59

See also https://tails.boum.org/blueprint/reproducible_builds/#custom-Debian-packages.

#15 Updated by Anonymous 2018-08-19 08:46:49

Inspiration: https://auto.debian.net/

#19 Updated by intrigeri 2019-09-19 06:24:55

It might be that once we have Feature #7036, there’s very few reasons left (if any) to bother doing what this very ticket is about.