Bug #17572

Remove unused packages from our custom APT repository

Added by intrigeri 2020-03-28 11:51:56 . Updated 2020-04-25 05:45:39 .

Status:
Resolved
Priority:
Normal
Assignee:
intrigeri
Category:
Build system
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

I think we could do that:

for dist in testing devel; do
   for pkg in tails-iuk tails-perl5lib tails-persistence-setup virtualbox; do
      ssh reprepro@incoming.deb.tails.boum.org reprepro removesrc "$dist" "$pkg"
   done
   ssh reprepro@incoming.deb.tails.boum.org reprepro remove "$dist" tor tor-geoipdb
done

This should not impact the build of our ISO/IMG.
It will merely clean unused things up and avoid some potential future confusion and misunderstandings.

anonym, kibi: if you agree this makes sense, please run this yourself or ask me to :)


Subtasks


Related issues

Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

History

#1 Updated by CyrilBrulebois 2020-03-28 12:53:32

While I can see where this is coming from for tails-* because I reviewed their removal (either before or after they got merged), I have no clue about virtualbox.

Regarding tor, and tor-geoip, maybe we caught up with upstream with the last bump in 4.4.1, but I’d rather see some context (even if only a link to some other place).

#2 Updated by intrigeri 2020-03-28 14:07:11

> I have no clue about virtualbox.

We now install virtualbox 6.1.4-dfsg-1 from sid, while our custom repo has:

$ ssh reprepro@incoming.deb.tails.boum.org reprepro list testing virtualbox
testing|contrib|amd64: virtualbox 6.0.12-dfsg-1
testing|contrib|source: virtualbox 6.0.12-dfsg-1

> Regarding tor, and tor-geoip, maybe we caught up with upstream with the last bump in 4.4.1, but I’d rather see some context (even if only a link to some other place).

We currently install tor 0.4.x built for Buster, while our custom repo has 0.3.x built for Stretch:

$ ssh reprepro@incoming.deb.tails.boum.org reprepro list testing tor
testing|main|amd64: tor 0.3.4.9-1~d90.stretch+1

#3 Updated by intrigeri 2020-04-06 14:25:22

#4 Updated by anonym 2020-04-06 14:45:50

  • Assignee set to anonym
  • Target version changed from Tails_4.5 to Tails_4.6

#5 Updated by anonym 2020-04-14 14:04:10

  • Status changed from Needs Validation to In Progress

Applied in changeset commit:tails|ab3402db9e0e9eb1b3ded34841b2fb8f85341ac5.

#6 Updated by anonym 2020-04-14 15:04:41

So I ran the above suggested cleanup command, although I also added the stable suite. Actually, stable == testing == devel for their APT suites, and they were all modified like this:

@@ -88,19 +88,6 @@
 amd64: squashfs-tools-dbg 1:4.3-3.0tails4
 amd64: squashfs-tools-dbgsym 1:4.3-8.0tails1
 amd64: tails-installer 5.0.16+dfsg-0tails1
-amd64: tails-iuk 4.0.3-1
-amd64: tails-perl5lib 2.0.3-1
-amd64: tails-persistence-setup 2.2.1-1
-amd64: tor 0.3.4.9-1~d90.stretch+1
-amd64: tor-geoipdb 0.3.4.9-1~d90.stretch+1
-amd64: virtualbox 6.0.12-dfsg-1
-amd64: virtualbox-dkms 6.0.12-dfsg-1
-amd64: virtualbox-guest-dkms 6.0.12-dfsg-1
-amd64: virtualbox-guest-source 6.0.12-dfsg-1
-amd64: virtualbox-guest-utils 6.0.12-dfsg-1
-amd64: virtualbox-guest-x11 6.0.12-dfsg-1
-amd64: virtualbox-qt 6.0.12-dfsg-1
-amd64: virtualbox-source 6.0.12-dfsg-1
 amd64: whisperback 1.8.3
 source: desktop-file-utils 0.23-4.0tails1
 source: fontconfig 2.13.1-2.0tails1
@@ -115,8 +102,4 @@
 source: po4a 0.47-2
 source: squashfs-tools 1:4.3-8.0tails1
 source: tails-installer 5.0.16+dfsg-0tails1
-source: tails-iuk 4.0.3-1
-source: tails-perl5lib 2.0.3-1
-source: tails-persistence-setup 2.2.1-1
-source: virtualbox 6.0.12-dfsg-1
 source: whisperback 1.8.3

LGTM!

Also, I made a backup of the stable APT suite in stable-apt-suite-backup so we can restore if there is a fuckup, and easily compare to how it was before.

Also, I think there are more packages to remove… to be continued!

#7 Updated by anonym 2020-04-14 17:34:53

  • Status changed from In Progress to Needs Validation

In my quest for more packages to delete, I devised this shitty script to my aid:

ssh reprepro@incoming.deb.tails.boum.org reprepro list stable | \
  grep -Ev -- "-dbg(sym)?\b" | \
  while read x p v; do
    if ! grep --silent -oPz "package: ${p}\n\s*version: $(echo ${v} | sed -e 's/+/\\+/g' -e 's/\./\\./g')\n" tails-amd64-4.5.build-manifest; then
      echo ${p} ${v}
    fi
  done

It will list all package-version pairs from the APT suite that are not listed in the build manifest, i.e. candidates for removal. For me the output is:

gir1.2-networkmanager-1.0 1.6.2-3+deb9u2+0.tails1
gjs-tests 1.54.3-1.0tails1
gtk-3-examples 3.24.5-1.0tails1
gvfs-bin 1.38.1-5.0tails1
gvfs-fuse 1.38.1-5.0tails1
ibus-doc 1.5.19-4+deb10u1.0tails1
ibus-wayland 1.5.19-4+deb10u1.0tails1
libfontconfig1-dev 2.13.1-2.0tails1
libgail-3-0 3.24.5-1.0tails1
libgail-3-dev 3.24.5-1.0tails1
libgail-3-doc 3.24.5-1.0tails1
libgjs-dev 1.54.3-1.0tails1
libgjs0e 1.46.0-1.0tails2
libgtk-3-bin 3.24.5-1.0tails1
libgtk-3-dev 3.24.5-1.0tails1
libgtk-3-doc 3.24.5-1.0tails1
libibus-1.0-dev 1.5.19-4+deb10u1.0tails1
libnm-dev 1.14.6-2.0tails1
libnm-glib-dev 1.6.2-3+deb9u2+0.tails1
libnm-glib-vpn-dev 1.6.2-3+deb9u2+0.tails1
libnm-glib-vpn1 1.6.2-3+deb9u2+0.tails1
libnm-glib4 1.6.2-3+deb9u2+0.tails1
libnm-util-dev 1.6.2-3+deb9u2+0.tails1
libnm-util2 1.6.2-3+deb9u2+0.tails1
live-boot-doc 1:20170112
network-manager-config-connectivity-debian 1.14.6-2.0tails1
network-manager-dev 1.14.6-2.0tails1
po4a 0.47-2
squashfs-tools 1:4.3-8.0tails1
gtk+3.0 3.24.5-1.0tails1
po4a 0.47-2
squashfs-tools 1:4.3-8.0tails1

These are mostly false positives, e.g. sure we don’t install libibus-1.0-dev, but it’s still ok since it was part of the upload of ibus, which we do install. However, at least these three candidates look like packages we really do not use at all and should remove:

  • gir1.2-networkmanager-1.0 1.6.2-3+deb9u2+0.tails1 ← seems like a leftover from the stretch days. My guess is that since this package doesn’t exist in buster, it was silently kept while the other packages were replaced when uploading the patched buster version to our APT suite.
  • po4a 0.47-2 ← not used during build at all, but perhaps we wanted to make this version easily available for our translators that do their work inside Tails? If so, it can still be removed as part of Bug #17005, so yay!
  • squashfs-tools 1:4.3-8.0tails1 (+ corresponding squashfs-tools-*) ← seems like a leftover since we started installing it from Sid in Bug #16637.

I deleted those, and the final diff for our APT suites is:

 amd64: fontconfig-dbgsym 2.13.1-2.0tails1
 amd64: gir1.2-gtk-3.0 3.24.5-1.0tails1
 amd64: gir1.2-ibus-1.0 1.5.19-4+deb10u1.0tails1
-amd64: gir1.2-networkmanager-1.0 1.6.2-3+deb9u2+0.tails1
 amd64: gir1.2-nm-1.0 1.14.6-2.0tails1
 amd64: gjs 1.54.3-1.0tails1
 amd64: gjs-dbgsym 1.54.3-1.0tails1
@@ -83,24 +82,7 @@
 amd64: network-manager-dbgsym 1.14.6-2.0tails1
 amd64: network-manager-dev 1.14.6-2.0tails1
 amd64: onioncircuits 0.6-0.0tails1
-amd64: po4a 0.47-2
-amd64: squashfs-tools 1:4.3-8.0tails1
-amd64: squashfs-tools-dbg 1:4.3-3.0tails4
-amd64: squashfs-tools-dbgsym 1:4.3-8.0tails1
 amd64: tails-installer 5.0.16+dfsg-0tails1
-amd64: tails-iuk 4.0.3-1
-amd64: tails-perl5lib 2.0.3-1
-amd64: tails-persistence-setup 2.2.1-1
-amd64: tor 0.3.4.9-1~d90.stretch+1
-amd64: tor-geoipdb 0.3.4.9-1~d90.stretch+1
-amd64: virtualbox 6.0.12-dfsg-1
-amd64: virtualbox-dkms 6.0.12-dfsg-1
-amd64: virtualbox-guest-dkms 6.0.12-dfsg-1
-amd64: virtualbox-guest-source 6.0.12-dfsg-1
-amd64: virtualbox-guest-utils 6.0.12-dfsg-1
-amd64: virtualbox-guest-x11 6.0.12-dfsg-1
-amd64: virtualbox-qt 6.0.12-dfsg-1
-amd64: virtualbox-source 6.0.12-dfsg-1
 amd64: whisperback 1.8.3
 source: desktop-file-utils 0.23-4.0tails1
 source: fontconfig 2.13.1-2.0tails1
@@ -112,11 +94,5 @@
 source: live-boot 1:20170112
 source: network-manager 1.14.6-2.0tails1
 source: onioncircuits 0.6-0.0tails1
-source: po4a 0.47-2
-source: squashfs-tools 1:4.3-8.0tails1
 source: tails-installer 5.0.16+dfsg-0tails1
-source: tails-iuk 4.0.3-1
-source: tails-perl5lib 2.0.3-1
-source: tails-persistence-setup 2.2.1-1
-source: virtualbox 6.0.12-dfsg-1
 source: whisperback 1.8.3

Looks legit! I’ll make sure Tails still builds and get identical .packages and .build-manifest to before, otherwise I’ll reset the suites to the backed up state.

#8 Updated by anonym 2020-04-14 18:29:49

anonym wrote:
> * po4a 0.47-2 ← not used during build at all, but perhaps we wanted to make this version easily available for our translators that do their work inside Tails? If so, it can still be removed as part of Bug #17005, so yay!

Whoops, seems I forgot about the uncertainty in this one, so I restored it to the APT suites.

#9 Updated by anonym 2020-04-14 18:49:10

  • Assignee changed from anonym to intrigeri

anonym wrote:
> I’ll make sure Tails still builds and get identical .packages and .build-manifest to before, otherwise I’ll reset the suites to the backed up state.

Stable’s run #3238, whose built was started after my modification to stable’s APT suite, has identical .build-manifest (modulo Debian security snapshot) and .packages to the earlier builds, so this look good!

So, intrigeri, once we remove po4a (see my comment on Bug #17005) then we can conclude this ticket (and remove the stable_apt_suite_backup as a cleanup), right?

(I also realized that the 4.5 (tagged) APT suite is identical to stable-apt-suite-backup so we even have a long-term backup.)

#10 Updated by anonym 2020-04-15 08:50:22

Removed po4a from the base branches’ APT suites again after getting clarifications on Bug #17005.

#11 Updated by intrigeri 2020-04-15 10:44:27

  • Status changed from Needs Validation to Resolved

LGTM!

#12 Updated by intrigeri 2020-04-25 05:45:39

anonym wrote:
> So, intrigeri, once we remove po4a (see my comment on Bug #17005) then we can conclude this ticket (and remove the stable_apt_suite_backup as a cleanup), right?

I’ve now deleted the stable_apt_suite_backup branch.