Feature #15291

Remove less popular packages that users could install themselves

Added by anonym 2018-02-05 16:32:50 . Updated 2019-05-23 21:20:44 .

Status:
Resolved
Priority:
Normal
Assignee:
CyrilBrulebois
Category:
Target version:
Tails_3.14
Start date:
2018-04-17
Due date:
% Done:

100%

Feature Branch:
feature/15291-remove-software, https://salsa.debian.org/tails-team/tails/merge_requests/10
Type of work:
End-user documentation
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

… with Additional Software Packages version 2!

Subtasks

Related issues

Related to Tails - Feature #15281: Stack one single SquashFS diff when upgrading Resolved 2016-04-13
Related to Tails - Feature #15632: Consider distributing an alternative and minimal ISO image Rejected 2018-06-03 2019-01-01
Related to Tails - Bug #15536: Smaller ISO image Rejected 2018-04-16
Related to Tails - Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs Resolved 2018-08-18
Related to Tails - Bug #16290: Remove Scribus Resolved 2019-01-05
Related to Tails - Bug #16353: Decide what to do wrt. Traverso in Tails 4.0 Resolved 2019-01-13
Related to Tails - Bug #16387: hOpenpgp-tools doesn't work since GnuPG 2.1 Resolved 2019-01-24
Related to Tails - Bug #16571: Tor Browser does not list spell checker dictionaries installed via APT In Progress 2019-03-18
Related to Tails - Bug #15945: Drop Python 2 in Tails 5.0 Confirmed 2015-08-24
Related to Tails - Feature #16399: Write release notes for 3.14 Resolved 2019-01-29
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed
Blocks Tails - Feature #15941: Core work 2018Q4 → 2019Q2: Technical writing Resolved 2018-09-11

History

#1 Updated by anonym 2018-02-05 16:33:12

  • related to Feature #15281: Stack one single SquashFS diff when upgrading added

#2 Updated by anonym 2018-02-05 16:33:58

This is related to Feature #15281 (1BigIUK) because of the larger IUKs. By removing software they won’t ever be updated which otherwise would increase the IUK size.

#3 Updated by anonym 2018-02-05 16:34:37

  • Assignee set to sajolida
  • QA Check set to Info Needed

sajolida, would you like to work on a list of candidates for removal?

#4 Updated by anonym 2018-02-05 16:35:48

#5 Updated by anonym 2018-02-05 19:02:40

BTW, as far as Feature #15281 is concerned, only packages with a history of frequent updates during Debian cycles (for security reasons or otherwise) matter; packages that are not upgraded through a Debian cycle does not impact the IUK in any way.

#6 Updated by sajolida 2018-02-09 06:47:38

  • Target version deleted (2018)
  • QA Check deleted (Info Needed)

We use “Target version: 2018” for stuff that we agreed to put on our roadmap. I don’t think this was part of it.

In terms of timeline, I want to work on this only have we have a very solid implementation of Additional Software (Feature #14568) and it has seen some adoption. A better bet would be 2019. By then we might have more research data that helps us select packages for removal.

But yes, I can do that as part of core UX.

#7 Updated by sajolida 2018-04-18 10:34:40

  • Feature Branch set to feature/15291-remove-software
  • Document edition
    • bookletimposer
    • pdf-redact-tools
    • poedit
  • Sound and video
    • traverso
    • pitivi
    • sound-juicer
    • brasero
    • cdrdao
  • OpenPGP tools
    • hopenpgp-tools
    • keyringer
    • paperkey
  • Misc
    • gobby
    • liferea
    • monkeysign
    • monkeysphere
    • msva-perl
    • nautilus-gtkhash
    • pwgen
    • ssss

#8 Updated by intrigeri 2018-04-30 11:46:05

Agreed with this list modulo I’d rather handle Liferea differently (Bug #11082).

#9 Updated by sajolida 2018-05-04 15:10:30

  • Assignee deleted (sajolida)
  • Priority changed from Normal to Low

My branch now builds! The resulting ISO is 1121 MB while devel is 1168 MB. That’s 4% less.

I’m a bit disappointed by the small difference :(

Note that devel has been broken for 12 days so maybe its size would be slightly different with a more recent build.

#10 Updated by intrigeri 2018-06-11 08:57:58

  • related to Feature #15632: Consider distributing an alternative and minimal ISO image added

#11 Updated by Rover11 2018-06-11 12:08:18

sajolida wrote:
> * Document edition
> bookletimposer
> pdf-redact-tools
> poedit
…
_
Probably huge-size packages should be priorities for exclusion?
like: Inkscape, GIMP, Scribus, audacity

#12 Updated by intrigeri 2018-06-11 12:20:11

> Probably huge-size packages should be priorities for exclusion?

I don’t think that size should be the primary criterion but indeed, it does matter. IMO the criterion should be something like:

(size + frequency of security upgrades) / popularity

Size and popularity are obvious I guess. Frequency of security upgrades matter for two reasons:

  • packages that are often upgraded make automatic upgrades bigger for everyone;
  • whatever we ship in the ISO is essentially frozen, so users don’t benefit of Debian security upgrades until our next release; while packages installed via ASP get upgraded on users’ systems.

#13 Updated by sajolida 2018-06-14 06:57:47

#14 Updated by sajolida 2018-06-23 17:54:08

  • Assignee set to sajolida

I’ll try prepare an ISO image with even less packages and see how much we gain in size.

#15 Updated by sajolida 2018-07-19 15:50:50

I tried to remove even more packages with f4c21628f4 to see the impact on ISO size.

#16 Updated by sajolida 2018-08-03 12:39:10

This one went down to 989315072 bytes vs 1217341440 in devel: 19% smaller than devel.

Now I’ll experiment with an ISO with no internationalization :) Starting with b7b0122d57.

#17 Updated by sajolida 2018-08-06 18:57:19

Removing fonts I get 929458176 instead of 1217341440 in devel: 23% less.

I’m now trying to remove precompiled locales in bda974fa5d.

#18 Updated by intrigeri 2018-08-07 08:17:57

I’m glad you’re doing all these experiments, it’s great!

Let’s keep in mind that some of these changes may increase boot/login time substantially, which kinda conflicts with our “Make it easier to switch between a Tails contextual identity and another identity outside of Tails” goal (that prompted me to start looking into low-hanging fruits wrt. boot time optimizations, stay tuned :)

#19 Updated by sajolida 2018-08-07 18:32:53

Without locales-all the size is 921231360 vs 1217341440 in devel: 24% smaller than devel.

#20 Updated by Anonymous 2018-08-16 12:16:00

  • related to Feature #15543: Give a list of examples of nice software to install each time added

#21 Updated by Anonymous 2018-08-16 12:17:13

Feature #15543 was rejected but I think if we remove all these softwares that were previously shipped automatically, we absolutely need a list of recommended programs. I don’t expect users to know what package they need to install to be able to sign keys (monkeysphere) or to burn a CD (brasero).

#22 Updated by sajolida 2018-08-17 08:23:39

  • Feature #15543 has been reopen.
  • I don’t think we’ll ever remove all these packages. I’m basically doing some experiment on the size of the ISO image, pushing it to the limit of what could be a super minimal (and limited) Tails image, which is borderline with Feature #15632.

#23 Updated by intrigeri 2018-08-18 15:09:39

I recommend you (temporarily) delete config/chroot_local-hooks/04-change-gids-and-uids from your branch so that it builds again. Since there’s no plan to merge it, that should be fine :)

#24 Updated by intrigeri 2018-08-18 18:11:57

  • related to Feature #15807: Define & apply clear criteria for including dictionaries, fonts and language packs added

#25 Updated by nodens 2018-08-30 15:12:58

  • Target version set to 2019

#26 Updated by sajolida 2018-09-12 17:03:36

Summary:

devel 1168 MB
remove less popular packages 02535f38b9 1121 MB –4%
remove document editing f4c21628f4 943 MB –19%
remove fonts b7b0122d57 886 MB –24%
remove locales bda974fa5d 878 MB –25%

I won’t do any more experiment for now :)

#27 Updated by intrigeri 2019-01-10 14:06:32

#28 Updated by intrigeri 2019-01-13 12:51:41

  • related to Bug #16353: Decide what to do wrt. Traverso in Tails 4.0 added

#29 Updated by goupille 2019-01-24 14:12:54

  • related to Bug #16387: hOpenpgp-tools doesn't work since GnuPG 2.1 added

#30 Updated by intrigeri 2019-03-18 13:45:09

  • related to Bug #16571: Tor Browser does not list spell checker dictionaries installed via APT added

#31 Updated by intrigeri 2019-03-23 16:13:38

I think we can, and should:

  • ASAP, which could be in 4.0 or even earlier:
    • remove all the command line tools from your list: presumably, command line users are more at ease than others to find the package the need based on the name of the command
    • remove some of the other software on your list, when users who need them will know their name, e.g. Gobby, which makes it rather easy to find in Synaptic, as opposed to “I want to burn a CD, Synaptic gives me 150 results, which should I do?”
    • remove obsolete & unpopular software, such as pitivi, traverso and hopenpgp-tools
    • remove software that’s not very usable in the context of Tails, especially if it’s for a small part of our target userbase, such as monkeysign
    • remove highly specialized software that’s mostly relevant for Tails contributors, such as keyringer and poedit
    • keep software that satisfies common use cases, when it’s too hard to find what’s the best option in Synaptic because there are too many results, e.g. Brasero and maybe others
    • keep LibreOffice, GIMP and Inkscape for now; Scribus is more specialized and is going away in 4.0 already
    • the above covers most of your initial list (Feature #15291#note-7) already
  • Once we ship GNOME Software, assuming it will make it easier to find the best / most GNOMEish option, we can consider removing more graphical apps, e.g. some of commit:f4c21628f4bdfa218fe7b6d9bbf17b4111507569.

#32 Updated by sajolida 2019-03-27 11:19:55

  • Assignee changed from sajolida to intrigeri
  • QA Check set to Info Needed

I agree with your criteria. Criteria #2 and #6 are super smart!

Here is the list of all the packages I removed in my branch (removing localization and font packages):

audacity
bookletimposer
electrum
gimp
gnome-sound-recorder
gobby
hopenpgp-tools
inkscape
keyringer
liferea
pdf-redact-tools
monkeysign
monkeysphere
msva-perl
nautilus-gtkhash
paperkey
pitivi
poedit
pwgen
sane-utils
scribus
simple-scan
traverso
sound-juicer
ssss
brasero
cdrdao

I propose that we remove ASAP (3.14?):

gobby
hopenpgp-tools
keyringer
monkeysign
monkeysphere
msva-perl
paperkey
pitivi
pwgen
traverso
ssss
cdrdao
pdf-redact-tools

To address Ulrike’s concern from Feature #15543#note-21, I’ll have to update some doc and maybe find how to recommend for packages for certain tasks.

I propose that we wait to see what happens with this 1st batch of removal before removing:

poedit
scribus

According to your criteria we should keep for now:

audacity
bookletimposer
electrum
gimp
gnome-sound-recorder
inkscape
liferea
nautilus-gtkhash
sane-utils
simple-scan
sound-juicer
brasero

I could probably have some fun working on this branch myself but otherwise it looks more like a mission for the Foundations Team, no?

#33 Updated by intrigeri 2019-04-02 14:37:33

  • related to Bug #15945: Drop Python 2 in Tails 5.0 added

#34 Updated by intrigeri 2019-04-08 08:39:56

#35 Updated by intrigeri 2019-04-08 08:44:48

#36 Updated by intrigeri 2019-04-08 08:49:05

  • Target version changed from 2019 to Tails_3.14
  • QA Check changed from Info Needed to Dev Needed

> I propose that we remove ASAP (3.14?):

I’ll do that.

> I propose that we wait to see what happens with this 1st batch of removal before removing:

Agreed.

> I could probably have some fun working on this branch myself but otherwise it looks more like a mission for the Foundations Team, no?

Sure. I’ve renamed your branch to wip/feature/15291-remove-software-sajolida-experiment and will start a new feature/15291-remove-software branch that implements the above agreement. The wip/ prefix will make Jenkins ignore your branch for now; feel free to drop it whenever you want to resume experimenting with this :)

#37 Updated by intrigeri 2019-04-08 08:53:56

>> I propose that we remove ASAP (3.14?):

> I’ll do that.

Err, except cdrdao: removing it would break some Brasero features and most users won’t manage to find out they need to install cdrdao to get them back.

#38 Updated by intrigeri 2019-04-08 09:21:32

  • Status changed from Confirmed to In Progress
  • QA Check deleted (Dev Needed)
  • Type of work changed from Research to Code

On top of the list agreed upon above, I’m also removing libgfshare-bin, for the same reason we’re removing ssss.

My grep’ing while doing this work spotted some doc that needs updating:

  • doc/encryption_and_privacy/keyringer: remove this page? Or mention the fact one needs to install keyringer first?
  • doc/advanced_topics/paperkey: same as keyringer
  • doc/sensitive_documents/sound_and_video (pitivi, traverso)
  • doc/sensitive_documents/metadata
  • backlinks of the aforementioned pages, if any of them is removed

#39 Updated by intrigeri 2019-04-08 09:43:09

#40 Updated by intrigeri 2019-04-08 10:45:20

  • QA Check set to Ready for QA

#41 Updated by intrigeri 2019-04-09 05:53:02

  • Assignee changed from intrigeri to sajolida
  • QA Check changed from Ready for QA to Dev Needed
  • Type of work changed from Code to End-user documentation

I’ve seen all our automated tests pass at least once; the code part is ready to be reviewed’n’merged by one of my fellow FT colleagues as soon as the doc is updated on this branch. So @sajolida, the floor is yours :) See my note above wrt. the bits of doc I’ve spotted that need updating.

#42 Updated by intrigeri 2019-04-09 05:53:59

  • blocks Feature #15941: Core work 2018Q4 → 2019Q2: Technical writing added

#43 Updated by intrigeri 2019-04-09 06:39:47

  • Feature Branch changed from feature/15291-remove-software to feature/15291-remove-software, https://salsa.debian.org/tails-team/tails/merge_requests/10

#44 Updated by intrigeri 2019-04-09 14:30:31

intrigeri wrote:
> I’ve seen all our automated tests pass at least once; the code part is ready to be reviewed’n’merged by one of my fellow FT colleagues as soon as the doc is updated on this branch. So @sajolida, the floor is yours :) See my note above wrt. the bits of doc I’ve spotted that need updating.

My MR was approved on Salsa so feel free to merge straight into stable once you’re happy with the doc update :)

#45 Updated by intrigeri 2019-04-09 15:04:13

> My MR was approved on Salsa […]

Glossary: MR = merge request (in the GitLab context; on GitHub it’s PR = pull request).

#46 Updated by sajolida 2019-04-16 14:50:29

> Err, except cdrdao: removing it would break some Brasero features and most users won’t manage to find out they need to install cdrdao to get them back.

I trust you more than me on these kinds of things but I listed cdrdao
because it’s nowhere in the dependencies of Brasero:

Depends: libbrasero-media3-1 (= 3.12.1-4), libc6 (>= 2.4), libcairo2 (>=
1.2.4), libgdk-pixbuf2.0-0 (>= 2.22.0), libglib2.0-0 (>= 2.37.3),
libgstreamer-plugins-base1.0-0 (>= 1.0.0), libgstreamer1.0-0 (>= 1.0.0),
libgtk-3-0 (>= 3.0.0), libnautilus-extension1a (>= 3.21.92-3~),
libpango-1.0-0 (>= 1.14.0), libtotem-plparser18 (>= 3.10.0),
libtracker-sparql-1.0-0 (>= 0.10.0), libxml2 (>= 2.7.4),
gstreamer1.0-plugins-base (>= 0.11.92), gvfs, brasero-common (= 3.12.1-4)
Recommends: yelp, brasero-cdrkit
Suggests: vcdimager, libdvdcss2, tracker

#47 Updated by intrigeri 2019-04-17 06:37:47

> I trust you more than me on these kinds of things but I listed cdrdao because it’s nowhere in the dependencies of Brasero:

Dependencies are transitive :) brasero depends on libbrasero-media3-1, which in turn has Recommends: cdrdao.

#48 Updated by sajolida 2019-04-17 08:37:41

> Dependencies are transitive :) brasero depends on libbrasero-media3-1, which in turn has Recommends: cdrdao.

Ha! I quickly checked level-2 dependencies of brasero but didn’t think
that a “lib.*” package could depend on a non-“lib.*” package :)

#49 Updated by sajolida 2019-05-10 08:17:17

  • Assignee changed from sajolida to cbrownstein

@cbrownstein: We’re removing these packages:

gobby
hopenpgp-tools
keyringer
monkeysign
monkeysphere
msva-perl
paperkey
pitivi
pwgen
traverso
ssss
cdrdao
pdf-redact-tools

We should remove all mentions of them in the doc.

#50 Updated by cbrownstein 2019-05-14 17:13:15

  • Assignee changed from cbrownstein to sajolida
  • QA Check changed from Dev Needed to Ready for QA

I’ve pushed a branch for review:

https://0xacab.org/cbrownstein/tails/tree/feature/15291-remove-software

It’s based on origin/feature/15291-remove-software

Mentions were found by running git grep -i $package_name

#51 Updated by intrigeri 2019-05-14 17:37:41

@cbrownstein or sajolida, it might be useful to check on Feature #15291#note-38 that nothing I had listed escaped your git grep’ing :)

#52 Updated by sajolida 2019-05-14 20:36:36

  • Assignee changed from sajolida to CyrilBrulebois
  • QA Check changed from Ready for QA to Pass

Cool!

I removed 1 extra thing with 198c9a3e13 (as per Feature #15291#note-38).

@CyrilBrulebois: I didn’t see a recent merge of this branch into stable, so I’ll let you do whatever is needed for this to be in 3.14, right?

#53 Updated by CyrilBrulebois 2019-05-17 13:28:38

  • Status changed from In Progress to Fix committed
  • % Done changed from 0 to 100

Applied in changeset commit:tails|f173fab62e515b5b457742145ca0a3118c25841d.

#54 Updated by CyrilBrulebois 2019-05-17 13:30:30

@sajolida:

I’ve just reviewed the changes between the merge base of this feature branch and stable, and just merged feature/15291-remove-software into stable.

#55 Updated by CyrilBrulebois 2019-05-23 21:20:44

  • Status changed from Fix committed to Resolved