Feature #15281

Stack one single SquashFS diff when upgrading

Added by anonym 2018-02-05 15:45:31 . Updated 2020-01-05 16:15:59 .

Status:
Resolved
Priority:
High
Assignee:
segfault
Category:
Installation
Target version:
Tails_4.2
Start date:
2016-04-13
Due date:
% Done:

100%

Feature Branch:
feature/15281-single-squashfs-diff, iuk:feature/15281-single-squashfs-diff, perl5lib:feature/15281-single-squashfs-diff
Type of work:
Code
Blueprint:

https://tails.boum.org/blueprint/Endless_upgrades/#single-squashfs-diff

Starter:
Affected tool:
Upgrader
Deliverable for:

Description

That is, address a number of UX problems in our upgrade process with an hopefully cheap and temporary solution.

Subtasks

Feature #15279: Refresh Tails signing key before each upgrade check Resolved segfault

100
Feature #15282: Add a prompt right before applying an IUK Resolved

100
Feature #15283: Implement the "one single SquashFS diff" scheme in Tails Upgrader Resolved segfault

100
Feature #15284: Adapt release process for the "one single SquashFS diff" upgrade scheme Resolved

100
Feature #15285: Adapt incremental upgrade manual test for the "one single SquashFS diff" upgrade scheme Resolved

100
Feature #15286: Adapt automated test suite for the "one single SquashFS diff" upgrade scheme Resolved segfault

100
Feature #15290: Reduce IUK size Resolved

10
Bug #15407: Prevent system user uid:s and gid:s from changing between releases Resolved

100

Related issues

Related to Tails - Feature #11131: Endless automatic upgrades Rejected 2015-01-05
Related to Tails - Feature #15291: Remove less popular packages that users could install themselves Resolved 2018-04-17
Related to Tails - Feature #15277: Update our survey of non-NIH system upgrade solutions Confirmed 2018-02-03
Related to Tails - Feature #11679: Rethink the installation process and upgrade process Resolved 2016-08-20
Related to Tails - Feature #7499: Extend the upgrader to allow full (self) upgrade Confirmed 2014-07-06
Related to Tails - Feature #15287: Make it possible to reproducibly generate IUKs in CI Resolved 2018-02-05
Related to Tails - Feature #15288: Document tricks for power users vs. bigger downloads for automatic upgrade Confirmed 2018-02-05
Related to Tails - Bug #17398: Follow-up to "Stack one single SquashFS diff when upgrading" Resolved
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

History

#1 Updated by anonym 2018-02-05 15:46:12

#2 Updated by anonym 2018-02-05 15:47:40

#3 Updated by anonym 2018-02-05 15:48:10

  • Target version set to Tails_3.6

We might be able to pull this off in time for Tails 3.6. Let’s see!

#4 Updated by anonym 2018-02-05 15:56:27

  • blocked by Feature #15282: Add a prompt right before applying an IUK added

#5 Updated by anonym 2018-02-05 16:18:24

  • blocks Feature #15288: Document tricks for power users vs. bigger downloads for automatic upgrade added

#6 Updated by anonym 2018-02-05 16:29:57

#7 Updated by anonym 2018-02-05 16:33:12

  • related to Feature #15291: Remove less popular packages that users could install themselves added

#8 Updated by anonym 2018-02-05 18:59:09

  • Feature Branch changed from https://tails.boum.org/blueprint/Endless_upgrades/ to feature/15281-1bigiuk
  • Blueprint set to https://tails.boum.org/blueprint/Endless_upgrades/

#9 Updated by intrigeri 2018-02-06 15:23:08

  • Description updated
  • Blueprint changed from https://tails.boum.org/blueprint/Endless_upgrades/ to https://tails.boum.org/blueprint/Endless_upgrades/#iuks-on-steroids

#10 Updated by intrigeri 2018-02-06 15:30:06

  • related to Feature #6876: Have the incremental upgrade process use less RAM added

#11 Updated by intrigeri 2018-02-06 15:30:27

  • related to Feature #15277: Update our survey of non-NIH system upgrade solutions added

#12 Updated by intrigeri 2018-02-06 15:31:22

  • related to deleted (Feature #8415: Migrate from aufs to overlayfs)

#13 Updated by intrigeri 2018-02-06 15:31:30

#14 Updated by intrigeri 2018-02-06 15:34:41

  • Feature Branch changed from feature/15281-1bigiuk to feature/11131-endless-upgrade, iuk:feature/11131-endless-upgrade, perl5lib:feature/11131-endless-upgrade

#15 Updated by intrigeri 2018-02-06 15:38:16

  • blocks deleted (Feature #15282: Add a prompt right before applying an IUK)

#16 Updated by intrigeri 2018-02-06 15:53:59

  • related to Feature #11679: Rethink the installation process and upgrade process added

#17 Updated by intrigeri 2018-02-06 16:19:13

  • Description updated

#18 Updated by intrigeri 2018-02-06 16:50:16

  • Subject changed from Switch to the 1BigIUK upgrade scheme to Stack one single SquashFS diff when upgrading

#19 Updated by intrigeri 2018-02-06 16:51:59

  • Blueprint changed from https://tails.boum.org/blueprint/Endless_upgrades/#iuks-on-steroids to https://tails.boum.org/blueprint/Endless_upgrades/#single-squashfs-diff

#20 Updated by intrigeri 2018-02-06 16:52:17

  • Description updated

#21 Updated by anonym 2018-02-07 16:58:05

  • Feature Branch changed from feature/11131-endless-upgrade, iuk:feature/11131-endless-upgrade, perl5lib:feature/11131-endless-upgrade to feature/15281-single-squashfs-diff, iuk:feature/11131-endless-upgrade, perl5lib:feature/11131-endless-upgrade

#22 Updated by anonym 2018-02-08 09:49:41

  • Status changed from Confirmed to In Progress
  • Feature Branch changed from feature/15281-single-squashfs-diff, iuk:feature/11131-endless-upgrade, perl5lib:feature/11131-endless-upgrade to feature/15281-single-squashfs-diff, iuk:feature/15281-single-squashfs-diff, perl5lib:feature/15281-single-squashfs-diff

#23 Updated by intrigeri 2018-02-24 14:23:25

Some of the design doc changes on feature/11131-endless-upgrade were relevant but are not on the new branch.

#24 Updated by intrigeri 2018-03-02 08:23:19

  • Target version changed from Tails_3.6 to Tails_3.7

#25 Updated by intrigeri 2018-03-28 09:22:11

  • Target version changed from Tails_3.7 to Tails_3.8

#26 Updated by intrigeri 2018-03-28 09:23:55

#27 Updated by intrigeri 2018-05-25 13:28:06

  • Target version changed from Tails_3.8 to Tails_3.10.1

#28 Updated by intrigeri 2018-06-28 14:00:19

  • blocked by deleted (Feature #15139: Core work 2018Q2: Foundations Team)

#29 Updated by intrigeri 2018-06-28 14:00:40

#30 Updated by intrigeri 2018-06-28 20:55:43

  • Target version changed from Tails_3.10.1 to Tails_3.11

#31 Updated by intrigeri 2018-06-28 20:59:28

  • blocked by deleted (Feature #15334: Core work 2018Q3: Foundations Team)

#32 Updated by intrigeri 2018-06-28 21:00:01

#33 Updated by intrigeri 2018-08-20 10:08:10

  • related to Feature #7499: Extend the upgrader to allow full (self) upgrade added

#34 Updated by intrigeri 2018-09-12 06:37:33

  • Assignee changed from anonym to intrigeri

segfault, I’ve taken over most subtasks. I expect some of them can be reassigned to you (depending on how much of Feature #8415 you take) but I have to take a look at the current status first.

#35 Updated by intrigeri 2018-10-11 09:14:15

  • related to deleted (Feature #6876: Have the incremental upgrade process use less RAM)

#36 Updated by intrigeri 2018-11-05 14:45:47

  • Target version changed from Tails_3.11 to Tails_3.12

#37 Updated by intrigeri 2018-11-06 15:04:45

  • Target version changed from Tails_3.12 to Tails_3.13

#38 Updated by intrigeri 2018-12-02 21:55:07

#39 Updated by intrigeri 2018-12-02 21:55:26

  • blocked by deleted (Feature #15506: Core work 2018Q4: Foundations Team)

#40 Updated by intrigeri 2019-01-25 16:32:08

  • Target version changed from Tails_3.13 to 2019

#41 Updated by intrigeri 2019-02-06 14:08:52

#42 Updated by intrigeri 2019-02-06 14:08:55

  • blocked by deleted (Feature #15507: Core work 2019Q1: Foundations Team)

#43 Updated by sajolida 2019-03-14 17:03:36

  • related to #15885 added

#44 Updated by intrigeri 2019-04-05 16:07:59

  • Assignee deleted (intrigeri)

#45 Updated by intrigeri 2019-08-31 16:11:39

  • Description updated

#46 Updated by intrigeri 2019-11-24 08:48:42

  • blocked by deleted (Feature #8415: Migrate from aufs to overlayfs)

#47 Updated by intrigeri 2019-11-24 11:39:05

intrigeri wrote:
> Some of the design doc changes on feature/11131-endless-upgrade were relevant but are not on the new branch.

I’ve salvaged these changes except those that are about the IUK format v2, which I’ll salvage later as part of Feature #6876.

#48 Updated by intrigeri 2019-11-24 11:41:50

  • Assignee set to intrigeri

I could definitely use some help on the non-Perl subtasks, but since most of the work is on my plate, I want to have the parent ticket clearly on my radar.

#49 Updated by intrigeri 2019-12-01 11:01:31

  • Target version changed from 2019 to Tails_4.3

Most of this has to be completed in January 2020, as per contract.

#50 Updated by intrigeri 2019-12-01 11:16:24

  • Priority changed from Normal to High

#51 Updated by intrigeri 2019-12-24 12:25:50

Hi @segfault! As a heads up:

  • I’m done with the bulk of the work here (see subtasks).
  • The corresponding automated tests pass locally already.
  • I’m only waiting for Jenkins results before sending onto your plate for review.
  • You can now start your review whenever you have time for it. The deadline for merging in Jan 5.
  • I’ll try to implement the testing strategy that anonym & I came up with on Feature #15283, if time allows.

#52 Updated by intrigeri 2019-12-24 12:31:58

  • Status changed from In Progress to Needs Validation

#53 Updated by intrigeri 2019-12-24 12:34:10

  • related to Feature #15287: Make it possible to reproducibly generate IUKs in CI added

#54 Updated by intrigeri 2019-12-24 12:35:36

  • related to Feature #15288: Document tricks for power users vs. bigger downloads for automatic upgrade added

#55 Updated by intrigeri 2019-12-24 12:37:16

  • Target version changed from Tails_4.3 to Tails_4.2

(I’ve moved out of this ticket the subtasks that don’t necessarily have to be done in time for 4.2, so it’s easier to track here what the blockers are for 4.2 and merging.)

#56 Updated by intrigeri 2019-12-25 10:18:51

  • Assignee changed from intrigeri to segfault

#57 Updated by segfault 2020-01-05 16:09:26

  • Status changed from Needs Validation to Resolved
  • % Done changed from 81 to 100

Applied in changeset commit:tails|e5e98537e7415b7c6a2c5e7ab1beb4f656333814.

#58 Updated by segfault 2020-01-05 16:15:59

Note that I did not review:

  • The commits by anonym from February 2018, because I assumed that intrigeri already reviewed those.
  • The code in config/chroot_local-includes/usr/src that was imported from iuk.git and perl5lib.git.

I will create a follow-up ticket for the few remarks I have.

#59 Updated by segfault 2020-01-05 16:18:21

  • related to Bug #17398: Follow-up to "Stack one single SquashFS diff when upgrading" added