Bug #17423

Decide which incremental upgrade paths we want to manually test as part of our release process

Added by intrigeri 2020-01-11 12:07:16 . Updated 2020-03-19 16:28:34 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Discuss
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Before commit:1696d48cbdfab990b09ad7ac45a2078dc904e7fe we would test all incremental upgrade paths. To take an extreme example, when we’ll release Tails 4.20 there will be more than 2 dozens such upgrade paths. I felt this would take too much of our resources to me, so in commit:1696d48cbdfab990b09ad7ac45a2078dc904e7fe I changed the doc to only test the “current stable installed from scratch → version being prepared” upgrade path (which uses the v2 upgrader, v2 UDFs, and a v2 IUK). But perhaps that’s not good enough and we need to find a middle ground.

Another question is whether we want to test “Tails << 4.2 → 4.2 or 4.2.1 → version being prepared”, which is a bit different: the first step uses v1 UDFs and IUKs, while the 2nd step switches to v2 UDFs and IUKs.

After the 4.2.1 release, depending on how things go (now that the IUKs and UDFs have been fixed, presumably, and now that all the UDFs are on the test channel), we can discuss what a suitable trade-off could look like.


Subtasks


History

#1 Updated by CyrilBrulebois 2020-01-11 17:48:25

Food for thoughts: Bug #17425

#2 Updated by intrigeri 2020-01-13 19:04:27

CyrilBrulebois wrote:
> Food for thoughts: Bug #17425

Yep, that’s definitely something to take into account.

FWIW, here’s an update on this front: we now have regression tests in place for Bug #17425, one in the Upgrader codebase itself, the other in the Tails integration test suite (commit:f46d5c504fac878a43de91e74ad9e000c68cb137 and commit:f122cc078ef4421f1f4d94ecf716620993e5a2ea). Had they been added as part of Feature #15281, we would not have hit Bug #17425 at the worst possible time.

#3 Updated by anonym 2020-03-19 16:28:34

  • Status changed from Confirmed to Resolved

During the RM meeting on 2020-03-19 we decided:

  • We thinks it’s now enough to do the minimal (as documented) test as long as the Upgrader and IUK generation do not change
  • For less simple situations, we try to figure out in advance what can go wrong, with developers + RM + other relevant stakeholders, we test risky cases before merging, and then the RM adds whatever extra manual tests they feel are needed during the release process.