Bug #12299

Make up our mind wrt. what old ISO our test suite should use on Jenkins

Added by intrigeri 2017-03-05 17:27:48 . Updated 2017-09-07 14:28:32 .

Status:
Resolved
Priority:
Normal
Assignee:
anonym
Category:
Continuous Integration
Target version:
Tails_3.2
Start date:
2017-03-05
Due date:
% Done:

100%

Feature Branch:
Type of work:
Discuss
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

https://git-tails.immerda.ch/puppet-tails/commit/files/jenkins/slaves/isotesters/wrap_test_suite?id=6f18e697ad102b18de44f391ee38f78e205222eb was meant to be a temporary measure, to be reverted after the 2.7 release, but I could not find where the follow-ups are tracked, and I wonder if there was a good reason not to revert it, hence this ticket.

So, do we want to pass the last released ISO to --old-iso when testing release branches, or do we want to use the default (i.e. use the to-be-tested ISO)?

Subtasks

Related issues

Blocks Tails - Feature #13239: Core work 2017Q3: Test suite maintenance Resolved 2017-06-29

History

#1 Updated by intrigeri 2017-03-05 17:34:41

  • Status changed from Confirmed to In Progress
  • Assignee set to anonym
  • Target version set to Tails_3.0
  • % Done changed from 0 to 10

IMO it makes sense to test (on the stable, testing and devel branches) the upgrade paths that our users will actually go through. Sometimes we’ll need to temporarily disable this logic, as happened twice in 2016, e.g. due to changes in the Installer or Upgrader. So I propose that:

  1. We pass a parameter to wrap_test_suite somewhere (jenkins-jobs.git:macros/test_Tails_ISO.yaml seems a better place than the script itself to store this bit of configuration), that indicates whether this logic shall be enabled. This way, we don’t have to keep adjusting the code in wrap_test_suite.
  2. Whenever we temporarily disable this logic, we file a ticket to re-enable it when we’re ready, so we don’t forget.
  3. We could maybe give our RMs write access to jenkins-jobs.git, so they can tweak this new setting as needed.

Thoughts?

Setting a close enough target version, as the temporary situation has lasted for 5 months now, and it should be pretty easy to make a decision and implement it.

#2 Updated by bertagaz 2017-03-08 12:27:27

Eeek, I thought I actually did re-enable this, don’t know why it isn’t the case (well, I have an idea about that, missing ticket)…

intrigeri wrote:
> Thoughts?

You seem to propose to implement that only for the base branches, is there a reason why we shouldn’t do it for other branches? Other than that your proposal sounds good to me at first.

#3 Updated by intrigeri 2017-03-08 12:58:17

> You seem to propose to implement that only for the base branches, is there a reason why we shouldn’t do it for other branches? Other than that your proposal sounds good to me at first.

The implementation we’re talking about (that’s currently disabled) is for base branches only. I don’t know why, but presumably there was a good reason to it?
Off the top of my head: it’s for release branches that we care about the upgrade path most. Non-release ones are more likely to expose the kind of issues that force us to disable this logic. So to keep our overhead low, it makes sense to me to only do this for release branches.

#4 Updated by bertagaz 2017-03-08 13:19:43

intrigeri wrote:
> The implementation we’re talking about (that’s currently disabled) is for base branches only. I don’t know why, but presumably there was a good reason to it?
> Off the top of my head: it’s for release branches that we care about the upgrade path most. Non-release ones are more likely to expose the kind of issues that force us to disable this logic. So to keep our overhead low, it makes sense to me to only do this for release branches.

Right, makes sense, I should have give that a bit more thoughts, my bad…

#5 Updated by intrigeri 2017-05-27 08:27:53

  • Target version changed from Tails_3.0 to Tails_3.1

This has been waiting for input since 3 months, so at this point it doesn’t make much of a difference to postpone it a little bit more => please focus on 3.0 blockers.

#6 Updated by anonym 2017-06-29 13:32:27

  • blocks Feature #13239: Core work 2017Q3: Test suite maintenance added

#7 Updated by anonym 2017-07-06 14:15:06

  • Target version changed from Tails_3.1 to Tails_3.2

#8 Updated by anonym 2017-09-07 13:37:56

intrigeri wrote:
> Non-release ones are more likely to expose the kind of issues that force us to disable this logic.

Such breaking changes will always (at least according to our process) be introduced in a feature/bugfix branch first, so this will only avoid the problem for that feature/bugfix branch; once merged, the base branch will also be affected. So we’ll have to react any way, so I’m not sure it’s worth dealing with these specially.

#9 Updated by intrigeri 2017-09-07 14:28:32

  • Status changed from In Progress to Resolved
  • % Done changed from 10 to 100

At the CI meeting we decided to:

  1. revert the commit this ticket is about (Bug #14608)
  2. move the kill switch to jenkins-jobs.git, making it global and have the script default to passing the last release to --old-iso unless the kill switch is enabled; document this in a place where anonym may find it in N months (Bug #14609)
  3. give anonym read-write access to jenkins-jobs.git (Feature #14610)
  4. evaluate in 9 months how it went (Feature #14611)