Feature #13436

Have Jenkins jobs that reproduce ISOs when a branch ticket is Ready for QA

Added by bertagaz 2017-07-07 12:27:50 . Updated 2018-04-08 17:13:01 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Continuous Integration
Target version:
Tails_3.7
Start date:
2017-07-07
Due date:
% Done:

100%

Feature Branch:
puppet-tails:feature/12633-lower-reproducible-builds-workload;jenkins-jobs:feature/12633-lower-reproducible-builds-workload
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:
301

Description

Following the discussion on Feature #12715, we settled on trying to reproduce builds for branches which Redmine ticket is Ready for QA.

Subtasks

Related issues

Related to Tails - Feature #12715: Decide what builds we will try to reproduce in Jenkins Resolved 2017-06-15
Related to Tails - Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs Resolved 2017-10-22
Related to Tails - Bug #14871: Deduplicate the ticket_qa_check Redmine query function into the Tails pythonlib Confirmed 2017-10-20

History

#1 Updated by bertagaz 2017-07-07 12:28:07

  • related to Feature #12715: Decide what builds we will try to reproduce in Jenkins added

#2 Updated by intrigeri 2017-07-07 15:40:58

  • Target version changed from Tails_3.1 to Tails_3.2

I think this will be much easier once I’ve done Feature #12633, so postponing. It’s clearly less urgent that the robustness issues anyway, so it can wait :)

#3 Updated by intrigeri 2017-07-07 15:41:09

  • blocked by Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs added

#4 Updated by intrigeri 2017-07-07 15:42:00

  • Subject changed from Add a Jenkins job to reproduce ISOs when a branch ticket is Ready for QA to Have Jenkins jobs that reproduce ISOs when a branch ticket is Ready for QA

#5 Updated by intrigeri 2017-07-18 12:22:01

  • Deliverable for changed from 289 to 301

#6 Updated by intrigeri 2017-07-30 08:32:21

  • blocks Bug #11680: Upgrade server hardware (2017-2019 edition) added

#7 Updated by intrigeri 2017-07-30 08:50:17

  • Target version changed from Tails_3.2 to Tails_3.3

Hi bertagaz!

First of all I plan to tackle Feature #12633, that blocks this ticket, on October 9-10. So this one can’t be done during the 3.2 cycle, and thus postponing it. If you want to work on this earlier, then please go ahead and take over Feature #12633, I won’t mind at all.

But the main point I want to discuss today is the timeline of the next steps: in order to have enough relevant data for Bug #11680 wrt. the impact of reproducible builds on the performance / developer experience of our CI infra, I’ll need this very ticket to be done and then wait at least a few weeks. I would like to spend a few days on Bug #11680 mid-December, so ideally this ticket would be solved (as in: deployed and working well in production, not a seldom tested initial draft) by mid-November. Can you realistically commit to that, i.e. doing all this work between October 11 and November 15?

If no: that’s fine, then please tell me what your realistic ETA would be and I’ll reschedule my Bug #11680 sprint. I depend on this to schedule my own work so I prefer you to be pessimistic/conservative/realistic (and delay Bug #11680 a bit) than to have to reschedule my sprint one month before it starts, so please take into account the time it sometimes takes to polish the bits that perform less well than expected after the initial deployment :)

#8 Updated by bertagaz 2017-10-19 09:56:45

  • related to Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs added

#9 Updated by bertagaz 2017-10-19 09:58:48

  • Status changed from Confirmed to In Progress
  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 0 to 30
  • Feature Branch set to puppet-tails:feature/12633-lower-reproducible-builds-workload;jenkins-jobs:feature/12633-lower-reproducible-builds-workload

intrigeri wrote:
> Can you realistically commit to that, i.e. doing all this work between October 11 and November 15?

It’s currently solved with Feature #12633 hopefully.

#10 Updated by intrigeri 2017-10-20 08:44:48

  • QA Check set to Ready for QA

(Same as Feature #12633, making the metadata match my understanding of what I’m being asked to do.)

#11 Updated by intrigeri 2017-10-20 09:38:12

  • Assignee changed from intrigeri to bertagaz
  • QA Check changed from Ready for QA to Dev Needed

I was asked to review this in batch with Feature #12633 so some bits of my review (Feature #12633#note-16) apply to this ticket.

#12 Updated by bertagaz 2017-10-30 18:13:31

  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 30 to 60
  • QA Check changed from Dev Needed to Ready for QA

bertagaz wrote:
> intrigeri wrote:
> > Can you realistically commit to that, i.e. doing all this work between October 11 and November 15?
>
> It’s currently solved with Feature #12633 hopefully.

And so is also RfQA.

#13 Updated by intrigeri 2017-11-08 17:33:20

  • Assignee changed from intrigeri to bertagaz
  • QA Check changed from Ready for QA to Dev Needed

Review on Feature #12633.

#14 Updated by intrigeri 2017-11-10 12:45:43

I’ve just noticed, when working on Bug #14933, a slightly problematic implication of our design: the only way I have to evaluate a branch meant to fix a RB problem is to flag its ticket as Ready for QA, which feels hackish. It’s no big deal because I can keep the ticket assigned to me. Ideally we would have a way to force Jenkins to test reproducibility, e.g. a “Force Reproducibility Testing” boolean custom field or similar. Another approach I’ve seen used on GitHub is to give instructions to Jenkins in comments instead of via static metadata, e.g. one could tell “Jenkins, please test reproducibility” which would trigger one RB job. Anyway, that’s merely food for thought :)

#15 Updated by intrigeri 2017-11-15 09:41:05

decide_if_reproduce seems to be buggy when building from a tag: if I understand https://jenkins.tails.boum.org/view/RM/job/reproducibly_build_Tails_ISO_stable/30/console right (not so easy as there’s no explicit indication of what’s going on), it decided that it was not worth rebuilding from this tag, which feels sad: it’s one of the situations (last pre-release steps) in which it would be most valuable to try to rebuild. I assume it’s easy to fix so I’m not filing a dedicated ticket, but if you want me to give a hand (e.g. on this specific problem), let me know and I’ll track this separately.

#16 Updated by intrigeri 2017-11-15 09:46:43

Similarly, https://jenkins.tails.boum.org/view/RM/job/reproducibly_build_Tails_ISO_devel/8/console didn’t try to reproduce the build. Any clue why?

#17 Updated by bertagaz 2017-11-15 11:25:48

intrigeri wrote:
> Similarly, https://jenkins.tails.boum.org/view/RM/job/reproducibly_build_Tails_ISO_devel/8/console didn’t try to reproduce the build. Any clue why?

Yes, a mistake I made even after triple checking. Should be fixed in jenkins-jobs:baa46b9. I’ll start a job right now to see if that was the problem.

#18 Updated by anonym 2017-11-15 11:30:47

  • Target version changed from Tails_3.3 to Tails_3.5

#19 Updated by intrigeri 2017-11-28 09:25:29

  • blocked by deleted (Bug #11680: Upgrade server hardware (2017-2019 edition))

#20 Updated by Anonymous 2018-01-16 14:44:42

  • related to Bug #14871: Deduplicate the ticket_qa_check Redmine query function into the Tails pythonlib added

#21 Updated by anonym 2018-01-23 19:52:28

  • Target version changed from Tails_3.5 to Tails_3.6

#22 Updated by bertagaz 2018-03-14 11:32:05

  • Target version changed from Tails_3.6 to Tails_3.7

#23 Updated by intrigeri 2018-04-08 17:10:35

  • blocks deleted (Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs)

#24 Updated by intrigeri 2018-04-08 17:13:01

  • Status changed from In Progress to Resolved
  • Assignee deleted (bertagaz)
  • % Done changed from 60 to 100
  • QA Check changed from Dev Needed to Pass

I think we’re done here: we have these Jenkins jobs and the (post-deployment) review discussion lives on Feature #12633. So I don’t think this very ticket is useful anymore. If I missed something, please revert this change.