Feature #12715

Decide what builds we will try to reproduce in Jenkins

Added by bertagaz 2017-06-15 09:38:43 . Updated 2017-07-07 16:09:11 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Continuous Integration
Target version:
Start date:
2017-06-15
Due date:
% Done:

100%

Feature Branch:
Type of work:
Discuss
Blueprint:

Starter:
Affected tool:
Deliverable for:
289

Description

Switching our Jenkins isobuilders to the vagrant build system was implemented so that we can test the reproducibility of our ISOs in Jenkins. We still did not decide what we will try to reproduce there. Should this be base branches only, or all active branches?


Subtasks


Related issues

Related to Tails - Feature #13436: Have Jenkins jobs that reproduce ISOs when a branch ticket is Ready for QA Resolved 2017-07-07
Related to Tails - Feature #9719: Configure Jenkins notifications to our ticket tracker Confirmed 2015-07-10

History

#1 Updated by bertagaz 2017-06-15 09:39:39

  • blocks Feature #12002: Estimate hardware cost of reproducible builds in Jenkins added

#2 Updated by intrigeri 2017-06-19 11:30:20

IMO all active branches, otherwise sooner or later the RM will merge a branch right before the freeze, notice after merging that it breaks reproducibility, and will have to fix it her/himself in a hurry. The way I see it, in order to be merged, a branch MUST NOT break reproducibility, and the only way we have to guarantee that is to actually test it.

#3 Updated by intrigeri 2017-07-04 09:59:23

  • Priority changed from Normal to High

This transitively blocks very high prio stuff, so let’s make up our mind ASAP, ideally during the next CI meeting if we don’t reach a conclusion earlier.

#4 Updated by intrigeri 2017-07-04 09:59:46

  • Assignee changed from bertagaz to anonym
  • QA Check set to Info Needed

anonym, what do you think?

#5 Updated by intrigeri 2017-07-05 10:12:45

  • blocked by deleted (Feature #12002: Estimate hardware cost of reproducible builds in Jenkins)

#6 Updated by anonym 2017-07-06 15:38:50

intrigeri wrote:
> IMO all active branches, otherwise sooner or later the RM will merge a branch right before the freeze, notice after merging that it breaks reproducibility, and will have to fix it her/himself in a hurry. The way I see it, in order to be merged, a branch MUST NOT break reproducibility, and the only way we have to guarantee that is to actually test it.

Me and intrigeri discussed this during the 2017-07-06 CI meeting. We both agree with the need of having the reproducibility of individual branches verified before merging for the above reasons. We simply want to optimize the above with: instead of reproducing all active branches, we just reproduce all active branches marked Ready for QA in Redmine. After all, before that a branch is not considered ready in any way, so attempting to reproduce it is just a wast of our limited resources.

Example: when a developer thinks her/his branch is ready, s/he marks it Ready for QA. Then Jenkins will automatically pick up this status change and, because of it, try to reproducibly build the branch, and also test it. If both are successful, Jenkins will change the QA status to “Ready for QA + Jenkins likes it” (name improvements welcome!); otherwise Jenkins will set it to Dev needed and probably assign it to the last committer.

#7 Updated by intrigeri 2017-07-06 15:44:52

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to bertagaz
  • Priority changed from High to Elevated
  • % Done changed from 0 to 50
  • QA Check changed from Info Needed to Ready for QA

anonym wrote:
> intrigeri wrote:
> > IMO all active branches, otherwise sooner or later the RM will merge a branch right before the freeze, notice after merging that it breaks reproducibility, and will have to fix it her/himself in a hurry. The way I see it, in order to be merged, a branch MUST NOT break reproducibility, and the only way we have to guarantee that is to actually test it.

> Me and intrigeri discussed this during the 2017-07-06 CI meeting. We both agree with the need of having the reproducibility of individual branches verified before merging for the above reasons. We simply want to optimize the above with: instead of reproducing all active branches, we just reproduce all active branches marked Ready for QA in Redmine. After all, before that a branch is not considered ready in any way, so attempting to reproduce it is just a wast of our limited resources.

ACK, I think this resolves this ticket. bertagaz, if you agree with our conclusions, please create whatever follow-up tickets are needed and close this one :)

Downgrading priority a bit since we’ve removed the blocker.

> Example: when a developer thinks her/his branch is ready, s/he marks it Ready for QA. Then Jenkins will automatically pick up this status change and, because of it, try to reproducibly build the branch, and also test it. If both are successful, Jenkins will change the QA status to “Ready for QA + Jenkins likes it” (name improvements welcome!); otherwise Jenkins will set it to Dev needed and probably assign it to the last committer.

I believe that’s material for Feature #9719, and not blocking what this ticket is about.

#8 Updated by bertagaz 2017-07-07 12:28:08

  • related to Feature #13436: Have Jenkins jobs that reproduce ISOs when a branch ticket is Ready for QA added

#9 Updated by bertagaz 2017-07-07 12:32:49

  • related to Feature #9719: Configure Jenkins notifications to our ticket tracker added

#10 Updated by bertagaz 2017-07-07 12:34:09

  • Assignee changed from bertagaz to intrigeri
  • Priority changed from Elevated to High
  • % Done changed from 50 to 0
  • QA Check changed from Ready for QA to Info Needed

intrigeri wrote:
> anonym wrote:
> > intrigeri wrote:
> > > IMO all active branches, otherwise sooner or later the RM will merge a branch right before the freeze, notice after merging that it breaks reproducibility, and will have to fix it her/himself in a hurry. The way I see it, in order to be merged, a branch MUST NOT break reproducibility, and the only way we have to guarantee that is to actually test it.
>
> > Me and intrigeri discussed this during the 2017-07-06 CI meeting. We both agree with the need of having the reproducibility of individual branches verified before merging for the above reasons. We simply want to optimize the above with: instead of reproducing all active branches, we just reproduce all active branches marked Ready for QA in Redmine. After all, before that a branch is not considered ready in any way, so attempting to reproduce it is just a wast of our limited resources.
>
> ACK, I think this resolves this ticket. bertagaz, if you agree with our conclusions, please create whatever follow-up tickets are needed and close this one :)

That’s a great idea! Created Feature #13436.

> > Example: when a developer thinks her/his branch is ready, s/he marks it Ready for QA. Then Jenkins will automatically pick up this status change and, because of it, try to reproducibly build the branch, and also test it. If both are successful, Jenkins will change the QA status to “Ready for QA + Jenkins likes it” (name improvements welcome!); otherwise Jenkins will set it to Dev needed and probably assign it to the last committer.
>
> I believe that’s material for Feature #9719, and not blocking what this ticket is about.

Agreed. Linked both tickets together as a reminder.

Two remarks though:

  • I guess we assume we’ll keep on trying to reproduce every builds for the base branches.
  • We talked about a reproducible ISO job set up for new tags pushed in the Tails repo. I’m not sure we decided something about that. Is it still a plan? If it is, I’ll create another ticket.

#11 Updated by bertagaz 2017-07-07 12:34:42

  • Priority changed from High to Elevated
  • % Done changed from 0 to 50

#12 Updated by intrigeri 2017-07-07 15:45:10

  • Status changed from In Progress to Resolved

>> ACK, I think this resolves this ticket. bertagaz, if you agree with our conclusions, please create whatever follow-up tickets are needed and close this one :)

> That’s a great idea! Created Feature #13436.

Closing, then.

> * I guess we assume we’ll keep on trying to reproduce every builds for the base branches.

Yes, sure. We already do that, and Feature #12633 will improve it :)

> * We talked about a reproducible ISO job set up for new tags pushed in the Tails repo. I’m not sure we decided something about that. Is it still a plan? If it is, I’ll create another ticket.

I don’t think we need any new job, we just need to fix the existing ones (Bug #12681). Note: Feature #5630 and https://labs.riseup.net/code/projects/tails/issues?query_id=249 are useful tools whenever you wonder what’s the current status of things.

#13 Updated by intrigeri 2017-07-07 15:45:36

  • Assignee deleted (intrigeri)
  • % Done changed from 50 to 100
  • QA Check changed from Info Needed to Pass

#14 Updated by bertagaz 2017-07-07 15:56:31

intrigeri wrote:
> > * We talked about a reproducible ISO job set up for new tags pushed in the Tails repo. I’m not sure we decided something about that. Is it still a plan? If it is, I’ll create another ticket.
>
> I don’t think we need any new job, we just need to fix the existing ones (Bug #12681). Note: Feature #5630 and https://labs.riseup.net/code/projects/tails/issues?query_id=249 are useful tools whenever you wonder what’s the current status of things.

Ok, so building from the tagged commit is enough, no need to build from the tag itself. Fine with me.

#15 Updated by intrigeri 2017-07-07 16:09:02

> Ok, so building from the tagged commit is enough, no need to build from the tag itself. Fine with me.

We do build from the tag already.

#16 Updated by intrigeri 2017-07-07 16:09:11

> Ok, so building from the tagged commit is enough, no need to build from the tag itself. Fine with me.

Last time I checked, we did build from the tag already.