Bug #14875

Build reproducibility Jenkins tests: confusing UX and implementation

Added by bertagaz 2017-10-22 11:47:18 . Updated 2019-09-22 05:56:09 .

Status:
Resolved
Priority:
Normal
Assignee:
intrigeri
Category:
Continuous Integration
Target version:
Tails_4.0
Start date:
2017-10-22
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Currently we always trigger a build of this job once we’ve built an ISO, and then this job decides whether it has been triggered for a good reason, writes this down using a flag file, and then N build steps each test whether they should do something or not (<=> whether the flag file exists), and if not, the build is marked as succeeded even when it has actually not tested anything. We should decide once for all whether a reproducibly_build_Tails_ISO downstream job shall be run for a given ISO.
It seems the conditional-buildstep plugin would be a “nicer solution” to use this plugin to determine if the job needs to reproduce an ISO or not, rather than the “ugly workaround” implemented first by using steps. For that to work, we need to have a newer version of Jenkins, given the dependencies of this plugin.

But it might not be the best option, especially wrt. reporting to developers/reviewers, see below.

Subtasks

Related issues

Related to Tails - Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs Resolved 2017-10-22
Related to Tails - Bug #16959: Gather usability data about our current CI In Progress

History

#1 Updated by bertagaz 2017-10-22 11:47:32

  • related to Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs added

#2 Updated by bertagaz 2017-10-22 11:47:49

  • blocked by Bug #10068: Upgrade to Jenkins 2.x, using upstream packages added

#3 Updated by intrigeri 2017-10-22 12:00:21

  • Subject changed from Use conditionnal-build plugin in reproducibly_build_Tails_ISO jobs to Use conditional-buildstep plugin in reproducibly_build_Tails_ISO jobs

#4 Updated by intrigeri 2017-10-22 12:01:13

(Fixed typo — s/conditionnal/conditional/ — and plugin ID.)

#5 Updated by intrigeri 2017-10-29 08:22:39

  • Deliverable for changed from 289 to 301

(This is scheduled to be done after the end of the contract.)

#6 Updated by intrigeri 2017-12-16 21:16:21

  • Tracker changed from Feature to Bug
  • Subject changed from Use conditional-buildstep plugin in reproducibly_build_Tails_ISO jobs to Decide once for all whether a reproducibly_build_Tails_ISO downstream job shall be run for a given ISO
  • Description updated

While reading about Jenkins stuff for unrelated reasons I came to think about this and after taking a step back, I don’t think it should be the responsibility of a reproducibly_build_Tails_ISO build to decide whether it shoud do anything at all. Because the way I understand Jenkins semantics, a job succeeds or fails, and it has no way to say “oh actually I don’t feel I’m needed today, I’ll pass”; so as a developer/reviewer, when looking at the Jenkins dashboard, a reproducibly_build_Tails_ISO job whose last run is marked as successful is confusing: to know whether my branch has actually been tested, I need to look at the “Last Duration” field; meh. As a developer/reviewer, I would like to be able to treat the results of this job like every other Jenkins job: Jenkins should tell me whether it has run, and whether it has succeeded, in a way that means what I’m interested about (i.e. the ISO was rebuilt identically success, or was rebuilt differently failure, or the ISO was not rebuilt at all == there’s been no build of that job).

So I think a build_Tails_ISO build should be the one deciding whether it’s worth checking if the ISO it has built should be tested for reproducibility. The good news is that apparently there’s a way to do it, that’s already supported by the version of the Parameterized Trigger plugin we use:

  • during the (first) ISO build, check if the ISO shall be tested for reproducibility and iff. yes, create a dummy properties file (whose content actually does not matter, it’ll be used as a flag file)
  • have the trigger-parameterized-builds for project: reproducibly_build_Tails_ISO_{name} “read properties from a file”, i.e. use property-file in jjb lingo; and enable fail-on-missing for project: reproducibly_build_Tails_ISO_{name} in there, so that the downstream job is not triggered when our flag file has not been created; I think that the “fail-on” phrasing is misleading BTW, as the Jenkins UI merely says “Don’t trigger if any files are missing”
  • and then all the conditional handling in reproducibly_build_Tails_ISO can be removed, and a given build of that job, when it’s been triggered, can focus on doing what it is about, and not on determining whether it was triggered for a good reason or not.

=> I think we should not use the conditional-buildstep plugin at all in this case. I think the resulting UX will be much better (and consistent with what we have been trained to) and the code+config will be simpler than if we used conditional-buildstep. And then this ticket should not blocked by Bug #10068 :)

What do you think?

#7 Updated by intrigeri 2017-12-16 21:18:08

  • Description updated

#8 Updated by anonym 2018-01-23 19:52:29

  • Target version changed from Tails_3.5 to Tails_3.6

#9 Updated by bertagaz 2018-03-14 11:32:05

  • Target version changed from Tails_3.6 to Tails_3.7

#10 Updated by intrigeri 2018-04-08 17:21:06

#11 Updated by intrigeri 2018-04-08 18:07:23

  • Subject changed from Decide once for all whether a reproducibly_build_Tails_ISO downstream job shall be run for a given ISO to Build reproducibility Jenkins tests: confusing UX and implementation
  • Description updated

#12 Updated by intrigeri 2018-04-08 18:09:26

#13 Updated by intrigeri 2018-04-08 18:09:36

  • related to Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs added

#14 Updated by intrigeri 2018-04-08 18:09:39

  • blocks deleted (Bug #10068: Upgrade to Jenkins 2.x, using upstream packages)

#16 Updated by bertagaz 2018-05-10 11:09:30

  • Target version changed from Tails_3.7 to Tails_3.8

#17 Updated by intrigeri 2018-06-26 16:28:03

  • Target version changed from Tails_3.8 to Tails_3.9

#18 Updated by intrigeri 2018-09-05 16:27:00

  • Target version changed from Tails_3.9 to Tails_3.10.1

#19 Updated by intrigeri 2018-10-24 17:03:42

  • Target version changed from Tails_3.10.1 to Tails_3.11

#20 Updated by CyrilBrulebois 2018-12-16 14:08:33

  • Target version changed from Tails_3.11 to Tails_3.12

#21 Updated by anonym 2019-01-30 11:59:21

  • Target version changed from Tails_3.12 to Tails_3.13

#22 Updated by Anonymous 2019-03-14 13:44:14

  • Assignee deleted (bertagaz)
  • Target version deleted (Tails_3.13)
  • Deliverable for deleted (301)

#23 Updated by intrigeri 2019-09-20 10:30:14

  • Category changed from Infrastructure to Continuous Integration

#24 Updated by intrigeri 2019-09-20 10:30:33

  • related to Bug #16959: Gather usability data about our current CI added

#25 Updated by intrigeri 2019-09-20 10:32:40

Looks like we can workaround the UX problem by marking the job as “unstable” when we actually don’t run any test at all. This is feasible since Jenkins 2.26: https://issues.jenkins-ci.org/browse/JENKINS-23786

#26 Updated by intrigeri 2019-09-20 10:33:01

  • blocked by Bug #10068: Upgrade to Jenkins 2.x, using upstream packages added

#27 Updated by intrigeri 2019-09-21 17:20:07

  • Status changed from Confirmed to Needs Validation
  • Assignee set to intrigeri
  • Target version set to Tails_4.0

intrigeri wrote:
> While reading about Jenkins stuff for unrelated reasons I came to think about this and after taking a step back, I don’t think it should be the responsibility of a reproducibly_build_Tails_ISO build to decide whether it shoud do anything at all. […]
> So I think a build_Tails_ISO build should be the one deciding whether it’s worth checking if the ISO it has built should be tested for reproducibility. The good news is that apparently there’s a way to do it, that’s already supported by the version of the Parameterized Trigger plugin we use:

I’ve implemented this in https://git.tails.boum.org/jenkins-jobs/commit/?id=4d6cc5fcc02a798bb89fd26522b3a48419710295, which fixes both the UX and the implementation problem. zen has reviewed these changes. It works fine so far on my local Jenkins and it was now deployed on lizard. Let’s see how it goes.

#28 Updated by intrigeri 2019-09-22 05:53:57

  • blocks deleted (Bug #10068: Upgrade to Jenkins 2.x, using upstream packages)