Feature #12654

Introduce more variations in our reproducibility CI tests

Added by intrigeri 2017-06-07 16:48:48 . Updated 2018-04-09 16:14:24 .

Status:
Resolved
Priority:
Normal
Assignee:
intrigeri
Category:
Continuous Integration
Target version:
Tails_3.7
Start date:
2017-06-07
Due date:
% Done:

100%

Feature Branch:
jenkins-jobs:feature/12654-more-build-env-variations-for-RB-tests
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Currently we test in the exact same VM, and the time varies very little between our two builds. IMO we should introduce as many variations as we can (easily). We have build options for CPU and time variations, that we should use in our second build.

See the Variations introduced when testing Debian packages on tests.r-b.o if we want more inspiration.

Subtasks

History

#1 Updated by intrigeri 2017-06-07 16:49:34

  • Assignee set to intrigeri
  • Deliverable for deleted (289)

I’ll take care of it unless bertagaz wants it bad.

#2 Updated by intrigeri 2017-06-07 16:50:02

  • related to Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs added

#3 Updated by bertagaz 2017-06-08 17:08:23

intrigeri wrote:
> I’ll take care of it unless bertagaz wants it bad.

I know you like that kind of area, so if that’s because you think you’ll enjoy it, feel free! Otherwise, we have time to decide who’s plate it’s on.

#4 Updated by intrigeri 2017-12-16 10:13:38

  • Description updated

#5 Updated by intrigeri 2017-12-16 10:34:13

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

I’ve looked at the variations introduced on t.r-b.o when building Debian packages and I think we fix all of them thanks to our Vagrant build system, except those we can adjust via build options:

  • dateoffset: let’s test building 42 days in the future (not 30 or 31 days as in some cases this gives the same day number; and we want to ensure we can reproduce an ISO at least 6 weeks after it’s been released)
  • cpus: let’s test with one less vcpu than the number of CPUs on the host system
  • cpumodel and machinetype (not all virtual CPUs can be plugged into a virtual Q35 system): harder to implement in a robust way without making too many assumptions about the host system’s ability to emulate something else, but I’ll give it a quick try, probably with one of the virtual CPUs supported by QEMU.

#6 Updated by intrigeri 2017-12-16 11:59:51

> * dateoffset: let’s test building 42 days in the future (not 30 or 31 days as in some
> cases this gives the same day number; and we want to ensure we can reproduce an ISO
> at least 6 weeks after it’s been released)

This won’t work as our time-based APT snapshot are only valid for 10 days by default, and we can’t change this easily as it is used as the basis for garbage collection. So I’m trying with +9 days instead.

#7 Updated by intrigeri 2017-12-16 12:30:10

  • % Done changed from 10 to 20
  • Feature Branch set to puppet-tails:feature/12654-more-build-env-variations-for-RB-tests

dateoffset + cpus variation: works fine on my local Jenkins.

#8 Updated by intrigeri 2017-12-16 13:22:59

  • Assignee changed from intrigeri to bertagaz
  • Target version set to Tails_3.5
  • % Done changed from 20 to 50
  • QA Check set to Ready for QA

cpumodel variation works fine here too.

#9 Updated by anonym 2018-01-23 19:52:28

  • Target version changed from Tails_3.5 to Tails_3.6

#10 Updated by bertagaz 2018-03-14 10:57:50

  • Target version changed from Tails_3.6 to Tails_3.7

#11 Updated by intrigeri 2018-04-08 16:50:07

  • Feature Branch changed from puppet-tails:feature/12654-more-build-env-variations-for-RB-tests to jenkins-jobs:feature/12654-more-build-env-variations-for-RB-tests

#12 Updated by intrigeri 2018-04-08 16:53:35

I’ve been happily running this code on my local Jenkins for almost 4 months now and I’ve received no feedback nor ETA here => I’ve eventually merged this myself. Let’s see how it goes on our shared Jenkins. A review would still be welcome though! :)

#13 Updated by intrigeri 2018-04-08 16:53:49

  • Assignee changed from bertagaz to intrigeri

#14 Updated by intrigeri 2018-04-08 17:19:28

  • related to deleted (Feature #12633: Lower the workload caused by reproducible builds Jenkins jobs)

#15 Updated by intrigeri 2018-04-09 16:14:25

  • Status changed from In Progress to Resolved
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

https://jenkins.tails.boum.org/view/RM/job/reproducibly_build_Tails_ISO_devel/186/ and https://jenkins.tails.boum.org/view/RM/job/reproducibly_build_Tails_ISO_stable/233/ passed since I’ve merged this change.

A review would still be welcome but let’s not block on this.