Bug #15993

Adjust infrastructure and CI for USB images

Added by Anonymous 2018-09-28 10:30:24 . Updated 2019-02-13 16:29:06 .

Status:
Resolved
Priority:
Normal
Assignee:
groente
Category:
Continuous Integration
Target version:
Tails_3.13
Start date:
2018-11-27
Due date:
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:
316

Description

Adjust CI wrt. more outputs from the build jobs => more inputs for the test suite and reproducibility tests

This includes at least:
- store .img* artifacts too when building
- copy .img* artifacts too to other jobs that need them
- garbage-collect .img* too
- symlink duplicate .img* too
- Deal with increased disk space requirements on Jenkins and in the ISO archive
- probably more stuff that we’ll discover along the way
- post-release bugfixing

This work is supposed to be equally shared by groente and intrigeri.

Subtasks

Bug #16155: increase jenkins and iso-archive diskspace Resolved

100
Bug #16157: Re-enable USE_LAST_RELEASE_AS_OLD_ISO=yes on Jenkins Resolved

100
Bug #16158: Ensure that old Jenkins artifacts are cleaned up Resolved

100
Bug #16159: reproducibly_build_Tails_ISO_* jobs are broken Resolved

100
Bug #16160: test_Tails_ISO_* fail on Jenkins due to lack of disk space Rejected

100
Bug #16162: Test reproducibility of USB images for all branches Resolved groente

100
Bug #16174: reproducibly_build_Tails_ISO_* jobs leave *.img around even when successful Resolved

100
Bug #16377: bittorrent.lizard lacks space for uploading both .img and .iso Resolved groente

0
Bug #16395: rsync.lizard lacks space for uploading both .img and .iso + IUKs Resolved groente

100

History

#1 Updated by Anonymous 2018-09-28 10:30:36

#2 Updated by Anonymous 2018-09-28 10:30:49

#3 Updated by Anonymous 2018-09-28 10:31:21

@intrigeri: I trust you create corresponding subtickets for this task yourself and assign them between yourself and groente. Thanks!

#4 Updated by intrigeri 2018-11-19 09:24:21

  • Subject changed from Adjust infrastructure and sysadmin for USB images to Adjust infrastructure and CI for USB images

#5 Updated by intrigeri 2018-11-19 09:24:29

  • Category set to Continuous Integration

#6 Updated by intrigeri 2018-11-19 09:26:29

  • Assignee changed from intrigeri to groente

During the sprint I’ll act as a mentor & rubber-duck both for groente (on this ticket) and kibi (on Bug #16002), while hopefully finding time to do my own work => formalizing who will take the lead on this ticket.

#7 Updated by intrigeri 2018-11-27 11:32:44

I expect the bulk of the work that’s needed is in:

  • jenkins-jobs.git (mostly in macros; don’t touch auto-generated stuff)
  • puppet-tails.git

#8 Updated by intrigeri 2018-11-27 12:02:04

> - store .img* artifacts too when building

We do this already. Note that the artifacts file naming scheme is changing a bit:

  • before we had .iso and .iso.{apt-sources,buildlog,etc.}
  • once Feature #16154 is merged we’ll have .{img,iso,apt-sources,buildlog,etc.} i.e. the extra artifacts (build log and friends) lose the .iso component in their filename: that component does not make sense anymore since these files apply just as well to the .img

#9 Updated by groente 2018-11-27 12:28:01

  • Assignee changed from groente to intrigeri
  • QA Check set to Ready for QA
  • Feature Branch set to jenkins-jobs:bugfix-15993-adjust-ci-for-usb-images

i’ve added img where it seemed to make sense in jenkins-jobs macros, can you check if it actually makes sense? :)

#10 Updated by intrigeri 2018-11-27 12:51:36

  • Assignee changed from intrigeri to groente
  • QA Check deleted (Ready for QA)

groente wrote:
> i’ve added img where it seemed to make sense in jenkins-jobs macros, can you check if it actually makes sense? :)

Perfect! I’ve pushed a minor nitpicking commit on top. Please merge into master once I’m done with Feature #16154.

#11 Updated by groente 2018-11-27 16:20:53

  • Assignee changed from groente to intrigeri
  • QA Check set to Ready for QA
  • Feature Branch changed from jenkins-jobs:bugfix-15993-adjust-ci-for-usb-images to puppet-tails:bugfix-15993-adjust-ci-for-usb-images

and some more changes to review!

#12 Updated by intrigeri 2018-11-27 17:03:52

  • Status changed from Confirmed to In Progress
  • Assignee changed from intrigeri to groente
  • QA Check changed from Ready for QA to Dev Needed
  • deduplicate_reproducible_build_jobs_upstream_ISOs:
    • -regex '*/archive/build-artifacts/1/*.i(so|mg)' does not do what you mean, I think (. and * mean something else). All in all I suspect that -path '*/archive/build-artifacts/1/*.iso' -o -path '*/archive/build-artifacts/1/*.img' will end up being more readable.
    • mixed tabs & space indentation; there were already some similar issues so I’ve fixed them all at once
  • I’ve pushed another improvement (writing the commit message was just as simple as explaining it here).
  • compare_artifacts: please DRY the duplicated code and make echo "Running diffoscope" tell which kind of things we’re running it on. Or ask me too, I have some time booked on this budget line too and I’m fine dealing with my own expectations myself here :)

#13 Updated by groente 2018-11-27 17:32:07

  • Assignee changed from groente to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

regexp should be fixed now, as well as the repetition in compare_artifacts

#14 Updated by intrigeri 2018-11-27 17:43:37

  • Assignee changed from intrigeri to groente

Pushed 1 bugfix + 1 nitpicking commit, please review and deploy if happy.

#15 Updated by groente 2018-11-27 17:51:54

  • Assignee changed from groente to intrigeri
  • QA Check deleted (Ready for QA)
  • Feature Branch deleted (puppet-tails:bugfix-15993-adjust-ci-for-usb-images)

merged & deployed

#16 Updated by intrigeri 2018-11-27 20:45:50

Grown each isotesterN-data by 3GB, otherwise once it’s got its upstream job’s artifacts downloaded, there’s no disk space left to wget the ISO + IMG from our ISO history repo.

#17 Updated by intrigeri 2018-11-29 13:39:25

  • Target version changed from Tails_3.11 to Tails_3.13

Status update: all necessary adjustments we’ve spotted are done and we’ve spotted and solved all known regressions before other Tails folks reported them to us. The remaining follow-up subtasks are tiny things that can only be done later (e.g. revert temporary hack XYZ once Tails 3.12 is out).

#18 Updated by intrigeri 2019-01-25 16:32:22

  • Target version changed from Tails_3.13 to 2019

#19 Updated by intrigeri 2019-01-25 16:32:36

  • Target version changed from 2019 to Tails_3.13

#20 Updated by intrigeri 2019-02-06 20:15:04

  • Assignee changed from intrigeri to groente

(All remaining subtasks are on groente’s plate.)

#21 Updated by groente 2019-02-13 15:38:04

  • Status changed from In Progress to Resolved

i think we’re done here.

#22 Updated by intrigeri 2019-02-13 16:29:06

Woohoo, mission accomplished \o/ :)))