Feature #12505

Switch isobuilders to vagrant-libvirt in Puppet

Added by bertagaz 2017-05-03 13:11:52 . Updated 2019-05-06 18:15:37 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Start date:
2017-05-03
Due date:
% Done:

100%

Feature Branch:
puppet-tails:feature/11972-use-vagrant-in-jenkins
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:
301

Description

This ticket is meant to track the puppet part of the implementation of vagrant-libvirt on our Jenkins isobuilders. The tails.git part has been achieved in Feature #11972. Feature #11972#note-5 and following notes contain some initial review from intrigeri, that needs to be addressed. The complete switch in puppet also needs to be finished in a dedicated branch of puppet-tails.


Subtasks


History

#1 Updated by bertagaz 2017-05-03 13:12:35

  • related to Feature #11972: Switch our Jenkins ISO build system to vagrant-libvirt added

#2 Updated by bertagaz 2017-05-03 13:43:54

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Dev Needed to Info Needed

I think all remarks made in the initial review (Feature #11972#note-5 to Feature #11972#note-13) has been addressed.

All the Puppet code is also already written in the feature branch, as the plan is to re-install all Jenkins isobuilders to stretch one by one, each time setting them to tails::jenkins::slave::iso_builder::use_vagrant: true in hiera so that they switch to vagrant based builds.
Once they all have migrated, the use_vagrant condition in ::tails::jenkins::slave::iso_builder will be removed, as well as any traces of ::tails::builder.

Does this make sense?

One black hole in this is how to manage the sib isobuilder. What’s your plan regarding this?

#3 Updated by intrigeri 2017-05-03 13:50:00

Don’t wait for me before you start implementing everything you want in a Puppet Git branch, otherwise everything will be delayed by possibly 1 week.

#4 Updated by intrigeri 2017-05-03 13:51:30

  • Assignee changed from intrigeri to bertagaz
  • QA Check changed from Info Needed to Dev Needed

(As explained above, don’t block on me at a time when I’ve announced I would be unavailable: go ahead and I’ll review your stuff either in Git or once deployed.)

#5 Updated by intrigeri 2017-05-10 11:40:51

bertagaz wrote:
> Once they all have migrated, the use_vagrant condition in ::tails::jenkins::slave::iso_builder will be removed, as well as any traces of ::tails::builder.
>
> Does this make sense?

Yes => please prepare this in a (preferably tested) branch and submit it for review.

> One black hole in this is how to manage the sib isobuilder. What’s your plan regarding this?

Just let me know what manual migration steps I have to do, on top of what Puppet does.

#6 Updated by bertagaz 2017-05-21 16:03:11

  • Target version changed from Tails_3.0~rc1 to Tails_3.0

#7 Updated by intrigeri 2017-05-27 08:46:17

  • Target version changed from Tails_3.0 to Tails_3.1

Please focus on actual breakage for now, this refactoring can wait a bit :)

#8 Updated by intrigeri 2017-06-30 19:13:41

  • Priority changed from Elevated to Normal
  • Deliverable for set to 289

#9 Updated by intrigeri 2017-06-30 19:13:54

  • related to deleted (Feature #11972: Switch our Jenkins ISO build system to vagrant-libvirt)

#10 Updated by intrigeri 2017-06-30 19:14:06

#11 Updated by intrigeri 2017-07-06 14:53:08

  • Target version changed from Tails_3.1 to Tails_3.2

intrigeri wrote:
> Please focus on actual breakage for now, this refactoring can wait a bit :)

Same :)

#12 Updated by intrigeri 2017-07-18 12:22:42

  • Deliverable for changed from 289 to 301

#13 Updated by anonym 2017-09-28 18:29:14

  • Target version changed from Tails_3.2 to Tails_3.3

#14 Updated by bertagaz 2017-10-21 13:52:08

  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 0 to 50
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:
> bertagaz wrote:
> > Once they all have migrated, the use_vagrant condition in ::tails::jenkins::slave::iso_builder will be removed, as well as any traces of ::tails::builder.
>
> Yes => please prepare this in a (preferably tested) branch and submit it for review.
>
> > One black hole in this is how to manage the sib isobuilder. What’s your plan regarding this?
>
> Just let me know what manual migration steps I have to do, on top of what Puppet does.

Ok, pushed two commits in the puppet-tails repo branch fb72d65 and 304db9e.

The first one removes the use_vagrant condition and tails::builder call from the tails::jenkins::slaves::iso_builder manifest.

The second cleans up loeftovers from tails::builder and tails::website_builder in case some Stretch install out there is still running tails::builder.

I’ve taken the big hammer approach here, rather than customizing the to be deleted tails::builder manifest so that it would support $ensure = absent for all its resources. To migrate/test, one just have to merge this branch in its puppet-tails repo master.

#15 Updated by intrigeri 2017-10-22 06:24:16

  • Assignee changed from intrigeri to bertagaz
  • % Done changed from 50 to 70
  • QA Check changed from Ready for QA to Dev Needed

README still references tails::builder.

Wrt. “when we decide every installations had time to switch”: if that’s about isobuilder1.sib, forget about it (I had to fully reinstall it when we switched to Vagrant anyway, so there’s nothing to clean up there); and IIRC isobuilder*.lizard have been fully reinstalled too, so perhaps this code won’t be needed in the end… which may avoid having to track this “XXX” at all :) If it’s really needed: the $preferences_snippets, $packages and $files variables in the if $cleanup_pre_vagrant_build_deps block have too generic names. But they seem useless anyway so please just get rid of them.

Please sort lines in $builder_packages (tails::iso_builder).

Other than this, looks good, feel free to deploy when you’re in a good position to deal with the potential fallout.

Meta: I have no idea if the proposed branch has been tested, because you didn’t tell and it’s unclear to me what’s the default/implicit in this area these days. So please either clearly tell me what’s your default/implicit (and then only communicate about specific exceptions on a case by case basis), or communicate explicitly about the tests that were done every time you submit a branch. E.g. for code changes, the default is that one submits branches for QA after building & testing them, and occasionally we skip this step but make it clear when submitting for QA that it’s not been tested. This seems like a reasonable approach to me and avoids reviewers wasting time wondering/commenting about stuff that cannot possibly work. YMMV.

#16 Updated by anonym 2017-11-15 11:30:47

  • Target version changed from Tails_3.3 to Tails_3.5

#17 Updated by anonym 2018-01-23 19:52:27

  • Target version changed from Tails_3.5 to Tails_3.6

#18 Updated by bertagaz 2018-03-14 11:32:05

  • Target version changed from Tails_3.6 to Tails_3.7

#20 Updated by bertagaz 2018-05-10 11:09:00

  • Target version changed from Tails_3.7 to Tails_3.8

#21 Updated by intrigeri 2018-06-26 16:27:37

  • Target version changed from Tails_3.8 to Tails_3.9

#22 Updated by intrigeri 2018-09-05 16:26:47

  • Target version changed from Tails_3.9 to Tails_3.10.1

#23 Updated by intrigeri 2018-10-24 17:03:29

  • Target version changed from Tails_3.10.1 to Tails_3.11

#24 Updated by CyrilBrulebois 2018-12-16 13:54:46

  • Target version changed from Tails_3.11 to Tails_3.12

#25 Updated by intrigeri 2019-01-09 17:47:08

#26 Updated by anonym 2019-01-30 11:59:16

  • Target version changed from Tails_3.12 to Tails_3.13

#27 Updated by Anonymous 2019-03-15 10:13:04

  • Assignee changed from bertagaz to intrigeri

Main problem left to solve here: we have two different Puppet classes to manage Jenkins ISO builders, one of them is obsolete since we switched to Vagrant.

Remaining things to do: process review (Feature #12505#note-15), which seems to boil down to dropping the HEAD of the topic branch, sorting a few lines, and deploying it.

#28 Updated by intrigeri 2019-03-15 10:14:34

  • Target version changed from Tails_3.13 to Tails_3.14

#29 Updated by intrigeri 2019-03-20 09:57:54

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 70 to 100
  • QA Check deleted (Dev Needed)

Done, deployed (as expected it’s a no-op). I’ll handle the fallout if needed.

#30 Updated by intrigeri 2019-05-05 08:23:52

  • Target version changed from Tails_3.14 to Tails_3.13.2

#31 Updated by anonym 2019-05-06 15:03:12

  • Target version changed from Tails_3.13.2 to Tails_3.14

#32 Updated by intrigeri 2019-05-06 18:15:37

  • Target version changed from Tails_3.14 to Tails_3.13.2