Bug #16460

Upgrade puppet-git.lizard to Buster

Added by intrigeri 2019-02-12 07:47:42 . Updated 2019-07-31 19:17:23 .

Status:
Resolved
Priority:
Normal
Assignee:
intrigeri
Category:
Infrastructure
Target version:
Start date:
2019-02-12
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

It’s getting too hard to run PuppetDB on Stretch.

What we have to do:

  • check release notes & upgrade doc
  • check modules compatibility

This is a rough plan that might work:

  • Move /var/lib/gitolite to its own (virtual) disk. Now we’ll have 3 gitolite disks.
  • Clone the current puppet master vm (change its mac address) — creates new master.
  • Temporarily rename sshd onion service on new VM
  • Follow important part of the doc on creating new vm.
  • Add it as a node to itself.
  • Pin the puppetdb version.
  • Upgrade to buster
  • Ensure PuppetDB data goes through schema migration; run puppetdb upgrade -c /etc/puppetdb/conf.d/config.ini if needed.
  • Make sure we have the pg_trgm PostgreSQL extension enabled.
  • Unpin the puppetdb version and downgrade.
  • Check puppetmaster logs for errors and deprecations and fix them.
  • Point every agent to the new master once with --noop to identify obvious issues.
  • Deal with issues by pushing fixes to new master.
  • Migrate from hiera() to lookup().
  • [until this point we can easily revert]
  • Shutdown old vm and mark as no autostart.
  • Disable Puppet agent on every system.
  • Point the new system drive to the old VM.
  • Merge the fixes in topic branches into master/production branches.
  • Revert 73966287, 0cfcfd7f, 40111d15 and 18f135c4 in the manifests repo.
  • Start puppet-git
  • Push all updated submodules and the main manifests repo.
  • Rename sshd onion service back to its official name on the new VM.
  • Re-enable Puppet agent on every system, one after the other.
  • Move new VM’s disks to the correct PV (= the same as the old VM’s disks).
  • Delete temporary disks created for puppet-git-buster.

Subtasks


Related issues

Related to Tails - Bug #16927: Upgrade to Hiera 5 Confirmed
Blocks Tails - Feature #13284: Core work: Sysadmin (Adapt our infrastructure) Confirmed 2017-06-30

History

#1 Updated by intrigeri 2019-02-12 07:50:54

  • Description updated

#2 Updated by intrigeri 2019-02-12 08:33:34

  • blocks Feature #13284: Core work: Sysadmin (Adapt our infrastructure) added

#3 Updated by intrigeri 2019-02-15 16:57:48

  • Assignee changed from intrigeri to bertagaz
  • QA Check set to Info Needed

I’ve proposed a course of action on our mailing list => please comment there. (I can’t assign to 2 people so I picked the one I’ve solicited the least recently.)

#4 Updated by CyrilBrulebois 2019-03-20 14:34:12

  • Target version changed from Tails_3.13 to Tails_3.14

#5 Updated by intrigeri 2019-04-03 05:28:55

As reported on https://bugs.debian.org/926291, the PuppetDB 6.2 release notes say that “puppetdb 6 requires puppet-master to be 6.0.0 or later”.

#6 Updated by CyrilBrulebois 2019-05-23 21:23:31

  • Target version changed from Tails_3.14 to Tails_3.15

#7 Updated by intrigeri 2019-06-02 15:12:11

  • Status changed from Confirmed to Needs Validation
  • Assignee changed from bertagaz to groente
  • QA Check deleted (Info Needed)

groente, maybe?

#8 Updated by intrigeri 2019-07-05 13:23:40

  • Status changed from Needs Validation to Confirmed
  • Assignee changed from groente to intrigeri
  • Target version changed from Tails_3.15 to Tails_3.16

I’ll schedule a work session to do it together with zen.

#10 Updated by intrigeri 2019-07-29 20:25:37

intrigeri wrote:
> Beware of https://bugs.debian.org/932135

We want PuppetDB 6.2.0-4 (currently in sid, maintainer won’t file a stable update request yet, he’s first waiting for a proper fix on the PostgreSQL side: https://bugs.debian.org/932247) to do the DB schema migration. And then, likely, we can downgrade to the Buster package.

#11 Updated by intrigeri 2019-07-29 21:30:30

  • Description updated

#12 Updated by zen 2019-07-29 21:33:06

  • Description updated

#13 Updated by intrigeri 2019-07-29 22:14:23

  • Description updated

#14 Updated by intrigeri 2019-07-29 22:56:52

  • Description updated

#15 Updated by intrigeri 2019-07-29 22:57:33

  • Description updated

#16 Updated by intrigeri 2019-07-30 13:26:57

  • Description updated

#17 Updated by intrigeri 2019-07-30 13:57:35

  • Description updated

#18 Updated by intrigeri 2019-07-30 14:13:18

  • Description updated

#19 Updated by intrigeri 2019-07-30 14:55:40

  • Description updated

#20 Updated by intrigeri 2019-07-30 15:24:49

  • Description updated
  • Status changed from Confirmed to In Progress

#21 Updated by zen 2019-07-30 16:32:57

  • Description updated

#22 Updated by zen 2019-07-30 16:41:27

  • Description updated

#23 Updated by zen 2019-07-30 16:50:03

  • Description updated

#24 Updated by zen 2019-07-30 17:02:16

  • Description updated

#25 Updated by zen 2019-07-30 17:32:54

  • Description updated

#26 Updated by zen 2019-07-30 19:55:03

  • Description updated

#27 Updated by zen 2019-07-30 20:08:52

  • Description updated

#28 Updated by intrigeri 2019-07-31 01:11:05

#29 Updated by intrigeri 2019-07-31 01:22:06

  • Description updated

#30 Updated by intrigeri 2019-07-31 01:36:07

  • Description updated

#31 Updated by zen 2019-07-31 13:10:11

  • Description updated

#32 Updated by intrigeri 2019-07-31 13:48:44

  • Description updated

#33 Updated by zen 2019-07-31 14:47:25

  • Description updated

#34 Updated by intrigeri 2019-07-31 14:52:42

  • Description updated

#35 Updated by zen 2019-07-31 17:13:06

  • Description updated

#36 Updated by zen 2019-07-31 17:15:38

  • Description updated

#37 Updated by intrigeri 2019-07-31 17:29:26

  • Description updated

#38 Updated by intrigeri 2019-07-31 17:45:27

  • Description updated

#39 Updated by intrigeri 2019-07-31 18:08:25

  • Description updated

#40 Updated by intrigeri 2019-07-31 19:16:54

  • Description updated

#41 Updated by intrigeri 2019-07-31 19:17:23

  • Status changed from In Progress to Resolved