Bug #12119

Shrink lizard's jenkins-data LV

Added by intrigeri 2017-01-07 17:04:59 . Updated 2017-03-12 15:55:01 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Tails_2.12
Start date:
2017-01-07
Due date:
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

As noted on Bug #10396 already, and rediscovered today while doing Feature #11806, we use about half of the space allocated to that LV. My estimates say that we could shrink it down to 300GB and still be fine until the end of 2018.

Subtasks

Related issues

Related to Tails - Feature #11806: Update server storage planning needs for at least 2017 Resolved 2016-09-19
Related to Tails - Bug #16025: jenkins-data-disk is running out of diskspace again Resolved 2018-10-03

History

#1 Updated by intrigeri 2017-01-07 17:16:15

  • related to Feature #11806: Update server storage planning needs for at least 2017 added

#2 Updated by intrigeri 2017-03-12 07:01:10

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

Recipe:

  1. shut the Jenkins service down
  2. shrink the ext4 FS
  3. shrink the jenkins LV in the vgdata VG
  4. shrink the /dev/vdb1 PV
  5. shrink the /dev/vdb1 partition
  6. shut jenkins.lizard down
  7. shrink the jenkins-data LV in the lizard VG

At this point, we’ll have achieve the main goal of this operation, i.e. freeing 200GB. One optional follow-up could be to simplify the storage stack for this data: move the data from a “FS inside LV inside VG inside LV” setup to a simpler “FS on a LV” one.

In any case, the final step is ensuring that the extents are on the physical drives we want them to be on.

#3 Updated by intrigeri 2017-03-12 07:15:43

> One optional follow-up could be to simplify the storage stack for this data: move the data from a “FS inside LV inside VG inside LV” setup to a simpler “FS on a LV” one.

It’ll be easier and safer to combine both operations actually:

  1. shut the Jenkins service down
  2. clean up enough artifacts to make the data fit into the free space we have in the lizard VG
  3. create a new LV in the lizard VG and make it available in jenkins.lizard
  4. create a FS on this new LV, mount it and set correct permissions/ownership
  5. copy the data to this FS
  6. unmount the old /var/lib/jenkins
  7. have Puppet mount the new FS (with its future new name!) on /var/lib/jenkins, in place of the old LV
  8. shut jenkins.lizard down
  9. delete the old /dev/lizard/jenkins-data LV and remove it from libvirt config
  10. rename the new LV to jenkins-data and adjust libvirt config
  11. resize the new LV + FS to 300GB
  12. start jenkins.lizard
  13. move the extents to the /dev/mapper/md2_crypt PV

#4 Updated by intrigeri 2017-03-12 15:55:01

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 10 to 100

Completed.

#5 Updated by bertagaz 2018-10-03 09:56:43

  • related to Bug #16025: jenkins-data-disk is running out of diskspace again added