Bug #12606

Better balance our isobuilders' I/O load over all available SSDs

Added by intrigeri 2017-05-26 11:00:02 . Updated 2017-05-26 12:12:44 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Continuous Integration
Target version:
Start date:
2017-05-26
Due date:
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:
289

Description

They hit almost exclusively md1. I want their load to be better spread over md1 and md3, at least for their -data and their -libvirt volumes, like I did for isotesters already. I’ll take care of it myself as it seems I’m the one who cares most about this topic.


Subtasks


History

#1 Updated by intrigeri 2017-05-26 12:12:44

  • Subject changed from Better spread our isobuilders' I/O over available SSDs to Better balance our isobuilders' I/O load over all available SSDs
  • Status changed from Confirmed to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 0 to 100

First, I’ve deleted isotesters 7 and 8 (that have been off for a long time anyway) and the corresponding LVs in order to have some margin to work with. Then I’ve moved some unrelated stuff around to make some more room where I needed it, and finally:

  • re-balanced isotester*-data to have 3 of them on md1 and the 3 others on md3
  • re-balanced isobuilder*-data to have 2 of them on md1 and the 2 others on md3

So calling this done.

Next time we need more space on SSDs, we could consider moving some system LVs to rotating drives: we’ll make boot performance a bit worse, but once services are started the system partition shouldn’t be heavily accessed. But I’ll want to check Munin first to ensure our rotating drives are not I/O-saturated already.