Feature #11009

Improve ISO building and testing throughput and latency

Added by intrigeri 2016-01-27 17:27:19 . Updated 2016-04-29 13:01:31 .

Status:
Resolved
Priority:
High
Assignee:
Category:
Infrastructure
Target version:
Tails_2.4
Start date:
2016-01-26
Due date:
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

https://tails.boum.org/blueprint/hardware_for_automated_tests_take2/

Starter:
Affected tool:
Deliverable for:

Description

As discussed on Feature #9264, Feature #10996 and Bug #10999, it would be good to:

  • be able to run the test suite more frequently on our infrastructure
  • improve the latency for ISO builds and tests

As elaborated on the blueprint, a first step would be to upgrade lizard v2 to give it more RAM, which will allow us to run more ISO testers and builders on it.

Subtasks

Bug #10999: Parallelize our ISO building workload on more builders Resolved

100
Feature #11010: Give lizard v2 more RAM Resolved

100
Feature #11011: Design a 2nd ISO testing machine Rejected intrigeri

100
Feature #11113: Run more ISO testers Resolved

100
Feature #11175: Decrease I/O load created by isotesters on lizard Resolved

100

Related issues

Related to Tails - Feature #9264: Consider buying more server hardware to run our automated test suite Resolved 2015-12-15

History

#1 Updated by intrigeri 2016-01-27 17:28:18

  • related to Feature #9264: Consider buying more server hardware to run our automated test suite added

#3 Updated by intrigeri 2016-01-27 17:48:44

  • Target version deleted (Tails_2.2)

#4 Updated by intrigeri 2016-02-29 13:02:02

  • Status changed from Confirmed to In Progress

#5 Updated by intrigeri 2016-02-29 13:10:54

  • Target version set to Tails_2.3

Over the last few days, we’ve improved our test suite runs throughput on lizard from 3.3 to 8 runs / hour, and decreased its worst case run time by 17% (Feature #11113, Feature #11175). Let’s evaluate how it goes in a month or so. Will now focus on the building side.

#6 Updated by intrigeri 2016-03-25 22:14:35

  • Target version changed from Tails_2.3 to Tails_2.4

intrigeri wrote:
> Over the last few days, we’ve improved our test suite runs throughput on lizard from 3.3 to 8 runs / hour, and decreased its worst case run time by 17% (Feature #11113, Feature #11175). Let’s evaluate how it goes in a month or so.

According to https://jenkins.tails.boum.org/plugin/cluster-stats/, ISO test jobs essentially never have to wait for an available isotester VM anymore, and the duration of these jobs hasn’t changed measurably since we added ISO testers, so on the testing side we’re all good!

> Will now focus on the building side.

That’s Bug #10999, which looks good as well. I’ll come back to it in a month, and then I bet we can close this ticket.

#7 Updated by intrigeri 2016-04-29 12:57:44

intrigeri wrote:
> According to https://jenkins.tails.boum.org/plugin/cluster-stats/, ISO test jobs essentially never have to wait for an available isotester VM anymore, and the duration of these jobs hasn’t changed measurably since we added ISO testers, so on the testing side we’re all good!

Confirmed in April :)

> > Will now focus on the building side.
>
> That’s Bug #10999, which looks good as well. I’ll come back to it in a month, and then I bet we can close this ticket.

I’ll do that now.

#8 Updated by intrigeri 2016-04-29 13:01:31

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)