Bug #9097

Automatic test: measure boot time

Added by BitingBird 2015-03-23 00:53:15 . Updated 2019-03-08 14:25:20 .

Status:
Confirmed
Priority:
Low
Assignee:
Category:
Test suite
Target version:
Start date:
2015-03-23
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

From manual test suite:

Measure boot time (from syslinux menu the GNOME dektop ready - quickly press enter in the greeter), then on some reference bare metal hardware, and compare with previous version. The new one should not be significantly slower to start.

The test suite needs to have access to data for previous releases, and should be allowed to increase a certain pourcentage (20-30 % ?).


Subtasks


Related issues

Related to Tails - Feature #12318: Have our test suite track detailed boot-up performance Confirmed 2017-03-11

History

#1 Updated by intrigeri 2015-03-23 14:45:22

> The test suite needs to have access to data for previous releases, and should be allowed to increase a certain pourcentage (20-30 % ?).

Thanks for filing this ticket while I was too tired to do it!

After thinking a bit more about it, my initial idea was:

  • on a given system, the test suite could store boot time data (on a per-branch basis) somehow;
  • when running the test suite, previous boot time could be compared to current one, and raise an error if it’s much longer.

Both could be done either purely via cucumber, or via Jenkins (there are Jenkins plugins to let it understand cucumber concepts, and IIRC there also are plugins to let it monitor job runtime and raise errors if it takes too long).

But now I’m less sure that it’s worth doing it specifically for boot time: whatever operation we automatically test should generally not take way longer in Tails N+1 than in Tails N. If it does, maybe it’s a bug. So perhaps leveraging Jenkins’ cucumber support to monitor per-scenario runtime would be a nice generalization of my initial idea. And then, it becomes a sysadmin task rather than a Test suite one.

Thoughts?

#2 Updated by intrigeri 2016-07-16 06:03:43

#3 Updated by spriver 2017-06-11 09:39:34

  • Assignee set to anonym
  • QA Check set to Info Needed

Would the output of systemd-analyze or systemd-analyze blame be useful here? At least it’d be possible to detect services that take unusually long to start (and could be compared to a previous version).

#4 Updated by intrigeri 2019-03-08 14:25:20

  • Assignee deleted (anonym)
  • QA Check deleted (Info Needed)

spriver wrote:
> Would the output of systemd-analyze or systemd-analyze blame be useful here?

@spriver, yes.

#5 Updated by intrigeri 2019-08-30 10:28:14

  • related to Feature #12318: Have our test suite track detailed boot-up performance added