Bug #11089

Deal with August 2016 false positive scenarios

Added by bertagaz 2016-02-09 11:03:51 . Updated 2016-09-02 04:30:12 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Continuous Integration
Target version:
Tails_2.6
Start date:
2016-08-05
Due date:
2016-09-05
% Done:

100%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
0
Affected tool:
Deliverable for:

Description

As defined in Bug #10993, check is there are still some of them and tag them as fragile if any (see Bug #10288)

Subtasks

History

#1 Updated by bertagaz 2016-02-09 11:17:27

  • Target version set to Tails_2.6

#2 Updated by intrigeri 2016-02-09 18:37:57

  • Subject changed from Deal with August false positive scenarios to Deal with August 2016 false positive scenarios
  • Deliverable for set to 270

#3 Updated by intrigeri 2016-02-09 18:38:56

  • Deliverable for deleted (270)

#4 Updated by intrigeri 2016-08-27 09:15:20

  • Assignee changed from bertagaz to intrigeri

In practice I’ve been doing this continuously so I can as well complete it. So I’ll trade this ticket against Bug #11091.

#5 Updated by intrigeri 2016-08-27 09:29:31

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 50

#6 Updated by intrigeri 2016-09-02 04:30:12

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 50 to 100
  • QA Check deleted (Dev Needed)

Done! Here’s my report, skipping issues that happened less than twice on stable/devel in August:

  • a few failures to check for upgrades, most of them during release process, i.e. not really a test suite problem
  • quite a few Bug #11711 => marked as fragile
  • on devel, since feature/from-intrigeri-for-2.6 was merged: lots of issues that look like Bug #10576; I guess it’s because of the 2nd CPU given to TailsToaster i.e. Feature #6729, or simply because having USB tests running more intensively overloads lizard enough that such bugs are triggered more often; so I’ve started working on fixing the root causes of Bug #10576 (Bug #8310, Bug #8312) on feature/stretch, and might backport these changes to Tails 2.x later; nothing we can do about it on the test suite side at this point; if we still see a lot of such problems next months, we should measure if reverting Feature #6729 would fix that and whether it would cause other problems
  • some “timeout after clicking Login” (but not so much on our release branches), let’s keep this one in mind; I’m starting to think that we should save the entire systemd Journal whenever a test fails; this would help debug such problems.