Bug #11546

Some test jobs are getting stucked in Jenkins

Added by bertagaz 2016-06-25 04:25:33 . Updated 2016-07-04 02:49:28 .

Status:
Duplicate
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
2016-06-24
Due date:
% Done:

0%

Feature Branch:
Type of work:
Research
Blueprint:

Starter:
0
Affected tool:
Deliverable for:

Description

We’ve noticed that some test jobs were taking too much time to run in Jenkins. We had to abort them in order to have more free isotesters. Symptom is that no scenarios go further than waiting for the Tails Greeter to show up, but the ISO never boot up to it, so every scenario reaches the 30 minute timeout. With every scenario taking 30 minutes to get aborted, the entire test job is taking a lot of time to run.

Booting by hand one of such ISO, it appears that the live-config boot script is stuck without any limit, so it’s blocking the boot process, and the Tails greeter never shows up.

We’ve seen that on this particular jobs:

https://jenkins.tails.boum.org/job/test_Tails_ISO_feature-10779-start-haveged-earlier/
https://jenkins.tails.boum.org/job/test_Tails_ISO_feature-10298-linux-4.x-aufs-kaslr/ (started to fail on job 16)
https://jenkins.tails.boum.org/job/test_Tails_ISO_feature-10298-linux-4.x-aufs/ (started to fail on job 22)
https://jenkins.tails.boum.org/job/test_Tails_ISO_feature-stretch/
https://jenkins.tails.boum.org/job/test_Tails_ISO_feature-8230-greeter-revamp-phase1/

The later is expected, as the test suite can’t handle the new greeter yet. The feature/stretch is WIP and always had this problem too.

Regarding the linux-4.x feature branches, this problem seems to have appeared on both of them due to a new kernel version.
Comparing the packages list for each branch of a working ISO and a faulty one, the diff shows only:

$ diff -Naur tails-i386-feature_10298-linux-4.x-aufs-2.6-20160619T2233Z-bbb7c37+devel@435cf17.iso.packages tails-i386-feature_10298-linux-4.x-aufs-2.6-20160620T2233Z-bbb7c37+devel@435cf17.iso.packages 
--- tails-i386-feature_10298-linux-4.x-aufs-2.6-20160619T2233Z-bbb7c37+devel@435cf17.iso.packages   2016-06-20 01:30:23.000000000 +0200
+++ tails-i386-feature_10298-linux-4.x-aufs-2.6-20160620T2233Z-bbb7c37+devel@435cf17.iso.packages   2016-06-21 01:29:04.000000000 +0200
@@ -1477,8 +1477,8 @@
 liferea-data   1.10.12-1
 linux-base 4.3
 linux-image-4.5.0-0.bpo.2-686  4.5.4-1~bpo8+1
-linux-image-4.5.0-0.bpo.2-amd64    4.5.4-1~bpo8+1
-linux-image-amd64  4.5+73~bpo8+1
+linux-image-4.6.0-0.bpo.1-amd64    4.6.1-1~bpo8+1
+linux-image-amd64  4.6+74~bpo8+1
 live-boot  4.0.2-1
 live-boot-initramfs-tools  4.0.2-1
 live-config    4.0.4-1

That’s the only visible difference, other than that their ISO are built from the same commit. Building at home an ISO with the 4.5.4-1 kernel and another same but with the only the newer 4.6.1-1 kernel resulted in the same problem.

It seems that the test_Tails_ISO_feature-10779-start-haveged-earlier job always exhibited this problem (and doesn’t use a 4.x kernel), for this one the root cause is probably different. This would need some more debugging though.


Subtasks


Related issues

Is duplicate of Tails - Feature #11449: Decrease boot timeout in the test suite Resolved 2016-05-19

History

#1 Updated by intrigeri 2016-07-04 02:43:37

  • related to Feature #11449: Decrease boot timeout in the test suite added

#2 Updated by intrigeri 2016-07-04 02:46:32

  • related to deleted (Feature #11449: Decrease boot timeout in the test suite)

#3 Updated by intrigeri 2016-07-04 02:46:55

  • is duplicate of Feature #11449: Decrease boot timeout in the test suite added

#4 Updated by intrigeri 2016-07-04 02:49:28

  • Status changed from Confirmed to Duplicate
  • Assignee deleted (bertagaz)
  • Target version deleted (Tails_2.5)

This has been a problem for a while, and I’ve reported it to you folks as Feature #11449. I don’t think there’s anything to do here except solve Feature #11449. Then we’ll see if there’s still another problem to solve.