Bug #17678

Test suite: aborted builds show no summary

Added by CyrilBrulebois 2020-05-05 00:38:31 . Updated 2020-05-07 08:47:01 .

Status:
Confirmed
Priority:
Normal
Assignee:
Category:
Continuous Integration
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Possibly due to ongoing networking issues, this build was aborted: https://jenkins.tails.boum.org/job/test_Tails_ISO_stable/2324/

Here’s the tail of the log:

@product
Feature: Browsing the web using the Tor Browser
  As a Tails user
  when I browse the web using the Tor Browser
  all Internet traffic should flow only through Tor

  Scenario: The Tor Browser cannot access the LAN                                  # features/torified_browsing.feature:7
    Given I have started Tails from DVD and logged in and the network is connected # features/step_definitions/snapshots.rb:170
    And a web server is running on the LAN                                         # features/step_definitions/common_steps.rb:834
    And I capture all network traffic                                              # features/step_definitions/common_steps.rb:107
Build timed out (after 60 minutes). Marking the build as failed.
Build was aborted
[PostBuildScript] - [INFO] Executing post build scripts.
Returning node parameter for isotester3
Triggering projects: keep_node_busy_during_cleanup
[test_Tails_ISO_stable] $ /bin/sh -xe /tmp/jenkins8434305340889444262.sh
+ sleep 30
[test_Tails_ISO_stable] $ /bin/sh -xe /tmp/jenkins7792186801622141263.sh
+ /usr/local/bin/post_test_cleanup /var/lib/jenkins/workspace/test_Tails_ISO_stable /tmp/TailsToaster
+ WORKSPACE=/var/lib/jenkins/workspace/test_Tails_ISO_stable
+ TEMP_DIR=/tmp/TailsToaster
+ [ ! -z /var/lib/jenkins/workspace/test_Tails_ISO_stable ]
+ [ ! -z /tmp/TailsToaster ]
+ [ -d /var/lib/jenkins/workspace/test_Tails_ISO_stable ]
+ [ -d /tmp/TailsToaster ]
+ sudo -n chown -R jenkins /tmp/TailsToaster/TailsToasterStorage /tmp/TailsToaster/already_run /tmp/TailsToaster/chutney-data /tmp/TailsToaster/debug_log_pseudo_fifo /tmp/TailsToaster/last_opencv_match.png /tmp/TailsToaster/no-network-logged-in-snapshot.memstate /tmp/TailsToaster/pulse-PKdhtXMmr18n /tmp/TailsToaster/run-2020-05-04_17:45:38_+0000-git_4.6_67f5651-tz0JkB /tmp/TailsToaster/sniffer.pcap /tmp/TailsToaster/tails-greeter-snapshot.memstate /tmp/TailsToaster/with-network-logged-in-snapshot.memstate
+ mkdir -p /var/lib/jenkins/workspace/test_Tails_ISO_stable/build-artifacts
+ find /tmp/TailsToaster ( -type d -a -name sikuli_candidates ) -print0
+ xargs --null --no-run-if-empty mv --target-directory=/var/lib/jenkins/workspace/test_Tails_ISO_stable/build-artifacts/
+ + xargs --null --no-run-if-empty cp -r --target-directory=/var/lib/jenkins/workspace/test_Tails_ISO_stable/build-artifacts/
find /tmp/TailsToaster ( -name *.png -o -name *.mkv -o -name *.pcap -o -name debug.log -o -name cucumber.json -o -name *_chutney-data -o -name *.htpdate -o -name *.journal -o -name *.tor ) -print0
+ sudo -n rm -rf /tmp/TailsToaster/TailsToasterStorage /tmp/TailsToaster/already_run /tmp/TailsToaster/chutney-data /tmp/TailsToaster/debug_log_pseudo_fifo /tmp/TailsToaster/last_opencv_match.png /tmp/TailsToaster/no-network-logged-in-snapshot.memstate /tmp/TailsToaster/pulse-PKdhtXMmr18n /tmp/TailsToaster/run-2020-05-04_17:45:38_+0000-git_4.6_67f5651-tz0JkB /tmp/TailsToaster/sniffer.pcap /tmp/TailsToaster/tails-greeter-snapshot.memstate /tmp/TailsToaster/with-network-logged-in-snapshot.memstate
Archiving artifacts
[CucumberReport] Using Cucumber Reports version 4.10.0
[CucumberReport] JSON report directory is "build-artifacts"
[CucumberReport] Copied 1 json files from workspace "/var/lib/jenkins/workspace/test_Tails_ISO_stable/build-artifacts" to reports directory "/var/lib/jenkins/jobs/test_Tails_ISO_stable/builds/2324/cucumber-html-reports/.cache"
[CucumberReport] Copied 0 properties files from workspace "/var/lib/jenkins/workspace/test_Tails_ISO_stable/build-artifacts" to reports directory "/var/lib/jenkins/jobs/test_Tails_ISO_stable/builds/2324/cucumber-html-reports/.cache"
[CucumberReport] Processing 1 json files:
[CucumberReport] /var/lib/jenkins/jobs/test_Tails_ISO_stable/builds/2324/cucumber-html-reports/.cache/cucumber.json
[CucumberReport] Missing report result - report was not successfully completed
[CucumberReport] Build status is left unchanged
Sending e-mails to: tails-rm@boum.org
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] done
Triggering a new build of reboot_node
Finished: FAILURE

It’s a little sad to see no stats regarding how many tests were actually run, and how many succeeded/failed. Meaning one gets to dive into/grep the logs to figure that out.

Depending on how many times we’re going to experience such issues, it might be nice to try and see if we can get some stats when exiting after such a timeout.


Subtasks


History

#1 Updated by intrigeri 2020-05-07 08:40:01

  • Category changed from Test suite to Continuous Integration

Triaging this:

  • “Build timed out (after 60 minutes)” is a Jenkins setting so adjust Category.
  • I don’t know if there’s a way for an aborted cucumber process to report about scenarios that did run already. Best case, it can report that info if killed nicely (SIGTERM) and perhaps the Jenkins build timeout thing could be adjusted to kill the process more nicely. Worst case, cucumber lacks this feature entirely and then it’s a harder problem to solve. In other words, someone needs to figure out if this would be better solved on the sysadmin side or on the test suite side. I don’t know who is responsible to do that and I’m slightly worried that developers and sysadmins could equally enter a deadlock, waiting for each other.

#2 Updated by intrigeri 2020-05-07 08:47:01

Silly me. Now sending to @anonym for triaging.