Feature #10445

Pass --artifacts-base-uri in Jenkins test suite wrapper

Added by bertagaz 2015-10-28 10:15:57 . Updated 2015-11-05 04:15:38 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Continuous Integration
Target version:
Tails_1.8
Start date:
2015-10-28
Due date:
% Done:

100%

Feature Branch:
puppet-tails:feature/10445-pass-artifacts-base-url-to-jenkins
Type of work:
Code
Blueprint:

Starter:
0
Affected tool:
Deliverable for:
268

Description

With Feature #10151, a new artifacts-base-url option has been implemented, so that when a scenario fails its artifacts location are nicely printed in the logs.

We should set this option in the test suite wrapper in Jenkins as explained in Feature #10151#note-11.

Subtasks

Related issues

Blocked by Tails - Feature #10151: Organize automated test suite artifacts in per-run directories Resolved 2015-09-02

History

#1 Updated by bertagaz 2015-10-28 10:16:29

  • blocked by Feature #10151: Organize automated test suite artifacts in per-run directories added

#2 Updated by bertagaz 2015-10-28 10:17:15

#3 Updated by intrigeri 2015-10-31 05:35:09

  • blocks #8668 added

#4 Updated by intrigeri 2015-10-31 05:35:25

  • Description updated

#5 Updated by intrigeri 2015-10-31 05:37:21

  • Subject changed from Configure artifacts-base-url in Jenkins test suite wrapper to Pass --artifacts-base-uri in Jenkins test suite wrapper

#6 Updated by bertagaz 2015-11-04 02:36:12

  • Status changed from Confirmed to In Progress
  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 0 to 80
  • QA Check set to Ready for QA
  • Feature Branch set to puppet-tails:feature/10445-pass-artifacts-base-url-to-jenkins

bertagaz wrote:
> With Feature #10151, a new artifacts-base-url option has been implemented, so that when a scenario fails its artifacts location are nicely printed in the logs.
>
> We should set this option in the test suite wrapper in Jenkins as explained in Feature #10151#note-11.

Merged and deployed the feature branch. A first job is already running with this option activated. But it’s running fine until now… :)

#7 Updated by bertagaz 2015-11-04 07:28:59

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 80 to 100
  • QA Check deleted (Ready for QA)

A bit bold maybe, but the test job is over, and the links to capture files in the logs works, so I say myself that it works. :)

#8 Updated by intrigeri 2015-11-05 02:53:49

Hi,

(please drop the ticket from Cc on next reply.)

> Status changed from In Progress to Resolved

> A bit bold maybe, but the test job is over, and the links to capture files in the logs works, so I say myself that it works. :)

> Feature Feature #10445: Pass —artifacts-base-uri in Jenkins test suite wrapper
> https://labs.riseup.net/code/issues/10445#change-48069

Thanks! Indeed I see links there:
https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/38/console

But these links are not where I would personally love to see them:
https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/38/cucumberTestReport/i2p/connecting-to-the-_i2p-irc-channel-with-the-pre-configured-account/

That is, if I want to understand a test failure, likely I’ll use
the “Cucumber Test Results” page instead of the raw console output.

Dear CI team, is that easy to fix?

Cheers!

#9 Updated by bertagaz 2015-11-05 03:59:42

intrigeri wrote:
> Thanks! Indeed I see links there:
> https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/38/console
>
> But these links are not where I would personally love to see them:
> https://jenkins.tails.boum.org/job/test_Tails_ISO_experimental/38/cucumberTestReport/i2p/connecting-to-the-_i2p-irc-channel-with-the-pre-configured-account/
>
> That is, if I want to understand a test failure, likely I’ll use
> the “Cucumber Test Results” page instead of the raw console output.

I think I have not been put in the email at first, but here’s my 2 cents: I’m not sure that when you’re looking at a failure in the log of the tests (where this links are) clicking on a link to the cucumber report page will give more information about that failure, as you already have the trace in sight. Having a direct link to the artifacts related to that failure you’re looking at is maybe more relevant?

#10 Updated by intrigeri 2015-11-05 04:15:38

Hi,

[re-adding -ci@ to the Cc list, please drop it on next reply.]

> I think I have not been put in the email at first,

See your mail from the CI team list :)

> I’m not sure that when you’re looking at a failure in the log of the tests (where this links are) clicking on a link to the cucumber report page will give more information about that failure, as you already have the trace in sight.

I agree.

To clarify my usecase, I was talking of looking at a failed test job from Jenkins interface, that is:

1. I’m on https://jenkins.tails.boum.org/view/RM/
2. I see a test run failed, e.g. test_Tails_ISO_stable so I hover its “Last failure” column and click “Cucumber Test Result” (since that’s the nicest interface I know of to analyze such problems).
3. I land on https://jenkins.tails.boum.org/view/RM/job/test_Tails_ISO_stable/lastFailedBuild/cucumberTestReport/.
4. I click on one of the failures and land on https://jenkins.tails.boum.org/view/RM/job/test_Tails_ISO_stable/lastFailedBuild/cucumberTestReport/using-totem/watching-mp4-videos-stored-on-the-persistent-volume-should-work-as-expected-given-our-apparmor-confinement/

=> I don’t have direct access to the artifacts.

> Having a direct link to the artifacts related to that failure you’re looking at is maybe more relevant?

Yes!