Feature #10382

Implement the specified notification system for test suite failures on Jenkins

Added by intrigeri 2015-10-16 03:03:16 . Updated 2015-12-20 03:54:12 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Continuous Integration
Target version:
Start date:
2015-10-16
Due date:
% Done:

100%

Starter:
Affected tool:
Deliverable for:
267

Description

This is what follow from Feature #10296 once we deem it’s time to go ahead.


Subtasks


Related issues

Blocked by Tails - Feature #10296: Reevaluate how the email notifications for failed automatic tests scale Resolved 2015-09-28

History

#1 Updated by intrigeri 2015-10-16 03:03:25

#2 Updated by intrigeri 2015-10-16 03:03:34

  • Tracker changed from Bug to Feature

#3 Updated by intrigeri 2015-10-16 03:03:43

  • blocks #8668 added

#4 Updated by intrigeri 2015-10-16 03:03:53

  • blocked by Feature #10296: Reevaluate how the email notifications for failed automatic tests scale added

#5 Updated by intrigeri 2015-10-16 03:10:13

  • Blueprint set to https://tails.boum.org/blueprint/automated_builds_and_tests/automated_tests_specs/

#6 Updated by bertagaz 2015-10-16 09:04:30

FYI this will be achieved by reverting commit 6dc5489 from the puppet-tails repo.

Then the test jobs in our JJB repo will still have the email notification activated, but the generation script will output only one email to be notified for the build and test jobs, which will be the same that we set up for the build jobs (tails-rm or last commiter).

#7 Updated by bertagaz 2015-11-03 02:25:39

  • Priority changed from Normal to Elevated
  • Target version changed from Tails_1.7 to Tails_1.8

Postponing and raising priority. Will work on that on the coming days.

#8 Updated by intrigeri 2015-11-06 07:32:25

  • Deliverable for changed from 268 to 267

#9 Updated by intrigeri 2015-12-05 16:06:40

IMO this ticket should be marked as blocked by… whatever blocks it. Feature #10296 is useful in itself, but it’s simply too vague to express what we have to do before we can turn on the big red switch. I guess most of the blockers are subtasks of Bug #10288. Once they’re marked as blocking this ticket, then it’ll be easier to sort them apart of the mass of Bug #10288 subtasks, which will help a lot anonym and kytv organize their work :)

#10 Updated by intrigeri 2015-12-06 14:36:15

intrigeri wrote:
> IMO this ticket should be marked as blocked by… whatever blocks it. […] I guess most of the blockers are subtasks of Bug #10288.

For what it’s worth, I’ve had a quick look and I’m not so sure: the concerning amount of jobs that are aborted due to inactivity seems to be the top reason of non-successful test suite runs (out of the last 10 runs of test_Tails_ISO_devel, 6 aborted that way, 2 were succesful, and 2 failed; on stable it’s “only” 3 aborted jobs out of 10).

I don’t know if email notifications go out for aborted jobs, so I don’t know if/how this bug is blocking this very ticket, or rather Feature #5288. bertagaz?

In any case, as requested a month ago (Bug #9157#note-18): the aborted jobs bug seems to be very relevant to making Feature #5288 deliver what it promised, so it’s a pretty serious one, please track it with a ticket; its exact initial priority/timeline is a factor of the above question.

#11 Updated by intrigeri 2015-12-06 15:16:09

(Created Bug #10717, gonna post some research there soon.)

#12 Updated by bertagaz 2015-12-15 03:37:41

  • Target version changed from Tails_1.8 to Tails_2.0

Postponing

#13 Updated by intrigeri 2015-12-18 09:11:18

Given Bug #10502#note-16 and Bug #10717#note-6, I think we should now go ahead and enable notifications for non-stable branches. We’ll do that on Tuesday unless there are objections, or earlier if e.g. anonym and bertagaz say they agree.

#14 Updated by anonym 2015-12-18 11:45:13

intrigeri wrote:
> Given Bug #10502#note-16 and Bug #10717#note-6, I think we should now go ahead and enable notifications for non-stable branches.

+1

#15 Updated by intrigeri 2015-12-19 07:53:23

  • Status changed from Confirmed to In Progress
  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA

Deployed, we’ll see in a few days how it goes.

#16 Updated by bertagaz 2015-12-20 01:49:39

You’ll see that there was a tiny bug like in test_Tails_ISO_web-10400-farsi-only #4 :

06:50:16 Sending e-mails to: $NOTIFY_TO
06:50:16 ERROR: Invalid Addresses

I’ve fixed that in commit 7170700 of the puppet-tails repo.

I’ve started a new build to test it: https://jenkins.tails.boum.org/job/build_Tails_ISO_web-10400-farsi-only/6/

#17 Updated by intrigeri 2015-12-20 02:12:05

> I’ve fixed that in commit 7170700 of the puppet-tails repo.

Looks OK, let’s see how it goes :)

#18 Updated by intrigeri 2015-12-20 03:54:12

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

Works!