Feature #6216

Email notification on failed Jenkins build

Added by intrigeri 2013-08-07 13:03:11 . Updated 2015-06-19 04:24:21 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Continuous Integration
Target version:
Start date:
2013-08-07
Due date:
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
0
Affected tool:
Deliverable for:

Description


Subtasks


Related issues

Related to Tails - Bug #9660: Jenkins notifications doesn't cut our needs Resolved 2015-06-30

History

#1 Updated by Dr_Whax 2014-04-09 18:23:33

It seems that a plugin is needed in order to achieve this. A quick search pointed me to “email-ext”[1].

Email-ext about itself:

> This plugin extends Jenkins built in email notification functionality by giving you more control. It provides customization of 3 areas.
>
> Triggers - Select the conditions that should cause an email notification to be sent.
> Content - Specify the content of each triggered email’s subject and body.
> Recipients - Specify who should receive an email when it is triggered.

[1] https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin

Is this something we consider handy or is more information needed?

#2 Updated by intrigeri 2014-04-09 19:01:45

I would use whatever Holger picked for jenkins.debian.net: the INSTALL file in http://anonscm.debian.org/gitweb/?p=qa/jenkins.debian.net.git;a=summary says “The Jenkins EMail notification plugin is used”. Is it the same? If unsure, ask holger@debian.org :)

#3 Updated by Dr_Whax 2014-04-09 20:58:41

I asked Holger. He uses this plugin, which is a different one I previously listed: http://wiki.jenkins-ci.org/display/JENKINS/Mailer

He uses this plugin to also send irc notifications from the emails, apparently, it works nicer than kgb-bot. More scripts can be seen at: git://git.debian.org/git/qa/jenkins.debian.net.git

see bin/email2irc.sh
see bin/email2irc.sh and ./procmailrc

#4 Updated by intrigeri 2014-12-19 13:21:12

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

tails-dev@ is now notified on failed ISO builds, and -l10n@ notified when the PO check on the master branch fails. Let’s see how it goes.

#5 Updated by BitingBird 2015-01-04 03:28:57

I think we could mark this as resolved ?

#6 Updated by intrigeri 2015-01-04 12:41:03

  • blocked by Feature #8482: Survey usual committers about dropping the collective pseudonym for signing commits added

#7 Updated by intrigeri 2015-01-04 12:41:37

> I think we could mark this as resolved ?

No, the current setup is only a preliminary experiment. Now:

  • let’s wait a month or two, and see if tails-dev@ subscribers complain;
  • most likely, we’ll want more fine-grained notifications at some point (e.g. reporting to the person who broke stuff), as discussed on tails-dev@.

#8 Updated by intrigeri 2015-05-31 13:57:15

  • blocks deleted (Feature #8482: Survey usual committers about dropping the collective pseudonym for signing commits)

#9 Updated by intrigeri 2015-05-31 13:57:34

  • Assignee set to bertagaz
  • Target version set to Tails_1.4.1
  • Parent task set to Feature #6196

Re-parenting so that the info here is easily available to bertagaz when he works on Feature #6196.

#10 Updated by intrigeri 2015-06-12 20:09:47

https://gitweb.torproject.org/project/jenkins/jobs.git/tree/jobs.yaml might help: it has a quite extensive config for the email-ext plugin.

#11 Updated by bertagaz 2015-06-17 04:07:25

I’m not sure there really is a problem here. We’re not using the email-ext plugin, but the Mailer plugin (used also on jenkins.debian.net) shipped with the Debian package.

We also hardcode the email that should be notified in our jobs configuration, so for automatic builds the notification on failure should be sent to the right recipient.

I’m not sure for the build triggered through the Jenkins web interface though.

We’ll see once the automatic build feature is deployed I think, this needs some live testing.

#12 Updated by intrigeri 2015-06-18 03:39:21

> I’m not sure there really is a problem here. […] We also hardcode the email that should be notified in our jobs configuration, so for automatic builds the notification on failure should be sent to the right recipient.

That’s thanks to this code snippet, right?

        elif branch['job_recipients'] == 'whoever_broke_the_build':
            self.job['publishers'].extend([
                {'email': {'recipients': '', 'sent-to-individuals': True}}
            ])

Was this tested and confirmed to do the right thing (as specified in the blueprint)?
(If not, please do test it, before we start spamming tons of people, in a way that should email you only if it works fine :)

#13 Updated by bertagaz 2015-06-18 05:23:45

intrigeri wrote:
> > I’m not sure there really is a problem here. […] We also hardcode the email that should be notified in our jobs configuration, so for automatic builds the notification on failure should be sent to the right recipient.
>
> That’s thanks to this code snippet, right?

Yes.

> […]
>
> Was this tested and confirmed to do the right thing (as specified in the blueprint)?
> (If not, please do test it, before we start spamming tons of people, in a way that should email you only if it works fine :)

Tested in build 148 of the build_Tails_ISO_feature-8415-overlayfs job.
As you can see in the logs after setting the job accordingly and introducing a build failure, I confirm I did get the notification in my mailbox.

#14 Updated by intrigeri 2015-06-18 07:09:41

  • % Done changed from 10 to 50
  • QA Check set to Dev Needed

OK, great! Perhaps enable those settings on all jobs now, so that it is tested a bit more before we deploy Feature #6196? Once this change is done, feel free to call this ticket resolved.

#15 Updated by bertagaz 2015-06-18 12:30:00

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

I’ve not enabled it for all jobs, I’ve updated the jobs config to reflect what would our setup be. Does it sound relevant?

#16 Updated by intrigeri 2015-06-19 01:56:37

  • Assignee changed from intrigeri to bertagaz
  • QA Check changed from Ready for QA to Info Needed

> I’ve not enabled it for all jobs, I’ve updated the jobs config to reflect what would our setup be. Does it sound relevant?

I’m sorry, I don’t understand what you mean. I see no send-to-individuals anywhere in our jenkins-jobs repo. May you please rephrase what exactly you did, and what I should check?

#17 Updated by bertagaz 2015-06-19 03:01:02

  • Assignee changed from bertagaz to intrigeri
  • QA Check changed from Info Needed to Ready for QA

Oops, my bad, forgot to push it. Done now, please have a look.

#18 Updated by intrigeri 2015-06-19 04:23:10

  • Status changed from In Progress to Resolved
  • % Done changed from 50 to 100

> Done now, please have a look.

Looks good. Added commit 1502146 on top to fix a mistake in there, please have a look.
Calling this done, we’ll file dedicated and more focused tickets if things go wrong.

#19 Updated by intrigeri 2015-06-19 04:24:21

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

#20 Updated by intrigeri 2015-06-30 00:33:29

  • related to Bug #9660: Jenkins notifications doesn't cut our needs added