Feature #14828

Notify LimeSurvey admins when a security update is available

Added by intrigeri 2017-10-11 08:35:39 . Updated 2017-11-21 14:08:57 .

Status:
Resolved
Priority:
High
Assignee:
Category:
Infrastructure
Target version:
Tails_3.5
Start date:
2017-10-11
Due date:
% Done:

50%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:
299

Description

Ensure the LimeSurvey service admins are notified, either automatically or by the sysadmin on duty (who shall learn about this automatically), when a LimeSurvey security update is released.

Bonus points if the update has been downloaded (in a somewhat secure way) on the VM already, and is waiting for the service administrator to apply it.

Notification over email is OK, via Redmine is better if it fits in the budget.

Subtasks

History

#1 Updated by intrigeri 2017-10-11 08:38:15

  • Description updated

#2 Updated by intrigeri 2017-10-11 09:32:05

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

To check for updates I’ll use Git. The content of the tarballs listed on https://github.com/LimeSurvey/LimeSurvey/releases is automatically generated from tags by GitHub, so basically I need to:

  1. git fetch
  2. check the release notes shipped in the last upstream released version (git show $(git describe --abbrev=0 --tags master):docs/release_notes.txt) for security updates in releases newer than the last one we’ve checked
  3. if any security update is found, notify the service admins
  4. save the output of git describe --abbrev=0 --tags master to a file so we can retrieve “the last one we’ve checked” from there next time we go through this process

This algorithm will avoid notifying multiple times for the same update.

And then we can actually use this local Git repo as the upstream for our LimeSurvey installation:

  • initially our LimeSurvey installation will be cloned from that local Git repo
  • the service admins can fetch/merge from it into the production Git repo in order to upgrade the production webapp => no tarball involved :)

I’ve set up this Git repo and initialized a production clone that can be used to run the webapp.

#3 Updated by intrigeri 2017-10-13 09:55:45

  • % Done changed from 10 to 20

Implemented a first version that should be suitable for running via cron, which will email whenever there’s something worth notifying about. I’ll deploy this and then, depending on the remaining time, I might add support for notifying via Redmine.

#4 Updated by sajolida 2017-10-13 19:22:21

Knowing me, I’d like to keep the email notification even after we have Redmine notification. I mean… if it can come for free not to remove the email notification.

#5 Updated by sajolida 2017-10-16 14:07:28

#6 Updated by intrigeri 2017-10-17 16:05:10

  • Assignee changed from intrigeri to sajolida
  • % Done changed from 20 to 50
  • QA Check set to Ready for QA

Deployed (with email notification by cron only).

In order to test the whole thing, I’ve deleted the recent tags in the local Git clone (~limesurvey/git) so within 6 hours you should get an email notification about 2.66.0 (and maybe another version) fixing security issues, and then nothing anymore about it. Please check that you indeed receive this email and that it’s understandable. Then close this ticket as resolved or tell me what’s wrong :)

I’ve configured the cronjob this way: https://git-tails.immerda.ch/puppet-tails/tree/manifests/limesurvey/releases_monitor.pp#n49. Patches welcome if you prefer a different check frequency. The only downside of checking more often is noise from cron when our server can’t reach GitHub or similar. The obvious upside of checking often is that you learn faster about security issues. I’ve set it to 4 times a day which is consistent with how we handle similar things everywhere else on our infra.

Whenever you receive one such email, the corresponding commits & tag have already been fetched in the ~limesurvey/git bare repo, so you can get a shell as www-data, then cd ~/limesurvey && git fetch and then merge whatever new tag you want to trigger the upgrade. I think I’ve already told you this somewhere else, but I’ll let you document this operating procedure somewhere to ensure you (or future LimeSurvey service admins) don’t have to ask me again :)

#7 Updated by sajolida 2017-10-26 12:52:54

  • Assignee changed from sajolida to intrigeri
  • QA Check changed from Ready for QA to Info Needed

> […] within 6 hours you should get an email notification about 2.66.0 (and maybe another version) fixing security issues, and then nothing anymore about it. Please check that you indeed receive this email and that it’s understandable. Then close this ticket as resolved or tell me what’s wrong :)

I never received that email I think. Any clue on how to spot it? From? Subject?

Also, from their release notes I see no security update since 2.67.2. The version in ~limesurvey today was 2.72.0+171010 I applied it anyway but do you consider that it was a security update?

> I’ve configured the cronjob this way: https://git-tails.immerda.ch/puppet-tails/tree/manifests/limesurvey/releases_monitor.pp#n49. Patches welcome if you prefer a different check frequency. The only downside of checking more often is noise from cron when our server can’t reach GitHub or similar. The obvious upside of checking often is that you learn faster about security issues. I’ve set it to 4 times a day which is consistent with how we handle similar things everywhere else on our infra.

4 times a day is fine.

> Whenever you receive one such email, the corresponding commits & tag have already been fetched in the limesurvey/git bare repo, so you can get a shell as www-data, then cd/limesurvey && git fetch and then merge whatever new tag you want to trigger the upgrade.

Tried all that and it works fine! Super easy!

> I think I’ve already told you this somewhere else, but I’ll let you document this operating procedure somewhere to ensure you (or future LimeSurvey service admins) don’t have to ask me again :)

→ Feature #14481

#8 Updated by intrigeri 2017-10-29 07:42:55

>> […] within 6 hours you should get an email notification about 2.66.0 (and maybe another version) fixing security issues, and then nothing anymore about it. Please check that you indeed receive this email and that it’s understandable. Then close this ticket as resolved or tell me what’s wrong :)

> I never received that email I think.

Weird, I’m 90% sure I had checked the logs to ensure this email had been delivered. Sadly the corresponding logs are gone and I can’t go back in past.

> Any clue on how to spot it? From? Subject?

It’s sent by cron. I’m not sure about the From/Subject so I’ll retry: I’ve just deleted the recent tags again and once I see the email has been delivered (this should happen today at 12:27 UTC) I’ll give you some more info.

> Also, from their release notes I see no security update since 2.67.2.

Correct.

> The version in ~limesurvey today was 2.72.0+171010 I applied it anyway but do you consider that it was a security update?

I see no hint about any security update since 2.67.2.

>> Whenever you receive one such email, the corresponding commits & tag have already been fetched in the limesurvey/git bare repo, so you can get a shell as www-data, then cd/limesurvey && git fetch and then merge whatever new tag you want to trigger the upgrade.

> Tried all that and it works fine! Super easy!

Excellent, thanks :)

#9 Updated by intrigeri 2017-10-29 16:51:47

intrigeri wrote:
> >> […] within 6 hours you should get an email notification about 2.66.0 (and maybe another version) fixing security issues, and then nothing anymore about it. Please check that you indeed receive this email and that it’s understandable. Then close this ticket as resolved or tell me what’s wrong :)
>
> > I never received that email I think.
>
> Weird, I’m 90% sure I had checked the logs to ensure this email had been delivered. Sadly the corresponding logs are gone and I can’t go back in past.

My testing protocol was buggy. Found the issue and started a new test.

#10 Updated by intrigeri 2017-10-29 18:44:32

This time the cronjob tried to send the email (as expected) but it got rejected on your side because its sender address was invalid. Fixed that on my side, retrying.

#11 Updated by intrigeri 2017-10-29 18:49:56

BTW the email should have From: limesurveylizard.tails.boum.org@.

#12 Updated by intrigeri 2017-10-30 08:01:28

  • Assignee changed from intrigeri to sajolida
  • QA Check changed from Info Needed to Ready for QA

You should have received a mail this time:

Oct 30 00:27:01 survey CRON[12613]: (limesurvey) CMD (/var/lib/limesurvey/monitor-releases/git/bin/tails-monitor-limesurvey-releases --git-dir=/var/lib/limesurvey/git --last-checked-cache-file=/var/lib/limesurvey/monitor-releases/last_checked_tag)
Oct 30 00:27:01 survey CRON[12612]: pam_unix(cron:session): session closed for user limesurvey
Oct 30 00:27:01 survey postfix/pickup[10820]: DBA2B435CD: uid=999 from=<limesurvey>
Oct 30 00:27:01 survey postfix/cleanup[12638]: DBA2B435CD: message-id=<20171030002701.DBA2B435CD@lizard.tails.boum.org>
Oct 30 00:27:01 survey postfix/qmgr[1851]: DBA2B435CD: from=<limesurvey@lizard.tails.boum.org>, size=1099, nrcpt=1 (queue active)
Oct 30 00:27:03 survey postfix/smtp[12640]: DBA2B435CD: to=<sajolida@pimienta.org>, relay=mail.poivron.org[91.194.60.101]:25, delay=1.1, delays=0.03/0.01/0.45/0.63, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 10896C04496)
Oct 30 00:27:03 survey postfix/qmgr[1851]: DBA2B435CD: removed

#13 Updated by sajolida 2017-11-14 13:19:29

  • Target version changed from Tails_3.3 to Tails_3.5

#14 Updated by sajolida 2017-11-21 14:08:58

  • Status changed from In Progress to Resolved
  • Assignee deleted (sajolida)
  • QA Check deleted (Ready for QA)

I got one email on October 30 and a second one on November 11.

Both had no security issues to fix (which I also checked on the release notes).

So I think we’re good and I can close this ticket!

Thanks a lot for the nice notification and update mechanism: I like it!