Feature #11480

Monitoring: check that /var/mail/ is empty

Added by intrigeri 2016-05-24 09:22:48 . Updated 2018-08-17 15:45:56 .

Status:
Confirmed
Priority:
Normal
Assignee:
georg
Category:
Infrastructure
Target version:
Start date:
2016-05-24
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
1
Affected tool:
Deliverable for:

Description

We constantly forget to add email redirects for new users we create, e.g. those that run cronjobs. So their mail lands into /var/mail/. And then, we have two problems: we don’t learn about problems, and /var/mail can use lots of disk space. I’ve just fixed a few such problems again, and I’ve grown tired of constantly catching them after the fact. IMO we should add a check that monitors that /var/mail/ is empty (or alternatively, configure Postfix to send all email to root, unless they are sent to an alias that has a fully-qualified domain part).

This does require a bit of shell (or whatever) scripting knowledge, and reading the documentation about adding new Icinga2 checks in our infrastructure. The Puppet part can be handled by our sysadmin team if this is a blocker for a contributor to work on that ticket.


Subtasks


History

#1 Updated by bertagaz 2016-08-18 08:53:28

  • Starter set to Yes

#2 Updated by bertagaz 2016-08-18 09:00:18

This does require a bit of shell (or whatever) scripting knowledge, and reading the documentation about adding new Icinga2 checks in our infrastructure. The puppet part can be handle by our sysadmin team if this is a blocker for a contributor to work on that ticket.

#3 Updated by intrigeri 2016-08-18 09:01:31

  • Description updated

#4 Updated by georg 2016-09-07 00:38:56

  • Assignee set to georg

#5 Updated by georg 2016-09-07 00:40:29

I will do it. Do you prefer a shell / bash or python or … script, or is it up to me?

#6 Updated by emmapeel 2016-09-07 01:28:08

Python!

#7 Updated by georg 2016-09-07 02:07:14

While thinking about this a bit more, and being unsure how regularly this problem occurs: Only checking that /var/mail/ is empty is just dealing with the symptoms, because then one has to manually (?) solve this. So, in my opinion, both proposals (script to check that /var/mail/ is empty and configure Postfix to deliver all mails which aren’t sent to aliases with a fully-qualified-domain part to root) should be implemented. Any opinions on this?

#8 Updated by bertagaz 2016-09-07 04:14:18

georg wrote:
> While thinking about this a bit more, and being unsure how regularly this problem occurs: Only checking that /var/mail/ is empty is just dealing with the symptoms, because then one has to manually (?) solve this. So, in my opinion, both proposals (script to check that /var/mail/ is empty and configure Postfix to deliver all mails which aren’t sent to aliases with a fully-qualified-domain part to root) should be implemented. Any opinions on this?

This problem occurs from time to time, it’s not mailbombing us that much. It’s mainly happening when a cronjob ran with a user that doesn’t have a correct email alias fails, e.g on unusual occasions like a VM being rebooted.

I agree with your conclusion, though if Postfix is setup to deliver all such kind of emails, it’s likely that the /var/mail/ filling problem will disappear. But if you’re motivated to implement this check, we won’t refrain you. So far none of us have commited to implement the Postfix catch all configuration any time soon. :)

Regarding the language, probably the most important is using one you feel at ease to write good enough code with (and we feel at ease enough to review). Shell or python are fine on our side.

#9 Updated by intrigeri 2017-03-22 09:14:27

Hi georg! Are you still interested in working on this? Do you need more info?

#10 Updated by georg 2017-04-24 11:16:23

> Hi georg! Are you still interested in working on this? Do you need
> more info?

Yes, absolutely! Sorry for the delay, quite busy with work etc.

Before writing new code, I’ve searched trough the Internets if there was
a plugin available, and I found one at [1]. This is written in Perl,
which wouldn’t have been my choice, but anyway, it works as expected.

It needs the libnagios-plugin-perl package.

PoC:

$ ls -al /tmp/foo/
total 16
drwxr-xr-x 2 georg georg 4096 Apr 24 11:19 .
drwxrwxrwt 18 root root 12288 Apr 24 11:19 ..

$ ./check_dir -d /tmp/foo -w 0 -c 0
CHECK_DIR OK - /tmp/foo=0 | /tmp/foo=0;0;0

If this looks like it could work for you, I could add a corresponding
icinga2 command and service code.

Awaiting feedback,
all the best,
Georg

[1] https://raw.githubusercontent.com/skyscrapers/monitoring-plugins/master/check_dir

#11 Updated by intrigeri 2017-04-29 11:16:36

> It needs the libnagios-plugin-perl package.

… that was removed from Debian as it’s been superseded by another library.
So we need a plugin that uses the new library instead.

> If this looks like it could work for you, I could add a corresponding icinga2 command and service code.

Sounds good, assuming there’s a plugin that doesn’t rely on obsolete libraries :)

#12 Updated by georg 2017-11-08 09:09:43

Sorry again for the delay, just wanted to check quickly: Would it still be nice to have this done? If so, I’ll do it during the upcoming week.

#13 Updated by intrigeri 2017-11-08 10:06:23

> Sorry again for the delay, just wanted to check quickly: Would it still be nice to have this done? If so, I’ll do it during the upcoming week.

Yes, please go ahead :)

#14 Updated by georg 2017-11-20 12:36:58

Sorry for the delay: I had to recover from a surgery, which took longer than expected.

New ETA: End of this week.

#15 Updated by intrigeri 2017-11-20 17:54:01

> Sorry for the delay: I had to recover from a surgery, which took longer than expected.

Take care!

#16 Updated by Anonymous 2018-08-17 15:45:56

Hi!

is this still on your plate? If not feel free to unassign yourself from this ticket.