Bug #11368

Check if Icinga2 perfdata are garbage collected

Added by intrigeri 2016-04-25 04:11:30 . Updated 2016-04-28 14:09:45 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Tails_2.4
Start date:
2016-04-25
Due date:
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:
268

Description

https://icingaweb2.tails.boum.org/monitoring/service/show?host=ecours.tails.boum.org&service=system_disks shows that 80% of inodes are used on ecours’ system partition. That’s because of /var/spool/icinga2/perfdata, that uses no less than 174K inodes today. Are those very many files cleaned up at some point?

Subtasks

History

#1 Updated by bertagaz 2016-04-26 03:48:55

  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 0 to 20
  • QA Check set to Info Needed

It seems that Icinga2 doesn’t clean up by itself this perfdata files, but rather expect an external software that uses it (e.g. pnp4nagios to draw perfdata graphs) to take care of that (as stated in this thread and other places).

We’re actually not using this data history, so maybe we could just deactivate the perfdata feature on our monitoring master and delete this files?

#2 Updated by intrigeri 2016-04-26 05:21:43

  • Assignee changed from intrigeri to bertagaz

Thanks for looking into it promptly!

> We’re actually not using this data history, so maybe we could just deactivate the perfdata feature on our monitoring master and delete this files?

I’ve no clue what these files can be useful for, so please either sum it up for me, or make the decision yourself :)

#3 Updated by bertagaz 2016-04-26 17:07:00

  • Status changed from Confirmed to In Progress
  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 20 to 70
  • QA Check changed from Info Needed to Ready for QA

intrigeri wrote:
> Thanks for looking into it promptly!
>
> > We’re actually not using this data history, so maybe we could just deactivate the perfdata feature on our monitoring master and delete this files?
>
> I’ve no clue what these files can be useful for, so please either sum it up for me, or make the decision yourself :)

Perfdatas are statistics collected by Icinga2 about the checks. For example for the ping checks, the perfdata collected is the response time. For the memory check, they will sum-up the total, free and cache memory. That’s the “Performance data” part you can see when you open a specific service check in Icingaweb2.

The files that are stored in /var/spool/icinga2/perfdata are the history of this perfdata. This statistic history is not used by Icinga2 itself, but it stores them so that other services can use them.

Typically that would be a service like pnp4nagios, which can draw graphs from this data. There’s a plugin to Icingaweb2 on github to integrate PNP4nagios graphs in the web interface, but it’s not packaged into Debian.

We also didn’t consider to setup graphs in icingaweb2, and it may not be interesting since we have Munin doing that.

So I removed this files, as well as disabled this Icinga2 feature. It will still be possible to re-activate it in the future anyway, so if you disagree now or later, it still can be rolled back. :)

That’s commit puppet-tails:5449457 and puppet-tails:83d4760.

#4 Updated by intrigeri 2016-04-28 14:09:45

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 70 to 100
  • QA Check changed from Ready for QA to Pass