Feature #17679
Dangling download directories from failed runs of check-mirrors
0%
Description
Suggestions:
- Ensure check-mirrors cleans up after itself [mirrors]
- Ensure the cronjob that runs check-mirrors cleans up if check-mirrors fails to do it itself [sysadmins]
- Ensure systemd tmpfiles is configured to garbage collect such leftovers after a little while [sysadmins]
Subtasks
Related issues
| Related to Tails - Bug #17702: Ensure check-mirrors.rb does not run forever | Confirmed |
History
#1 Updated by sajolida 2020-05-05 01:05:07
- Status changed from Confirmed to Needs Validation
I referred to the doc of the Ruby stdlib to try to make check-mirror cleans better after itself but failed to find a way to do so that resists a SIGKILL. So I pushed e1be807 and hopefully we won’t have to ask anything to our sysadmins before getting a working check-mirrors again on [misc].
I’ll make sure that this commits works on [misc] before asking someone else to review my approach.
#2 Updated by intrigeri 2020-05-05 08:53:20
> I referred to the doc of the Ruby stdlib to try to make check-mirror cleans better after itself but failed to find a way to do so that resists a SIGKILL.
AFAIK, nothing can resist a SIGKILL: as opposed to a SIGTERM, with SIGKILL, the kernel kills the process immediately, without giving it a chance to do such clean up operations.
I’m curious why this matters though: are there real-world cases where check-mirrors is sent a SIGKILL without having first been sent a SIGTERM?
#3 Updated by sajolida 2020-05-05 15:10:15
> nothing can resist a SIGKILL
This sounds like a quote from an action movie :)))
> as opposed to a SIGTERM, with SIGKILL, the kernel kills the process immediately, without giving it a chance to do such clean up operations.
Thanks for the additional info. I knew this already and should have done
more tests yesterday.
> I’m curious why this matters though: are there real-world cases where check-mirrors is sent a SIGKILL without having first been sent a SIGTERM?
So… I did more tests today :)
Sending SIGTERM, check-mirrors seems to always clean after itself.
I tried both syntax from
https://ruby-doc.org/stdlib-2.5.3/libdoc/tmpdir/rdoc/Dir.html#method-c-mktmpdir
(using a block and using an ‘ensure’ clause) and they seem to behave in
the same way. After all, I think that they are equivalent; at least the
doc doesn’t make is clear that one is more resilient than the other.
And of course, sending SIGKILL, check-mirrors never cleans up after himself.
So it seems like, somehow on rare occasions, check-mirrors crashes more
like from a SIGKILL and doesn’t clean after itself. Maybe if some child
process like wget or whatever behaves badly, maybe for other reasons.
My e1be807 should solve these weird cases. The only downside is that you
might not be able to run 2 check-mirrors in parallel. Good enough given
my coding skills and that check-mirrors.rb is the only place when I can
change things myself.
But first let’s see if the cron job works tonight :)
#4 Updated by sajolida 2020-05-06 14:57:56
- Assignee changed from sajolida to intrigeri
It worked!
sajolida@misc:~$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 235M 0 235M 0% /dev
tmpfs 50M 5.4M 44M 11% /run
/dev/mapper/vg1-root 6.6G 2.1G 4.2G 34% /
tmpfs 247M 0 247M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 247M 0 247M 0% /sys/fs/cgroup
/dev/vda1 228M 61M 156M 28% /boot#5 Updated by intrigeri 2020-05-08 11:23:39
- Status changed from Needs Validation to Resolved
- Assignee deleted (
intrigeri)
I like this self-healing approach! Code LGTM.
#6 Updated by intrigeri 2020-05-10 07:19:44
- related to Bug #17702: Ensure check-mirrors.rb does not run forever added