apt-cacher-ng expiration cronjob fails on apt-proxy.lizard
Since July 02, the cronjob running apt-cacher-ng’s expiration script run daily by cron reports failure to the sysadmins of this kind:
/etc/cron.daily/apt-cacher-ng Error: cannot fetch http://localhost:3142/acng-report.html?doExpire=Start+Expiration&abortOnErrors=aOe, HTTP/1.1 500 Connection timeout
Blocks Tails -
#4 Updated by intrigeri 2017-07-06 15:42:09
- Subject changed from Apt-cacher-ng expiration cronjob fails on apt-proxy.lizard to apt-cacher-ng expiration cronjob fails on apt-proxy.lizard
- Status changed from Confirmed to In Progress
- % Done changed from 0 to 10
The cronjob seems to have worked fine today according to
/var/log/apt-cacher-ng/maint_1499322301.log.html whose mtime is 06:38 UTC. But we got this error from cron earlier, at 06:27 UTC.
And yesterday at 06:26:30 UTC the OOM killer had to kill
apt-show-versions (deployed as part of
Feature #11523 on June 25, which confirms my guess that this problem started before July 2), 25 seconds before the email error about acng timing out occurred. I guess that’s
So I think we should give a bit more memory to that system.
#5 Updated by intrigeri 2017-07-07 07:22:23
- Assignee changed from intrigeri to bertagaz
- % Done changed from 10 to 50
- QA Check set to Ready for QA
The cronjob worked just fine today. It seemed wasteful to allocate RAM that’ll be useful for very limited amounts of time every second day or so, so instead I’ve added some swap:
virsh vol-create-as lvm apt-proxy-swap 1G && sudo mkswap /dev/lizard/apt-proxy-swap # verified the backing PV makes sense virsh attach-disk apt-proxy /dev/lizard/apt-proxy-swap vdc --config --live --driver qemu --iothread 3 --cache directsync virsh edit apt-proxy # set io='native', not sure how to do it in the previous command line
Then I’ve declared a swap
mount resource for the node in our Puppet manifests, pushed, applied and finally
ssh apt-proxy.lizard sudo swapon -a.
Please close if you’re fine with this solution and if the problem doesn’t come up again in the next few days :)
#6 Updated by bertagaz 2017-07-10 10:16:51
- Status changed from In Progress to Resolved
- Assignee deleted (
- % Done changed from 50 to 100
- QA Check changed from Ready for QA to Pass
> Please close if you’re fine with this solution and if the problem doesn’t come up again in the next few days :)
Sounds good. No new email since then, let’s close this ticket.