Tails

#1 Updated by bertagaz 2018-10-02 22:15:42

I’ve noticed that too locally, acng is quite sloppy at shrinking to the maximum amount it is given. It always do to a somewhat higher value. I “solved” that by just lowering the number to get closer to my needs.

#2 Updated by anonym 2018-10-03 09:12:11

That didn’t work for me — no matter the size argument I gave nothing was removed. It is as if a too high ratio of non-debs (APT indices, TBB tarballs) makes its calculations flip out and nothing is freed.

#3 Updated by anonym 2018-10-08 15:24:37

intri suggested that acng’s daily cronjob should be able to clean it up, but that it takes a long time: “(it fetches all dists again to identify obsolete packages) so I doubt we can do that at every build”.

#4 Updated by segfault 2018-10-08 15:25:23

I had the same issue (see Bug #16032)

#5 Updated by anonym 2018-10-08 15:27:33

anonym wrote:
> intri suggested that acng’s daily cronjob should be able to clean it up, but that it takes a long time: “(it fetches all dists again to identify obsolete packages) so I doubt we can do that at every build”.

Or maybe not:

(17:25:53) intrigeri: ah ah on lizard, we do
"rm -rf /var/cache/apt-cacher-ng/*.tails.boum.org"
weekly, so no, the cronjob is not what saves us there.

#9 Updated by CyrilBrulebois 2019-03-21 07:30:11

Let’s look at our code calling acngtool shrink (vagrant/provision/assets/build-tails):

if [ "${TAILS_PROXY_TYPE}" = "vmproxy" ]; then
    # The apt-cacher-ng cache disk is 15G, so let's ensure at most 10G
    # of it is used there is 5G before each build, which should be
    # enough for any build, even if we have to download a complete set
    # of new packages for a new Debian release.
    /usr/lib/apt-cacher-ng/acngtool shrink 10G -f || \
        echo "The clean-up of apt-cacher-ng's cache failed: this is" \
             "not fatal and most likely just means that some disk" \
             "space could not be reclaimed -- in order to fix that" \
             "situation you need to manually investigate " \
             "/var/cache/apt-cacher-ng/apt-cacher-ng-log/main_*.html" >&2
fi

It seems pretty straightforward, catching issues and displaying an error message when that happens (while carrying on).

But upstream code (source/acngtool.cc in apt-cacher-ng 2-2) has:

        if(verbose)
                cout << "Found " << totalSize << " bytes of relevant data, reducing to " << wantedSize << endl;
        while(!delQ.empty())
        {
                bool todel = (totalSize > wantedSize);
                totalSize -= delQ.top().size;
                const char *msg = 0;
                if(verbose || dryrun)
                        msg = (todel ? "Delete: " : "Keep: " );
                auto& delpath(delQ.top().path);
                if(msg)
                        cout << msg << delpath << endl << msg << delpath << ".head" << endl;
                if(todel && apply)
                {
                        unlink(delpath.c_str());
                        unlink(mstring(delpath + ".head").c_str());
                }
                delQ.pop();
        }
        return 0;

Notice the utter lack of error handling and the mandatory return 0;? That’s why we have been missing this for so long: that command never fails. In verbose mode, it seems there’s much work going on, with many “Delete:” and a few “Keep:” entries. But the filesystem is left untouched.

Prepending the command with as_root_do fixes the shrinking…