Feature #8615

Clean up files from obsolete APT suites

Added by intrigeri 2015-01-09 12:48:09 . Updated 2015-03-22 12:10:58 .

Status:
Resolved
Priority:
Normal
Assignee:
intrigeri
Category:
Infrastructure
Target version:
Tails_1.3.1
Start date:
2015-01-09
Due date:
2015-02-24
% Done:

100%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

When we delete obsolete branches, we’ll want to clean up the APT repo too.

Subtasks

History

#1 Updated by intrigeri 2015-01-09 13:49:34

#2 Updated by intrigeri 2015-01-09 14:03:25

  • Target version changed from Sustainability_M1 to Tails_1.3.2

#3 Updated by intrigeri 2015-02-21 12:04:16

  • Status changed from Confirmed to In Progress

#4 Updated by intrigeri 2015-02-21 12:40:30

  • % Done changed from 0 to 20
  • QA Check set to Info Needed

APT suites corresponding to vanished Git branches are now automatically removed from reprepro’s configuration, which means the scalability issue (on reprepro config and dists/* update) is solved. This was the main goal of this ticket.

However, do we also want to delete:

  • dists/$REMOVED_BRANCH? These files are tiny and harmless, so I say the advantage of removing it isn’t worth the risk of deleting useful data if our removal code was buggy.
  • Packages that were in the deleted APT suite, and are not referenced by any other APT suite? It depends on the algorithm we use to delete old branches (Feature #8616), but if we only delete merged branches, in theory their APT suite has also been merged into another one, and then there should simply be no such package to clean up. The only exception I can think of is topic branches that get thrown away for some reason, and that might ship packages in the corresponding APT suite. Sounds like a corner case, disk space is cheap, and here again, the advantage of removing these files doesn’t seem worth the risk of deleting useful data if our removal scripts were not perfect.

anonym, bertagaz, DrWhax: what do you think?

#5 Updated by intrigeri 2015-02-21 13:56:41

After discussing it with DrWhax, it appears that there’s actually value in keeping these dist indices and packages around: we’re going to delete merged Git branches, but thanks to the merge commit, we’ll still have their latest commit ID somewhere, and it can happen that we need to check it out and rebuild from it (especially once we have a freezable APT repo) to try and reproduce a bug in the state of such a branch. That’s only possible if we keep the corresponding dist indices and packages around.

So, I’m now even more convinced that we shouldn’t delete files corresponding to deleted APT suites, now that the main scalability problem has been solved. Unless someone comes up with good reasons to delete these files in the next few days, I’ll close this ticket as resolved.

#6 Updated by intrigeri 2015-02-21 13:57:06

  • % Done changed from 20 to 50

#7 Updated by intrigeri 2015-02-22 20:09:40

  • Due date set to 2015-02-24

#8 Updated by intrigeri 2015-02-24 17:41:00

  • Status changed from In Progress to Resolved
  • % Done changed from 50 to 100
  • QA Check deleted (Info Needed)

bertagaz says (on IRC) he agrees => closing.

#9 Updated by BitingBird 2015-03-22 12:10:58

  • Target version changed from Tails_1.3.2 to Tails_1.3.1