Feature #6439

Fix cleanup of nightly built ISO images

Added by intrigeri 2013-11-25 06:25:34 . Updated 2015-08-05 09:31:31 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Continuous Integration
Target version:
Start date:
2013-11-25
Due date:
2015-07-15
% Done:

100%

Feature Branch:
puppet-tails:bugfix/6439-fix-cleanup-of-nightly-built-ISO-images
Type of work:
Code
Blueprint:

Starter:
0
Affected tool:
Deliverable for:
266

Description

http://nightly.tails.boum.org/build_Tails_ISO_experimental/ and http://nightly.tails.boum.org/build_Tails_ISO_devel/ only have artifacts for the last few days. In case it matters, these branches didn’t build for a few days, so perhaps the script expects more files and was a little lost?


Subtasks


Related issues

Blocks Tails - Feature #6196: Build all active branches Resolved 2013-08-07 2015-07-15

History

#1 Updated by sajolida 2014-07-10 14:48:23

  • Assignee deleted (bertagaz)

#2 Updated by hyas 2014-07-10 15:30:58

Non-blocking ticket for 2.0

#3 Updated by intrigeri 2014-08-17 00:35:41

#4 Updated by intrigeri 2014-10-13 05:00:49

I suspect the correct algorithm would:

  • keep one ISO for each of the last 8 days => we always have the most recent ISO from last week
  • keep one ISO per week over the last 5 weeks => we always have an ISO from ~ last month

But anyway, I’m pretty sure this is a classical problem, that has well-known good solutions.

#5 Updated by intrigeri 2015-01-09 17:21:13

  • Assignee set to Dr_Whax
  • Target version changed from Sustainability_M1 to Tails_1.4

#7 Updated by intrigeri 2015-05-09 02:16:49

  • Target version changed from Tails_1.4 to Tails_1.4.1

Postponing. Note that we can’t postpone it once more, so if you doubt you can do it in time, please say so ASAP so we can reallocate this work without going crazy at the last minute.

The code lives as files/jenkins/slave/iso_builder/clean_old_jenkins_artifacts.rb in the tails Puppet module. Any of Python, Ruby or Modern Perl are acceptable languages, as long as the code is good and the dependencies in Wheezy + backports (we’ve not migrated our builders to Jessie yet).

#8 Updated by intrigeri 2015-05-28 15:01:14

  • Assignee changed from Dr_Whax to intrigeri

#9 Updated by intrigeri 2015-05-29 11:32:06

  • blocks #8668 added

#10 Updated by intrigeri 2015-05-31 07:34:59

  • Priority changed from Normal to Elevated

#11 Updated by intrigeri 2015-06-06 15:20:39

For the record, a potential new contributor might want to take this one. If this doesn’t happen by next Wednesday, I’ll try hard to take care of it.

#12 Updated by intrigeri 2015-06-11 20:49:07

  • Status changed from Confirmed to In Progress
  • Assignee changed from intrigeri to bertagaz
  • % Done changed from 0 to 50
  • Feature Branch set to puppet-tails:bugfix/6439-fix-cleanup-of-nightly-built-ISO-images

I’ve pushed something that seems to work on a local test data set. The algorithm may not be perfect yet, and I didn’t bother refactoring the whole thing, but it seems to be better (given how the current algorithm deletes most ISOs we would like to keep, it can’t be much worse anyway). Please review, and if happy merge and deploy!

#13 Updated by bertagaz 2015-06-17 05:28:44

  • Assignee changed from bertagaz to intrigeri
  • % Done changed from 50 to 80

I’ve tested it on a local dataset too. It seems to work well, and have a better retention strategy than the previous version for sure, congrats!

So I’ve merged, deployed it, and tested it live. So far the build goes fine, and it doesn’t seem to mess up with the artifacts.

If you believe this does close this ticket, please do. I’m not sure if we should wait some time now that the code is live to see how it behaves, even if I’m confident it works better.

#14 Updated by intrigeri 2015-06-18 03:01:36

  • Status changed from In Progress to Resolved
  • % Done changed from 80 to 100

> So I’ve merged, deployed it, and tested it live. So far the build goes fine, and it doesn’t seem to mess up with the artifacts.

Yay!

> If you believe this does close this ticket, please do.

Done.

> I’m not sure if we should wait some time now that the code is live to see how it behaves, even if I’m confident it works better.

We can still reopen this ticket (or create a new, more specific, one) if needed.

#15 Updated by intrigeri 2015-06-18 03:01:53

  • Assignee deleted (intrigeri)

#16 Updated by sajolida 2015-08-05 09:31:31

  • Due date set to 2015-07-15