Tails

#1 Updated by bertagaz 2015-09-03 01:33:48

• Parent task set to Feature #10153

#2 Updated by bertagaz 2015-09-03 01:34:12

• blocks Feature #10154: Store automated test videos with other artifacts added

#4 Updated by intrigeri 2015-09-22 12:41:56

• Deliverable for set to 268

#5 Updated by bertagaz 2015-09-25 16:03:42

• Assignee changed from bertagaz to intrigeri
• QA Check set to Info Needed

intrigeri wrote:
> bertagaz wrote:
> > Videos archived as artifacts will take quite some disk space, so we should rotate them, probably removing those that are older than one week.
>
> With the branch for Feature #10148 now merged, passing --capture will only save videos for the specific scenarios that failed. Combined with the improvements of Bug #10001, maybe the size of videos to store won’t be that big :)

Seems files are mostly 500K on average for basic scenarios that fail quickly after the greeter or without much changes on the screen. Peaks goes to something like 4M worst seen.
In the end it seems reasonable to assume a run with a lot of failing scenarios has around 20M of video captures to be saved. So one week of 16 runs per day for 25 branches would mean 56G of data. With this average, it means keeping them for a full release would mean something around 336G.

So we probably can’t afford a full release, but maybe raise a bit more than one week if we want so.

> If all we want to express is “older than a week”, then we need to choose between:
>
> * erase all artifacts from “old” test suite runs, and not only videos: then we can use Jenkins’ own archives expiration support and not bother handling it ourselves
> * erase only videos: then indeed we probably need to adapt our script and do it our own way.

I’m unsure if people will one day need any test suite artifacts more than one week after the failure. If we feel this might not really happen or may not be so bad if the all the artifacts (but the logs in Jenkins) are not there anymore in this case, then let’s just drop them all. If not, then go for the second.

I’m inclined to vouch for the first proposal.

#8 Updated by intrigeri 2015-09-30 08:08:59

• Assignee changed from intrigeri to bertagaz

#10 Updated by bertagaz 2015-09-30 08:58:00

• QA Check changed from Info Needed to Dev Needed

#12 Updated by bertagaz 2015-10-10 12:24:10

• related to Bug #10354: Check the size of video captures artifacts after a while added

#13 Updated by bertagaz 2015-10-10 12:28:09

• Assignee changed from bertagaz to intrigeri
• QA Check changed from Dev Needed to Ready for QA

Since run 9 for of the test_Tails_ISO_devel job the failing scenarios video captures are archived.

My first calculations was wrong, for 20M of video artifacts worst case, 16 times per day, gives 2240 MB per week.
So we probably can afford keeping them for a whole release process, as asked by anonym. We can try for a first release cycle and evaluate how sustainable it seems.

To do that, the Jenkins own artifact rotation method is enough, and can be configured by adding:

    logrotate:
      daysToKeep: 50
      artifactDaysToKeep: 50

as done int the test_Tails_ISO_{devel,stable,experimental} jobs (and the build jobs too).

I’ve updated the blueprint accordingly in, and opened ticket Bug #10354 to remind me to check that.

#14 Updated by intrigeri 2015-10-14 00:17:16

• Assignee changed from intrigeri to bertagaz
• QA Check changed from Ready for QA to Info Needed

> Since run 9 for of the test_Tails_ISO_devel job the failing scenarios video captures are archived.

Confirmed, I could download one such artifact :)

> To do that, the Jenkins own artifact rotation method is enough, and can be configured by adding:

>

>     logrotate:
>       daysToKeep: 50
>       artifactDaysToKeep: 50
> 

> as done int the test_Tails_ISO_{devel,stable,experimental} jobs

The artifactDaysToKeep looks good, assuming it’s OK to delete the JSON test suite output as part of this cleanup.

> (and the build jobs too).

I was not able to find what you’re referring to here. Any pointer?

> I’ve updated the blueprint accordingly in, and opened ticket Bug #10354 to remind me to check that.

:)

#15 Updated by intrigeri 2015-10-14 00:20:35

• blocked by deleted (Feature #10154: Store automated test videos with other artifacts)

#16 Updated by bertagaz 2015-10-14 02:30:29

• Assignee changed from bertagaz to intrigeri

intrigeri wrote:
> > Since run 9 for of the test_Tails_ISO_devel job the failing scenarios video captures are archived.
>
> Confirmed, I could download one such artifact :)

And the test jobs can use the copyartifacts plugin to do so too!

> The artifactDaysToKeep looks good, assuming it’s OK to delete the JSON test suite output as part of this cleanup.

I don’t think we need to keep them longer than the other artifacts. And the cucumber plugin is storing its parsed output in a different xml file, so we won’t loose its report on the build page.

> > (and the build jobs too).
>
> I was not able to find what you’re referring to here. Any pointer?

Right, it’s a default configuration, not a build one, so in the default.yaml file of the JJB repo.

Shall I adapt the test jobs definition? I think we can wait for Bug #10354 to see if we need to change that value and thus use this field differently.

#17 Updated by intrigeri 2015-10-14 05:43:54

• Assignee changed from intrigeri to bertagaz

>> The artifactDaysToKeep looks good, assuming it’s OK to delete the JSON test suite output as part of this cleanup.

> I don’t think we need to keep them longer than the other artifacts. And the cucumber plugin is storing its parsed output in a different xml file, so we won’t loose its report on the build page.

Thanks, this removes the main (silent) concern I had.

>> I was not able to find what you’re referring to here. Any pointer?

> Right, it’s a default configuration, not a build one, so in the default.yaml file of the JJB repo.

OK, seen it. Thanks.

> Shall I adapt the test jobs definition?

I don’t understand. What do you mean?

> I think we can wait for Bug #10354 to see if we need to change that value and thus use this field differently.

Yes.

#18 Updated by bertagaz 2015-10-15 00:22:32

• Assignee changed from bertagaz to intrigeri

intrigeri wrote:
> Thanks, this removes the main (silent) concern I had.

Silently heard it. :)

> > Shall I adapt the test jobs definition?
>
> I don’t understand. What do you mean?

I was proposing to remove this lines from the test jobs template to get them using the default, but:

> > I think we can wait for Bug #10354 to see if we need to change that value and thus use this field differently.

So putting this Ready For QA again.

#19 Updated by intrigeri 2015-10-15 02:35:52

• Status changed from Confirmed to Resolved
• Assignee deleted (intrigeri)
• % Done changed from 0 to 100
• QA Check changed from Info Needed to Pass

I have no strong opinion wrt. whether these settings should be made explicit in the test jobs or be inherited from the defaults, thus closing. If some day you have a strong opinion then I trust you’ll propose to change how it is done currently.