Bug #17439
Enable the cachewebsite build option by default, including on our CI
100%
Description
Enabling the new cachewebsite
build option (Feature #15342) by default should lower by 25% the total runtime of a Tails images build job on our CI.
It would take up to 5 extra GiB on isobuilders’ /var/lib/libvirt/images
.
Note that this option is automatically disabled when building a release (i.e. from a tag), to be on the safe side.
Here’s the plan:
Ask our developers to use thedone (2020-02-04)cachewebsite
option locally, in order to iron out whatever bugs are left in the implementation (I think I’ll do that anyway)Ensure our isobuilders have enough space to store the cachedone- While waiting for feedback from developers, enable this on my local Jenkins for a few weeks, in order to learn about potential Jenkins-specific problems this may cause
- On a topic branch, enable
cachewebsite
by default and add acachewebsite=no
option to allow disabling this feature
Subtasks
Related issues
Blocked by Tails - |
Resolved | 2018-02-22 | |
Blocks Tails - Feature #16209: Core work: Foundations Team | Confirmed | ||
Blocked by Tails - |
Resolved |
History
#1 Updated by intrigeri 2020-01-14 19:46:54
- blocked by
Feature #15342: Add option to our build system for caching the wiki added
#2 Updated by intrigeri 2020-01-14 20:06:38
- Description updated
#3 Updated by anonym 2020-01-15 12:35:16
intrigeri wrote:
> Enabling the new cachewebsite
build option (Feature #15342) should lower by 25% the total runtime of a Tails images build job on our CI.
> It would take up to 5 extra GiB on isobuilders’ /var/lib/libvirt/images
.
A great trade-off, IMHO!
> Any reasons why we should not enable this option?
I don’t think so. Well, this adds another instance of where a caching bug can occur, which generally is tricky to even notice, but I don’t think that should be a deterrent (imagine where we would be without any caching). It is very clear to me that the code that disables this cache for releases is correct, and that’s the most import bit. Let’s just keep it that way!
> If not, I could:
>
> * ask our developers to use the cachewebsite
option locally, in order to iron out whatever bugs are left in the implementation (I think I’ll do that anyway)
> * enable this on my local Jenkins for a few weeks, in order to learn about potential Jenkins-specific problems this may cause
Sounds good! In fact, I think we should consider making cachewebsite
the default since it will improve the developer UX story and we get less fragmented options among developers. In my experience, opt-in for a nice feature generally results in more fragmentation compared to opt-out — with opt-in for a desirable feature, only users that read the docs will know about it and thus possibly/likely enable it.
#4 Updated by intrigeri 2020-01-15 12:46:27
> In fact, I think we should consider making cachewebsite
the default since it will improve the developer UX story and we get less fragmented options among developers. In my experience, opt-in for a nice feature generally results in more fragmentation compared to opt-out — with opt-in for a desirable feature, only users that read the docs will know about it and thus possibly/likely enable it.
Agreed. I made it opt-in to start with only out of humility and because I did not want to affect our CI so soon.
So perhaps this ticket becomes “Consider enabling the cachewebsite build option by default, including on our CI”.
And then we’ll need a nocachewebsite
option, I guess.
#5 Updated by segfault 2020-01-15 17:50:55
anonym wrote:
> intrigeri wrote:
> > Enabling the new cachewebsite
build option (Feature #15342) should lower by 25% the total runtime of a Tails images build job on our CI.
> > It would take up to 5 extra GiB on isobuilders’ /var/lib/libvirt/images
.
>
> A great trade-off, IMHO!
>
> > Any reasons why we should not enable this option?
>
> I don’t think so. Well, this adds another instance of where a caching bug can occur, which generally is tricky to even notice, but I don’t think that should be a deterrent (imagine where we would be without any caching).
I agree.
#6 Updated by intrigeri 2020-02-04 10:38:31
- Subject changed from Consider enabling the cachewebsite build option on our CI to Enable the cachewebsite build option by default, including on our CI
- Description updated
- Assignee set to intrigeri
- Target version set to Tails_4.4
#7 Updated by intrigeri 2020-02-04 10:38:51
- blocks Feature #16209: Core work: Foundations Team added
#8 Updated by intrigeri 2020-02-04 11:47:46
- Description updated
- Status changed from Confirmed to In Progress
intrigeri wrote:
> Enable this on my local Jenkins for a few weeks, in order to learn about potential Jenkins-specific problems this may cause
Enabled this today and already enjoying the benefits :)
> Ensure our isobuilders have enough space to store the cache
They don’t: during a build, there’s only 2.9G free space there.
#9 Updated by intrigeri 2020-02-04 12:13:40
- Description updated
#10 Updated by intrigeri 2020-02-04 12:36:52
- Description updated
#11 Updated by intrigeri 2020-02-04 12:38:13
- Parent task set to Bug #16960
- Type of work changed from Sysadmin to Code
(The sysadmin part is done.)
#12 Updated by intrigeri 2020-02-15 17:37:20
- Feature Branch set to feature/17439-enable-cachewebsite-by-default
#13 Updated by intrigeri 2020-02-16 10:17:34
- blocked by
Bug #17481: Refresh our patches to apply on top of thunderbird 1:68.5.0-1~deb10u1 added
#14 Updated by intrigeri 2020-02-16 12:17:29
- Status changed from In Progress to Needs Validation
- Assignee deleted (
intrigeri)
Looks good so far. It indeed cuts the build time down by almost 30% on Jenkins, in the best case scenario: https://jenkins.tails.boum.org/job/build_Tails_ISO_feature-17439-enable-cachewebsite-by-default/buildTimeTrend
#15 Updated by segfault 2020-02-21 10:41:57
- Status changed from Needs Validation to Resolved
- % Done changed from 0 to 100
Applied in changeset commit:tails|7951787568f5229d0481c9432742c7f5be43d940.