Tails

#5 Updated by Anonymous 2018-11-02 20:38:28

If we don’t want to disable caching, we should use a technique similar to the one used in 4e30a3e59b7c97cf90b5a87bc6df1ece568f4335 whenever we make important changes. I think that would be reasonable to update the “timestamp” if needed - but I agree it can be forgotten..

#7 Updated by intrigeri 2018-11-03 06:32:37

> I created this ticket as a way to have our sysadmins reflect on how we can prevent such problems in the past. I don’t know anything about caching, what the industry standard is and I don’t want to learn about this because it’s way out of my skillset and attributions in the project. If you think it’s a bad idea please reject this ticket (or adjust the Redmine metadata as you please) but please don’t count on me to take a technical decision on this.

Fair enough.

The thing is, there’s simply no single person/team in the project who is obviously responsible for this problem, because the typical solution requires collaboration between web developers (who know about this sort of things), sysadmins (who can bring complementary knowledge and operational requirements), and tech writers (who can bring another set of requirements because their work will be affected by the decision).

So:

• Sysadmins may not be able to fix that alone, at least not without creating other kinds of problems. Still, I’ll see what I can do :)
• I’m sorry I am still operating under the flawed assumption that “tech writer” implies “responsible for all aspects of the HTML we generate from wiki/src”, i.e. our tech writers are also web developers. I should really stop implicitly assuming that: it’s unrealistic, unfair, and has caused all kinds of problems in the past, such as the kind of Redmine ping pong we’re playing here :/
• We should think about how to fix the “nobody feels skilled and responsible for the web development aspect of our website” problem, at the very least because even once I’ve fixed my flawed assumptions and stop reassigning you things that are indeed outside of your skillset & attributions, in order to address this kind of tickets I’ll still need someone who’s ready to do the web site part of the work. Perhaps it boils down to broadening the scope of our quest for a solution to “lack of JS developers”. And perhaps we’ll need to add this to the core budget somehow. Food for thought!

#9 Updated by sajolida 2018-11-04 04:00:56

> * I’m sorry I am still operating under the flawed assumption that “tech writer” implies “responsible for all aspects of the HTML we generate from wiki/src”, i.e. our tech writers are also web developers. I should really stop implicitly assuming that: it’s unrealistic, unfair, and has caused all kinds of problems in the past, such as the kind of Redmine ping pong we’re playing here :/

Right, technical writers are responsible for the documentation only.
Our description of “UX designer” also overlaps a bit with web dev as
they “do UX design (graphical interface, interactions, language, etc.)
on the maintenance or important improvements […] to the core pages of
our website”.

But we have nobody responsible for the good working of our website in
general. Though I sometime do it myself or even bend a bit my assigned
roles (eg. Feature #14922 in UX as per fundraising.git:final/ISC/1/unofficial.mdwn).

> Perhaps it boils down to broadening the scope of our quest for a solution to “lack of JS developers”.
> And perhaps we’ll need to add this to the core budget somehow. Food
> for thought!

Yeap. That’s why I mentioned the need for “Front-end web development”,
at the summit. See summit-2018.git:core_days/hiring_strategy.mdwn.

#10 Updated by Anonymous 2018-11-05 11:30:11

> Yeap. That’s why I mentioned the need for “Front-end web development”,
> at the summit. See summit-2018.git:core_days/hiring_strategy.mdwn.

All of this lies in my skillset and I’m happy to help out with these things.

#12 Updated by Anonymous 2018-11-05 11:52:02

To get back to the issue of this ticket: I think it is super useful to cache CSS files and not make people download them over and over again. However, sometimes we want to force them to redownload the files because, as in the case of the donation campaign, we made a change that should be visible to users.

https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching#invalidating-and-updating-cached-responses advises to use versioned filenames such as local.20181011.css.

In order not to change filenames by hand we could set up a htaccess that redirects versioned files to the original filename see https://css-tricks.com/strategies-for-cache-busting-css.

Or, as I did in my commit, use query strings, also mentioned in https://css-tricks.com/strategies-for-cache-busting-css/#article-header-id-1. This seems like an easy solution to me, although it requires manual intervention from frontend devs/fundraisers/tech writers.

#14 Updated by Anonymous 2018-11-09 13:20:02

> So next step is to evaluate our options on the website front. Happy to work on this (with my “I kind of know some ikiwiki stuff” hat on, not sysadmin) with u at some point.

Sure! I think I’d prefer to do that in a call rather than on this ticket, and then report the result on the ticket.

#15 Updated by geb 2018-11-10 21:09:34

Hi,

intrigeri wrote:
> As part of Feature #14588, I’ve configured the webserver that will soon host our website so it allows browsers to cache CSS, JS, fonts and images for up to 1 day (instead of 1 week on the current production website); I’ve ensure ETag headers are sent so browsers can revalidate their cache cheaply. This is meant as a temporary mitigation until our website supports invalidating browser cache when needed (using one of the techniques u has mentioned earlier). Once these techniques and processes are in place, we should increase the expiration delay.
>

If I may, I speed some of time working on HTTP Caching and, while u’s trick about versionned files names (or URL arguments like 4e30a3e59b) is great (did not thought about it for Bug #16049), its a bit constraining if not included in the framework. However, HTTP offers some nice caching mecanisms you could use to force longer expiration delay while asking or forcing revalidation :

Actually on the website i see :

Expire: current + 1 day
Cache-Control: max-age=86400
Etags

The use of etags actually let the browser revalidate for every request (its not explicit, so its browser decision, my firefox revalidate every CSS/JSS, but don’t always ask for revalidation of the images) but let it use stale content for up to one day, even if the website is unavailable, and should be equivalent to:

Expire: current + 1 day
Cache-Control: max-age=86400
Cache-Control: min-fresh=0
[NoEtags]

Depending what you want to do it should be possible to :

• Let the browser use cached content for longer time without revalidating.
• Let the browser use cached content for longer time after revalidation.
• Let the browser use cached content for longer time if revalidation fails.

For example :

Expire: current + 1 month
Cache-Control: max-age= 1 month
Cache-Control: max-stale= 1 mouth
Cache-Control: min-fresh=0
[NoEtags]

Will still ask the client to revalidate the content for every request, but keep the file in cache for up to one month which would decrease the server load.

Or

Expire: current + 1 month
Cache-Control: max-age= 1 month
Cache-Control: max-stale= 1 mouth
Cache-Control: min-fresh= 1 day
[NoEtags]

Will let the browser use cached content (even if revalidation fails) for up to 1 month and ask only for revalidation after one day, which would also decrease the server load (but may be not desirable).

Good reading :
https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching Not fully complete (some things like max-stale etc are missing)
https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching same
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control complete (even include useless experimental features)

If you would like to play a bit with HTTP caching, feel free to ping me.

(Side note: its less KISS, but maybe images could be cached for longer than CSS)

#16 Updated by geb 2018-11-10 22:44:18

Another side note :

Cache theory and web users habits make it clear than even if a really small/short cache of one or two hours would have a huge impact on performances while allowing quick updates.

Thus if I had to propose a policy, I would suggest something like :

• Use files from cache without revalidation for 1 or 2 hours
• Use files from cache after revalidation for 1 week or 1 month

It would allow :

• Quick (1, or 2 hours) changes without complexes revalidation mechanisms like ikiwi changes (and the possibility to rely manually on them using URL arguments if really necessay)
• Subsequents requests for a given visit (to different pages) to be served directly from cache, reducing the server load and giving user faster access
• Use of cached files for longer time than actually, reducing the server load and giving user faster access.

Which would give thoses HTTP headers :

Expire: current + 1 week
Cache-Control: max-age= 1 week
Cache-Control: max-stale= 1 week
Cache-Control: min-fresh= 2 hours
[NoEtags]

Or

Expire: current + 1 week
Cache-Control: max-stale= 1 week
Cache-Control: max-age= 2 hours
[NoEtags]

Which as far i remind, may have the same scemantic (need testing) and is more KISS.

As images are less likely to change, (and URL tricks are still possible), another policy using 1-2 days before revalidation and 1 week or 1 month for after revalidation sounds also raisonnable.

#17 Updated by sajolida 2018-11-13 09:54:00

> Thus if I had to propose a policy, I would suggest something like :
> * Use files from cache without revalidation for 1 or 2 hours
> * Use files from cache after revalidation for 1 week or 1 month

Thanks geb for joining the discussion with such detailed information!