Feature #11304

Make sure that DAVE honors Expires and Cache-Control header

Added by intrigeri 2016-04-03 13:14:17 . Updated 2016-12-07 08:07:43 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Installation
Target version:
Start date:
2016-04-03
Due date:
% Done:

100%

Feature Branch:
Type of work:
Research
Blueprint:

Starter:
Affected tool:
ISO Verification Extension
Deliverable for:

Description

Our website now (hopefully) sets proper caching headers on the IDF. Next step is to ensure that DAVE honors them.


Files


Subtasks


History

#1 Updated by intrigeri 2016-04-03 13:14:44

  • blocks #8538 added

#2 Updated by intrigeri 2016-04-03 13:17:23

A cursory look at the access logs shows me no request replied with 304, only 200. While I see 304 replies from browsers that access e.g. local.css. Could it be that this code disables caching in DAVE:

req.channel.loadFlags |= Ci.nsIRequest.LOAD_ANONYMOUS | Ci.nsIRequest.LOAD_BYPASS_CACHE | Ci.nsIRequest.INHIBIT_PERSISTENT_CACHING;

?

#3 Updated by ma1 2016-05-25 22:41:53

Sorry, I just noticed this ticket, and indeed it might be the culprit.
Please try applying the attached patch, thanks.

#4 Updated by intrigeri 2016-05-26 13:31:21

  • Status changed from Confirmed to In Progress
  • Assignee changed from ma1 to intrigeri
  • Target version set to Tails_2.5
  • QA Check set to Ready for QA

Sure, I’ll take it.

#5 Updated by intrigeri 2016-06-01 11:37:46

  • Assignee changed from intrigeri to ma1
  • % Done changed from 0 to 50
  • QA Check deleted (Ready for QA)

I’ve tested and confirmed that this patch works, so I’ve pushed it to Git. Thanks! So, to make this live we need you to release a new version of the add-on. However, our URL transformation code (Feature #11109) is not fully ready yet, so likely you’ll want to base the release on commit e807906 + cherry-pick 66b2b60. Alternatively, just wait for Feature #11109 to be in a better shape, and then do a single release that includes both this bugfix and that feature. Just let me know what you prefer to do, and I’ll adjust ticket relationships accordingly :)

#6 Updated by ma1 2016-06-02 15:28:43

If that’s not a problem I’d prefer the second option, i.e. coalescing into one single release.

#7 Updated by intrigeri 2016-06-02 15:46:21

> If that’s not a problem I’d prefer the second option, i.e. coalescing into one single release.

Sure, no problem!

#8 Updated by intrigeri 2016-08-02 09:31:57

  • Target version changed from Tails_2.5 to Tails_2.6

#9 Updated by anonym 2016-09-20 16:54:06

  • Target version changed from Tails_2.6 to Tails_2.7

#10 Updated by ma1 2016-11-04 17:13:03

  • Status changed from In Progress to Fix committed
  • Assignee changed from ma1 to sajolida
  • QA Check set to Ready for QA

2.8rc2 and 2.8 are on AMO, see Bug #11300.

#11 Updated by sajolida 2016-11-06 19:27:45

  • Assignee changed from sajolida to intrigeri

Yeap. The patch is applied in 2.8 from AMO.

intrigeri: do you want to check anything else? shall I perform some test? how?

#12 Updated by intrigeri 2016-11-07 09:10:33

  • Assignee changed from intrigeri to sajolida

sajolida wrote:
> Yeap. The patch is applied in 2.8 from AMO.
>
> intrigeri: do you want to check anything else?

I want to test that the fix works once deployed in production.

> shall I perform some test?

Feel free to do it yourself, it doesn’t have to be me.

> how?

IIRC Giorgio provided instructions in the thread referenced in the ticket description, and I later provided some feedback about these instructions.

#13 Updated by bertagaz 2016-11-08 12:35:37

  • Status changed from Fix committed to In Progress

Fixing some conflicting metadata: ticket can’t be RfQA AND Fix committed.

#14 Updated by sajolida 2016-11-10 17:26:23

  • Assignee changed from sajolida to intrigeri
  • QA Check changed from Ready for QA to Info Needed

I tried that but I don’t get it.

I read “https://mailman.boum.org/pipermail/tails-dev/2016-May/010681.html”:

  • When I do Ctrl+K from my browser nothing happens, same when I do Ctrl+K from the developer tools. So I assumed that Giorgio was referring to the Console → Net tab of the developers tools.
  • So I tried to start a fresh Tails, go to https://tails.boum.org/install/download and check the download of https://tails.boum.org/install/v1/Tails/i386/stable/latest.yml, which is the main URL that DAVE should be downloading and for which we should disable caching. I couldn’t see any line corresponding to this. Neither in the Network tab. So I don’t know how to test that latest.yml is actually downloaded only once if I don’t know how to verify if it’s downloaded at all.
  • In the email, intrigeri you were referring to “DAVE fetches the body every time”, the body of what? Anything else than latest.yml?

I’m sorry but I need more guidance…

#15 Updated by intrigeri 2016-11-12 07:49:51

  • Target version changed from Tails_2.7 to Tails_2.9.1

#16 Updated by intrigeri 2016-12-04 14:52:20

  • Assignee changed from intrigeri to sajolida

> * So I tried to start a fresh Tails, go to https://tails.boum.org/install/download and […]

This report makes me wonder: did you install DAVE in that browser?

#17 Updated by sajolida 2016-12-05 15:07:58

  • File no-idf.png added
  • Assignee changed from sajolida to intrigeri

I don’t remember so much about what I did last time but this time I tried to:

#18 Updated by intrigeri 2016-12-06 12:41:37

  • Status changed from In Progress to Resolved
  • QA Check changed from Info Needed to Pass

> I don’t remember so much about what I did last time but this time I tried to:

> * Install DAVE from https://tails.boum.org/install/download
> * Shutdown Tor Browser
> * Restart Tor Browser, open the Network tab of the developers tools
> * Go to https://tails.boum.org/install/download
> * And I still don’t see any fetch of the IDF, see screenshot

OK, it took me some time but I got it. I’ve already been bitten by essentially the same problem for the web console vs. the browser console: the “normal” web dev tools don’t display all activity that’s caused by extensions (as opposed as activity happening due to the web content). So, I’ve used an instance of Firefox that has the recommended setup for extensions dev, and in there I did Tools → Web Developer → Browser Toolbox. It has a Network tab that looks extremely similar to the one found in the normal dev tools (that you used), but this one includes all activity caused by extensions.

This allowed me to confirm that DAVE doesn’t fetch latest.yml once they’ve been cached locally.

#19 Updated by intrigeri 2016-12-06 12:49:08

  • Assignee deleted (intrigeri)
  • % Done changed from 50 to 100

#20 Updated by sajolida 2016-12-07 08:07:43

> OK, it took me some time but I got it. I’ve already been bitten by
> essentially the same problem for the web console vs. the browser
> console: the “normal” web dev tools don’t display all activity that’s
> caused by extensions (as opposed as activity happening due to the web
> content).

This makes sense and reflects pretty much what I’ve seen.

> So, I’ve used an instance of Firefox that has the
> recommended setup for extensions
> dev
,
> and in there I did Tools → Web Developer → Browser Toolbox. It has a
> Network tab that looks extremely similar to the one found in the
> normal dev tools (that you used), but this one includes all activity
> caused by extensions.
>
> This allowed me to confirm that DAVE doesn’t fetch latest.yml once
> they’ve been cached locally.

I would have felt super lazy to experiment with that, so I’m glad you
did it in the end!