Feature #11123

Have the Upgrader use the new mirror pool design

Added by intrigeri 2016-02-13 15:40:59 . Updated 2016-06-26 11:17:55 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Tails_2.4
Start date:
2016-02-13
Due date:
% Done:

100%

Feature Branch:
{tails,iuk,perl5lib}:feature/11123-new-mirror-pool
Type of work:
Code
Blueprint:

https://tails.boum.org/blueprint/HTTP_mirror_pool/

Starter:
Affected tool:
Deliverable for:
269

Description

The status quo, if we don’t do anything, would be that we go on pointing to http://dl.amnesia.boum.org/ in UDFs, and then all IUK downloads will be pulled from the remaining DNS round-robin pool. Is it acceptable, or do we want the Upgrader to use the new mirror pool?

Subtasks

Related issues

Blocks Tails - Feature #11284: Update the DNS round-robin mirror pool to only include a few fast and reliable HTTP mirrors Resolved 2016-03-25

History

#1 Updated by intrigeri 2016-03-03 15:08:08

  • Assignee set to intrigeri

Oops, seems that I failed to put this on my radar.

#2 Updated by intrigeri 2016-03-08 13:47:28

  • Target version changed from Tails_2.2 to Tails_2.3

#3 Updated by intrigeri 2016-03-25 21:33:09

  • Priority changed from Normal to High

Rationale: if we decide that the Upgrader needs to use the new mirror pool, then the corresponding code will need to be written early enough to go into Tails 2.4. I’d like to know ASAP if I need to schedule time for this.

#4 Updated by intrigeri 2016-04-08 07:16:13

  • Subject changed from Design how the Upgrader will use the new mirror pool design to Have the Upgrader use the new mirror pool design

I’ve started drafting the code, and will push this a bit further. If it gets too complicated I might reconsider using the fallback DNS pool for downloading IUKs, but so far I’m confident I can get this done in time for 2.4.

#5 Updated by intrigeri 2016-04-08 07:16:31

  • Status changed from Confirmed to In Progress
  • Type of work changed from Discuss to Code

#6 Updated by intrigeri 2016-04-16 14:59:47

  • Target version changed from Tails_2.3 to Tails_2.4

#7 Updated by intrigeri 2016-04-29 05:46:58

  • Feature Branch set to {tails,iuk,perl5lib}:feature/11123-new-mirror-pool

#8 Updated by intrigeri 2016-04-29 07:35:33

  • blocked by Feature #11384: Record the current state of our mirror pool in JSON added

#9 Updated by intrigeri 2016-04-29 07:36:51

  • % Done changed from 0 to 20

So, I have something that works in the context of the iuk.git’s test suite. Next step is to build Debian packages and test in a running Tails, which will only be possible once Feature #11384 is done.

#10 Updated by intrigeri 2016-04-30 06:58:06

FTR, Feature #11384 has seen enough progress so it’s not a blocker for testing this. But of course, it remains a blocker for merging this.

#11 Updated by intrigeri 2016-05-04 08:45:29

Tested in Tails 2.2, after installing tails-{iuk,perl5lib} built from their debian_feature-11123-new-mirror-pool branch, and manually applying the changes from feature/11123-new-mirror-pool: it fails to run tails-transform-mirror-url because one construct used in mirror-dispatcher.js is not supported by Jessie’s nodejs (while it is on sid).

#12 Updated by intrigeri 2016-05-04 09:15:11

  • Assignee changed from intrigeri to anonym
  • % Done changed from 20 to 50
  • QA Check set to Ready for QA

tl;dr: it now works! Packages uploaded.

That bug was fixed by adding fallback code to mirror-pool-dispatcher.js, to support this use case. And now I see Tails Upgrader use a mirror URL it got using mirror-pool-dispatcher.js, in the testing environment described above.

anonym, can you please review the 3 topic branches and then reassign to me? We can’t merge until the blocking ticket is fully done, but still you might want to save yourself (and perhaps me) from a future rush, with an early review :)

#13 Updated by Anonymous 2016-05-10 02:46:06

@intrigeri, could you please add a comment to bin/tails-transform-mirror-url in mirror-pool-dispatcher.git, so that we know what this file is for and by what it’s used? I think that would be useful. Thanks!

#14 Updated by intrigeri 2016-05-10 05:54:51

> @intrigeri, could you please add a comment to bin/tails-transform-mirror-url in mirror-pool-dispatcher.git, so that we know what this file is for and by what it’s used? I think that would be useful. Thanks!

Good idea! Done.

#15 Updated by intrigeri 2016-05-21 15:47:16

Updated the submodule so that it points to the latest JS code.

#16 Updated by intrigeri 2016-05-23 12:40:12

  • blocks deleted (Feature #11384: Record the current state of our mirror pool in JSON)

#17 Updated by intrigeri 2016-05-23 12:42:57

intrigeri wrote:
> FTR, Feature #11384 has seen enough progress so it’s not a blocker for testing this. But of course, it remains a blocker for merging this.

Feature #11384 is basically done, and our mirror pool is ready to receive requests => please merge this for 2.4 (otherwise, we have a problem: these branches are not exactly of the kind we want to merge during the 2.5 point-release cycle, and until they are merged, Tails Upgrader will use the fallback DNS pool at dl.a.b.o; but that pool will very soon be reduced to ~5 mirrors, and we want to use it as little as possible, so it would be much better if downloads from the Upgrader started to use our new mirror pool before).

#18 Updated by intrigeri 2016-05-23 12:43:50

  • blocks Feature #11284: Update the DNS round-robin mirror pool to only include a few fast and reliable HTTP mirrors added

#19 Updated by anonym 2016-05-25 19:36:32

  • Status changed from In Progress to Fix committed
  • Assignee deleted (anonym)
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

#20 Updated by anonym 2016-06-08 01:25:34

  • Status changed from Fix committed to Resolved

#21 Updated by BitingBird 2016-06-26 11:17:55

  • Priority changed from High to Normal