Feature #12407

Ensure our mirror pool is ready for temporary surge of downloads

Added by intrigeri 2017-03-29 11:04:47 . Updated 2019-03-12 16:21:27 .

Status:
Confirmed
Priority:
Normal
Assignee:
intrigeri
Category:
Infrastructure
Target version:
Start date:
2017-03-29
Due date:
% Done:

0%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Problem: assume many people suddenly want to try Tails for the first time, e.g. after a successful press article; given we distribute the download load equally over almost all our mirrors, that have vastly different capacity, chances are that the slowest ones will quickly get overloaded ⇒ downloads are super slow or fail.

To solve this, we need:

  • either many more mirrors, but even if we can find them easily, then it would be a PITA to maintain the pool
  • or spread the load according to the capacity of our current mirrors: benchmark all mirrors, and adjust their weight according to the download bandwidth they can provide.

Subtasks

Related issues

Related to Tails - Feature #7864: Have check-mirrors archive speed statistics Confirmed 2014-08-31
Has duplicate Tails - Feature #14604: Implement mirror weights Duplicate 2017-09-05

History

#1 Updated by intrigeri 2017-06-05 17:19:23

  • Priority changed from Normal to Low

#2 Updated by intrigeri 2017-09-07 06:06:51

#3 Updated by intrigeri 2017-09-07 06:09:28

  • Assignee deleted (intrigeri)
  • Priority changed from Low to Normal
  • Target version set to Tails_3.5

(As per Feature #14604.)

#4 Updated by Anonymous 2018-01-18 18:46:12

  • Target version changed from Tails_3.5 to Tails_3.6

#5 Updated by Anonymous 2018-01-19 15:50:54

  • related to Feature #7864: Have check-mirrors archive speed statistics added

#6 Updated by geb 2018-02-10 19:13:24

As some mirrors are up to 10Gb/s, you may need/want to test them from a 10Gb/s connected server. I may be able to provide one. Please let me know if needed.

#7 Updated by Anonymous 2018-03-12 15:56:02

  • Target version changed from Tails_3.6 to Tails_3.7

I’ve just made some tests from a home connection right now. Here is what i get:

name        avg dl speed in kb
umu         ????
mipnamic    !ssl
kernel      down

secure.voy  0216

pplacid     1308
26.dl       1372
tetaneutral 1413
metalgamer  1539
berkeley    1672
adamas      1774

wikimedia   1940
24.dl       1956
ybti        1995
mirroservic 1998
strsbg      2068
bl0m        2089
stewart     2127
alpix       2150
void        2178
interpipe   2179
ukfast      2202
unixcorn    2206
dotsrc      2230
codingrat   2250
fr33tux     2254
wubim       2257
hactar      2259
hirlimann   2270
as1101      2283
fau         2283
ipph        2290
urown       2294
25.dl       2298

nluug       2302
bytemark    2304
c3l         2308
netcolog    2314
2cryptoreb  2316
3cryptoreb  2316
1cryptoreb  2317
dustri      2318
braingap    2322
ph3x        2324
bgadmin     2340
halifax     2348


@geb: I’m interested in having access to this server.

#8 Updated by Anonymous 2018-03-12 15:58:20

  • Status changed from Confirmed to In Progress

#9 Updated by geb 2018-03-12 17:09:01

> @geb: I’m interested in having access to this server.

Sent privately.

For the record, I may be able to provide access to well connected servers in Europe, for other Tails related needs, but no one in US.

#10 Updated by bertagaz 2018-05-10 11:09:18

  • Target version changed from Tails_3.7 to Tails_3.8

#11 Updated by intrigeri 2018-06-26 16:27:56

  • Target version changed from Tails_3.8 to Tails_3.9

#12 Updated by Anonymous 2018-09-03 17:46:22

  • Target version changed from Tails_3.9 to Tails_3.11

#13 Updated by CyrilBrulebois 2018-12-16 13:54:33

  • Target version changed from Tails_3.11 to Tails_3.12

#14 Updated by anonym 2019-01-30 11:59:16

  • Target version changed from Tails_3.12 to Tails_3.13

#15 Updated by Anonymous 2019-03-12 14:49:56

  • Assignee set to intrigeri
  • QA Check set to Info Needed

#18 Updated by intrigeri 2019-03-12 16:19:01

  • Description updated

@u, while it’s debatable whether the problem this is meant to fix has much chance to happen any time soon, I don’t think the fact we have many mirrors solves it at all. I’m clarifying why I think this way in the ticket description. Sorry I didn’t do that earlier :)

#19 Updated by intrigeri 2019-03-12 16:21:27

  • Status changed from In Progress to Confirmed
  • Target version deleted (Tails_3.13)
  • QA Check deleted (Info Needed)

I’ll keep this on my plate for now in case I have time for it on a rainy Sunday. It shouldn’t take more than 2 hours of focussed work to get a first set of measurements (at least from 2 servers on both sides of the Atlantic) and adjust weights accordingly.

Anyone interested, feel free to steal it from me :)