Tails

#1 Updated by intrigeri 2014-05-05 11:17:27

The Tor ticket I’ve created is apparently a duplicate of https://trac.torproject.org/projects/tor/ticket/4734, which clearly won’t be resolved quickly. So, we really have to move another way to point to, or redirect to, mirrors.

#2 Updated by Anonymous 2014-05-11 05:26:26

Here are some alternatives I could think of. Some of them might not really be feasible, though:

• Modify the DNS server, so that it only serves 29 randomly selected A records from a possibly much larger pool
• Create an interstitial website which lists all available mirrors, or which automatically selects one of them for an automatic download (SourceForge does that, for example)
• Create A records for a.dl.amnesia.boum.org, b.dl.amnesia.boum.org, … - each of them should have at most 29 A records. The DNS server for dl.amnesia.boum.org (which probably needs to be modified for that) now only needs to serve a random CNAME record to one of these subdomains. dl.amnesia.boum.org might point to a.dl.amnesia.boum.org, and that could point to 29 IP addresses.
• Instead of creating an interstitial website, replace all occurences of links to dl.amnesia.boum.org with [randomcharacter].dl.amnesia.boum.org; imho best implemented using JavaScript and a fallback to dl.amnesia.boum.org
• Create a script which automatically replaces the current A records with 29 new, random ones from a possibly much larger pool, every x minutes/hours/days, using a simple crontab entry. This might be the easiest solution, and I guess that it would actually be quite powerful. Especially for large .iso images, changing the A records every minute will probably actually have a load distributing effect similar to a huge number of A records.

#3 Updated by Sagi 2014-05-12 05:26:31

Just my 2 cents, some questions:
To keep up with increasing demand, does tails intend to:

1. spread its download infrastructure as widely as possible (say, bazillion mirrors)
   or
2. up the requirements for the limited set of mirrors

Some considerations:

• The disadvantage of uniformly selecting mirrors is that capacity constraints are not taken into account, which becomes a problem once demand is greater than the smallest of mirrors.
• The disadvantagea of a (large) mirror pool without some form of monitoring is the increasing chance of unmaintained/intermittently failing mirrors getting traffic.

Have you guys taken a look at something like mirrorbrain [1]? I suppose the geo-location functionality is less noticable over Tor, but the ability to assign weights to mirrors and/or specific locations to files might prove useful.

[1] http://mirrorbrain.org/

#4 Updated by sajolida 2014-05-12 05:50:32

> Have you guys taken a look at something like mirrorbrain [1]? I suppose the geo-location functionality is less noticable over Tor, but the ability to assign weights to mirrors and/or specific locations to files might prove useful.
>
> [1] http://mirrorbrain.org/

That looks cool.

http://mirrorbrain.org/features/

Unfortunately, it is not available in Debian. But there are unofficially
Debian packages:

http://download.opensuse.org/repositories/Apache:/MirrorBrain/Debian_7.0/

#5 Updated by Sagi 2014-05-12 05:58:00

I guess the disadvantage of such a centralized system is its resilience to interference relative to e.g. DNS-records with high TTL’s. But I guess that’s part of the problem space here. I’m not sure what the relevant trade-offs are for Tails.

#6 Updated by foob 2014-05-12 20:04:44

I opened a thread on tails-dev@ with some additional informations to this ticket: https://mailman.boum.org/pipermail/tails-dev/2014-May/005733.html . If you would like to participate … :)

#7 Updated by intrigeri 2014-05-13 03:01:23

Thanks, foob!

For the record, I believe tails-dev@ will be a way better place to discuss this than a ticket: I expect the discussion to need a few dozen emails, and we’ll definitely want sub-threads, that are not supported in Redmine.

#11 Updated by Anonymous 2014-08-07 17:50:35

(% Done can be at least 50% now imho)

I created the needed JavaScript code now; all that would be left to be done is telling the mirror owners to do a small configuration change, depending on what solution you prefer. :)

There is also an example website for both solutions linked on the blueprint page.

https://tails.boum.org/blueprint/HTTP_mirror_pool/#index3h1

Best regards,
Tobias Frei

#12 Updated by intrigeri 2014-08-08 21:30:14

Replied on tails-dev@. Let’s keep the discussion there for now :)

#20 Updated by Anonymous 2015-09-26 15:02:02

We decided to implement a two-way strategy for this feature:

• keep DNS as is, serving 3 highly reliable mirrors only. This will be the fallback for people who do not use JS.
• use Vanilla JS (no frameworks) to modify the link on the download page. This JS will be hosted in a git overlay, so not every contributor can interfere with it. It’ll load a JSON file, probably hosted on another overlay, with a list of mirrors and an assigned weight. weight 0 would mean that the mirror is currently not active. We’ll use JSON and not YAML to avoid 3rd party parsers.

I’ve updated the blueprint with my working prototype.