Feature #11079

Publish a call for more HTTP mirrors

Added by intrigeri 2016-02-08 11:03:49 . Updated 2016-06-26 11:13:04 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Start date:
2016-02-08
Due date:
% Done:

100%

Feature Branch:
Type of work:
Communicate
Blueprint:

Starter:
Affected tool:
Deliverable for:
269

Description

We’re aiming to have 35 mirrors 3 months after the new mirrors infra is up. It’ll likely support mirrors with arbitrary vhost names so perhaps it’ll be time to reach out to e.g. mirrors.kernel.org etc.


Subtasks


Related issues

Blocked by Tails - Feature #11054: Update contribute/how/mirror documentation with examples for the web servers we support Resolved 2016-02-04
Blocked by Tails - Feature #11334: Document how HTTP mirrors get a unique virtualhost name Resolved 2016-04-08

History

#1 Updated by geb 2016-02-08 16:06:19

Hi,

I got a list of big mirrors hosters (Research networks, universities etc). This list includes some private contact informations, so i cannot share it publicly, but i can either share it privately or use it to send requests once the call is published.

For the call itself, maybe can we reuse part of the text we used to ask for support for the tails hackfest ?

> About Tails
> ===========
> 
> Tails is a live operating system that aims at preserving privacy and
> anonymity. In a nutshell, Tails is providing a platform to solve many
> surveillance problems by "doing the right thing" out of the box by
> default, protecting even less tech-savvy users from the most likely
> and highest impact risks. It helps to:
> 
> * use the Internet anonymously everywhere, on any computer: all
> connections to the Internet are forced to go through the Tor
> network; this allows to circumvent censorship and defeat
> surveillance on the Internet;
>
> * leave no trace on the computer unless the users asks it
> explicitly; this protects against after-the-fact forensics of the
> computer that was used;
>
> * use state-of-the-art cryptographic tools to encrypt files, email
> and instant messaging.
> 
> Tails has been around since 2009, and has been mostly run by
> volunteers since the beginning. We collaborate a lot with The Tor
> Project, we contribute to Debian GNU/Linux and to many other great
> Free Software projects upon which Tails is built. Thanks to other
> partners, we are connected with on-the-ground groups who make use of
> Tails, and can provide us with useful feedback and user experience.
> 
> Goals and values we share
> =========================
> 
> You can read more about Tails below, but in a nutshell, here is why we
> believe the Internet community may happily, and even proudly, want to
> support our work:
> 
> * We open the Internet, and make it more accessible: thanks to
> Tails, a lot of people can use Internet for things they could not
> do without putting themselves at risk otherwise.
> 
> * We are focused on providing the best possible privacy and
> security, but without harming usability: we believe it's the best
> way to empower people in their daily use of the Internet.
> 
> * We value education: our documentation for end-users is often
> praised, and we believe it is on par with the best one can find
> among open source IT security tools.
> 
> * We believe in community; Tails is a free software project
> developed in the open; the huge majority of the work has been done
> by volunteers since the beginning.
> 
> * We are good citizens of the FOSS community: we contribute to many
> FOSS projects we use or ship, including Debian and Tor; we are
> committed to contribute our improvements upstream.
> 
> * We make a difference: we've been around since 2009, and we're
> alive and kicking. Our user base doubles every 6-9 months; as we 
> are speaking, a Tails is started every 10 seconds, somewhere. #TODO: to be updated

How does that sound ? It is obviously missing a paragraph about mirrors. I can start writing it, but IHMO this introduction text can be a good base and be mostly keeped as it is now.

#2 Updated by intrigeri 2016-02-08 22:39:44

> I got a list of big mirrors hosters (Research networks, universities etc). This list includes some private contact informations, so i cannot share it publicly, but i can either share it privately

Yes: to me and u, please :)

> or use it to send proposal once the call is published.

It would indeed be awesome if we share the work to get in touch with potential new mirror operators, once we’re there.
I’m not sure we’re ready to give up our “OpenPGP key required” policy yet.
If you think we should give up this policy, can you please raise this topic on tails-dev@?

> For the call itself, maybe can we reuse part of the text we used to ask for support for the tails hackfest ?

Frankly, I don’t remember what was in there, and can’t find it anymore, so I can’t possibly have an opinion.

#3 Updated by Anonymous 2016-02-09 18:16:25

Thanks for working on this. I like to reuse the text indeed, as copied here above.

The call for mirrors text - I’d be glad if you could start writing it. Feel free to reassign this ticket to me for helping with this.

#4 Updated by intrigeri 2016-03-04 18:40:49

  • blocked by Feature #11054: Update contribute/how/mirror documentation with examples for the web servers we support added

#5 Updated by intrigeri 2016-03-04 18:41:22

Hey geb, did I miss an email with your list of big mirrors hosters?

#6 Updated by geb 2016-03-04 20:30:18

Hi,

Sorry, i had no time to check for it, but will try to do it within the next week.

#7 Updated by intrigeri 2016-03-25 21:23:26

Pinged geb over private email.

#8 Updated by intrigeri 2016-03-25 21:28:07

  • Priority changed from Normal to Elevated

Time has passed, bumping priority.

#9 Updated by Anonymous 2016-03-26 18:06:40

I started a draft here: https://pad.riseup.net/p/call-for-mirrors

#10 Updated by geb 2016-03-29 19:54:23

Hi,

- Intrigeri, you and u, should both have an email with a list of big reliable mirrors hosters.
- U, Thanks for the pad, sounds good. Got a few questions / suggests ; better to talk about that directly.

Thanks a lot for following this work.

#11 Updated by geb 2016-03-30 16:00:22

Hi again,

> I’m not sure we’re ready to give up our “OpenPGP key required” policy yet.
> If you think we should give up this policy, can you please raise this topic on tails-dev@?

I am sorry i won’t have time to open and follow this topic on tails-dev@. If you would like to open it, it would be great.
Otherwise we can give a try without, but the principal argument for dropping this requirement was that most of big mirrors operators don’t use GPG, so if we want to integrate them, its very likely that either we will have to drop this requirement, either they will all have to start using GPG … not sure it will work.

Sorry for my lack of havability to follow this topic.

u, I am adding a few comments to the pad. thanks !

#12 Updated by intrigeri 2016-04-08 01:08:11

  • blocked by Feature #11334: Document how HTTP mirrors get a unique virtualhost name added

#13 Updated by intrigeri 2016-04-08 07:18:29

  • Target version changed from Tails_2.3 to Tails_2.4

#14 Updated by intrigeri 2016-04-08 07:18:47

  • Assignee deleted (intrigeri)

#16 Updated by intrigeri 2016-05-05 04:37:47

I think we need to clarify our strategy, i.e. whether we’re going to issue a public, generic call for more mirrors (e.g. as a blog post), and/or a more target one sent privately to a list of selected mirror operators. I bet that the communication style would be quite different depending on which one we want to do.

I’m personally in favour of doing the latter (private email to selected mirror operators) first, in order to target fast mirrors that provide HTTPS. It’ll make Feature #10295 easier, and given enough mirror bandwidth with HTTPS enabled, it’ll be easier to drop mirrors that have no TLS support.

Still, per-contract our metrics of success is the raw number of mirrors we get in the end, so if that’s doesn’t give us enough mirrors, we can:

  • either do the blog post as a second step;
  • or, explain the sponsor that the new mirror pool setup allowed us to integrate a bunch of fast mirrors (thanks to dropping the requirement on a custom dl.a.b.o vhost), which nicely solves the problem just as well as having more mirrors (that we can also have now, thanks to escaping Tor’s DNS reply size limit).

#17 Updated by Anonymous 2016-05-10 02:02:58

  • Assignee set to intrigeri

intrigeri wrote:
> I think we need to clarify our strategy, i.e. whether we’re going to issue a public, generic call for more mirrors (e.g. as a blog post), and/or a more target one sent privately to a list of selected mirror operators. I bet that the communication style would be quite different depending on which one we want to do.

ack.

> I’m personally in favour of doing the latter (private email to selected mirror operators) first, in order to target fast mirrors that provide HTTPS. It’ll make Feature #10295 easier, and given enough mirror bandwidth with HTTPS enabled, it’ll be easier to drop mirrors that have no TLS support.

I agree completely. so, when you say selected mirror operators you mean from the list we have been given, correct?

> Still, per-contract our metrics of success is the raw number of mirrors we get in the end, so if that’s doesn’t give us enough mirrors, we can:
>
> * either do the blog post as a second step;

ack with this.

> * or, explain the sponsor that the new mirror pool setup allowed us to integrate a bunch of fast mirrors (thanks to dropping the requirement on a custom dl.a.b.o vhost), which nicely solves the problem just as well as having more mirrors (that we can also have now, thanks to escaping Tor’s DNS reply size limit).

that’s a second possibility. can we confirm this through metrics somehow?

intrigeri, please tell me if you want me to take care of publishing this call?

#18 Updated by intrigeri 2016-05-10 06:11:25

  • Assignee deleted (intrigeri)

> so, when you say selected mirror operators you mean from the list we have been given, correct?

Yes, that list is a good starting point. But it should not prevent us from thinking of potential mirrors that are not on that list :)

>> * or, explain the sponsor that the new mirror pool setup allowed us to integrate a bunch of fast mirrors (thanks to dropping the requirement on a custom dl.a.b.o vhost), which nicely solves the problem just as well as having more mirrors (that we can also have now, thanks to escaping Tor’s DNS reply size limit).

> that’s a second possibility. can we confirm this through metrics somehow?

Yes, we can measure mirrors speed (e.g. https://tails.boum.org/blueprint/HTTP_mirror_pool/#speed) and report about it.

> intrigeri, please tell me if you want me to take care of publishing this call?

Yes, please: IIRC, last time we revamped how the work on this deliverable is split, you took this one and I took over some of the communication with existing mirror operators. Speaking of which, I’m fine with taking care of leading Feature #8635 to completion. Will comment about it there. Works for you?

#19 Updated by Anonymous 2016-05-16 09:32:24

  • % Done changed from 0 to 10

That works. I’m sending you the call i wrote per email for review.

Also, I’d like to ask what you had in mind when talking about selected mirror operators. In the list i do indeed see some which i’d preferrably write to, but maybe you had something else in mind?

#20 Updated by intrigeri 2016-05-16 12:15:37

  • Assignee set to intrigeri
  • QA Check set to Ready for QA

> That works. I’m sending you the call i wrote per email for review.

OK, will review privately then. (Meta: given geb is willing to send the call personally to some of his contacts, and has been helping a bit so far, I would slightly prefer if we included him in the drafting process, but well: your call!)

> Also, I’d like to ask what you had in mind when talking about selected mirror operators. In the list i do indeed see some which i’d preferrably write to, but maybe you had something else in mind?

What I meant was the mirror operators we have on that list, that’s all :)

#21 Updated by intrigeri 2016-05-16 12:22:09

  • Status changed from Confirmed to In Progress
  • Assignee deleted (intrigeri)
  • % Done changed from 10 to 20
  • QA Check changed from Ready for QA to Dev Needed

> OK, will review privately then.

Done!

#22 Updated by Anonymous 2016-05-17 09:11:35

intrigeri wrote:
> > That works. I’m sending you the call i wrote per email for review.
>
> OK, will review privately then. (Meta: given geb is willing to send the call personally to some of his contacts, and has been helping a bit so far, I would slightly prefer if we included him in the drafting process, but well: your call!)

I did and based my work on his previous comments :)

But I’ll send him the current draft.

> > Also, I’d like to ask what you had in mind when talking about selected mirror operators. In the list i do indeed see some which i’d preferrably write to, but maybe you had something else in mind?
>
> What I meant was the mirror operators we have on that list, that’s all :)

Ack!

#23 Updated by Anonymous 2016-05-17 09:31:08

  • QA Check deleted (Dev Needed)

Resent a new text for review. I’d like to send this call very soonish so I hope i’ll get feedback soon from geb.

#24 Updated by Anonymous 2016-05-17 10:49:00

  • Assignee set to intrigeri
  • QA Check set to Ready for QA

Could you review the last version please? The only real modification is the beginning, everything else has not changed.
waiting for some follow-up communication by email.

#25 Updated by Anonymous 2016-05-17 11:06:03

  • Assignee deleted (intrigeri)
  • QA Check deleted (Ready for QA)

thanks, done. taking back the ticket.

#26 Updated by Anonymous 2016-05-17 11:14:53

  • % Done changed from 20 to 30

waiting for some details by email.

#27 Updated by intrigeri 2016-05-17 11:24:43

Replied, so reassigning.

#28 Updated by Anonymous 2016-05-17 17:32:05

  • Assignee set to geb

Assigning to geb for the missing mirrors. Please reassing this back to me when you’re done. Thanks!

#29 Updated by Anonymous 2016-05-18 12:30:45

I’ve reupdated the list and now sent to everybody on my list and even received some answers :)

#30 Updated by intrigeri 2016-05-21 10:24:50

geb: please make sure you Cc: tails-mirrors@boum.org when you send emails on your side :)

#31 Updated by Anonymous 2016-05-23 21:25:49

I’ve added notes about people answering this call in the first colum. The plus sign means that we’ve added the mirror to the pool.

#32 Updated by intrigeri 2016-05-25 10:52:45

geb, just a nitpick: we’re not asking to mirror tails.boum.org, so I suggest you adjust your intro for the next batch of email you send :)

#33 Updated by geb 2016-05-25 13:33:36

Hi,

Intrigeri. Ok, sorry, will try to make it more clear the next time. But according return i had from different people, it was clear enough thanks to the rest of the mail.

But, I just eddited the pad, because of a typo in the mail address to contact: it was tail-mirrors@boum.org instead of tails-mirrors@boum.org. If i may, maybe would i suggest to setup a temporary alias to have tail-mirrors@boum.org: tails-mirrors@boum.org to not miss answers.

Sorry to just discover that now.

#34 Updated by intrigeri 2016-06-06 03:03:37

geb, any status update?

#35 Updated by geb 2016-06-06 03:21:37

I am waiting from new from a couple of operators who told them privately they are interested.

For other, maybe should we consider sending a second request in a while.

Will try to discuss and sync with u for both those points.

#36 Updated by anonym 2016-06-08 01:34:53

  • Target version changed from Tails_2.4 to Tails_2.5

#37 Updated by Anonymous 2016-06-08 03:13:47

I think we can close this ticket, because the call has been sent.

#38 Updated by intrigeri 2016-06-08 04:54:45

> I think we can close this ticket, because the call has been sent.

I’d rather keep it open to track the fallout of sending the call, i.e. a last round of pings and handling replies (that’s not worth creating another ticket just for that IMO). As discussed over email this should be completed in a couple weeks.

#39 Updated by geb 2016-06-08 05:09:13

Hi,

> I’d rather keep it open to track the fallout of sending the call, i.e. a last round of pings and handling replies

I tend to agree. It may worth to send a last ping at a moment, at least for a really interesting mirroirs (the ones that offer >10G for example).

And, if i may, shouldn’t we publish the call directly in a new on the website ?

#40 Updated by intrigeri 2016-06-08 05:14:33

> And, if i may, shouldn’t we publish the call directly in a new on the website ?

We’ve now got enough mirrors, so I think it’s not worth it at this point (see Feature #11079#note-16 for the discussion we had about it).

#41 Updated by Anonymous 2016-06-26 10:03:22

  • Status changed from In Progress to Resolved
  • Assignee deleted (geb)
  • % Done changed from 30 to 100

I’ve relaunched one person who said they want to propose a mirror and one persone who needs to update their configuration. I think that we can now close this ticket, as i consider the last round of pinging done. We’ve now 42 mirrors :)

#42 Updated by BitingBird 2016-06-26 11:13:04

  • Priority changed from Elevated to Normal