Tails

#1 Updated by intrigeri 2014-11-08 09:08:32

• related to Feature #7980: Support obfs4 bridges added

#2 Updated by intrigeri 2014-11-08 09:08:43

• related to Feature #7909: Support ScrambleSuit bridges added

#3 Updated by intrigeri 2015-02-10 11:50:36

• Target version set to Tails_1.4

Given the feedback we got on tor-talk (“Which PTs shall we prioritize for inclusion in Tails?” thread), this should be our top-priority in this area, along with Feature #7980 (obfs4).

Now, we haven’t reached a practically usable conclusion in the “keeping up with pluggable transports by using TBB’s Tor and tor-launcher” thread, so it’s not clear how meek support should be implemented.

#4 Updated by BitingBird 2015-05-09 02:45:04

• Target version changed from Tails_1.4 to Tails_1.5

#5 Updated by BitingBird 2015-07-06 13:26:35

• Target version deleted (Tails_1.5)

#10 Updated by Anonymous 2017-06-29 14:42:49

• Type of work changed from Code to Wait

For now, we simply have to wait.

#11 Updated by Anonymous 2017-06-30 14:25:21

• related to Feature #8287: Explain what is required to have new pluggable transports added

#13 Updated by intrigeri 2017-09-14 12:54:52

• related to Bug #8775: Tor Launcher should be able to resolve DNS host names for the proxy configuration added

#14 Updated by intrigeri 2017-09-14 12:57:41

• Type of work changed from Wait to Research

#19 Updated by intrigeri 2017-12-12 08:33:18

• related to Bug #14555: Migrate to Tor Launcher compatible with Firefox ESR60 added

#23 Updated by iry 2018-01-03 16:09:39

• related to Feature #15144: Consider migrating from Tor Launcher to anon-connection-wizard added

#24 Updated by anonym 2018-01-18 20:50:54

• Status changed from Confirmed to In Progress
• Assignee set to intrigeri
• Target version set to Tails_3.6
• % Done changed from 0 to 40
• QA Check set to Ready for QA
• Feature Branch set to feature/8243-meek

The feature branch implements this ticket and its blocker, Bug #8775. Since you’ve already had an initial look I don’t think I have to say much about it, and I think you’ll be pretty happy with its current state, except:

• there’s no design doc changes or automated test suite updates yet. I’ll wait until Feature #15197 is merged in order to save me some work.

• commit:f5cbdd2d9008b79901ce5114d613a499b609697b enables Tor’s default Meek bridges in our Tor Launcher. My understanding is that it is the stealthiest pluggable transport (in practice, among the ones we support at least) despite requiring no user input, so it feels sane to ship them since users otherwise will be manually inputting the same text any way (and probably screwing up the “meek_lite” part). In fact, AFAIK there is no user-friendly way to learn about meek bridges. OTOH, my worry is that Tails might overload the meek bridges, since it is the only type of bridge we support. Thoughts?

So, I’m not asking for a merge, just for input on the above question, and a review and manual testing!

#26 Updated by intrigeri 2018-01-19 11:09:11

• related to Feature #8825: Provide default bridges added

#28 Updated by intrigeri 2018-01-19 12:41:40

• Assignee changed from intrigeri to anonym
• QA Check changed from Ready for QA to Dev Needed

intrigeri wrote:
> anonym wrote:
> > * commit:f5cbdd2d9008b79901ce5114d613a499b609697b enables Tor’s default Meek bridges in our Tor Launcher. My understanding is that it is the stealthiest pluggable transport (in practice, among the ones we support at least) despite requiring no user input, so it feels sane to ship them since users otherwise will be manually inputting the same text any way (and probably screwing up the “meek_lite” part). In fact, AFAIK there is no user-friendly way to learn about meek bridges. OTOH, my worry is that Tails might overload the meek bridges, since it is the only type of bridge we support. Thoughts?
>
> I have no idea how Tor Launcher looks like when only 1 type of default bridges is enabled => in order to understand if and how this is related to our reasoning and decision on Feature #8825, I’ll need to see this thing in action => I’ll build an ISO and will try it out.

I’ve tried it and ouch, that’s a tough one. I think I agree with your description of the problem. I’m not sure about what conclusion we can/should draw and how. To make sure we’re on the same page let me phrase how I see things using my own words:

• On the one hand, the reasoning that lead us (Feature #8825) to not propose default Tor bridges applies here even stronger: back then we were considering proposing all default bridges from TBB, which would have spread the load somewhat. So I don’t think we (you and I) can dismiss that alone and implement something that’s totally against something we’ve decided at the project scale.
• On the other hand, your UX argument makes sense: it would feel stupid to say “we now support meek but to use them, you need to locate the corresponding bridge addresses in the Tor Browser Git repo or some hidden wiki page; and then every time you start Tails, you’ll need to type them correctly and of course, don’t forget to replace ”meek" with “meek_lite”; what, wasn’t that obvious?".

So I propose we do this:

1. Once the other known issues are resolved, merge this without the default meek bridges. Don’t announce this as a new feature except in changelog. Rationale: the code is basically ready, let’s not let it bitrot. It might even be useful to have this code around, merged and tested for moat support, even without the default set of bridges; we’ll see, I have no idea how moat support will look like.
2. Wait for moat support in Tor Launcher to be ready for us to test it. Rationale: it might make this discussion moot (I wonder if the default bridges will still be a thing in TB once moat is there; and even if they’re not a thing anymore, for Tails users the benefits of having default bridges available becomes almost null once there’s a nice UI that lets them fetch their own bridges automatically).
3. If moat integration in Tor Launcher does not magically make this problem disappear, go back to the drawing board (i.e. reopen the project-wide discussion on Feature #8825 and/or seek advice from UX people).

I think this plan only makes sense if we are strongly committed to make step 2 real. It might be that we have to (and then it’s FT core work), it might be that we don’t have to (and then we need to do it as volunteers). Either way, count me in but not alone.

#29 Updated by Anonymous 2018-01-19 16:34:16

• related to Feature #15197: Upgrade to Tor Browser 7.5 added

#30 Updated by anonym 2018-01-26 11:44:29

• Type of work changed from Research to Code

intrigeri wrote:
> anonym wrote:
> > […] default Meek bridges […]

Let’s continue this discussion on Feature #8825.

> > a review
>
> Looks great to me modulo:
>
> * Let’s please not introduce new non-extended regexps. Sorry for being like a broken record.

Fixed in feature/8825-default-meek-bridges.

> * The “Remove TBB’s default bridges. If we want them, we want it in Tor Launcher’s own profile” comment is confusing: it seems to conflict with the fact we’re enabling the default meek bridges in /etc/xul-ext/tor-launcher.js. Or I misunderstood, which proves it’s confusing.

commit:53131b0a30cb09e8eaf1368cc9f7f02509312eb1 should make it clearer.

> * If you don’t know about grep’s --line-regexp, I find it useful to move complexity out of regexps, which can be interesting given the definition of ^ and $ varies accross the regexp languages (beginning/end of string vs. beginning/end of line).

I know about it, but personally prefer ^$ over some tool-specific option.

> * I suspect one of your commits fixes Bug #7903 (untested) => if I’m guessing right, please mark that ticket as RfQA to and consider rewriting history to encode this in the corresponding commit.

Actually it doesn’t: notice how /etc/resolv-over-clearnet.conf is copied into the chroot. A bind-mount should solve it, but might get tricky to get right given the mount namespace. I’ll have a quick look.

#31 Updated by anonym 2018-01-26 12:31:32

• Assignee changed from anonym to intrigeri
• QA Check changed from Dev Needed to Ready for QA

anonym wrote:
> > * I suspect one of your commits fixes Bug #7903 (untested) => if I’m guessing right, please mark that ticket as RfQA to and consider rewriting history to encode this in the corresponding commit.
>
> Actually it doesn’t: notice how /etc/resolv-over-clearnet.conf is copied into the chroot. A bind-mount should solve it, but might get tricky to get right given the mount namespace. I’ll have a quick look.

Never mind the part about “mount namespaces”, that’s for tor only. So the fix is a trivial bind-mount, with the “hard” thing being how to deal with that mount vs try_cleanup_browser_chroot(). findmnt to the rescue! Yup, the problem Bug #7903 wants to document a workaround for is now solved!

I have rewritten the history of feature/8243-meek, but I’ve only rebased on devel, reordered commits and squashed commits via fixup. You have reviewed up to and including commit:a993d3eb30f88bf0a7a085db7e63653979846502.

#32 Updated by intrigeri 2018-01-30 08:06:25

• Assignee changed from intrigeri to bertagaz

#33 Updated by intrigeri 2018-02-20 08:23:25

• related to Feature #15331: Support automatic bridge retrieval in Tor Launcher (Moat) added

#35 Updated by bertagaz 2018-02-21 12:48:50

• Assignee changed from bertagaz to anonym
• QA Check changed from Ready for QA to Info Needed

This branch automated test suite runs always end up with a failure of the “Starting the Unsafe Browser without a network connection results in a complaint about no DNS server being configured” scenario, in Jenkins and at home. It does not seem related to Bug #8775 though. Maybe Bug #7903? Did I miss something?

#39 Updated by anonym 2018-02-24 00:03:31

• Assignee changed from anonym to bertagaz
• QA Check changed from Info Needed to Ready for QA

bertagaz wrote:
> Pushed a commit installing systemd from stretch-backports.

Crap! The plan was for me to fix that before assigning it for review to you. I’m very sorry for the time wasted on debugging this! :/

Reassigning it back to you then! :)

#40 Updated by bertagaz 2018-02-24 10:33:07

• Assignee changed from bertagaz to anonym
• QA Check changed from Ready for QA to Info Needed

bertagaz wrote:
> This branch automated test suite runs always end up with a failure of the “Starting the Unsafe Browser without a network connection results in a complaint about no DNS server being configured” scenario, in Jenkins and at home. It does not seem related to Bug #8775 though. Maybe Bug #7903? Did I miss something?

You did not reply about this though. What’s the plan? I guess it’s expected that this test now fails (should have think twice before commenting) and we should just remove it?

#41 Updated by anonym 2018-02-24 12:50:47

• Assignee changed from anonym to bertagaz
• % Done changed from 40 to 50
• QA Check changed from Info Needed to Ready for QA

bertagaz wrote:
> bertagaz wrote:
> > This branch automated test suite runs always end up with a failure of the “Starting the Unsafe Browser without a network connection results in a complaint about no DNS server being configured” scenario, in Jenkins and at home. It does not seem related to Bug #8775 though. Maybe Bug #7903? Did I miss something?
>
> You did not reply about this though. What’s the plan? I guess it’s expected that this test now fails (should have think twice before commenting) and we should just remove it?

Gah! So it is expected because the error we look for has moved to after the Unsafe Browser verification. I updated the test for this, but cannot find the change (I even tried a bit using git reflog). I suspect I have done some work on this branch that has been lost… any way, now I think the branch should be actually ready for you to look at and possibly merge. Sorry for the messy review! :(

#42 Updated by bertagaz 2018-02-24 16:40:02

• % Done changed from 50 to 90
• QA Check changed from Ready for QA to Pass

anonym wrote:
> Gah! So it is expected because the error we look for has moved to after the Unsafe Browser verification. I updated the test for this, but cannot find the change (I even tried a bit using git reflog). I suspect I have done some work on this branch that has been lost… any way, now I think the branch should be actually ready for you to look at and possibly merge. Sorry for the messy review! :(

That’s fine. I’ve done more tests, and finished the code reviewing. I’m ok with it now, I think we’re good to release that in 3.6 so I’ll merge it, yay!

#43 Updated by bertagaz 2018-02-24 19:00:20

• Status changed from In Progress to Fix committed
• % Done changed from 90 to 100

Applied in changeset commit:4bd6d7b4e61e34dcc384b15f2322b7130fce6fc9.

#44 Updated by bertagaz 2018-02-24 19:02:31

• Assignee deleted (bertagaz)

And so is merged!

#45 Updated by intrigeri 2018-02-26 11:23:06

• Status changed from Fix committed to In Progress
• Assignee set to anonym
• % Done changed from 100 to 90
• QA Check changed from Pass to Dev Needed

Ouch, while reviewing ..origin/devel I just noticed that commit:bf317f15289f3c3beb966a9967d58f96debd0cd4 is incomplete: it only pins a subset of the binary packages built from src:systemd. E.g. in the .packages file I see libudev1:amd64 232-25+deb9u2. I really don’t think that mixing 2 different versions of systemd is a good idea.

Also, I believe the corresponding comment might be incorrect: we need systemd >= v233, not systemd > v233. Finally, commenting out that text with # is not supported. I’m surprised it seemingly works and I’d rather see us 1. be consistent with the rest of this file; 2. only use documented syntax instead of relying on luck.

#46 Updated by intrigeri 2018-02-26 11:23:27

• Priority changed from Normal to Elevated

#47 Updated by anonym 2018-02-26 11:36:36

• QA Check changed from Dev Needed to Ready for QA
• Feature Branch changed from feature/8243-meek to feature/8243-meek feature/8243-meek+force-all-tests

intrigeri wrote:
> Ouch, while reviewing ..origin/devel I just noticed that commit:bf317f15289f3c3beb966a9967d58f96debd0cd4 is incomplete: it only pins a subset of the binary packages built from src:systemd.

Good catch!

E.g. in the .packages file I see libudev1:amd64 232-25+deb9u2. I really don’t think that mixing 2 different versions of systemd is a good idea.

Agreed! Fixed! And I also pushed a +force-all-tests branch so we get to see the results of a full run given the udev bump.

> Also, I believe the corresponding comment might be incorrect: we need systemd >= v233, not systemd > v233. Finally, commenting out that text with # is not supported. I’m surprised it seemingly works and I’d rather see us 1. be consistent with the rest of this file; 2. only use documented syntax instead of relying on luck.

Fixed!

#49 Updated by anonym 2018-02-26 19:33:01

• QA Check changed from Ready for QA to Dev Needed

We have one run of each branch:

• https://jenkins.tails.boum.org/job/test_Tails_ISO_feature-8243-meek/1/
• https://jenkins.tails.boum.org/job/test_Tails_ISO_feature-8243-meek-force-all-tests/1/

Notably, the only scenario that failed in the normal (only @fragile) run was Boot with a hardware clock set way in the past and make sure that Tails sets the clock to the source date, which also failed in the force-all-tests run. It seems like the udev bump might have caused issues with at least the RTC used by our test suite VM. I will retry locally to get some more clarity.

#53 Updated by anonym 2018-02-27 14:57:39

• Assignee changed from anonym to bertagaz
• QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:
> bertagaz wrote:
> > It uses systemd’ build time to set the clock. Having a look at what time is set in the failed scenario (Sun Jan 28 15:58:39 UTC 2018.), it seems to correspond to systemd’s release/build date, so I think that’s the explanation. Not sure what to do about that though.
>
> FTR I see this exact same failure, with that exact same time, on a test suite run for an ISO built from a branch built on top of current devel. So it seems this regression was already introduced there and is not caused by the follow-up fixes anonym is working on. That commit first appeared in systemd v229 so we’re only been “lucky” not to be hit by that test suite bug earlier.

Indeed! I sneaked the fix into this branch any way. :)

#54 Updated by bertagaz 2018-02-28 13:37:18

• Status changed from In Progress to Fix committed
• Assignee deleted (bertagaz)
• % Done changed from 90 to 100
• QA Check changed from Ready for QA to Pass

anonym wrote:
> intrigeri wrote:
> > FTR I see this exact same failure, with that exact same time, on a test suite run for an ISO built from a branch built on top of current devel. So it seems this regression was already introduced there and is not caused by the follow-up fixes anonym is working on. That commit first appeared in systemd v229 so we’re only been “lucky” not to be hit by that test suite bug earlier.
>
> Indeed! I sneaked the fix into this branch any way. :)

Works fine so it’s merged into devel. Let’s time this time we’re good. :)

#55 Updated by bertagaz 2018-03-14 11:06:27

• Status changed from Fix committed to Resolved

#56 Updated by mercedes508 2018-08-24 15:46:08

• has duplicate Bug #15836: Meek-lite added

#57 Updated by mercedes508 2018-10-15 17:53:20

• has duplicate Feature #15972: Adding meek bridges to Tails added

#58 Updated by emmapeel 2019-06-11 08:29:08

• Status changed from Resolved to Confirmed
• Target version deleted (Tails_3.6)

after some discussion on XMPP, we don’t see this issue as solved. reopening thus.

#59 Updated by intrigeri 2019-08-08 11:15:06

• related to Feature #5461: Persistence preset: Tor configuration added

#60 Updated by anonym 2019-08-11 20:43:59

• Status changed from Confirmed to In Progress

Applied in changeset commit:tails|e4115249b2bdf26690d39f6cb30252f079d948db.

#61 Updated by intrigeri 2019-08-11 20:51:01

• Status changed from In Progress to Confirmed

#63 Updated by johanbluecreek 2020-02-06 17:29:56

• Feature Branch changed from feature/8243-meek feature/8243-meek+force-all-tests to https://gitlab.com/johanbluecreek/tails.git:feature/8243-meek

This branch is based on wip/feature/15331-moat, which was trying to use Moat. Moat, that is selecting “Request a bridge from torproject.org” in configuring bridges part of the Tor Launcher, does not work for Tails, as that requires internet connection to receive a captcha and the get the bridges.

Instead, this branch gives support for Meek. As suggested by @intrigeri , it makes use of the meek_lite that is a part of the Tor Browser’s obfs4proxy. The debian package obfs4proxy is removed, and the radio-button suggesting Moat could work is removed.

The branch has been built and tested. The bridges works fine, which is controlled by comparing the fingerprint reported for the circuits listed by clicking on the Onion-applet, once tor has come up, and the fingerprint of the line added to /etc/tor/torrc for Meek. I also “simulated” a Chinese user where the clock would have been misconfigured, simply by `date -s +8hours` before the Tor Launcher was started to configure the bridges. This test also worked and time was synced and bridged circuits were established.

#65 Updated by johanbluecreek 2020-02-07 09:09:52

• Status changed from Confirmed to Needs Validation

intrigeri wrote:
> What’s the next step here?
>
> If it’s “someone should push this to our CI and review the branch”, please set “Status” to “Needs Validation” :)

Sorry, yeah; push to CI and review. I’m not sure what to set the “% Done” to, but switched “Status”.

#66 Updated by anonym 2020-02-07 09:33:30

• Status changed from Needs Validation to In Progress
• Assignee set to johanbluecreek

johanbluecreek wrote:
> […] and the radio-button suggesting Moat could work is removed.

This will break our automated test suite’s bridge scenarios. They all use this step:

When /^I configure some (\w+) pluggable transports in Tor Launcher$/ do |bridge_type|
  @screen.wait_and_click('TorLauncherConfigureButton.png', 10)
  @screen.wait_and_click('TorLauncherBridgeCheckbox.png', 10)
  @screen.wait_and_click('TorLauncherBridgeList.png', 10)
[...]

You can find the corresponding images in features/images. This step assumes that clicking the “Tor is censored” checkbox will make the “custom bridges” text area visible, but with your branch it will require an extra click, something like:

When /^I configure some (\w+) pluggable transports in Tor Launcher$/ do |bridge_type|
  @screen.wait_and_click('TorLauncherConfigureButton.png', 10)
  @screen.wait_and_click('TorLauncherBridgeCheckbox.png', 10)
  @screen.wait_and_click('TorLauncherBridgeCustomRadio.png', 10)
  @screen.wait_and_click('TorLauncherBridgeList.png', 10)
[...]

where TorLauncherBridgeCustomRadio.png is an image of the radio button and label saying “Provide a bridge I know”. I don’t think there is much of a point pushing this to our infra before this is fixed.

Are you up for this? Otherwise I could do this part.

Also, before we can merge this our bridge docs (wiki/src/doc/first_steps/startup_options/bridge_mode.mdwn) needs to be updated to reflect that the default choice after selecting “Tor is censored” now is Meek (not custom bridges), and doesn’t need further configuration. And I guess we need some explanations/comparisons of the options? A job for @sajolida, I think! (?)

However, this made me start thinking. This change in Tails probably will start driving a lot more users to the Meek bridges, which could be a problem. Vanilla Tor Launcher, after all, has bundled obfs4 bridges as the default, not Meek. I think we (I volunteer) should ask tor-dev@ what they think about this, in particular if they think the Meek bridges can handle this. Do we have some sort of data about bridge usage in Tails? I guess we have some from Whisperback reports, but I wonder how representative that is.

Worst case, if we cannot use Meek as the default, maybe we just have to give up on our opinion that “public bridges == bad” and provide the bundled obfs4 bridges as default as well, i.e. exactly the same as in vanilla Tor Launcher.

@intrigeri, what do you think?

#69 Updated by anonym 2020-03-18 13:43:37

• Assignee changed from johanbluecreek to anonym

sajolida wrote:
> I’m very happy to see support for meek bridges move forward but I’ll refrain from preparing updates to the doc until the code is ready.

Cool! I think this whole thing also is blocked by my fears in Feature #8243#note-66 about overloading the Meek bridges. I’m gonna ask the Tor devs what they think, but I want to provide them some sort of ballpark figure; see your inbox, @sajolida! :)

johanbluecreek wrote:
> sajolida wrote:
> > @anonym: If you reassigned this ticket to johanbluecreek for the updates to the test suite, I’d suggest not blocking on them anymore (6 weeks have passed).
>
> My bad. I forgot to push and tell you about those updates. Updated branch on my gitlab has a fix for the test suite. The meek support requires an extra radio-button click to be able to run the test for the step of manually adding bridges, and that has been added to this branch.

I’ve pulled and pushed to our CI, so let’s see what Jenkins thinks! :) I’ll report back here.

I assigned the ticket to myself since I’m the blocker for moving this forward.

#70 Updated by anonym 2020-03-24 11:43:16

• Assignee changed from anonym to johanbluecreek

Jenkins is happy! Out of 4 attempts, none had any related failures!

However, I have consulted tor-dev@ about the Meek bridge capacity issue, and the answer is that its already overloaded. This thread has the details: https://lists.torproject.org/pipermail/tor-dev/2020-March/014189.html

So, in order to add Meek support we need to add another default that is ~equally convenient. Our options are:

• a default set of bridges (e.g. the one shipped in Tor Browser) which we want to avoid due to our threat model.
• Moat, which we so far have ignored since it requires clearnet access.

So I’d like us to investigate Moat support and consider adding it as the default. I think all we have to do is to let the tor-launcher user access to whatever CDN Meek is hosted (if you are confused: Moat fetches the bridges using Meek as the transport! :)). There’s also a captcha involved, but I hope the image is forwarded through Moat and not served via something like recaptcha.google.com so we have to make a firewall exception for that as well.

@johanbluecreek, are you interested in digging into this?

#75 Updated by sajolida 2020-04-24 18:21:04

• Description updated