Tails

#1 Updated by intrigeri 2013-07-25 07:17:26

• Parent task set to Feature #5479
• Starter set to No

#2 Updated by intrigeri 2013-07-25 07:24:10

• Parent task changed from Feature #5479 to Feature #5920

#3 Updated by intrigeri 2013-12-08 14:33:18

• Assignee set to anonym

#4 Updated by intrigeri 2014-03-05 01:43:38

• Assignee changed from anonym to sajolida
• % Done changed from 0 to 50
• QA Check set to Ready for QA

anonym says: “shouldn’t be relevant since Tor Launcher (started by the NM hook before the one starting the time syncing) blocks until settings that actually makes Tor build a circuit are set.”

Reassigning to sajolida who’s testing that stuff, then, so that he can confirm.

#5 Updated by intrigeri 2014-03-06 16:29:38

• Subject changed from bridge mode vs tordate timeouts to bridge mode vs. time syncing
• Assignee changed from sajolida to anonym
• % Done changed from 50 to 10
• QA Check changed from Ready for QA to Dev Needed

In 0.23~rc1, time syncing in bridge mode still has quite some issues.

#6 Updated by intrigeri 2014-04-06 09:38:13

• Tracker changed from Feature to Bug
• Subject changed from bridge mode vs. time syncing to Fix time syncing in bridge mode

#7 Updated by anonym 2014-04-10 12:45:20

• Assignee changed from anonym to intrigeri
• QA Check changed from Dev Needed to Info Needed

intrigeri wrote:
> In 0.23~rc1, time syncing in bridge mode still has quite some issues.

Can you clarify on how “bridge mode still has quite some issues”?

It works quite well for me in 0.23, and I’ve done some extensive testing with different time skews: /- 10 years,/- 1 year, /- 1 month,/- 3 days, /- 6 hours, and no skew. Tor always manages to bootstrap and build a circuit for me. I even tested at different times of the hour to take into account the age of the consensus (since I have vague suspicions that there are corner cases related to this in general, not just in bridge mode). In total I saw around 40 successful Tails boots Tor bootstraps and no failure.

The only issue I’ve found is the following, which we knew about even before 0.23-rc1: if the clock is way off (outside the picked authority’s certificate lifetime), we reload Tor (i.e. send a `SIGHUP` to it). Tor Launcher isn’t designed to handle this, so it’ll show an error stating that “Tor failed to establish a Tor network connection”. In contrast to that error, Tor will bootstrap just fine, but the error will be shown until Tor Launcher is killed by the 60-tor-ready-notification.sh hook, which definitely should be enough time for the user to read it and get confused. There’s no really good fix for this in our time syncing mess. I suppose it could be made less confusing by killing Tor Launcher before reloading Tor, but that’s still inconsistent with the behaviour in the other cases, and hence potentially confusing. Still “clock is way off” is a (supposedly) rare corner case, so I wonder if we should care.

Similarly, when the time syncing script restarts Tor (e.g. if the clock is not way off, but outside the consensus’ validity interval), Tor Launcher’s bootstrap progress meter will get stuck until it’s killed by the 60-tor-ready-notification.sh hook later. While not pretty, I don’t think there’s much room for confusion since there will be two other indicators that Tor is working (Vidalia icon and GNOME notification).

#9 Updated by intrigeri 2014-04-11 09:29:56

• Assignee changed from intrigeri to anonym
• QA Check deleted (Info Needed)

#10 Updated by intrigeri 2014-04-15 14:51:01

• QA Check set to Info Needed

#12 Updated by anonym 2014-04-29 11:29:17

• Target version changed from Tails_1.0 to Tails_1.1

#13 Updated by intrigeri 2014-06-10 10:46:18

• Assignee deleted (anonym)
• Target version deleted (Tails_1.1)

#14 Updated by intrigeri 2014-06-10 10:46:42

• Category set to Tor configuration
• Parent task deleted (Feature #5920)

#16 Updated by anonym 2015-02-19 12:02:09

• Assignee set to anonym
• QA Check changed from Info Needed to Dev Needed

I believe I’ve found the issue while developing the automated bridge tests (yay!), where I also introduce time syncing tests while in bridge mode. To explain why I couldn’t reproduce it, it may have been because I only tested with one bridge, but those who could see this tested with >1. Yeah… 20-date.sh is good stuff.

When the time is so wrong that tor_cert_lifetime_invalid fails, we set the time according to tor_cert_valid_after. The latter will parse Tor’s log and extract the time from the last certificate lifetime runs from $DATE line it finds. If multiple bridges are given, we will see several of these lines, but only one, the last, will be picked for setting the date. If that one has a valid-after date which is before all of the other ones, we may end up with a problem: it seems Tor will not necessarily retry all of the bridges, and may only end up retrying the bridges we didn’t set our date according to. In that case our system clock may be before the other bridges’ certificates’ valid-after, and we’re stuck.

The solution is to some how wait until we’ve gotten the valid-after for all our bridges certificates, and then pick the highest date. I’m gonna give this a try…

#17 Updated by Tails 2015-02-19 13:10:48

• Status changed from Confirmed to In Progress

Applied in changeset commit:4b52e279833b75346009c4d8eeb3ac1ebca6442f.

#18 Updated by anonym 2015-02-19 16:27:53

• % Done changed from 10 to 30
• Feature Branch set to bugfix/5394-time_syncing_in_bridge_mode

Pushed something that’s not been tested at all. Could even be syntax errors in it.

#19 Updated by anonym 2015-02-19 21:36:34

• Assignee deleted (anonym)
• % Done changed from 30 to 50
• QA Check changed from Dev Needed to Ready for QA

The branch seems to fix the problem as I understand it. It helps to run time_syncing.feature on an image built from this branch (bugfix/5394-time_syncing_in_bridge_mode) with the test/6305-tor-bridges branch (not merged) checked out. To verify the error, also try running some other ISO with 3 or so branches configured in features/config/local.yml.

#20 Updated by intrigeri 2015-02-20 09:04:33

• Assignee set to intrigeri
• Target version set to Tails_1.3

#21 Updated by intrigeri 2015-02-20 09:17:52

• Assignee changed from intrigeri to anonym
• QA Check changed from Ready for QA to Dev Needed

Here’s a quick initial code review:

• typo in set_date_accoring_to_tor_certs
• maybe it’s worth updating the corresponding design doc?
• nr_bridges=$(grep '^Bridge' "${TOR_RC}" | wc -l) would be more robust if the regexp had \s+ appended (needs -E option), as there are other torrc options whose name starts with “Bridge”

#22 Updated by anonym 2015-02-20 09:45:34

• Assignee changed from anonym to intrigeri
• QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:
> Here’s a quick initial code review:
>
> * typo in set_date_accoring_to_tor_certs
> * maybe it’s worth updating the corresponding design doc?

Fixed.

> * nr_bridges=$(grep '^Bridge' "${TOR_RC}" | wc -l) would be more robust if the regexp had \s+ appended (needs -E option), as there are other torrc options whose name starts with “Bridge”

Fixed, but I didn’t see the need for the +.

#23 Updated by intrigeri 2015-02-20 15:32:54

• % Done changed from 50 to 60

Code review now passes.

#24 Updated by intrigeri 2015-02-20 21:26:16

• Assignee changed from intrigeri to anonym
• QA Check changed from Ready for QA to Info Needed

anonym wrote:
> The branch seems to fix the problem as I understand it. It helps to run time_syncing.feature on an image built from this branch (bugfix/5394-time_syncing_in_bridge_mode) with the test/6305-tor-bridges branch (not merged) checked out.

I did that and it passed.

> To verify the error, also try running some other ISO with 3 or so branches configured in features/config/local.yml.

I did that, with an ISO built from testing branch 2 days ago, and it passed too. It shouldn’t, right?

#26 Updated by anonym 2015-02-22 13:12:45

• Assignee deleted (anonym)
• Target version deleted (Tails_1.3)
• % Done changed from 60 to 0
• QA Check changed from Info Needed to Dev Needed
• Type of work changed from Code to Research

intrigeri wrote:
> anonym wrote:
> > To verify the error, also try running some other ISO with 3 or so branches configured in features/config/local.yml.
>
> I did that, with an ISO built from testing branch 2 days ago, and it passed too. It shouldn’t, right?

Well, it seemed like there was an element of randomness to it, e.g. that we happened to pick a bridge whose certificate had the lowest valid-after date, and that Tor chose to not retry that particular bridge for whatever reason. It did happen quite often for me with a certain set of (3) bridges, though. However, when I retry the exact same thing today, I cannot reproduce it. I do see completely different valid-after dates for these bridges today, though. Hmm.

> > I did that, with an ISO built from testing branch 2 days ago, and it passed too. It shouldn’t, right?
>
> Same with 1.3~rc1, so there’s definitely a problem somewhere.

Sadly, yes…

> Are there any requirements about these bridges? I’ve used 3 regular bridges, and didn’t check if they are all working right now.

As I understood it, you’d need >1 bridges with distinct valid-after dates on their certs. That’s all, but it seems incorrect.

#30 Updated by intrigeri 2016-07-29 07:48:54

• related to Bug #11589: Time syncing over bridge is fragile added

#31 Updated by Anonymous 2017-06-28 08:13:49

• Assignee set to anonym

Hi @anonym, could you please check how relevant this ticket still is? Has this branch been merged? Do we still need it? Thanks!

#32 Updated by intrigeri 2019-10-20 07:48:35

• Assignee changed from anonym to intrigeri
• Target version set to Tails_4.1
• Type of work changed from Research to Code

This branch was never merged in the end. My understanding is that it’s because the branch was never sent back to the reviewer after anonym provided extra info about if/how one could reproduce the problem this ticket was originally about. AFAICT this problem can still happen and it would be nice to fix it, both for our users and our test suite (Bug #11589, which is one of the last reasons why we need +force-all-tests for every branch that affects Tor Launcher, which made me land here: our developer & reviewer experience is seriously harmed these days by the fact developers need to reason about "do I need to add +force-all-tests and sometimes forget).

I’ll resurrect this branch and will try to lead this to completion. If that’s enough to fix Bug #11589, I’ll call this very ticket done too, and then we’ll keep our ears open to see if there are other cases when time sync’ing is broken in bridge mode for actual users (outside of our test suite).

#33 Updated by intrigeri 2019-11-08 18:53:45

• Feature Branch changed from bugfix/5394-time_syncing_in_bridge_mode to bugfix/5394-time_syncing_in_bridge_mode+force-all-tests

#34 Updated by intrigeri 2019-11-08 19:03:27

• Subject changed from Fix time syncing in bridge mode to Time syncing can break Tor bootstrap when using several bridges with different certificate lifespans
• Description updated

#36 Updated by intrigeri 2019-11-08 19:51:56

• Assignee deleted (intrigeri)
• Target version deleted (Tails_4.1)

I’ve tried to reproduce the bug described in Bug #5394#note-16 with Tails 4.0, using:

• 6 different bridges (2 I’m used to using + 4 ones I freshly got from BridgeDB) each time
• a VM with a hardware clock set 24h after the current UTC time

I’ve started Tails 8 times in this configuration. Every time, the clock was corrected and Tor bootstrapped just fine. So I don’t see how I can make progress here without looking at the certificate lifetime of each of these bridges. Either way, this problem seems unlikely to affect real-world users often ⇒ I’m tempted to classify this as a test suite problem, and to reject this ticket in favor of Bug #11589 (which might itself be solved by Bug #16792). Other opinions welcome wrt. what to do with this old ticket.

Anyway, given my initial motivation for working on this is moot (see previous comment), so I’m giving up.