Bug #10497

wait_until_tor_is_working helper is fragile

Added by anonym 2015-11-06 10:36:31 . Updated 2016-06-08 01:30:47 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Test suite
Target version:
Start date:
2015-11-06
Due date:
% Done:

100%

Feature Branch:
test/10497-tor-bootstrap-is-fragile
Type of work:
Research
Blueprint:

Starter:
Affected tool:
Deliverable for:
270

Description

We did what we could, basically, with Feature #9516. The real fix is something like Bug #10238.

commit:f8e5fb9 and commit:1c48ca26 tagged these tests as fragile.


Subtasks


Related issues

Blocked by Tails - Feature #9521: Use the chutney Tor network simulator in our test suite Resolved 2016-04-15
Blocks Tails - Bug #10376: The "the Tor Browser loads the (startup page|Tails roadmap)" step is fragile Resolved 2015-10-15
Blocks Tails - Bug #10381: The "I open the address" steps are fragile Resolved 2015-10-15
Blocks Tails - Bug #10442: Totem "Watching a WebM video over HTTPS" test never passes on Jenkins Resolved 2015-10-28
Blocks Tails - Bug #10440: Time syncing scenarios are fragile Resolved 2015-10-28
Blocked by Tails - Bug #11401: robust_notification_wait sometimes opens the Applications menu which breaks tests Resolved 2016-05-05
Blocked by Tails - Bug #11398: Florence sometimes hides other windows, which breaks tests Resolved 2016-05-05

History

#1 Updated by anonym 2015-11-06 10:36:40

  • blocks #8668 added

#2 Updated by anonym 2015-11-06 10:36:54

  • blocked by Bug #10238: Investigate why Tor sometimes is slow at or even fails bootstrapping added

#3 Updated by anonym 2015-11-06 10:37:11

  • Assignee changed from amette to anonym

Whoops, wrong assignee! :)

#4 Updated by anonym 2015-11-06 10:46:09

#5 Updated by anonym 2015-11-06 12:18:37

  • Target version set to Tails_2.3

#6 Updated by bertagaz 2015-11-16 04:06:31

  • Target version changed from Tails_2.3 to Tails_1.8

This bug has happened 19 times since Nov 6, 2015 and the last @fragile tagging, and has become one of the most important. It should probably be worked on quickly, so I’m updating its target version.

#7 Updated by intrigeri 2015-11-16 04:18:49

  • Type of work changed from Wait to Research

(Let’s not encode the “blocked by” relationship in two different ways.)

#8 Updated by intrigeri 2015-12-05 13:25:36

  • Target version changed from Tails_1.8 to Tails_2.0

(We’re going to mark as fragile all tests that depend on Tor to have bootstrapped for the moment => not so urgent.)

#9 Updated by bertagaz 2015-12-07 08:14:12

We marked the related scenarios as fragile in commit:f8e5fb9

#10 Updated by intrigeri 2015-12-07 08:38:30

  • Description updated

#11 Updated by anonym 2015-12-18 04:13:48

  • Status changed from Confirmed to Fix committed
  • % Done changed from 0 to 100

Applied in changeset commit:cd53d12c651c3713289d983e2c09551f8c14a898.

#12 Updated by intrigeri 2015-12-18 12:10:27

  • Status changed from Fix committed to Confirmed
  • % Done changed from 100 to 0

(Last change was caused by a typo.)

#13 Updated by anonym 2016-01-27 14:36:49

  • Target version changed from Tails_2.0 to Tails_2.2

#14 Updated by anonym 2016-02-20 13:19:34

  • Priority changed from Normal to Elevated
  • Target version changed from Tails_2.2 to Tails_2.3

Hoping that Bug #10238 will lead us to the solution. Otherwise simulating the Tor network locally with Chutney (Feature #9521) is an alternative.

#15 Updated by intrigeri 2016-02-20 13:20:24

  • Priority changed from Elevated to Normal
  • Target version changed from Tails_2.3 to Tails_2.2
  • Feature Branch set to test/10497-tor-bootstrap-is-fragile

#16 Updated by intrigeri 2016-02-20 13:21:25

  • Priority changed from Normal to Elevated
  • Target version changed from Tails_2.2 to Tails_2.3

#17 Updated by intrigeri 2016-02-20 14:58:51

  • blocked by deleted (#8668)

#18 Updated by intrigeri 2016-02-20 14:59:19

  • blocked by Feature #9521: Use the chutney Tor network simulator in our test suite added

#19 Updated by intrigeri 2016-02-20 14:59:29

  • Target version changed from Tails_2.3 to Tails_2.4

#20 Updated by intrigeri 2016-05-18 13:16:54

  • Assignee changed from anonym to intrigeri

Will give it a try.

#21 Updated by intrigeri 2016-05-18 14:57:06

  • blocks deleted (Bug #10238: Investigate why Tor sometimes is slow at or even fails bootstrapping)

#22 Updated by intrigeri 2016-05-18 15:02:58

  • Status changed from Confirmed to In Progress
  • Assignee changed from intrigeri to anonym
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA

(or should it be for bertagaz, since I took this over from anonym, and bertagaz committed to do reviews’n’merges for anonym’s test suite stuff?)

So, I’ve made sure that all scenarios that have failed on this branch since we have chutney are flagged as fragile (on the devel branch), except the too general ones we didn’t want to mark as fragile (e.g. those caused by Bug #11398), then I’ve merged devel into this branch.

I’m confident that the test results will be good, but please give it a few days and look at https://jenkins.tails.boum.org/job/test_Tails_ISO_test-10497-tor-bootstrap-is-fragile/ to make sure it’s all right. If it isn’t, please reassign to me.

#23 Updated by intrigeri 2016-05-18 15:10:09

  • blocks Bug #10376: The "the Tor Browser loads the (startup page|Tails roadmap)" step is fragile added

#24 Updated by intrigeri 2016-05-18 15:10:32

  • blocks Bug #10381: The "I open the address" steps are fragile added

#25 Updated by intrigeri 2016-05-18 15:13:41

  • blocks Bug #10442: Totem "Watching a WebM video over HTTPS" test never passes on Jenkins added

#26 Updated by intrigeri 2016-05-18 15:29:15

  • blocks Bug #10440: Time syncing scenarios are fragile added

#27 Updated by intrigeri 2016-05-20 18:34:10

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Ready for QA to Dev Needed

Too many false positives on Jenkins, I’ll flag them as @fragile and file tickets before re-submitting. Sorry for the noise!

#28 Updated by intrigeri 2016-05-22 09:29:39

  • blocked by Bug #11401: robust_notification_wait sometimes opens the Applications menu which breaks tests added

#29 Updated by intrigeri 2016-05-22 10:59:55

  • blocked by Bug #11398: Florence sometimes hides other windows, which breaks tests added

#30 Updated by intrigeri 2016-05-25 10:23:10

  • % Done changed from 50 to 60

Now that I’ve marked a few more tests as fragile, this seems to be pretty robust. However, this branch has the one for Bug #11398 merged in, and it seems that this might introduce a new robustness problem, so I want to have that other one fixed before we merge test/10497-tor-bootstrap-is-fragile.

#31 Updated by intrigeri 2016-05-27 13:45:32

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Dev Needed to Ready for QA

#32 Updated by anonym 2016-06-01 19:53:56

  • Status changed from In Progress to Fix committed
  • Assignee changed from anonym to intrigeri
  • % Done changed from 60 to 100

It looks good => merged! However, I pushed the following tiny improvements on top which you might want to ack before closing this ticket properly:

22a81c3 Optimize scenario by not using the network.
3fc0368 Move feature-wide @fragile tag to the only affected scenario.
e036f60 Explain why the whole feature is @fragile.

#33 Updated by intrigeri 2016-06-02 13:21:33

  • QA Check changed from Ready for QA to Pass

> 22a81c3 Optimize scenario by not using the network.
> 3fc0368 Move feature-wide fragile tag to the only affected scenario. > e036f60 Explain why the whole feature is fragile.

ACK

#34 Updated by intrigeri 2016-06-02 13:21:53

  • Assignee deleted (intrigeri)

#35 Updated by anonym 2016-06-08 01:30:48

  • Status changed from Fix committed to Resolved