Bug #16555

Thunderbird "my Thunderbird inbox is non-empty" test consistently fails

Added by intrigeri 2019-03-14 13:55:43 . Updated 2019-03-20 14:28:49 .

Status:
Resolved
Priority:
Normal
Assignee:
CyrilBrulebois
Category:
Test suite
Target version:
Tails_3.13
Start date:
2019-03-14
Due date:
% Done:

90%

Feature Branch:
bugfix-16422-thunderbird-60.5.1
Type of work:
Research
Blueprint:

Starter:
Affected tool:
Email Client
Deliverable for:

Description

It seems that since March 3, this step of “Scenario: Thunderbird can download the inbox with POP3” fails every time it runs, both on our shared Jenkins and on my local one:

    Then my Thunderbird inbox is non-empty                                            # features/step_definitions/thunderbird.rb:219
      <false> is not true. (Test::Unit::AssertionFailedError)
      features/thunderbird.feature:49:in `Then my Thunderbird inbox is non-empty'

Some of the failures were caused by the mailbox being indeed empty (no idea why) but the test still fails while I’ve ensured there’s a message there (confirmed by the test video). I suspect the Dogtail code that checks this is buggy.

Subtasks

Related issues

Blocks Tails - Feature #13241: Core work: Test suite maintenance Rejected 2017-06-29
Blocked by Tails - Bug #16422: Upgrade Thunderbird to 60.5.1 Resolved 2019-02-05

History

#1 Updated by intrigeri 2019-03-14 13:56:02

#2 Updated by intrigeri 2019-03-14 13:56:18

  • related to Bug #16422: Upgrade Thunderbird to 60.5.1 added

#3 Updated by intrigeri 2019-03-18 06:08:08

  • Status changed from Confirmed to In Progress
  • Assignee set to intrigeri
  • Target version set to Tails_3.13
  • QA Check set to Ready for QA
  • Feature Branch set to bugfix-16422-thunderbird-60.5.1

I understand commit:b02cfe43722c7f1fdfb5d7a8a24c9fdca03cc588 may fix this :)

#4 Updated by intrigeri 2019-03-18 07:09:06

  • related to deleted (Bug #16422: Upgrade Thunderbird to 60.5.1)

#5 Updated by intrigeri 2019-03-18 07:09:12

  • blocked by Bug #16422: Upgrade Thunderbird to 60.5.1 added

#6 Updated by intrigeri 2019-03-18 07:09:40

  • % Done changed from 0 to 90
  • QA Check changed from Ready for QA to Pass

Yeah, the aforementioned commit does fix this scenario \o/

#7 Updated by intrigeri 2019-03-18 09:06:58

  • Assignee changed from intrigeri to CyrilBrulebois

@CyrilBrulebois, please mark as “Fix committed” once the branch for Bug #16422, that has this fix, is merged.

#8 Updated by CyrilBrulebois 2019-03-18 13:40:16

Sorry I missed this ticket; I suppose I didn’t actually search for it because we weren’t supposed to have 60.5.1 just yet? I suppose your failures weren’t on the stable branch?

#9 Updated by intrigeri 2019-03-18 14:20:12

> I suppose your failures weren’t on the stable branch?

My failures were on branches based on stable, that included the version of Thunderbird and its test suite that’s in 3.12.1. I have no clue why the problem started to occur on March 3. FWIW, I’ve initially suspected someone did $things in the shared test mailbox that changed the behaviour of the test cases. The only thing I’ve found that correlates wrt. timing is that I’ve set up a new Jenkins isotester around the time the problem appeared, but I don’t see how that’s related. Anyways :)

#10 Updated by CyrilBrulebois 2019-03-18 14:29:39

  • Status changed from In Progress to Fix committed

OK… Still curious but I don’t have time to go look at the sources to see what changed and when, so… I’ll let it go, let it goooooo.

Switching to fix-committed manually, since that wasn’t in the merge commit.

#11 Updated by CyrilBrulebois 2019-03-20 14:28:49

  • Status changed from Fix committed to Resolved