Bug #17125

"Persistent browser bookmarks" test suite scenario regression

Added by intrigeri 2019-10-06 03:33:19 . Updated 2020-04-15 06:02:26 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Test suite
Target version:
Tails_4.0
Start date:
Due date:
% Done:

50%

Feature Branch:
devel
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Welcome Screen
Deliverable for:

Description

I’ve just had a look at a few test suite runs on the devel branch and devel-based branches that completed in the last hours, and “Scenario: Persistent browser bookmarks” failed every time with “FindFailed: can not find TailsGreeterPersistenceUnlocked.png”.

The timing vs. the merge of Feature #17098, Feature #16095 and friends, makes me suspicious, although I can’t recall seeing this on that branch before I merged it yesterday.

Let’s at least investigate in time for 4.0. If that’s test suite fragility, “fine”; if that’s a potentially-user-visible regression, let’s try to fix it.

Files

04_58_18_Persistent_browser_bookmarks.mkv (809757 B) intrigeri, 2019-10-06 05:05:27

Subtasks

Related issues

Related to Tails - Bug #12461: Test suite fails to enable persistence in "Persistent browser bookmarks" In Progress 2017-04-19
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

History

#1 Updated by intrigeri 2019-10-06 03:33:40

#2 Updated by intrigeri 2019-10-06 05:02:52

  • Assignee set to intrigeri

(For the initial investigation.)

#3 Updated by intrigeri 2019-10-06 05:24:55

  • related to Bug #12461: Test suite fails to enable persistence in "Persistent browser bookmarks" added

#4 Updated by intrigeri 2019-10-06 05:28:48

The video (attached) shows:

  1. the Greeter is displayed a first time suspiciously early in the boot process (2:32)
  2. the screen goes black for about 15 seconds
  3. the blue background + GNOME Shell top bar are displayed
  4. a few seconds later the Greeter’s main window comes back

The Journal show the Greeter starting only once, with a timing that matches the second time we see it on the video.

So at this point, I think what happens is:

  • The first Greeter we see is a phantom image, resurrected from the graphics card’s memory: in the affected scenario, immediately before the failing step, we do a reboot of the VM, so it’s possible that the (virtual) hardware carries state from the previous boot.
  • Our test suite tries to interact with the first instance of the Greeter that it sees, which is not an actual GUI one can interact with.

That is, this is another failure mode with the same root cause as Bug #12461. I won’t merge this one as a duplicate though, because Bug #12461 has been entirely bearable recently, while this bug seems to now happen on every test suite run, which is vastly more annoying. Still, if whatever fix we find here also addresses Bug #12461, nobody will complain :)

To fix this, I think we should replace “I warm reboot the computer” with a cold reboot.

Note that I’ve seen this happen both with Linux 5.2 and 5.3.

#5 Updated by segfault 2019-10-08 15:56:26

  • Assignee set to segfault

#6 Updated by segfault 2019-10-08 15:56:29

  • Status changed from Confirmed to In Progress

Applied in changeset commit:tails|ec8c3541334e4e822a54042326318db300352b51.

#7 Updated by segfault 2019-10-08 15:56:59

  • Feature Branch set to bugfix/17125-fragile-persistent-bookmarks-test

#8 Updated by intrigeri 2019-10-09 07:23:54

  • Feature Branch changed from bugfix/17125-fragile-persistent-bookmarks-test to bugfix/17125-fragile-persistent-bookmarks-test+force-all-tests

I’ve renamed the branch so that the scenario this is about actually runs on Jenkins :)

#9 Updated by intrigeri 2019-10-09 07:30:11

Dear segfault, if Jenkins is happy with this today and you’re not around, may I merge?

#10 Updated by segfault 2019-10-09 07:32:25

intrigeri wrote:
> I’ve renamed the branch so that the scenario this is about actually runs on Jenkins :)

Argh, I never think about checking whether the scenario I want to run is fragile.

> Dear segfault, if Jenkins is happy with this today and you’re not around, may I merge?

Sure.

#11 Updated by intrigeri 2019-10-09 19:33:36

In one local test suite run and on Jenkins:

  • this branch fixes the problem this ticket is about ⇒ I’m going to merge it; it also uncovers a follow-up issues:
  • “FindFailed: can not find TorBrowserEFFBookmark.png” ⇒ later.

#12 Updated by intrigeri 2019-10-09 19:34:01

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset commit:tails|ecfad1b61082b5c6d9293c34126b2786f1475f84.

#13 Updated by intrigeri 2019-10-10 04:12:00

  • Status changed from Resolved to In Progress
  • Assignee changed from segfault to intrigeri
  • % Done changed from 100 to 50
  • Feature Branch changed from bugfix/17125-fragile-persistent-bookmarks-test+force-all-tests to devel

intrigeri wrote:
> * “FindFailed: can not find TorBrowserEFFBookmark.png” ⇒ later.

I thought this error was unrelated to segfault’s branch, but it wasn’t ⇒ pushed an extra commit on top to fix the regression it introduced, let’s see how it goes.

#14 Updated by intrigeri 2019-10-10 11:48:22

  • Status changed from In Progress to Resolved
  • Assignee deleted (intrigeri)

Jenkins is happy again: https://jenkins.tails.boum.org/view/RM/job/test_Tails_ISO_devel/1936/cucumber-html-reports/report-feature_32_1732831431.html

#15 Updated by intrigeri 2020-04-15 06:02:26

  • Affected tool changed from Greeter to Welcome Screen