"Persistent browser bookmarks" test suite scenario regression
I’ve just had a look at a few test suite runs on the devel branch and devel-based branches that completed in the last hours, and “Scenario: Persistent browser bookmarks” failed every time with “FindFailed: can not find TailsGreeterPersistenceUnlocked.png”.
Let’s at least investigate in time for 4.0. If that’s test suite fragility, “fine”; if that’s a potentially-user-visible regression, let’s try to fix it.
#4 Updated by intrigeri 2019-10-06 05:28:48
- File 04_58_18_Persistent_browser_bookmarks.mkv added
- Assignee deleted (
- Type of work changed from Research to Code
The video (attached) shows:
- the Greeter is displayed a first time suspiciously early in the boot process (2:32)
- the screen goes black for about 15 seconds
- the blue background + GNOME Shell top bar are displayed
- a few seconds later the Greeter’s main window comes back
The Journal show the Greeter starting only once, with a timing that matches the second time we see it on the video.
So at this point, I think what happens is:
- The first Greeter we see is a phantom image, resurrected from the graphics card’s memory: in the affected scenario, immediately before the failing step, we do a reboot of the VM, so it’s possible that the (virtual) hardware carries state from the previous boot.
- Our test suite tries to interact with the first instance of the Greeter that it sees, which is not an actual GUI one can interact with.
That is, this is another failure mode with the same root cause as Bug #12461. I won’t merge this one as a duplicate though, because Bug #12461 has been entirely bearable recently, while this bug seems to now happen on every test suite run, which is vastly more annoying. Still, if whatever fix we find here also addresses Bug #12461, nobody will complain :)
To fix this, I think we should replace “I warm reboot the computer” with a cold reboot.
Note that I’ve seen this happen both with Linux 5.2 and 5.3.
#10 Updated by segfault 2019-10-09 07:32:25
> I’ve renamed the branch so that the scenario this is about actually runs on Jenkins :)
Argh, I never think about checking whether the scenario I want to run is fragile.
> Dear segfault, if Jenkins is happy with this today and you’re not around, may I merge?
#13 Updated by intrigeri 2019-10-10 04:12:00
- Status changed from Resolved to In Progress
- Assignee changed from segfault to intrigeri
- % Done changed from 100 to 50
- Feature Branch changed from bugfix/17125-fragile-persistent-bookmarks-test+force-all-tests to devel
> * “FindFailed: can not find TorBrowserEFFBookmark.png” ⇒ later.
I thought this error was unrelated to segfault’s branch, but it wasn’t ⇒ pushed an extra commit on top to fix the regression it introduced, let’s see how it goes.
#14 Updated by intrigeri 2019-10-10 11:48:22
- Status changed from In Progress to Resolved
- Assignee deleted (