Bug #15006

Test suite regression: "I successfully start the Unsafe Browser" fails since Tails 3.3

Added by intrigeri 2017-11-29 08:59:23 . Updated 2019-06-22 13:37:39 .

Status:
Confirmed
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
2017-11-29
Due date:
% Done:

60%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Unsafe Browser
Deliverable for:

Description

I guess it’s caused by the new pure HTML Unsafe Browser home page. I see that on the stable branch, did not look elsewhere.

Files

01_45_15_The_Unsafe_Browser_can_be_used_in_all_languages_supported_in_Tails.mkv (1468019 B) intrigeri, 2017-12-10 07:12:41
02_40_45_The_Unsafe_Browser_will_not_make_any_connections_to_the_Internet_which_are_not_user_initiated.png (73306 B) intrigeri, 2018-01-05 07:23:50
UnsafeBrowserStartPage.png (2481 B) intrigeri, 2018-01-05 07:23:55

Subtasks

Related issues

Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed

History

#1 Updated by intrigeri 2017-11-29 08:59:38

  • blocks Feature #13240: Core work 2017Q4: Test suite maintenance added

#2 Updated by intrigeri 2017-12-05 14:42:42

  • blocks Feature #14817: Test suite improvements from buster sprint 1 added

#3 Updated by anonym 2017-12-08 16:10:38

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to intrigeri
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA

Bumped some images in stable:

b749e0441c Test suite: bump TailsHomepage.png.
fe0e759dc0 Test suite: bump the Unsafe Browser's start page image.

#4 Updated by intrigeri 2017-12-08 18:00:13

  • % Done changed from 50 to 60

LGTM, will now test.

#5 Updated by intrigeri 2017-12-09 06:23:47

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed

Works in my tests but “the Unsafe Browser works in all supported languages” is still broken if the locales sample includes a language for which we have a translated homepage: we used to look for a picture that didn’t vary per-language; since we don’t display that picture anymore you simplified things in commit:38662c3219333d4f2dea5d2936749daa133939f9, which makes sense, but now our test suite is essentially trying to verify that we see the English homepage in all languages, which of fails when the locales sample includes a language for which we have a translated homepage. As per our experiment (try to do such work in a less fine-grained way) I’m not filing another dedicated ticket about it, let’s generalize this one instead :)

#6 Updated by intrigeri 2017-12-09 06:23:59

  • blocked by deleted (Feature #14817: Test suite improvements from buster sprint 1)

#7 Updated by intrigeri 2017-12-09 07:06:11

Also, it seems your refactoring in commit:38662c3219333d4f2dea5d2936749daa133939f9 had a nasty side-effect that introduces a regression in “Upgrading Tails with Tails Upgrader through an incremental upgrade”: if the Unsafe Browser takes more than 10 seconds to start (which happens in the few recent runs on Jenkins I’ve seen) then we fail, while previously I think we were waiting up to 6 minutes.

#8 Updated by intrigeri 2017-12-10 07:15:36

I still a strange failure here on an ISO built from the branch for Bug #14819 at commit:6c23dc58e241abd46efba7f861baa1b4fdf2e811. I see “Unsafe Browser failed to launch in the following locale(s): en_US.utf8” but the Unsafe Browser did start in English; the test suite got confused, did not even start it in the 2nd language to be tested, and then successfully started it in the 3rd one. I’m attaching the video and here’s the debug output:

01:43:11.703914579: calling as root: test -d '/usr/lib/locale/zh_CN.utf8'
01:43:11.824896168: call returned: [0, "", ""]
01:43:11.825769087: calling as root: pidof -x -o '%PPID' gnome-terminal-server
01:43:11.985597426: call returned: [1, "", ""]
01:43:12.024865880: calling as amnesia: xdotool key Super
01:43:12.218533316: call returned: [0, "", ""]
01:43:13.429144086: [log]  TYPE "commandline"
01:43:13.497129847: [log] ( Ctrl )  TYPE "#ENTER."
01:43:18.520386107: [log]  TYPE "LANG=en_US.utf8 LC_ALL=en_US.utf8 sudo unsafe-browser#ENTER."
01:43:20.378738335: [log]  TYPE "#RIGHT.#ENTER."
01:43:30.932402796: calling as root: pidof -x -o '%PPID' gnome-terminal-server
01:43:31.387661570: call returned: [0, "12272\n", ""]
01:44:05.120614434: Found fuzzy candidate picture for GnomeTerminalWindow.png with similarity 0.4
01:44:05.875618000: [log] CLICK on L(894,130)@S(0)[0,0 1024x768]
01:44:38.765072649: [log]  TYPE "LANG=it_IT.utf8 LC_ALL=it_IT.utf8 sudo unsafe-browser#ENTER."
01:44:51.774493042: Found fuzzy candidate picture for GnomeQuestionDialogIcon.png with similarity 0.4
01:44:52.014205672: [log]  TYPE "#RIGHT.#ENTER."
01:44:52.015108795: calling as root: . /usr/local/lib/tails-shell-library/tor-browser.sh && echo ${TBB_INSTALL}/firefox
01:44:52.024615452: [log] ( Ctrl )  TYPE "q"
01:44:52.304498606: call returned: [0, "/usr/local/lib/tor-browser/firefox\n", ""]
01:44:52.304714124: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:44:52.515435092: call returned: [0, "", ""]
01:44:53.515771341: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:44:53.617212895: call returned: [0, "", ""]
01:44:54.617553746: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:44:54.746847177: call returned: [0, "", ""]
01:44:55.747130393: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:44:55.874164049: call returned: [0, "", ""]
01:44:56.874496324: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:44:57.009500894: call returned: [1, "", ""]
01:44:57.010509348: calling as root: pidof -x -o '%PPID' gnome-terminal-server
01:44:57.149766601: call returned: [0, "12272\n", ""]
01:44:57.995066829: [log] CLICK on L(226,585)@S(0)[0,0 1024x768]
01:45:02.307530592: [log]  TYPE "LANG=ar_EG.utf8 LC_ALL=ar_EG.utf8 sudo unsafe-browser#ENTER."
01:45:02.390330168: [log]  TYPE "#LEFT.#ENTER."
01:45:07.597363046: [log] ( Ctrl )  TYPE "q"
01:45:07.597754992: calling as root: . /usr/local/lib/tails-shell-library/tor-browser.sh && echo ${TBB_INSTALL}/firefox
01:45:07.890353732: call returned: [0, "/usr/local/lib/tor-browser/firefox\n", ""]
01:45:07.890586771: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:45:08.148060863: call returned: [0, "", ""]
01:45:09.148333387: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:45:09.275159227: call returned: [0, "", ""]
01:45:10.275515549: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:45:10.382848078: call returned: [0, "", ""]
01:45:11.383199829: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:45:11.510450811: call returned: [0, "", ""]
01:45:12.510821804: calling as root: test -d '/var/lib/unsafe-browser/chroot'
01:45:12.624946123: call returned: [1, "", ""]
    Then the Unsafe Browser works in all supported languages                       # features/step_definitions/unsafe_browser.rb:31
      Unsafe Browser failed to launch in the following locale(s): en_US.utf8.
      <false> is not true. (Test::Unit::AssertionFailedError)
      features/localization.feature:18:in `Then the Unsafe Browser works in all supported languages'

#9 Updated by intrigeri 2018-01-01 16:58:58

  • blocked by deleted (Feature #13240: Core work 2017Q4: Test suite maintenance)

#10 Updated by intrigeri 2018-01-01 16:59:02

#11 Updated by intrigeri 2018-01-05 07:23:59

I’ve seen this fail occasionally even with English locales. Attaching the screenshot + Sikuli candidate.

#12 Updated by anonym 2018-01-23 19:52:53

  • Target version changed from Tails_3.5 to Tails_3.6

#13 Updated by bertagaz 2018-03-14 11:32:27

  • Target version changed from Tails_3.6 to Tails_3.7

#14 Updated by intrigeri 2018-04-13 11:33:01

  • Assignee deleted (anonym)

#15 Updated by intrigeri 2018-04-13 15:55:48

  • Target version deleted (Tails_3.7)

#16 Updated by intrigeri 2018-07-07 08:16:55

intrigeri wrote:
> Also, it seems your refactoring in commit:38662c3219333d4f2dea5d2936749daa133939f9 had a nasty side-effect that introduces a regression in “Upgrading Tails with Tails Upgrader through an incremental upgrade”: if the Unsafe Browser takes more than 10 seconds to start (which happens in the few recent runs on Jenkins I’ve seen) then we fail, while previously I think we were waiting up to 6 minutes.

FTR commit:ae21d2665d4d4da8fabfe2c2676e99d1b2d9d1f5 bumped this timeout to 30 seconds but that’s not enough with Firefox 60 on lizard when the box is running tests at full capacity. Bumping to 60 seconds.

#17 Updated by intrigeri 2019-03-08 15:22:06

  • Status changed from In Progress to Confirmed

#18 Updated by intrigeri 2019-03-20 14:49:31

#19 Updated by intrigeri 2019-03-20 14:49:38

  • blocked by deleted (Feature #13241: Core work: Test suite maintenance)

#20 Updated by intrigeri 2019-06-22 13:37:39

  • Priority changed from Elevated to Normal

(I think this test is fragile but not totally broken on Jenkins.)