Tails

#5 Updated by intrigeri 2018-06-04 07:40:45

intrigeri wrote:
> I don’t know if that’s a regression since the new tails-documentation was introduced or if it’s a test suite bug.

Given the online flavour often fails too, I think this is “just” yet another problem with desktop icons.

#7 Updated by intrigeri 2018-08-07 15:47:13

intrigeri wrote:
> intrigeri wrote:
> > I don’t know if that’s a regression since the new tails-documentation was introduced or if it’s a test suite bug.
>
> Given the online flavour often fails too, I think this is “just” yet another problem with desktop icons.

Actually, commit:74536c6a04c850db69686588723a9fec5060be7b removed some robustnes improvements. I’ll reintroduced them via Feature #15720 and we’ll see.

#9 Updated by intrigeri 2018-08-17 07:28:38

I’ve found another failure mode for “the support documentation page opens in Tor Browser”. The browser does start and open the right page, but then the heading lookup fails:

02:26:58.195719964: executing Python as amnesia: node432 = node428.child('Tails - Hilfe & Support - Tor-Browser', roleName='frame')
02:26:58.247062647: execution complete
02:26:58.247209492: executing Python as amnesia: 
    nodes433 = node432.findChildren(dogtail.predicate.GenericPredicate(roleName='heading'))
    print(len(nodes433))
02:26:59.100786690: execution complete
    Then the support documentation page opens in Tor Browser                                                  # features/step_definitions/checks.rb:97
      <false> is not true. (Test::Unit::AssertionFailedError)
      features/localization.feature:13:in `Then the support documentation page opens in Tor Browser'

Looking at the code:

   step "\"#{expected_title}\" has loaded in the Tor Browser"
   if @language == 'German'
     browser_name = 'Tor-Browser'
   else
     browser_name = 'Tor Browser'
   end
   headings = @torbrowser
              .child(expected_title + " - #{browser_name}", roleName: 'frame')
              .children(roleName: 'heading')
   assert(
     headings.any? { |heading| heading.text == expected_heading }
   )

… this seems racy to me: after the “has loaded” step has passed, we’re immediately collecting a list of headings; but if Tor Browser is a bit slow, it may not have exposed the entire page content on the a11y bus yet. I think we should drop the assert and let dogtail look for the expected heading itself, so we benefit from its internal wait/retry magics. This might require extending our dogtail custom bindings to support the .text selector. If that’s not feasible then we should wrap the assert with some retry loop.

#12 Updated by intrigeri 2019-05-18 16:36:04

I’ve seen a case when the test suite actually clicked the “Tails-Dokumentation” icon instead of the “Einen Fehler melden” one: Found fuzzy candidate picture for DesktopReportAnError.png with similarity 0.6. So then obviously, in the next step Dogtail does not find the expected page title.

#13 Updated by intrigeri 2019-08-16 05:46:03

intrigeri wrote:
> I’ve seen a case when the test suite actually clicked the “Tails-Dokumentation” icon instead of the “Einen Fehler melden” one: Found fuzzy candidate picture for DesktopReportAnError.png with similarity 0.6. So then obviously, in the next step Dogtail does not find the expected page title.

Seen this failure mode again. That’s because the “Einen Fehler melden” icon is half-hidden behind the GNOME Shell top bar. This won’t happen anymore on Tails 4.x because the desktop icons there are managed by a GNOME Shell extension instead of Nautilus.