Tails

#1 Updated by intrigeri 2017-10-09 10:24:24

I’ve analyzed anonym’s run-2017-10-08_14:12:48_+0200-git_test-buster-with-more-sikuli-fuzziness_28e814b-SA5xr4:

• “Install packages using apt”: typed only 3 chars in the Greeter’s admin password field and 4 chars in the confirmation field passed, let’s file a test suite robustness ticket if we see it again
• “Install packages using Synaptic” fails at ’I install “cowsay” using Synaptic’, not clear why without the video ⇒ retry by hand or with —capture passed, let’s file a test suite robustness ticket if we see it again
• “Encryption and decryption using OpenPGP Applet”, “Signing and verification using OpenPGP Applet”, “Encryption/signing and decryption/verification using OpenPGP Applet”, “Symmetric encryption and decryption using OpenPGP Applet”: FindFailed: PinEntryPrompt.png, unclear why without a video; on some of the pictures it seems that scenario has worked just fine, so I dunno why the test suite is waiting for a Pinentry prompt ⇒ retried and they fail with FindFailed: GeditNewTab.png or FindFailed: GeditWindow.png, will investigate further fixed those + some more but a few scenarios need more work, let’s see after next run
• USB tests: skipped ⇒ retry after merging the fix for Bug #14809 and Bug #14816
• “MAC address spoofing is disabled”, “MAC address spoofing is successful”: fails at “I hotplug a network device and wait for it to be initialized”, that I’ve seen on my sid as well ⇒ retry on Jenkins (that is not affected) works on anonym’s system with the fix from Bug #14819
• “Thunderbird can download the inbox with POP3”: I see “Username or password invalid” after clicking the “Done” button, hard to tell what’s going on without a video ⇒ retry by hand or with —capture passed, let’s file a ticket if we ever see it again.
• “I can view a file stored in ”_{/Tor Browser" but not in}/.gnupg“: fails at the step immediately after ‘When I open the address “file:///home/amnesia/.gnupg/synaptic.html” in the Tor Browser’, i.e. Sikuli finds TorBrowserSynapticManual.png (”Found fuzzy candidate picture for TorBrowserSynapticManual.png with similarity 0.4") while it should not. So if/when we start running with --fuzzy-image-matching it’ll break. Looking at the actual picture, that’s because we have another tab still open from the previous similar test, with a part of the same text in the tab title, it makes sense ⇒ add more context to TorBrowserSynapticManual.png so the actual web page we’re looking for is not confused with similar stuff, or increase the required similarity level for this picture. done
• “Watching MP4 videos stored on the persistent volume should work as expected given our AppArmor confinement”: fail with “Call to virStorageVolCreateXML failed: cannot fork child process: Cannot allocate memory (Guestfs::Error)” ⇒ retry with a host system that works better once Bug #14816 is fixed
• “VirtualBox guest modules are available”: fail with “Call to virStorageVolCreateXML failed: cannot fork child process: Cannot allocate memory (Guestfs::Error)” ⇒ retry with a host system that works better passed
• “The Unsafe Browser cannot be configured to use Tor and other local proxies” ⇒ update UnsafeBrowserNetworkTabAlreadySelected.png done

#5 Updated by anonym 2017-10-11 16:24:30

Partial response while I have my finding from Feature #14817 fresh in mind:

intrigeri wrote:
> * 4 of the “Encryption and verification using GnuPG” scenarios fail with “FindFailed: can not find PinEntryPrompt.png”:
> in one case the operation that’s supposed to ask a passphrase was not even attempted, so that’s clearly a test suite bug

It’s not clear what this means to me, please clarify! Also note the “maybe” in maybe_deal_with_pinentry, which is explained by the comment if we fail to find PinEntryPrompt.png:

    # The passphrase was cached or we wasn't prompted at all (e.g. when
    # only encrypting to a public key)

Perhaps that explains why no pin entry prompt appeared, but the operation was in fact performed?

> in the 3 other cases the operation that’s being tested worked just fine, so that’s clearly a test suite bug too: looks like the rescue FindFailed part of maybe_deal_with_pinentry does not work anymore? I’ve had a quick look at commit:03f0967beec8acea070c35d8908a8f46bc83084b and it seems that some code should be guarded by if ! $config["SIKULI_FUZZY_IMAGE_MATCHING"], because I see “Failed to find fuzzy candidate picture for PinEntryPrompt.png” in the logs => test suite bug, I’ll reopen Feature #14817 accordingly.

Should be fixed, Feature #14817 is back on your plate.

#6 Updated by intrigeri 2017-10-12 04:51:15

> intrigeri wrote:
>> * 4 of the “Encryption and verification using GnuPG” scenarios fail with “FindFailed: can not find PinEntryPrompt.png”:
>> in one case the operation that’s supposed to ask a passphrase was not even attempted, so that’s clearly a test suite bug

> It’s not clear what this means to me, please clarify!

The test suite keeps did not even try to click on stuff to perform the operation being tested.

> Also note the “maybe” in maybe_deal_with_pinentry, which is explained by the comment if we fail to find PinEntryPrompt.png:

>

>     # The passphrase was cached or we wasn't prompted at all (e.g. when
>     # only encrypting to a public key)
> 

> Perhaps that explains why no pin entry prompt appeared, but the operation was in fact performed?

Sure. But then we should not be looking endlessly for a pinentry prompt.

#7 Updated by intrigeri 2017-11-07 14:52:26

intrigeri wrote:
> * “Browsing the web using the Tor Browser ǂ I can view a file stored in ”_{/Tor Browser" but not in}/.gnupg" fails with “FindFailed: can not find TorBrowserSynapticManual.png”; possibly I borked things when I updated this picture (FTR I see “Found fuzzy candidate picture for TorBrowserSynapticManual.png with similarity 0.8” => anonym, may you please have a look?

This might be an instance of Bug #14929 (that happens on the devel branch).