Tails

#1 Updated by intrigeri 2019-09-06 09:20:09

This happened 4 times in the last 10 runs on Jenkins (devel) branch ⇒ FT.

#4 Updated by intrigeri 2019-09-06 10:16:24

Actually, Sikuli is too fast: on the video, I see the mouse cursor moving towards the “Untitled document 1” tab before OpenPGP Applet had time to open its menu. Now, I think there’s a bug in our code. Fuzzy matching is used, that’s why Sikuli clicks on the (vaguely similar) tab instead of waiting any further:

03:19:20.989537354: Sikuli: calling wait("GpgAppletDecryptVerify.png")...
03:19:45.108976627: Found fuzzy candidate picture for GpgAppletDecryptVerify.png with similarity 0.4
03:19:45.109665405: FindFailedHookFailure was raised, re-running the failing Sikuli method.
03:19:45.172519400: Sikuli: calling click(M[158,82 176x22]@S(S(0)[0,0 1024x768]) S:0.42 C:246,93 [62 msec])...
03:19:45.902044690: [log] CLICK on L(246,93)@S(0)[0,0 1024x768]

It’s not the first time I notice a fuzzy image being used while I don’t expect it to be.
I don’t see “Using fuzzy candidate picture for #{picture}” in the debug log, which confirms that SIKULI_FUZZY_IMAGE_MATCHING is not true. But FindFailedHookFailure is invoked and on 2nd try, Sikuli is happy to match the fuzzy tab header (I assume “S:0.42” means similarity = 0.42). It looks exactly as if us retrying in findfailed_hook with lower similarity expectations reset the MinSimilarity setting for the following Sikuli action.

I see the same kind of behavior elsewhere, e.g.

03:26:45.644231746: Sikuli: calling wait("Gtk3PrintButton.png")...
03:27:02.733362618: Found fuzzy candidate picture for Gtk3PrintButton.png with similarity 0.6
03:27:02.733999556: FindFailedHookFailure was raised, re-running the failing Sikuli method.
03:27:02.803860495: Sikuli: calling click(M[569,103 81x36]@S(S(0)[0,0 1024x768]) S:0.69 C:609,121 [70 msec])...
03:27:03.527528786: [log] CLICK on L(609,121)@S(0)[0,0 1024x768]

Without diving into the rjb craziness, what we could do is:

• Call pattern.similar(0.9) at the end of each [0.80, 0.70, 0.60, 0.50, 0.40].each do |similarity| iteration, in the hope we revert the side effect.
• Skip [0.80, 0.70, 0.60, 0.50, 0.40].each do |similarity| entirely unless fuzzy image matching is enabled. For sure, this will make some of our work more difficult (one will have to run tests locally and enable fuzzy image matching) but IMO it’s worth it, in order to fix weird behavior that breaks the test suite in a way that affects every developer, in many runs.

#11 Updated by intrigeri 2019-09-06 18:26:55

Note that this test suite bug make us miss an actual, user-impacting, bug: Bug #17030.