Feature #15012

Review test suite changes for Tails 3.5

Added by anonym 2017-12-01 15:59:52 . Updated 2018-03-14 11:08:55 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
2018-01-17
Due date:
2018-01-22
% Done:

100%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

(Per our new arrangement.)


Subtasks


History

#1 Updated by intrigeri 2017-12-09 14:09:50

  • QA Check set to Ready for QA

(Not sure how best to track this ticket since it’s not ready for QA yet, but well.)

#2 Updated by intrigeri 2017-12-23 08:41:06

  • Status changed from Confirmed to In Progress

#3 Updated by intrigeri 2018-01-17 10:12:12

  • Assignee changed from intrigeri to anonym
  • % Done changed from 0 to 50
  • QA Check changed from Ready for QA to Dev Needed
  • commit:755515488b08f2a3cadc3a4bc90dc97073c86467: if I’m not mistaken we have no guarantee that the policy changes have been applied already when pkill returns due to signals’s asynchronous nature. I don’t recall having seen any issue with that during my own full test suite runs but I might have mistakenly dismissed such failures as transient network problems, and we don’t run these tests on Jenkins so we lack data. I don’t know if there’s a way to do this safely, perhaps dbus-daemon sends a D-Bus signal somewhere or logs something. Anyway, for now let’s not bother and just keep an eye on this when we see the affected tests fail.
  • I see we’ve dropped compatibility with older sikuli but wiki/src/contribute/release_process/test/setup.mdwn still relies on the old libsikuli-script-java, that’s a transitional package even in Stretch. I think we should install libsikulixapi-java directly instead because transitional packages get removed after a while (and we won’t notice our doc is broken until someone else than us hits the problem). Please fix that and then feel free to close this ticket.
  • I had already reviewed the changes made in features/support/helpers/sikuli_helper.rb so I’ve skipped it this time.

Apart of that, LGTM at commit:ed3637b435dbbcbe63a30f11e5b9648af6c573e7 :)

#4 Updated by anonym 2018-01-23 19:52:53

  • Target version changed from Tails_3.5 to Tails_3.6

#5 Updated by anonym 2018-02-02 14:20:34

  • Assignee changed from anonym to intrigeri
  • % Done changed from 50 to 80
  • QA Check changed from Dev Needed to Info Needed

intrigeri wrote:
> 755515488b08f2a3cadc3a4bc90dc97073c86467: if I’m not mistaken we have no guarantee that the policy changes have been applied already when pkill returns due to signals’s asynchronous nature. I don’t recall having seen any issue with that during my own full test suite runs but I might have mistakenly dismissed such failures as transient network problems, and we don’t run these tests on Jenkins so we lack data. I don’t know if there’s a way to do this safely, perhaps dbus-daemon sends a D-Bus signal somewhere or logs something. Anyway, for now let’s not bother and just keep an eye on this when we see the affected tests fail.

True. However, with this hack I’ve never seen the problem it crappily tries to work around, so it is still an improvement. Hypothesis: at least until we have Feature #11888. Hypothesis: the delay imposed by using the remote shell makes sending signals synchronous.

So, IMHO, let’s comment this (commit:476efac33e737ab73c99f3282dc3321fb6b7f934) and postpone the investigating for the perfect solution until my hypothesis fails (if ever). (Info needed: if you are fine with this, please close this ticket!)

> I see we’ve dropped compatibility with older sikuli but wiki/src/contribute/release_process/test/setup.mdwn still relies on the old libsikuli-script-java, that’s a transitional package even in Stretch. I think we should install libsikulixapi-java directly instead because transitional packages get removed after a while (and we won’t notice our doc is broken until someone else than us hits the problem). Please fix that and then feel free to close this ticket.

Also true! But this on i fixed, in commit:374e12685bfbef11fd26d783367217ccf11b8c80 (stable).

#6 Updated by intrigeri 2018-02-03 07:30:06

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Info Needed to Dev Needed

> So, IMHO, let’s comment this (commit:476efac33e737ab73c99f3282dc3321fb6b7f934) and postpone the investigating for the perfect solution until my hypothesis fails (if ever).

OK!

>> I see we’ve dropped compatibility with older sikuli but wiki/src/contribute/release_process/test/setup.mdwn still relies on the old libsikuli-script-java, that’s a transitional package even in Stretch. I think we should install libsikulixapi-java directly instead because transitional packages get removed after a while (and we won’t notice our doc is broken until someone else than us hits the problem). Please fix that and then feel free to close this ticket.

> Also true! But this on i fixed, in commit:374e12685bfbef11fd26d783367217ccf11b8c80 (stable).

Please update run_test_suite accordingly and then feel free to close this ticket.

I’ve updated our Puppet code to also install libsikulixapi-java directly.

#7 Updated by anonym 2018-02-10 16:54:13

  • Status changed from In Progress to Fix committed
  • Assignee deleted (anonym)
  • % Done changed from 80 to 100
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:
> Please update run_test_suite accordingly and then feel free to close this ticket.

Gah. Thanks! Fixed in commit:3e887ff98b3e6070423925ee67412154d19c484f.

> I’ve updated our Puppet code to also install libsikulixapi-java directly.

Great!

#8 Updated by bertagaz 2018-02-20 21:01:26

  • QA Check changed from Ready for QA to Pass

anonym wrote:
> intrigeri wrote:
> > Please update run_test_suite accordingly and then feel free to close this ticket.
>
> Gah. Thanks! Fixed in commit:3e887ff98b3e6070423925ee67412154d19c484f.

Seems it has been reviewed, doing so then.

#9 Updated by bertagaz 2018-03-14 11:08:55

  • Status changed from Fix committed to Resolved