Feature #8165

Make our automated test suite run in a Jessie environment

Added by intrigeri 2014-10-20 14:59:57 . Updated 2015-05-12 18:45:34 .

Status:
Resolved
Priority:
Elevated
Assignee:
Category:
Test suite
Target version:
Tails_1.4
Start date:
2014-10-20
Due date:
% Done:

100%

Feature Branch:
test/8165-run-test-suite-in-jessie
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Subtasks

Related issues

Blocked by Tails - Feature #7212: Try running our automated test suite in a Jessie environment Resolved 2014-05-10
Blocks Tails - Feature #5683: Puppet module to deploy systems able to run our test suite Resolved
Blocked by Tails - Bug #9074: python-xmpp broken in Jessie which breaks otr-bot Resolved 2015-03-18

History

#1 Updated by intrigeri 2014-10-20 15:00:07

  • blocked by Feature #7212: Try running our automated test suite in a Jessie environment added

#2 Updated by intrigeri 2014-10-20 15:54:22

  • Subject changed from Make our automated test suite in a Jessie environment to Make our automated test suite run in a Jessie environment

#3 Updated by anonym 2014-11-06 14:06:56

  • Status changed from Confirmed to In Progress
  • Assignee set to anonym
  • % Done changed from 0 to 20

While testing this (Feature #7212) I noticed the following:

  • Setup:
    - Unsurprisingly, all packages can be installed straight from Jessie, so no -t or $pkg/testing is needed, and we can drop the custom apt-pinnings. Yeah! We should make it clear that we need non-free sources since ovmf is there.
    - The libavcodec-extra-53 and ffmpeg packages are not in Jessie, so we need to migrate to libav-tools’ avconv.
    - The libvirt-ruby package has been renamed to ruby-libvirt.
  • Running:
    - We must get rid of the TailsBootSplash*PostReset.png workaround (the PostReset images should now become the default ones for non-UEFI boots) since virt-viewer now properly applies fullscreen mode on the first boot, not only after a reset. Good news!
    - Similar to the above point, something isn’t quite right with the TailsBootSplash*UEFI.png images, but I suspect they only need a trivial update.
    - In features/support/helpers/display_helper.rb, Display.active? always fails. Just removing the call of it in Display.start makes it all run (but may introduce races, especially when snapshots are restored). I tried running active?’s xprop command manually, and it works, so I’m not sure what the problem is. I’m confident we’ll find a solution given more than the three minutes I invested in this investigation.

All-in-all it looks like an easy transition.

Bumping the progress slightly to reflect this research.

#4 Updated by anonym 2014-11-06 14:24:08

anonym wrote:
> * Setup:
Almost forgot this one:
- Instead of syslinux we must install syslinux-utils since isohybrid (used by untrusted_partitions.feature) has moved there.

#5 Updated by intrigeri 2014-11-06 14:51:48

redmine@labs.riseup.net wrote (06 Nov 2014 14:06:56 GMT) :
> * Setup:
> - Unsurprisingly, all packages can be installed straight from Jessie, so no -t or
> $pkg/testing is needed, and we can drop the custom apt-pinnings. Yeah! We should
> make it clear that we need non-free sources since ovmf is there.
> - The libavcodec-extra-53 and ffmpeg packages are not in Jessie, so we need to migrate to libav-tools’ avconv.
> - The libvirt-ruby package has been renamed to ruby-libvirt.

IIRC I’ve already adjusted most of that in the feature/jessie branch.

> All-in-all it looks like an easy transition.

:)

#6 Updated by intrigeri 2014-11-10 13:33:51

I think I’ve now pushed all test suite setup doc adjustments to feature/jessie, except:

> The libavcodec-extra-53 and ffmpeg packages are not in Jessie, so we need to migrate to libav-tools’ avconv.

I suspect we’ll want libavcodec-extra-56 and avconv.

anonym, have you got something to push too, with the adaptations needed for runtime?

#7 Updated by anonym 2014-12-02 08:24:26

intrigeri wrote:
> anonym, have you got something to push too, with the adaptations needed for runtime?

No, I only (without committing) removed those deps from the dep checking, and let the steps depending on the video fail.

#8 Updated by intrigeri 2014-12-18 12:23:50

  • Priority changed from Normal to Elevated

#9 Updated by intrigeri 2015-01-06 12:56:57

  • blocks #8538 added

#10 Updated by anonym 2015-01-09 14:38:31

  • Target version changed from Tails_2.0 to Tails_1.5

#11 Updated by anonym 2015-01-10 17:29:41

  • Target version changed from Tails_1.5 to Tails_1.4

#12 Updated by intrigeri 2015-02-23 14:40:15

  • blocks Feature #5683: Puppet module to deploy systems able to run our test suite added

#13 Updated by anonym 2015-03-03 22:14:27

  • Target version changed from Tails_1.4 to Tails_1.3.2

#14 Updated by anonym 2015-03-18 16:11:34

  • blocked by Bug #9074: python-xmpp broken in Jessie which breaks otr-bot added

#15 Updated by anonym 2015-03-19 08:52:17

  • % Done changed from 20 to 40
  • Feature Branch set to test/8165-run-test-suite-in-jessie
  • Type of work changed from Code to Wait

In the feature branch I’ve fixed everything in Feature #8165#note-3 (and some more fixes), and I’ve seen all features pass completely, except pidgin.feature due to Bug #9074 (and transient issues due to Bug #9072) so what remains is waiting until it’s fixed.

#16 Updated by intrigeri 2015-03-19 09:49:58

  • Type of work changed from Wait to Code

(The “blocked by” relationship is already expressed in a better way.)

#17 Updated by anonym 2015-03-30 09:21:13

  • Assignee changed from anonym to intrigeri
  • % Done changed from 40 to 50
  • QA Check set to Info Needed

With Bug #9072 resolved, I think this branch is ready to be merged. However, Debian Jessie hasn’t been released yet, so I’m unsure if we want this to go into our stable branches just yet. In any case, just to avoid any problems this late in the Tails 1.3.2 release, I’d prefer to do this transition after the release.

What do you think?

#18 Updated by intrigeri 2015-03-30 09:30:41

> With Bug #9072 resolved, I think this branch is ready to be merged.

\o/

> However, Debian Jessie hasn’t been released yet, so I’m unsure if we want this to go into our stable branches just yet.

I say let’s merge it into devel after 1.3.2 is out, which will give everyone affected 4-5 weeks to upgrade before 1.4~rc1 has to be tested. It’s also likely that Jessie is released during our 1.4 dev cycle.

#19 Updated by intrigeri 2015-03-30 09:31:35

  • Assignee changed from intrigeri to anonym
  • Target version changed from Tails_1.3.2 to Tails_1.4
  • QA Check deleted (Info Needed)

#20 Updated by anonym 2015-03-31 19:39:02

  • Assignee changed from anonym to intrigeri
  • QA Check set to Ready for QA

#21 Updated by intrigeri 2015-04-01 16:39:24

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Ready for QA to Dev Needed

First problem I’ve seen: CTRL-C’ing a running test suite doesn’t seem to work anymore: the TightVNC display turns black, QEMU is still running, and cucumber is eating a full CPU core. I had to pkill -9 cucumber, and then virsh destroy TailsToaster.

I’ve now started a full run, will report back later if I find any other regression.

#22 Updated by intrigeri 2015-04-02 06:03:57

Note to myself — check what happens exactly wrt. the guestfs “appliance”:

  • on initial setup on Jessie
  • on upgrade from Wheezy

#23 Updated by intrigeri 2015-04-03 15:05:34

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:
> First problem I’ve seen: CTRL-C’ing a running test suite doesn’t seem to work anymore:

A 2nd CTRL-C is actually enough, so no problem.

> I’ve now started a full run, will report back later if I find any other regression.

Completed, I now need to analyze the results and sort false positives out..

#24 Updated by intrigeri 2015-04-04 12:41:47

  • Status changed from In Progress to Fix committed
  • % Done changed from 50 to 100

Applied in changeset commit:c035baf398d23f1d558511210d51d37a515700b0.

#25 Updated by intrigeri 2015-04-04 12:52:12

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

Merged into stable, devel, feature/jessie and experimental. Will now complete Feature #5683 and send a heads-up on -dev@ so that anyone affected upgrades their testing environment.

#26 Updated by BitingBird 2015-05-12 18:45:34

  • Status changed from Fix committed to Resolved