Bug #12731
Instructions to run the automated test suite fail
0%
Description
https://tails.boum.org/contribute/release_process/test/setup/ is incorrect and should be fixed as follows, assuming Debian GNU/Linux 9:
sudo apt-get update
sudo apt install \
cucumber \
devscripts \
dnsmasq-base \
gawk \
git \
i18nspector \
libav-tools \
libcap2-bin \
libsikuli-script-java \
libvirt-clients \
libvirt-daemon-system \
libvirt-dev \
libvirt0 \
obfs4proxy \
openssh-server \
ovmf \
pry \
python-jabberbot \
python-potr \
qemu-kvm \
qemu-system-common \
qemu-system-x86 \
qemu-utils \
redir \
ruby-guestfs \
ruby-json \
ruby-libvirt \
ruby-net-irc \
ruby-packetfu \
ruby-rb-inotify \
ruby-rjb \
ruby-rspec \
ruby-test-unit \
seabios \
tcpdump \
tor \
unclutter \
virt-viewer \
x11vnc \
tigervnc-viewer \
xvfb
$ sudo systemctl restart libvirtd
The proposed usage also needs fixing at https://tails.boum.org/contribute/release_process/test/usage/
$ sudo apt-get install git
$ git clone --recursive -b stable https://git-tails.immerda.ch/tails
$ sudo ./run_test_suite --view --iso ../*.iso features/apt.feature
but I was not able to figure out how to fix the last failure:
debian@test-loic-9-tests:~/tails$ sudo ./run_test_suite --view --iso ../*.iso features/apt.feature
Virtual X framebuffer started on display :0
VNC server running on: localhost:5900
[info] runcmd: lsb_release -i -r -s
@product
Feature: Installing packages through APT
As a Tails user
when I set an administration password in Tails Greeter
I should be able to install packages using APT and Synaptic
and all Internet traffic should flow only through Tor.
Scenario: APT sources are configured correctly # features/apt.feature:8
Given I have started Tails from DVD without network and logged in # features/step_definitions/snapshots.rb:172
FindFailed: can not find GnomeSystrayFlorence.png in S(0)[0,0 1024x768]
Line 2171, in file Region.java
(RuntimeError)
./features/step_definitions/common_steps.rb:273:in `/^I log in to a new session(?: in )?(|German)$/'
./features/step_definitions/snapshots.rb:154:in `block in reach_checkpoint'
./features/step_definitions/snapshots.rb:152:in `each'
./features/step_definitions/snapshots.rb:152:in `reach_checkpoint'
./features/step_definitions/snapshots.rb:173:in `/^I\ have\ started\ Tails\ from\ DVD\ without\ network\ and\ logged\ in$/'
features/apt.feature:9:in `Given I have started Tails from DVD without network and logged in'
Then the only hosts in APT sources are "vwakviie2ienjx6t.onion,sgvtcaew4bxjd7ln.onion,jenw7xbd6tf7vfhp.onion,sdscoq7snqtznauu.onion" # features/step_definitions/apt.rb:3
And no proposed-updates APT suite is enabled # features/step_definitions/apt.rb:17
Scenario failed at time 00:02:08
Screenshot: /tmp/TailsToaster/run-2017-06-18_17:35:45_+0000-git_stable_7e12308-7rWWw4/00:02:08_APT_sources_are_configured_correctly.png
@check_tor_leaks
Scenario: Install packages using apt # features/apt.feature:14
Given I have started Tails from DVD and logged in with an administration password and the network is connected # features/step_definitions/snapshots.rb:172
FindFailed: can not find GnomeSystrayFlorence.png in S(0)[0,0 1024x768]
Line 2171, in file Region.java
(RuntimeError)
./features/step_definitions/common_steps.rb:273:in `/^I log in to a new session(?: in )?(|German)$/'
./features/step_definitions/snapshots.rb:154:in `block in reach_checkpoint'
./features/step_definitions/snapshots.rb:152:in `each'
./features/step_definitions/snapshots.rb:152:in `reach_checkpoint'
./features/step_definitions/snapshots.rb:139:in `reach_checkpoint'
./features/step_definitions/snapshots.rb:173:in `/^I\ have\ started\ Tails\ from\ DVD\ and\ logged\ in\ with\ an\ administration\ password\ and\ the\ network\ is\ connected$/'
features/apt.feature:15:in `Given I have started Tails from DVD and logged in with an administration password and the network is connected'
When I configure APT to use non-onion sources # features/step_definitions/apt.rb:24
And I update APT using apt # features/step_definitions/apt.rb:41
And I install "cowsay" using apt # features/step_definitions/apt.rb:54
Then the package "cowsay" is installed # features/step_definitions/common_steps.rb:543
Scenario failed at time 00:02:59
Screenshot: /tmp/TailsToaster/run-2017-06-18_17:35:45_+0000-git_stable_7e12308-7rWWw4/00:02:59_Install_packages_using_apt.png
<del><a class='issue tracker-1 status-5 priority-4 priority-default closed child' href='/code/issues/12586' title='Synaptic test is fragile on Stretch'>Bug #12586</a></del>
@check_tor_leaks @fragile
Scenario: Install packages using Synaptic # features/apt.feature:23
Given I have started Tails from DVD and logged in with an administration password and the network is connected # features/step_definitions/snapshots.rb:172
FindFailed: can not find GnomeSystrayFlorence.png in S(0)[0,0 1024x768]
Line 2171, in file Region.java
(RuntimeError)
./features/step_definitions/common_steps.rb:273:in `/^I log in to a new session(?: in )?(|German)$/'
./features/step_definitions/snapshots.rb:154:in `block in reach_checkpoint'
./features/step_definitions/snapshots.rb:152:in `each'
./features/step_definitions/snapshots.rb:152:in `reach_checkpoint'
./features/step_definitions/snapshots.rb:139:in `reach_checkpoint'
./features/step_definitions/snapshots.rb:173:in `/^I\ have\ started\ Tails\ from\ DVD\ and\ logged\ in\ with\ an\ administration\ password\ and\ the\ network\ is\ connected$/'
features/apt.feature:24:in `Given I have started Tails from DVD and logged in with an administration password and the network is connected'
When I configure APT to use non-onion sources # features/step_definitions/apt.rb:24
And I start Synaptic # features/step_definitions/apt.rb:68
And I update APT using Synaptic # features/step_definitions/apt.rb:79
When I install "cowsay" using Synaptic # features/step_definitions/apt.rb:100
Then the package "cowsay" is installed # features/step_definitions/common_steps.rb:543
Scenario failed at time 00:03:49
Screenshot: /tmp/TailsToaster/run-2017-06-18_17:35:45_+0000-git_stable_7e12308-7rWWw4/00:03:49_Install_packages_using_Synaptic.png
Artifacts directory: /tmp/TailsToaster/run-2017-06-18_17:35:45_+0000-git_stable_7e12308-7rWWw4
Debug log: /tmp/TailsToaster/run-2017-06-18_17:35:45_+0000-git_stable_7e12308-7rWWw4/debug.log
Failing Scenarios:
cucumber features/apt.feature:8 # Scenario: APT sources are configured correctly
cucumber features/apt.feature:14 # Scenario: Install packages using apt
cucumber features/apt.feature:23 # Scenario: Install packages using Synaptic
3 scenarios (3 failed)
14 steps (3 failed, 11 skipped)
3m47.891s
Subtasks
History
#1 Updated by intrigeri 2017-06-23 11:22:54
- Subject changed from instructions to run the tests fail to Instructions to run the automated test suite fail
- Category set to Test suite
- Assignee set to intrigeri
- Type of work changed from Pass Test to Contributors documentation
#2 Updated by intrigeri 2017-06-23 11:39:18
- Status changed from New to Resolved
> https://tails.boum.org/contribute/release_process/test/setup/ is incorrect and should be fixed as follows, assuming Debian GNU/Linux 9:
Fixed, thanks. I was not expecting this would be needed, but why not :)
> The proposed usage also needs fixing at https://tails.boum.org/contribute/release_process/test/usage/
>
> $ sudo apt-get install git
> $ git clone --recursive -b stable https://git-tails.immerda.ch/tails
> $ sudo ./run_test_suite --view --iso ../*.iso features/apt.feature
>
I’ve added a link to our Git doc in commit:f769033, as I’d rather not duplicate such information all over the place.
Regarding sudo
: see the “Running the test suite as a non-root user” section on https://tails.boum.org/contribute/release_process/test/setup/. I’d rather not encourage everyone to run this as root when it’s not really needed.
> but I was not able to figure out how to fix the last failure:
I guess that’s Bug #11872, but I would need to see the screenshot or video to be sure.