Tails

#5 Updated by intrigeri 2016-07-13 15:07:26

Beware: time flies, and in a week the data will start disappearing.

#8 Updated by intrigeri 2016-07-18 08:39:25

> We’re in sync, I started this morning! The failures are tricky to debug though, will take a bit more time.

Yeah :)

I would hope that most of them are subtasks of Bug #10288, those that are not of the kind that we can flag fragile e.g. Bug #10463, #10465, and Bug #10479.

#10 Updated by intrigeri 2016-07-21 13:39:18

> I need some more researches and inputs from other -CI people before going further.

Sure. Please ask us what input you need specifically (keeping in mind that the data this is about already started disappearing).

> * devel #373 / stable #319
> cucumber features/time_syncing.feature:13 # Scenario: Clock with host's time in bridge mode
> ‘time has synced’ failure, at the /var/run/tordate check
> This does seem to be a (new?) bug in the bridge support.

Yes. Historically tordate + bridge mode has not been a very robust configuration. Please mark as fragile.

> * devel #400
> cucumber features/checks.feature:99 # Scenario: The Tails Greeter "disable all networking" option disables networking within Tails
> righteous error, the check found eth0 to be up. There may be a bug with this option.
> Happened only once in Jenkins memory

Wow, that’s scary. Please file a bug report about it. Debug log is:

Scenario: The Tails Greeter "disable all networking" option disables networking within Tails      # features/checks.feature:99
calling as root: echo 'hello?'
call returned: [0, "hello?\n", ""]
[log] CLICK on (1024,384)
calling as root: /sbin/ifconfig eth0 | grep -q 'inet addr'
call returned: [1, "", "eth0: error fetching interface information: Device not found\n"]
calling as root: date -s '@1465635115'
call returned: [0, "Sat Jun 11 08:51:55 UTC 2016\n", ""]
Given I have started Tails from DVD without network and stopped at Tails Greeter's login screen # features/step_definitions/snapshots.rb:199
[log] CLICK on (433,404)
And I enable more Tails Greeter options                                                         # features/step_definitions/common_steps.rb:308
[log] CLICK on (643,447)
And I disable all networking in the Tails Greeter                                               # features/step_definitions/checks.rb:240
[log] CLICK on (512,671)
[log] CLICK on (812,712)
calling as root: test -e '/etc/sudoers.d/tails-greeter' -o -e '/etc/sudoers.d/tails-greeter-no-password-lecture'
call returned: [1, "", ""]
calling as root: test -e '/etc/sudoers.d/tails-greeter' -o -e '/etc/sudoers.d/tails-greeter-no-password-lecture'
call returned: [0, "", ""]
calling as amnesia: gsettings set org.gnome.desktop.session idle-delay 0
call returned: [0, "", ""]
calling as amnesia: gsettings set org.gnome.desktop.interface toolkit-accessibility true
call returned: [0, "", ""]
calling as amnesia: xdotool search --all --onlyvisible --maxdepth 1 --classname 'Florence'
call returned: [1, "", ""]
And I log in to a new session                                                                   # features/step_definitions/common_steps.rb:292
calling as root: . /usr/local/lib/tails-shell-library/hardware.sh && get_all_ethernet_nics
call returned: [0, "eth0\n", ""]
Then no network interfaces are enabled                                                          # features/step_definitions/mac_spoofing.rb:75
<0> expected but was
<1>. (Test::Unit::AssertionFailedError)
./features/step_definitions/mac_spoofing.rb:79:in `/^(\d+|no) network interface(?:s)? (?:is|are) enabled$/'
features/checks.feature:104:in `Then no network interfaces are enabled'
Scenario failed at time 00:58:11

> * stable #326
> cucumber features/i2p.feature:18 # Scenario: I2P's AppArmor profile is in enforce mode
> raised ‘At least one unit failed to start’ error.
> This error happened 3 times in Jenkins memory:
> test_Tails_ISO_feature-10298-linux-4.x-aufs-kaslr/builds/2016-06-10_12-14-08
> test_Tails_ISO_stable/builds/2016-06-29_09-16-50
> test_Tails_ISO_test-10499-icmp-tor-enforcement-is-fragile/builds/2016-07-01_10-40-07

Indeed, and I’ve flagged it as fragile two days ago (Bug #11114). Looking at the debug log I see no unit in “failed” state, so I guess all this is about i2p.service, which is still in “activating” state.

> * devel #404
> cucumber features/torified_browsing.feature:18 # Scenario: The Tor Browser directory is usable > Last ignored exception was: ExecutionFailedInVM: Command failed: /usr/bin/python '/tmp/tmp.ygsOPWA8KD'
> dogtail problem, happened 8 times in Jenkins known history:
> test_Tails_ISO_bugfix-10494-retry-htpdate/builds/2016-07-04_04-06-03
> test_Tails_ISO_bugfix-10950-add-gnome-sound-recorder/builds/2016-07-10_21-37-38
> test_Tails_ISO_devel/builds/2016-06-15_07-53-32
> test_Tails_ISO_feature-10298-linux-4.x-aufs/builds/2016-06-09_23-37-44
> test_Tails_ISO_feature-5650-rngd/builds/2016-06-10_14-35-10
> test_Tails_ISO_test-10496-apt-get-scenarios-are-fragile/builds/2016-07-13_09-35-41
> test_Tails_ISO_test-10499-icmp-tor-enforcement-is-fragile/builds/2016-06-26_10-43-33
> test_Tails_ISO_test-10777-robust-boot-menu/builds/2016-06-21_12-18-54
> New ticket: ‘step the Tor Browser has started and loaded the startup page is fragile’?
> Dogtail can’t find the page opened in Tor Browser. It’s not the only place where dogtail shows troubles with the Tor Browser, see the next failure. Whether both are related to the same bug is still unclear to me.

Well, dogtail can’t find the page… because Tor Browser could not load it. So that’s not a problem with dogtail. I guess we need to add some retry logic in there ⇒ fragile (either under Bug #10381, or a new related ticket).

> * stable #322
> cucumber features/torified_browsing.feature:7 # Scenario: The Tor Browser cannot access the LAN > Then the Tor Browser shows the "Unable to connect" error Failure > Could not find the 'Unable to connect' error in the Tor Browser (RuntimeError)
> TorBrowser shows the error message, but dogtail does not find it.
> Happened 3 times in Jenkins known history
> test_Tails_ISO_stable/builds/2016-06-25_08-49-01
> test_Tails_ISO_test-10498-ssh-tests-are-fragile/builds/2016-07-13_17-07-57
> test_Tails_ISO_test-10504-allow-more-boot-devices/builds/2016-06-25_06-21-45

Debug log:

And I open a page on the LAN web server in the Tor Browser                     # features/step_definitions/common_steps.rb:847
calling as amnesia: mktemp
call returned: [0, "/tmp/tmp.CNToYRov0p\n", ""]
calling as root: rm -f '/tmp/tmp.CNToYRov0p'
call returned: [0, "", ""]
calling as amnesia: echo '#!/usr/bin/python
from dogtail import tree
from dogtail.config import config
config.searchShowingOnly = True
application = tree.root.application('"'"'Firefox'"'"')
application.child('"'"'Problem loading page'"'"', roleName='"'"'document frame'"'"')' >> '/tmp/tmp.CNToYRov0p'
call returned: [0, "", ""]
calling as amnesia: /usr/bin/python '/tmp/tmp.CNToYRov0p'
call returned: [0, "Creating logfile at /tmp/dogtail-amnesia/logs/tmp.CNToYRov0p_20160625-101308_debug ...\nDogtail: warning: omiting possibly broken at-spi application record\n", "\n** (tmp.CNToYRov0p:5721): WARNING **: AT-SPI: Error in GetItems, sender=org.freedesktop.DBus, error=The name :1.0 was not provided by any .service files\n"]
calling as root: rm -f '/tmp/tmp.CNToYRov0p'
call returned: [0, "", ""]
calling as amnesia: mktemp
call returned: [0, "/tmp/tmp.Hsw2U5t8Cf\n", ""]
calling as root: rm -f '/tmp/tmp.Hsw2U5t8Cf'
call returned: [0, "", ""]
calling as amnesia: echo '#!/usr/bin/python
from dogtail import tree
from dogtail.config import config
config.searchShowingOnly = True
application = tree.root.application('"'"'Firefox'"'"')
from dogtail import predicate
for n in application.child('"'"'Problem loading page'"'"', roleName='"'"'document frame'"'"').findChildren(predicate.GenericPredicate(roleName='"'"'heading'"'"')):
print(n.path)' >> '/tmp/tmp.Hsw2U5t8Cf'
call returned: [0, "", ""]
calling as amnesia: /usr/bin/python '/tmp/tmp.Hsw2U5t8Cf'
call returned: [0, "Creating logfile at /tmp/dogtail-amnesia/logs/tmp.Hsw2U5t8Cf_20160625-101311_debug ...\nDogtail: warning: omiting possibly broken at-spi application record\n/org/a11y/atspi/accessible/1153\n", "\n** (tmp.Hsw2U5t8Cf:5814): WARNING **: AT-SPI: Error in GetItems, sender=org.freedesktop.DBus, error=The name :1.0 was not provided by any .service files\n"]
calling as root: rm -f '/tmp/tmp.Hsw2U5t8Cf'
call returned: [0, "", ""]
calling as amnesia: mktemp
call returned: [0, "/tmp/tmp.bsXFyu1tbL\n", ""]
calling as root: rm -f '/tmp/tmp.bsXFyu1tbL'
call returned: [0, "", ""]
calling as amnesia: echo '#!/usr/bin/python
from dogtail import tree
from dogtail.config import config
config.searchShowingOnly = True
application = tree.root.application('"'"'Firefox'"'"')
from dogtail import predicate
node = None
for n in application.child('"'"'Problem loading page'"'"', roleName='"'"'document frame'"'"').findChildren(predicate.GenericPredicate()):
if str(n.path) == '"'"'/org/a11y/atspi/accessible/1153'"'"':
node = n
break
assert(node)
print(node.text)' >> '/tmp/tmp.bsXFyu1tbL'
call returned: [0, "", ""]
calling as amnesia: /usr/bin/python '/tmp/tmp.bsXFyu1tbL'
call returned: [0, "Creating logfile at /tmp/dogtail-amnesia/logs/tmp.bsXFyu1tbL_20160625-101314_debug ...\nDogtail: warning: omiting possibly broken at-spi application record\nUnable to connect\n", "\n** (tmp.bsXFyu1tbL:5906): WARNING **: AT-SPI: Error in GetItems, sender=org.freedesktop.DBus, error=The name :1.0 was not provided by any .service files\n"]
calling as root: rm -f '/tmp/tmp.bsXFyu1tbL'
call returned: [0, "", ""]
Then the Tor Browser shows the "Unable to connect" error                       # features/step_definitions/browser.rb:223
Could not find the 'Unable to connect' error in the Tor Browser (RuntimeError)
./features/step_definitions/browser.rb:231:in `/^the Tor Browser shows the "([^"]+)" error$/'
features/torified_browsing.feature:14:in `Then the Tor Browser shows the "Unable to connect" error'

Looks like dogtail found the page open, but not the expected heading.

Please mark any scenario that uses the “the Tor Browser shows the […] error” step as fragile + ticket.

> * stable #321
> cucumber features/time_syncing.feature:13 # Scenario: Clock with host's time in bridge mode > Unexpected connections were made: > 00:42:22.011 #<OpenStruct mac_saddr="50:54:00:de:5d:0d", mac_daddr="52:54:00:3d:60:5d", protocol="tcp", saddr="10.2.1.208", daddr="10.2.1.1", sport=48124, dport=5009>.
> Happened 4 times in Jenkins known memory
> test_Tails_ISO_feature-5650-rngd/builds/2016-06-10_17-37-00
> test_Tails_ISO_stable/builds/2016-06-24_08-48-32
> test_Tails_ISO_stable/builds/2016-07-13_19-42-53
> test_Tails_ISO_test-11463-notification-not-recognized/builds/2016-06-05_09-20-52
> That’s Bug #11521

I’ve updated the ticket accordingly. Bonus points if you attach the corresponding pcap files, unless you have time to analyze them and make sure they’re useless :)

> * devel #419
> cucumber features/erase_memory.feature:7 # Scenario: Anti-test: no memory erasure on a modern computer > FindFailed: can not find GnomeNotificationAppletOpened.png on the screen.
> Happened 7 times in Jenkins known history
> test_Tails_ISO_devel/builds/2016-06-30_07-54-19
> test_Tails_ISO_test-10441-synaptic-test-is-fragile/builds/2016-06-18_16-18-05
> test_Tails_ISO_test-10496-apt-get-scenarios-are-fragile/builds/2016-07-02_19-21-14
> test_Tails_ISO_test-10498-ssh-tests-are-fragile/builds/2016-07-13_17-07-57
> test_Tails_ISO_test-10499-icmp-tor-enforcement-is-fragile/builds/2016-06-15_15-06-08
> test_Tails_ISO_test-10504-allow-more-boot-devices/builds/2016-06-21_06-22-53
> test_Tails_ISO_test-6304-icedove-tests/builds/2016-06-09_18-42-22
> test_Tails_ISO_test-6304-icedove-tests/builds/2016-07-15_18-42-15
> That’s Bug #11464 I think, but is supposed to be resolved.

That’s sad, but the discussion on Bug #11464 makes it quite clear that the fix was best-effort and not perfect :/ Please reopen Bug #11464, and add debug logs + videos there, then.