Bug #10211

Sometimes the test suite fails with "Call to virNetworkCreate failed"

Added by kytv 2015-09-17 02:02:21 . Updated 2015-12-22 14:19:29 .

Status:
Rejected
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Tails_2.0
Start date:
2015-09-17
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
0
Affected tool:
Deliverable for:

Description

While running the test suite with both the new snapshot system (Feature #6094) and the old (Feature #6302) I have received the following:

Testing ISO image: tails-i386-feature_i2p-0.9.22-1.6-20150914.iso
Using old ISO image: tails-i386-feature_i2p-0.9.22-1.6-20150914.iso
@product
Feature: System memory erasure on shutdown
  As a Tails user
  when I shutdown Tails
  I want the system memory to be free from sensitive data.

  Scenario: Anti-test: no memory erasure on a modern computer             # features/erase_memory.feature:7
    Given a computer                                                      # features/step_definitions/common_steps.rb:112
    And the computer is a modern 64-bit system                            # features/step_definitions/erase_memory.rb:24
    And the computer has 8 GiB of RAM                                     # features/step_definitions/common_steps.rb:117
    And I set Tails to boot with options "debug=wipemem"                  # features/step_definitions/common_steps.rb:183
    And I start Tails from DVD with network unplugged and I login         # features/step_definitions/common_steps.rb:196
    Then the PAE kernel is running                                        # features/step_definitions/erase_memory.rb:47
    And at least 8 GiB of RAM was detected                                # features/step_definitions/erase_memory.rb:67
      Detected 8003 MiB of RAM
    And process "memlockd" is running                                     # features/step_definitions/common_steps.rb:510
    And process "udev-watchdog" is running                                # features/step_definitions/common_steps.rb:510
    And udev-watchdog is monitoring the correct device                    # features/step_definitions/erase_memory.rb:19
    When I fill the guest's memory with a known pattern without verifying # features/step_definitions/erase_memory.rb:100
    And I reboot without wiping the memory                                # features/step_definitions/erase_memory.rb:187
    Then I find many patterns in the guest's memory                       # features/step_definitions/erase_memory.rb:178
      Pattern coverage: 86.264% (6903 MiB)

  Scenario: Memory erasure on a modern computer                   # features/erase_memory.feature:22
    Given a computer                                              # features/step_definitions/common_steps.rb:112
    And the computer is a modern 64-bit system                    # features/step_definitions/erase_memory.rb:24
    And the computer has 8 GiB of RAM                             # features/step_definitions/common_steps.rb:117
    And I set Tails to boot with options "debug=wipemem"          # features/step_definitions/common_steps.rb:183
    And I start Tails from DVD with network unplugged and I login # features/step_definitions/common_steps.rb:196
    Then the PAE kernel is running                                # features/step_definitions/erase_memory.rb:47
    And at least 8 GiB of RAM was detected                        # features/step_definitions/erase_memory.rb:67
      Detected 8003 MiB of RAM
    And process "memlockd" is running                             # features/step_definitions/common_steps.rb:510
    And process "udev-watchdog" is running                        # features/step_definitions/common_steps.rb:510
    And udev-watchdog is monitoring the correct device            # features/step_definitions/erase_memory.rb:19
    When I fill the guest's memory with a known pattern           # features/step_definitions/erase_memory.rb:100
      Pattern coverage: 86.314% (6907 MiB)
    And I shutdown and wait for Tails to finish wiping the memory # features/step_definitions/erase_memory.rb:193
    Then I find very few patterns in the guest's memory           # features/step_definitions/erase_memory.rb:169
      Pattern coverage: 0.000% (0 MiB)

  Scenario: Anti-test: no memory erasure on an old computer               # features/erase_memory.feature:37
    Given a computer                                                      # features/step_definitions/common_steps.rb:112
    And the computer is an old pentium without the PAE extension          # features/step_definitions/erase_memory.rb:31
    And the computer has 8 GiB of RAM                                     # features/step_definitions/common_steps.rb:117
    And I set Tails to boot with options "debug=wipemem"                  # features/step_definitions/common_steps.rb:183
    And I start Tails from DVD with network unplugged and I login         # features/step_definitions/common_steps.rb:196
    Then the non-PAE kernel is running                                    # features/step_definitions/erase_memory.rb:53
    And at least 3500 MiB of RAM was detected                             # features/step_definitions/erase_memory.rb:67
      Detected 3548 MiB of RAM
    And process "memlockd" is running                                     # features/step_definitions/common_steps.rb:510
    And process "udev-watchdog" is running                                # features/step_definitions/common_steps.rb:510
    And udev-watchdog is monitoring the correct device                    # features/step_definitions/erase_memory.rb:19
    When I fill the guest's memory with a known pattern without verifying # features/step_definitions/erase_memory.rb:100
    And I reboot without wiping the memory                                # features/step_definitions/erase_memory.rb:187
    Then I find many patterns in the guest's memory                       # features/step_definitions/erase_memory.rb:178
      Pattern coverage: 73.379% (2603 MiB)

  Scenario: Memory erasure on an old computer                     # features/erase_memory.feature:52
    Given a computer                                              # features/step_definitions/common_steps.rb:112
    And the computer is an old pentium without the PAE extension  # features/step_definitions/erase_memory.rb:31
    And the computer has 8 GiB of RAM                             # features/step_definitions/common_steps.rb:117
    And I set Tails to boot with options "debug=wipemem"          # features/step_definitions/common_steps.rb:183
    And I start Tails from DVD with network unplugged and I login # features/step_definitions/common_steps.rb:196
    And the non-PAE kernel is running                             # features/step_definitions/erase_memory.rb:53
    And at least 3500 MiB of RAM was detected                     # features/step_definitions/erase_memory.rb:67
      Detected 3548 MiB of RAM
    And process "memlockd" is running                             # features/step_definitions/common_steps.rb:510
    And process "udev-watchdog" is running                        # features/step_definitions/common_steps.rb:510
    And udev-watchdog is monitoring the correct device            # features/step_definitions/erase_memory.rb:19
    When I fill the guest's memory with a known pattern           # features/step_definitions/erase_memory.rb:100
      Pattern coverage: 73.985% (2624 MiB)
    And I shutdown and wait for Tails to finish wiping the memory # features/step_definitions/erase_memory.rb:193
    Then I find very few patterns in the guest's memory           # features/step_definitions/erase_memory.rb:169
      Pattern coverage: 0.001% (0 MiB)

Testing ISO image: tails-i386-feature_i2p-0.9.22-1.6-20150914.iso
Using old ISO image: tails-i386-feature_i2p-0.9.22-1.6-20150914.iso
Call to virNetworkCreate failed: internal error: Child process (/sbin/ip addr add fc00::1/7 dev virbr10) unexpected exit status 2: RTNETLINK answers: Permission denied
 (Libvirt::Error)
/home/kytv/tails/features/support/helpers/vm_helper.rb:37:in `create'
/home/kytv/tails/features/support/helpers/vm_helper.rb:37:in `update'
/home/kytv/tails/features/support/helpers/vm_helper.rb:14:in `initialize'
/home/kytv/tails/features/support/hooks.rb:102:in `new'
/home/kytv/tails/features/support/hooks.rb:102:in `block in <top (required)>'
/home/kytv/tails/features/support/extra_hooks.rb:35:in `call'
/home/kytv/tails/features/support/extra_hooks.rb:35:in `invoke'
/home/kytv/tails/features/support/extra_hooks.rb:66:in `block in before_feature'
/home/kytv/tails/features/support/extra_hooks.rb:65:in `each'
/home/kytv/tails/features/support/extra_hooks.rb:65:in `before_feature'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:181:in `block in send_to_all'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:179:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:179:in `send_to_all'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:169:in `broadcast'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:26:in `visit_feature'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:28:in `block in accept'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:17:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:17:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:27:in `accept'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:170:in `broadcast'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:20:in `visit_features'
/usr/lib/ruby/vendor_ruby/cucumber/runtime.rb:49:in `run!'
/usr/lib/ruby/vendor_ruby/cucumber/cli/main.rb:42:in `execute!'
/usr/bin/cucumber:13:in `<main>'

Once this error is received, re-running the test suite will fail.

# ./run_test_suite   --view --iso ../isos/tails-i386-feature_i2p-0.9.22-1.6-20150914.iso  -- --format pretty --format debug --out /tmp/debug.log 
Virtual X framebuffer started on display :1
VNC server running on: localhost:5900
[info] Sikuli vision engine loaded.
Testing ISO image: tails-i386-feature_i2p-0.9.22-1.6-20150914.iso
Using old ISO image: tails-i386-feature_i2p-0.9.22-1.6-20150914.iso
Call to virNetworkCreate failed: internal error: Network is already in use by interface virbr10 (Libvirt::Error)
/home/kytv/tails/features/support/helpers/vm_helper.rb:37:in `create'
/home/kytv/tails/features/support/helpers/vm_helper.rb:37:in `update'
/home/kytv/tails/features/support/helpers/vm_helper.rb:14:in `initialize'
/home/kytv/tails/features/support/hooks.rb:102:in `new'
/home/kytv/tails/features/support/hooks.rb:102:in `block in <top (required)>'
/home/kytv/tails/features/support/extra_hooks.rb:35:in `call'
/home/kytv/tails/features/support/extra_hooks.rb:35:in `invoke'
/home/kytv/tails/features/support/extra_hooks.rb:66:in `block in before_feature'
/home/kytv/tails/features/support/extra_hooks.rb:65:in `each'
/home/kytv/tails/features/support/extra_hooks.rb:65:in `before_feature'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:181:in `block in send_to_all'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:179:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:179:in `send_to_all'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:169:in `broadcast'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:26:in `visit_feature'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:28:in `block in accept'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:17:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:17:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:27:in `accept'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:170:in `broadcast'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:20:in `visit_features'
/usr/lib/ruby/vendor_ruby/cucumber/runtime.rb:49:in `run!'
/usr/lib/ruby/vendor_ruby/cucumber/cli/main.rb:42:in `execute!'
/usr/bin/cucumber:13:in `<main>'

Once the device is removed the test suite can be run again.

# ip link del dev virbr10
# ./run_test_suite   --view --iso ../isos/tails-i386-feature_i2p-0.9.22-1.6-20150914.iso  -- --format pretty --format debug --out /tmp/debug.log 
Virtual X framebuffer started on display :1
VNC server running on: localhost:5900
[info] Sikuli vision engine loaded.
Testing ISO image: tails-i386-feature_i2p-0.9.22-1.6-20150914.iso
Using old ISO image: tails-i386-feature_i2p-0.9.22-1.6-20150914.iso
@product
Feature: Installing packages through APT
  As a Tails user
  when I set an administration password in Tails Greeter
  I should be able to install packages using APT and Synaptic
  and all Internet traffic should flow only through Tor.

  Background:                                                            # features/apt.feature:8
    Given a computer                                                     # features/step_definitions/common_steps.rb:112
    And I start the computer                                             # features/step_definitions/common_steps.rb:188

Subtasks

History

#1 Updated by intrigeri 2015-09-17 02:54:00

  • Assignee set to kytv
  • QA Check set to Info Needed

I think I’ve seen that in the last N days after some upgrade in sid. Not sure if I’ve restarted libvirt or rebooted or what since then, but never seen it again. Might you be in a similar situation? Perhaps restart libvirt or reboot and see if it happens again?

#2 Updated by kytv 2015-09-17 04:35:57

intrigeri wrote:
> I think I’ve seen that in the last N days after some upgrade in sid. Not sure if I’ve restarted libvirt or rebooted or what since then, but never seen it again. Might you be in a similar situation? Perhaps restart libvirt or reboot and see if it happens again?

I am, however, seeing this on Jessie (which in retrospect I probably should have mentioned).

I’ll try a bit more troubleshooting/diagnostics when/if I see it again.

#3 Updated by anonym 2015-09-17 05:16:20

I think test/6094-improved-snapshots may be the cause, because with --keep-snapshots the domain and network is not undefined after the test suite finishes, and when you re-run it with --keep-snapshots it tries to reuse that “left-over” domain and network. That’s not the case without test/6094-improved-snapshots — then the network is at least always undefined when a new run starts.

#4 Updated by kytv 2015-09-17 05:36:38

anonym wrote:
> I think test/6094-improved-snapshots may be the cause, because with --keep-snapshots the domain and network is not undefined after the test suite finishes, and when you re-run it with --keep-snapshots it tries to reuse that “left-over” domain and network. That’s not the case without test/6094-improved-snapshots — then the network is at least always undefined when a new run starts.

I’ll try confirming this theory but I’m not sure it is coming from Feature #6094.

IIRC I have seen that using --keep-snapshots with Feature #6094 and then trying to run the test suite on a branch which isn’t not based off of Feature #6094 that whether or not --keep-snapshots is used on the “non-6094-branch” you’ll have a problem trying to undefine the TailsToaster VM when run_test_suite is run.

When I saw the problem with undefining the VM (maybe I’m misremembering the text—I’ll confirm it later), I re-ran a single step from the test suite using a branch based off of Feature #6094 but without --keep-snapshots. When I got the failure in the OP it was after I ran the test suite without --keep-snapshots.

I’ll retrace my Feature #6094 => other branch results after the current test suite run completes.

#5 Updated by anonym 2015-09-17 05:49:20

Ah, indeed. When changing to/from the Feature #6094 branch, the libvirt state may be incompatible and have to be cleaned up manually.

#6 Updated by kytv 2015-09-19 17:44:04

It may actually be related to Feature #6094 after all. Since the last reboot I’ve stayed on the “old style” snapshot branches and haven’t seen this come up again.

#7 Updated by kytv 2015-09-20 09:21:15

  • Assignee deleted (kytv)
  • QA Check deleted (Info Needed)

kytv wrote:
> It may actually be related to Feature #6094 after all. Since the last reboot I’ve stayed on the “old style” snapshot branches and haven’t seen this come up again.

Not true. I just saw it on an ‘old style snapshot’ branch. I have not run the test suite with a Feature #6094-based branch since I last rebooted.


Testing ISO image: tails-i386-icedove-wip-1.6-20150920.iso
Using old ISO image: tails-i386-1.5.1.iso
Call to virNetworkCreate failed: internal error: Child process (/sbin/ip addr add fc00::1/7 dev virbr10) unexpected exit status 2: RTNETLINK answers: Permission denied
 (Libvirt::Error)
/home/kytv/tails/features/support/helpers/vm_helper.rb:36:in `create'
/home/kytv/tails/features/support/helpers/vm_helper.rb:36:in `update'
/home/kytv/tails/features/support/helpers/vm_helper.rb:14:in `initialize'
/home/kytv/tails/features/support/hooks.rb:102:in `new'
/home/kytv/tails/features/support/hooks.rb:102:in `block in <top (required)>'
/home/kytv/tails/features/support/extra_hooks.rb:35:in `call'
/home/kytv/tails/features/support/extra_hooks.rb:35:in `invoke'
/home/kytv/tails/features/support/extra_hooks.rb:66:in `block in before_feature'
/home/kytv/tails/features/support/extra_hooks.rb:65:in `each'
/home/kytv/tails/features/support/extra_hooks.rb:65:in `before_feature'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:181:in `block in send_to_all'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:179:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:179:in `send_to_all'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:169:in `broadcast'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:26:in `visit_feature'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:28:in `block in accept'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:17:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:17:in `each'
/usr/lib/ruby/vendor_ruby/cucumber/ast/features.rb:27:in `accept'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:170:in `broadcast'
/usr/lib/ruby/vendor_ruby/cucumber/ast/tree_walker.rb:20:in `visit_features'
/usr/lib/ruby/vendor_ruby/cucumber/runtime.rb:49:in `run!'
/usr/lib/ruby/vendor_ruby/cucumber/cli/main.rb:42:in `execute!'
/usr/bin/cucumber:13:in `<main>'

intrigeri wrote:
> Might you be in a similar situation?

I don’t think so.

#8 Updated by intrigeri 2015-12-14 09:40:17

  • Assignee set to kytv
  • Target version changed from Tails_1.7 to Tails_2.0
  • QA Check set to Info Needed

kytv, did you see it since last time we’ve discussed this? I haven’t.

#9 Updated by kytv 2015-12-22 14:02:32

  • Assignee deleted (kytv)
  • QA Check deleted (Info Needed)

intrigeri wrote:
> kytv, did you see it since last time we’ve discussed this? I haven’t.

Once, but it may have been my fault. I’m thinking that maybe I caused it by pressing CTRL+C at the “wrong time”. Previously I saw it in the middle of a run but that I haven’t experienced since the last time it was discussed.

I’ll update this ticket if I see it again without unclean interruptions coming into play.

#10 Updated by intrigeri 2015-12-22 14:19:29

  • Status changed from Confirmed to Rejected

> I’ll update this ticket if I see it again without unclean interruptions coming into play.

OK, please reopen if it happens :)