Bug #10988: Tails Installer workarounds for UDisks 2 bugs are not robust enough

Bug #10988

Tails Installer workarounds for UDisks 2 bugs are not robust enough

Added by intrigeri 2016-01-24 19:22:57 . Updated 2019-03-07 15:25:03 .

Status:

Rejected

Priority:

Normal

Assignee:

Category:

Installation

Target version:

Start date:

2015-12-07

Due date:

% Done:

100%

Feature Branch:

Type of work:

Code

Blueprint:

Starter:

Affected tool:

Installer

Deliverable for:

Description

As ~~Bug #9691~~ explains, due to bugs in UDisks 2 we had to add workarounds to Tails Installer. However, as seen e.g. on ~~Bug #10720~~ and ~~Bug #10987~~, our workarounds are fragile, to the point that we had to disable all automated tests that use Tails Installer, on Jenkins.

I suspect that the same technique(s) can be used to fix all these problems, hence this ticket to track them.

Subtasks

Bug #10720: Tails Installer freezes when calling system_partition.call_set_name_sync in partition_device

Resolved

100

Bug #10987: Tails Installer sometimes fails with: No support for modifying a partition a table of type `PMBR'

Resolved

intrigeri

Bug #11590: Improve Tails Installer robustness for 2.6

Resolved

100

Related issues

Related to Tails - ~~Bug #9691~~: Tails Installer has to workaround race conditions in UDisks2	Resolved	2015-07-05
Related to Tails - ~~Bug #15031~~: Installing to a device that previously had an hybrid ISO copied to it is fragile	Resolved	2017-12-09

History

#1 Updated by intrigeri 2016-01-24 19:23:10

related to ~~Bug #9691~~: Tails Installer has to workaround race conditions in UDisks2 added

#2 Updated by intrigeri 2016-01-24 19:34:33

Assignee set to alant
QA Check set to Info Needed

So, here’s an idea I had. In partition_device, currently we sync’, sleep a few seconds, rescan, and then we assume that everything is in the shape we expect it is. Unfortunately, as the subtasks show, this is too bold an assumption, and results in brittle behaviour. Could we instead wait and rescan until things actually are in the state we expect them to be? E.g. the criterion for ~~Bug #10987~~ would be that the partition table is seen as GPT by UDisks, and for ~~Bug #10720~~ we need to wait until UDisks sees the partition we’ve just created as something wit an org.freedesktop.UDisks2.Partition interface.

Alan, what do you think?

#3 Updated by alant 2016-02-18 13:28:10

Assignee changed from alant to intrigeri

intrigeri wrote:
> So, here’s an idea I had. In partition_device, currently we sync’, sleep a few seconds, rescan, and then we assume that everything is in the shape we expect it is. Unfortunately, as the subtasks show, this is too bold an assumption, and results in brittle behaviour.

I had a look at UDisks code and indeed the documentation of udisks_linux_block_object_reread_partition_table and udisks_linux_block_object_trigger_uevent in udiskslinuxblockobject.c (which are called amond others by rescan) both specifies that “This method does not wait for the event to be received.”

> Could we instead wait and rescan until things actually are in the state we expect them to be? E.g. the criterion for ~~Bug #10987~~ would be that the partition table is seen as GPT by UDisks, and for ~~Bug #10720~~ we need to wait until UDisks sees the partition we’ve just created as something wit an org.freedesktop.UDisks2.Partition interface.

That looks like a good idea to me. What do you expect me to do now?

#4 Updated by alant 2016-02-18 14:32:15

We might also be interested by https://udisks.freedesktop.org/docs/latest/UDisksClient.html#udisks-client-settle

#5 Updated by intrigeri 2016-02-18 16:28:31

Assignee changed from intrigeri to alant

alant wrote:
> What do you expect me to do now?

Well, if you want to try these new ideas (the settle one first I guess, and then perhaps mine) to try to make this udisks2 port robust enough for our needs, it would be super helpful. It would be nice to have this fixed in 2.3 (freeze = early April), and so I’d like to know soonish if I need to schedule time to work on this. Thanks in advance!

#6 Updated by alant 2016-02-18 17:21:39

Assignee changed from alant to intrigeri

> Well, if you want to try these new ideas (the settle one first I guess, and then perhaps mine) to try to make this udisks2 port robust enough for our needs, it would be super helpful.

I tried to replace a time.sleep by a self._udisksclient.settle() and it failed.

#7 Updated by alant 2016-02-18 18:37:14

I tried to check for the 2nd idea, but it’s less straightformward than I could imagine and I don’t promise I’ll find a fix on time.

#8 Updated by intrigeri 2016-02-18 20:40:14

Target version changed from Tails_2.2 to Tails_2.3
QA Check deleted (~~Info Needed~~)

Thanks!

#9 Updated by intrigeri 2016-04-16 15:41:04

Target version changed from Tails_2.3 to Tails_2.4

#10 Updated by elouann 2016-04-29 08:35:57

I’m not sure it’s related, but the “Error wiping newly created partition”, workarounded in ~~Bug #9691~~ was recently fixed upstream: https://bugs.freedesktop.org/show_bug.cgi?id=85477

#11 Updated by intrigeri 2016-05-16 13:23:54

Target version changed from Tails_2.4 to Tails_2.5

#12 Updated by BitingBird 2016-06-26 10:35:34

Status changed from Confirmed to In Progress

#13 Updated by intrigeri 2016-07-19 05:15:18

Target version deleted (~~Tails_2.5~~)

I lack info to work on one of the subtasks (~~Bug #10987~~) so it makes no sense to try and complete this within a fixed time frame.

#14 Updated by intrigeri 2016-07-21 12:23:41

I’ve been adding more kludges similar to the ones we had already (see ~~Bug #10720~~), with quite some success. If that’s not enough, IMO we should eventually go for the “retry and wait until the condition we need is satisfied” idea (~~Bug #10988#note-2~~).

#15 Updated by intrigeri 2016-12-19 14:57:12

I’ve seen that on Jenkins:

2016-10-02 09:02:50,056 [creator.py:1338 (get_extlinux_version)] WARNING: extlinux not found! Only FAT filesystems will be supported
2016-10-02 09:02:50,133 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/sda
2016-10-02 09:02:50,134 [creator.py:673 (detect_supported_drives)] DEBUG: {'bootable': None,
 'device': '/dev/sda',
 'free': None,
 'fstype': '',
 'fsversion': '',
 'is_device_big_enough': True,
 'is_optical': False,
 'label': 'QEMU-QEMU-HARDDISK-1-0000:00:05.0-2',
 'model': 'QEMU HARDDISK',
 'mount': None,
 'mounted_partitions': set([]),
 'parent': None,
 'parent_size': None,
 'parent_udi': None,
 'size': 4294967296L,
 'udi': '/org/freedesktop/UDisks2/block_devices/sda',
 'uuid': '',
 'vendor': 'QEMU'}
2016-10-02 09:02:50,135 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/drives/QEMU_DVD_ROM_QM00009
2016-10-02 09:02:50,135 [creator.py:593 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/drives/QEMU_DVD_ROM_QM00009 which is not a block device
2016-10-02 09:02:50,135 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/Manager
2016-10-02 09:02:50,135 [creator.py:593 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/Manager which is not a block device
2016-10-02 09:02:50,135 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/sr0
2016-10-02 09:02:50,136 [creator.py:625 (detect_supported_drives)] WARNING: Skipping device '/org/freedesktop/UDisks2/block_devices/sr0' connected to '' interface
2016-10-02 09:02:50,137 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/loop7
2016-10-02 09:02:50,137 [creator.py:599 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/block_devices/loop7 which has no associated drive
2016-10-02 09:02:50,137 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/loop6
2016-10-02 09:02:50,137 [creator.py:599 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/block_devices/loop6 which has no associated drive
2016-10-02 09:02:50,137 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/loop5
2016-10-02 09:02:50,137 [creator.py:599 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/block_devices/loop5 which has no associated drive
2016-10-02 09:02:50,137 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/loop4
2016-10-02 09:02:50,137 [creator.py:599 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/block_devices/loop4 which has no associated drive
2016-10-02 09:02:50,138 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/loop3
2016-10-02 09:02:50,138 [creator.py:599 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/block_devices/loop3 which has no associated drive
2016-10-02 09:02:50,138 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/loop2
2016-10-02 09:02:50,138 [creator.py:599 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/block_devices/loop2 which has no associated drive
2016-10-02 09:02:50,138 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/loop1
2016-10-02 09:02:50,139 [creator.py:599 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/block_devices/loop1 which has no associated drive
2016-10-02 09:02:50,139 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/block_devices/loop0
2016-10-02 09:02:50,139 [creator.py:599 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/block_devices/loop0 which has no associated drive
2016-10-02 09:02:50,139 [creator.py:591 (detect_supported_drives)] DEBUG: looking at /org/freedesktop/UDisks2/drives/QEMU_QEMU_HARDDISK_1_0000_3a00_3a05_2e0_2
2016-10-02 09:02:50,139 [creator.py:593 (detect_supported_drives)] DEBUG: skip /org/freedesktop/UDisks2/drives/QEMU_QEMU_HARDDISK_1_0000_3a00_3a05_2e0_2 which is not a block device
2016-10-02 09:02:50,139 [creator.py:692 (detect_supported_drives)] DEBUG: {}
2016-10-02 09:02:50,139 [gui.py:451 (add_devices)] DEBUG: drives: {'/dev/sda': {'size': 4294967296L, 'is_optical': False, 'vendor': 'QEMU', 'fsversion': '', 'parent': None, 'bootable': None, 'is_device_big_enough': True, 'mount': None, 'parent_size': None, 'mounted_partitions': set([]), 'free': None, 'label': 'QEMU-QEMU-HARDDISK-1-0000:00:05.0-2', 'fstype': '', 'parent_udi': None, 'device': '/dev/sda', 'model': 'QEMU HARDDISK', 'udi': '/org/freedesktop/UDisks2/block_devices/sda', 'uuid': ''}}
2016-10-02 09:02:55,076 [creator.py:514 (_set_drive)] DEBUG: /dev/sda selected: {'size': 4294967296L, 'is_optical': False, 'vendor': 'QEMU', 'fsversion': '', 'parent': None, 'bootable': None, 'is_device_big_enough': True, 'mount': None, 'parent_size': None, 'mounted_partitions': set([]), 'free': None, 'label': 'QEMU-QEMU-HARDDISK-1-0000:00:05.0-2', 'fstype': '', 'parent_udi': None, 'device': '/dev/sda', 'model': 'QEMU HARDDISK', 'udi': '/org/freedesktop/UDisks2/block_devices/sda', 'uuid': ''}
2016-10-02 09:02:57,399 [creator.py:783 (unmount_device)] DEBUG: Entering unmount_device for '/dev/sda'
2016-10-02 09:02:57,418 [creator.py:786 (unmount_device)] DEBUG: {'bootable': None,
 'device': '/dev/sda',
 'free': None,
 'fstype': '',
 'fsversion': '',
 'is_device_big_enough': True,
 'is_optical': False,
 'label': 'QEMU-QEMU-HARDDISK-1-0000:00:05.0-2',
 'model': 'QEMU HARDDISK',
 'mount': None,
 'mounted_partitions': set([]),
 'parent': None,
 'parent_size': None,
 'parent_udi': None,
 'size': 4294967296L,
 'udi': '/org/freedesktop/UDisks2/block_devices/sda',
 'uuid': '',
 'vendor': 'QEMU'}
2016-10-02 09:02:57,418 [creator.py:284 (popen)] DEBUG: sync
2016-10-02 09:03:00,449 [creator.py:284 (popen)] DEBUG: /sbin/sgdisk --print /dev/sda
2016-10-02 09:03:00,639 [creator.py:820 (partition_device)] INFO: Partitioning device /dev/sda
2016-10-02 09:03:00,639 [creator.py:823 (partition_device)] DEBUG: Creating partition table
2016-10-02 09:03:01,448 [creator.py:846 (partition_device)] DEBUG: Creating partition
2016-10-02 09:03:01,449 [gui.py:259 (run)] ERROR: 'NoneType' object has no attribute 'call_create_partition_sync'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/tails_installer/gui.py", line 196, in run
    self.live.partition_device()
  File "/usr/lib/python2.7/dist-packages/tails_installer/creator.py", line 849, in partition_device
    partition_table.call_create_partition_sync(
AttributeError: 'NoneType' object has no attribute 'call_create_partition_sync'
2016-10-02 09:03:01,450 [gui.py:260 (run)] DEBUG: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/tails_installer/gui.py", line 196, in run
    self.live.partition_device()
  File "/usr/lib/python2.7/dist-packages/tails_installer/creator.py", line 849, in partition_device
    partition_table.call_create_partition_sync(
AttributeError: 'NoneType' object has no attribute 'call_create_partition_sync'

#16 Updated by intrigeri 2017-12-09 15:02:11

related to ~~Bug #15031~~: Installing to a device that previously had an hybrid ISO copied to it is fragile added

#17 Updated by intrigeri 2019-03-07 15:25:03

Status changed from In Progress to Rejected
Assignee deleted (~~intrigeri~~)

I’ve not noticed such issues recently (be it on Jenkins or reported by help desk) and I don’t think we’ll ever prioritize this high enough to reproduce bugs, report them upstream, and hope someone else fixes them or do it ourselves.