Bug #16748
create-usb-image-from-iso uses the wrong version of syslinux
100%
Description
(This is only about my local runs since the results on Jenkins are broken because earlier steps fail, and this feature has inter-scenario dependencies.)
The step “Booting Tails from a USB drive upgraded from USB with persistence enabled” fails with booting looking like this:
I’ve causally looked at the volume in a VM, and it has the appropriate partitions, so only the boot bits seem to have been corrupted (not investigated that at all yet).
I cannot reproduce this manually: I installed to USB, cloned the volume, then booted one of them and upgraded the other, and that worked fine, which is essentially what the test suite does (it doesn’t copy, but redoes the same exact installation (we don’t supply an --old-iso so the same ISO will be used) so it shouldn’t matter).
Files
Subtasks
History
#1 Updated by anonym 2019-05-23 13:14:00
segfault, intrigeri,
Do you have any suggestions for how I can compare the “boot bits” of these volumes to try to narrow down what has happened?
Note that the Tails partitions are fine because the ‘the running Tails is installed on USB drive “to_upgrade”’ step succeeded, so the problem should be limited to the partition table/MBR/GPT header/similar. Oh, and to be clear, this is about legacy booting, not UEFI.
#2 Updated by anonym 2019-05-23 13:46:15
anonym wrote:
> Oh, and to be clear, this is about legacy booting, not UEFI.
So I tested UEFI boot, and that works. So it looks like the MBR (I guess?) is corrupted some how by the actions taken by the test suite.
#3 Updated by anonym 2019-05-23 13:55:12
- Description updated
#4 Updated by anonym 2019-05-23 15:08:05
The problem is that the .img (used by the test suite to create the “old” install) is generated with the syslinux used by the builder VM (currently Stretch’s) which might mismatch the version of syslinux we use in the installer (currently Buster’s). So yes there is a mismatch. The fix is to make auto/scripts/create-usb-image-from-iso some how use the correct version of syslinux, e.g. the one used in live-build’s chroot.
Is the chroot still available at the end of auto/build (where create-usb-image-from-iso is called)? If not, this might get awkward/difficult…
#5 Updated by anonym 2019-05-23 15:09:47
- Subject changed from Test suite cannot boot USB-to-USB upgraded Tails/Buster to create-usb-image-from-iso uses the wrong version of syslinux
- Category changed from Test suite to Installation
- Parent task deleted (
Bug #16281
#6 Updated by segfault 2019-05-23 15:28:20
- Category changed from Installation to Test suite
- Feature Branch set to bugfix/16748-use-syslinux-from-chroot
In a buildlog I see chroot/debootstrap still being accessed after live-build finished, so we might be good. I will just try using chroot/usr/bin/syslinux in create-usb-image-from-iso.
#7 Updated by segfault 2019-05-23 15:32:06
- Status changed from Confirmed to In Progress
Applied in changeset commit:tails|21182abe428422180dd993922a8182d5d6fcd91c.
#8 Updated by intrigeri 2019-05-23 15:36:23
> In a buildlog I see chroot/debootstrap still being accessed after live-build finished, so we might be good. I will just try using chroot/usr/bin/syslinux in create-usb-image-from-iso.
@segfault, note that the modules version has to match the syslinux binary version (there’s no ABI compatibility, which is the reason for some of the contrived things we do in Tails Installer about this sort of things). So you probably also should copy the modules from the chroot.
#9 Updated by segfault 2019-05-23 15:37:44
intrigeri wrote:
> > In a buildlog I see chroot/debootstrap still being accessed after live-build finished, so we might be good. I will just try using chroot/usr/bin/syslinux in create-usb-image-from-iso.
>
> @segfault, note that the modules version has to match the syslinux binary version (there’s no ABI compatibility, which is the reason for some of the contrived things we do in Tails Installer about this sort of things). So you probably also should copy the modules from the chroot.
ACK, will do that
#10 Updated by intrigeri 2019-05-24 07:13:40
- blocks Feature #16209: Core work: Foundations Team added
#11 Updated by anonym 2019-05-24 08:15:02
- Category changed from Test suite to Installation
- Assignee set to anonym
To me the least error prone approach would be to the these things inside the chroot instead. I’ll give it a quick try, and pass this ticket to segfault if I fail.
#12 Updated by anonym 2019-05-24 09:05:25
- Assignee changed from anonym to segfault
anonym wrote:
> To me the least error prone approach would be to the these things inside the chroot instead. I’ll give it a quick try, and pass this ticket to segfault if I fail.
I tested:
--- a/auto/build
+++ b/auto/build
@@ -181,4 +181,8 @@ echo "Generating build manifest..."
generate-build-manifest chroot/debootstrap "${BUILD_MANIFEST}"
echo "Creating USB image ${BUILD_USB_IMAGE_FILENAME}..."
-create-usb-image-from-iso "${BUILD_ISO_FILENAME}"
+cp auto/scripts/create-usb-image-from-iso chroot
+chmod a+x chroot/create-usb-image-from-iso
+mv "${BUILD_ISO_FILENAME}" chroot
+chroot chroot /create-usb-image-from-iso "${BUILD_ISO_FILENAME}"
+mv chroot/"${BUILD_ISO_FILENAME}" .but that failed with:
[...]
INFO:__main__:Setting up loop device
Traceback (most recent call last):
File "/create-usb-image-from-iso", line 340, in <module>
main()
File "/create-usb-image-from-iso", line 336, in main
image_creator.create_image()
File "/create-usb-image-from-iso", line 100, in create_image
with self.setup_loop_device():
File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/create-usb-image-from-iso", line 149, in setup_loop_device
with self.get_udisks_client() as udisks_client:
File "/usr/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/create-usb-image-from-iso", line 93, in get_udisks_client
client = UDisks.Client().new_sync()
gi.repository.GLib.Error: g-io-error-quark: Could not connect: No such file or directory (1)so I don’t think my approach is “less error-prone” any more. :) Nevermind!
#13 Updated by anonym 2019-05-24 13:05:45
I merged the current state of bugfix/16748-use-syslinux-from-chroot into feature/buster because it indeed fixes the original problem so now we can test beyond those failures.
#14 Updated by segfault 2019-05-24 20:52:47
anonym wrote:
> I merged the current state of bugfix/16748-use-syslinux-from-chroot into feature/buster because it indeed fixes the original problem so now we can test beyond those failures.
Cool! So, what’s the status here? fix committed? And what about stable? This didn’t hurt us yet because the vagrant box has the same syslinux version as Tails 3.x, and this will probably stay that way, but we could still merge this into stable.
#15 Updated by anonym 2019-05-27 07:19:25
- % Done changed from 0 to 40
- QA Check set to Dev Needed
segfault wrote:
> anonym wrote:
> > I merged the current state of bugfix/16748-use-syslinux-from-chroot into feature/buster because it indeed fixes the original problem so now we can test beyond those failures.
>
> Cool! So, what’s the status here? fix committed?
Let’s finish up with adding a CHROOT_ prefix to SYSLINUX_COM32MODULES_DIR for consistency, and also add a comment above those two vars about why we use the chroot.
> And what about stable? This didn’t hurt us yet because the vagrant box has the same syslinux version as Tails 3.x, and this will probably stay that way, but we could still merge this into stable.
Definitely!
#16 Updated by segfault 2019-05-27 14:13:50
- Assignee changed from segfault to anonym
- QA Check changed from Dev Needed to Ready for QA
anonym wrote:
> Let’s finish up with adding a CHROOT_ prefix to SYSLINUX_COM32MODULES_DIR for consistency, and also add a comment above those two vars about why we use the chroot.
Done
#17 Updated by anonym 2019-05-28 14:24:48
- Status changed from In Progress to Fix committed
- % Done changed from 40 to 100
Applied in changeset commit:tails|1bffb90c2f9d41c6ed66e459e0faa4dd2a8e1b0f.
#18 Updated by anonym 2019-05-28 14:26:25
- Assignee deleted (
anonym) - QA Check changed from Ready for QA to Pass
LGTM!
#19 Updated by intrigeri 2019-06-01 06:36:34
- Target version changed from Tails_4.0 to Tails_3.15
(This was merged into stable.)
#20 Updated by intrigeri 2019-07-03 08:13:38
- Status changed from Fix committed to Resolved
- Target version changed from Tails_3.15 to Tails_3.14.1