Feature #15806

Use GRUB for USB boot on EFI 64-bit

Added by intrigeri about 7 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Target version:
Tails_4.5
Start date:
Due date:
% Done:

100%

Feature Branch:
feature/6560-secure-boot+force-all-tests
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

We need this for Secure Boot (Feature #6560) and it’ll simplify quite a few parts of our code.

Making a Tails USB image boot with GRUB is straightforward but we need to:

  • Find another way to identify the boot device in our partitioning script: git grep -E 'FSUUID|sysappend'
  • Update our test suite accordingly: it currently expects the syslinux UI
  • Subtasks of this very issue
  • Fix whatever the tests below show as broken

Team: FT (intrigeri, segfault)

Manual testing plan

Note:

  • Tests done at commit:3c79a9cdbeb33d99a4cec7dc024091204b37db1a
  • Since this branch includes Feature #8415, the upgrade tests below also cover switching from aufs to overlayfs.
  • Automatic upgrade tests are done like this:
    • IUK generated using wrap_tails_create_iuks (just like in our release process)
    • IUK installed manually with tails-install-iuk (UDFs are not impacted by this branch, only IUK generation is)

Freshly installed Tails built from this branch

First, boots using isolinux from DVD: OK

Second, USB tests:

  1. boot a USB stick that’s been freshly installed from this branch
  2. verify that the system partition was resized
  3. boot a 2nd time to ensure that resizing the system partition did not break the boot
GNOME Disks (.img) Tails Installer (cloning) Etcher (.img)
boots using GRUB on EFI 64-bit + Secure Boot OK OK TODO
boots using GRUB on EFI 32-bit OK OK TODO
boots using syslinux on legacy BIOS OK OK TODO

Tails upgraded from older version that had no GRUB 64-bit EFI

with Tails Installer (cloning) with automatic upgrade
boots using GRUB on EFI 64-bit + Secure Boot OK OK
boots using GRUB on EFI 32-bit OK OK
boots using syslinux on legacy BIOS OK OK
syslinux EFI 64-bit was deleted OK OK

Tails upgraded from older Tails that already had GRUB 64-bit EFI

Note: the new image & the corresponding test IUK must modify EFI/debian/grub.cfg.

with Tails Installer (cloning) with automatic upgrade
boots using GRUB on EFI 64-bit + Secure Boot OK OK
boots using GRUB on EFI 32-bit OK OK
boots using syslinux on legacy BIOS OK OK
EFI/debian/grub.cfg was updated OK OK

Subtasks

Feature #17494: Update design doc for GRUB + Secure Boot Resolved

100

Related issues

Related to Tails - Feature #7422: Do not duplicate syslinux on the ISO root filesystem Rejected 2014-06-19
Related to Tails - Feature #15292: Distribute a USB image Resolved 2016-04-14 2019-01-29
Related to Tails - Feature #12440: Drop GRUB for 32-bit (ia32) UEFI Rejected 2017-04-12
Related to Tails - Feature #12439: Unify the syslinux directory & config file name between ISO and installed USB stick Rejected 2017-04-12
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed
Blocks Tails - Bug #16980: Increase size of random seed in the kernel command-line Confirmed
Blocks Tails - Bug #17049: Recent MacBook does not start: no boot menu, black screen Resolved

History

#1 Updated by intrigeri about 7 years ago

#2 Updated by intrigeri about 7 years ago

  • related to Feature #7422: Do not duplicate syslinux on the ISO root filesystem added

#3 Updated by intrigeri about 7 years ago

#4 Updated by intrigeri about 7 years ago

#5 Updated by intrigeri about 7 years ago

  • related to Feature #12439: Unify the syslinux directory & config file name between ISO and installed USB stick added

#6 Updated by intrigeri about 7 years ago

We won’t do this as part of the USB Image project.

#7 Updated by intrigeri about 7 years ago

  • Description updated
  • Target version set to 2019

#8 Updated by intrigeri about 6 years ago

#9 Updated by intrigeri about 6 years ago

  • Subject changed from Use GRUB for USB boot to Use GRUB for USB boot on EFI 64-bit
  • Description updated
  • Feature Branch set to wip/feature/6560-secure-boot

Clarifying the scope of this ticket; got a PoC that boots fine, documenting the next steps in the ticket description.

#10 Updated by intrigeri about 6 years ago

  • Description updated

#11 Updated by intrigeri about 6 years ago

  • Description updated

#12 Updated by intrigeri about 6 years ago

  • related to Bug #16229: Boot Loader Menu documentation does not support 32-bit UEFI added

#13 Updated by intrigeri about 6 years ago

  • Status changed from Confirmed to In Progress

Applied in changeset commit:tails|69157449b1d90b6f3fd170e4485b83c7134aa8f4.

#14 Updated by intrigeri about 6 years ago

  • Status changed from In Progress to Confirmed

#15 Updated by segfault about 6 years ago

  • blocks Bug #16980: Increase size of random seed in the kernel command-line added

#16 Updated by intrigeri about 6 years ago

  • related to Bug #17049: Recent MacBook does not start: no boot menu, black screen added

#17 Updated by intrigeri about 6 years ago

  • Feature Branch changed from wip/feature/6560-secure-boot to feature/6560-secure-boot

#18 Updated by intrigeri about 6 years ago

  • related to deleted (Bug #17049: Recent MacBook does not start: no boot menu, black screen)

#19 Updated by intrigeri about 6 years ago

  • blocks Bug #17049: Recent MacBook does not start: no boot menu, black screen added

#20 Updated by intrigeri about 6 years ago

Possibly relevant although we’re using live-build 2.x and might just do our GRUB config stuff ourselves without live-build’s help: https://bugs.debian.org/940846

#21 Updated by segfault about 5 years ago

> * Find another way to identify the boot device in our partitioning

I can’t find a way to make grub tell us the boot device. If we don’t find an option for that, the only alternative I see is to do the same thing live-boot does to choose the boot device, which is trying to mount all removable devices and choose the first one that contains a squashfs (or other filesystems) in /live/ (see find_livefs() in 9990-misc-helpers.sh.

#22 Updated by intrigeri about 5 years ago

> I can’t find a way to make grub tell us the boot device.

This might help:

probe --fs-uuid ($root)

Documentation: https://www.gnu.org/software/grub/manual/grub/grub.html#probe

But that only works if $root is already set to the partition where GRUB was loaded from, and usually one sets $root in grub.cfg by searching for the drive that has the already-known UUID where GRUB was installed. So perhaps there’s a chicken’n’egg problem here and this suggestion is not helpful. In our 64-bit UEFI + GRUB test images, are $prefix and $root set automatically by GRUB on startup?

> If we don’t find an option for that, the only alternative I see is to do the same thing live-boot does to choose the boot device, which is trying to mount all removable devices and choose the first one that contains a squashfs (or other filesystems) in /live/ (see find_livefs() in 9990-misc-helpers.sh.

It seems that in the end, we decided to repartition only on first boot. Then, what about this: at build time we know what the initial UUID for the FAT filesystem is going to be, so we can hard-code this somewhere in the FAT partition or in the initrd, and then use that in the partitioning script? This can cause trouble if 2 never-booted-yet Tails devices are plugged in, but I believe our current implementation already has this problem.

#23 Updated by segfault about 5 years ago

intrigeri wrote:
> > I can’t find a way to make grub tell us the boot device.
>
> This might help:
>
> […]
>
> Documentation: https://www.gnu.org/software/grub/manual/grub/grub.html#probe

Thanks, I also found that yesterday :)

>
> But that only works if $root is already set to the partition where GRUB was loaded from, and usually one sets $root in grub.cfg by searching for the drive that has the already-known UUID where GRUB was installed. So perhaps there’s a chicken’n’egg problem here and this suggestion is not helpful. In our 64-bit UEFI + GRUB test images, are $prefix and $root set automatically by GRUB on startup?

Yes, $root is set up automatically by GRUB.

#24 Updated by segfault about 5 years ago

  • Status changed from Confirmed to In Progress

Applied in changeset commit:tails|99fd55d467aae03c1c3c67321444af9906324309.

#25 Updated by segfault about 5 years ago

  • Description updated

I pushed a commit which uses GRUB’s probe command to set FSUUID on the kernel command-line. But I had to install grub from sid for that, the grub from buster doesn’t seem to support the probe command (which is strange because the command is older than the 2.02 release that Buster’s grub is based on, and I see grub-core/commands/probe.c in the Debian packaging repo on tag debian/2.02+dfsg1-20).

#26 Updated by intrigeri about 5 years ago

> the grub from buster doesn’t seem to support the probe command (which is strange because the command is older than the 2.02 release that Buster’s grub is based on, and I see grub-core/commands/probe.c in the Debian packaging repo on tag debian/2.02+dfsg1-20).

I think this can be explained by:

grub2 (2.04-3) unstable; urgency=medium

[…]
 * Add probe module to signed UEFI images (closes: #936082).

:)

#27 Updated by intrigeri about 5 years ago

  • Priority changed from Normal to High
  • Target version changed from 2019 to Tails_4.5

Some of this has to be done by May, some of it by July.

#28 Updated by intrigeri about 5 years ago

  • Description updated

#29 Updated by intrigeri about 5 years ago

  • Feature Branch changed from feature/6560-secure-boot to feature/6560-secure-boot+force-all-tests

#30 Updated by intrigeri about 5 years ago

  • Description updated

#31 Updated by intrigeri about 5 years ago

  • Description updated

#32 Updated by intrigeri about 5 years ago

  • Description updated

#33 Updated by intrigeri about 5 years ago

  • Description updated

#34 Updated by intrigeri about 5 years ago

  • Description updated

#35 Updated by intrigeri about 5 years ago

  • Description updated

#36 Updated by intrigeri about 5 years ago

  • Description updated

#37 Updated by intrigeri about 5 years ago

  • Description updated

(Drafted manual testing plan.)

#38 Updated by intrigeri about 5 years ago

  • Description updated

(Add a few manual tests.)

#39 Updated by intrigeri about 5 years ago

  • Description updated

(Describe test protocol more precisely, report some results.)

#40 Updated by intrigeri about 5 years ago

  • Description updated

#41 Updated by intrigeri about 5 years ago

  • Description updated

#42 Updated by intrigeri about 5 years ago

  • Description updated

#43 Updated by intrigeri about 5 years ago

  • Description updated

(Report success for all non-Etcher initial install scenarios.)

#44 Updated by intrigeri about 5 years ago

  • Description updated

(Clarify & fix how upgrade tests are done.)

#45 Updated by intrigeri about 5 years ago

  • Description updated

(Reporting success for all non-Etcher tests.)

#46 Updated by intrigeri about 5 years ago

  • Description updated

#47 Updated by intrigeri about 5 years ago

#48 Updated by intrigeri about 5 years ago

#49 Updated by intrigeri about 5 years ago

  • Status changed from In Progress to Needs Validation
  • Assignee set to sajolida

@sajolida, could you please do the Etcher tests listed as “TODO” in the ticket description? I see no reason why there could be a regression (we’re copying a disk image bit-by-bit after all!) but still, better safe than sorry.

If it works fine, please close this ticket as resolved :)

#50 Updated by intrigeri about 5 years ago

  • Status changed from Needs Validation to Resolved

Applied in changeset commit:tails|7ca99d6f0e1670dc0c5daedf13a449528d5a7783.

#51 Updated by sajolida about 5 years ago

  • Assignee deleted (sajolida)