Feature #15806

Use GRUB for USB boot on EFI 64-bit

Added by intrigeri 2018-08-18 14:04:50 . Updated 2020-03-21 23:00:06 .

Status:
Resolved
Priority:
High
Assignee:
Category:
Target version:
Tails_4.5
Start date:
Due date:
% Done:

100%

Feature Branch:
feature/6560-secure-boot+force-all-tests
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

We need this for Secure Boot (Feature #6560) and it’ll simplify quite a few parts of our code.

Making a Tails USB image boot with GRUB is straightforward but we need to:

  • Find another way to identify the boot device in our partitioning script: git grep -E 'FSUUID|sysappend'
  • Update our test suite accordingly: it currently expects the syslinux UI
  • Subtasks of this very issue
  • Fix whatever the tests below show as broken

Team: FT (intrigeri, segfault)

Manual testing plan

Note:

  • Tests done at commit:3c79a9cdbeb33d99a4cec7dc024091204b37db1a
  • Since this branch includes Feature #8415, the upgrade tests below also cover switching from aufs to overlayfs.
  • Automatic upgrade tests are done like this:
    • IUK generated using wrap_tails_create_iuks (just like in our release process)
    • IUK installed manually with tails-install-iuk (UDFs are not impacted by this branch, only IUK generation is)

Freshly installed Tails built from this branch

First, boots using isolinux from DVD: OK

Second, USB tests:

  1. boot a USB stick that’s been freshly installed from this branch
  2. verify that the system partition was resized
  3. boot a 2nd time to ensure that resizing the system partition did not break the boot
GNOME Disks (.img) Tails Installer (cloning) Etcher (.img)
boots using GRUB on EFI 64-bit + Secure Boot OK OK TODO
boots using GRUB on EFI 32-bit OK OK TODO
boots using syslinux on legacy BIOS OK OK TODO

Tails upgraded from older version that had no GRUB 64-bit EFI

with Tails Installer (cloning) with automatic upgrade
boots using GRUB on EFI 64-bit + Secure Boot OK OK
boots using GRUB on EFI 32-bit OK OK
boots using syslinux on legacy BIOS OK OK
syslinux EFI 64-bit was deleted OK OK

Tails upgraded from older Tails that already had GRUB 64-bit EFI

Note: the new image & the corresponding test IUK must modify EFI/debian/grub.cfg.

with Tails Installer (cloning) with automatic upgrade
boots using GRUB on EFI 64-bit + Secure Boot OK OK
boots using GRUB on EFI 32-bit OK OK
boots using syslinux on legacy BIOS OK OK
EFI/debian/grub.cfg was updated OK OK

Subtasks

Feature #17494: Update design doc for GRUB + Secure Boot Resolved

100

Related issues

Related to Tails - Feature #7422: Do not duplicate syslinux on the ISO root filesystem Rejected 2014-06-19
Related to Tails - Feature #15292: Distribute a USB image Resolved 2016-04-14 2019-01-29
Related to Tails - Feature #12440: Drop GRUB for 32-bit (ia32) UEFI Rejected 2017-04-12
Related to Tails - Feature #12439: Unify the syslinux directory & config file name between ISO and installed USB stick Rejected 2017-04-12
Blocks Tails - Feature #16209: Core work: Foundations Team Confirmed
Blocks Tails - Bug #16980: Increase size of random seed in the kernel command-line Confirmed
Blocks Tails - Bug #17049: Recent MacBook does not start: no boot menu, black screen Resolved

History

#1 Updated by intrigeri 2018-08-18 14:04:58

#2 Updated by intrigeri 2018-08-18 14:12:17

  • related to Feature #7422: Do not duplicate syslinux on the ISO root filesystem added

#3 Updated by intrigeri 2018-08-18 14:12:44

#4 Updated by intrigeri 2018-08-18 14:18:40

#5 Updated by intrigeri 2018-08-22 11:35:58

  • related to Feature #12439: Unify the syslinux directory & config file name between ISO and installed USB stick added

#6 Updated by intrigeri 2018-08-25 10:36:09

We won’t do this as part of the USB Image project.

#7 Updated by intrigeri 2018-09-14 11:38:11

  • Description updated
  • Target version set to 2019

#8 Updated by intrigeri 2019-04-05 16:10:07

#9 Updated by intrigeri 2019-04-11 11:05:51

  • Subject changed from Use GRUB for USB boot to Use GRUB for USB boot on EFI 64-bit
  • Description updated
  • Feature Branch set to wip/feature/6560-secure-boot

Clarifying the scope of this ticket; got a PoC that boots fine, documenting the next steps in the ticket description.

#10 Updated by intrigeri 2019-04-11 11:40:56

  • Description updated

#11 Updated by intrigeri 2019-04-11 13:50:08

  • Description updated

#12 Updated by intrigeri 2019-04-11 13:50:20

  • related to Bug #16229: Boot Loader Menu documentation does not support 32-bit UEFI added

#13 Updated by intrigeri 2019-04-11 13:58:31

  • Status changed from Confirmed to In Progress

Applied in changeset commit:tails|69157449b1d90b6f3fd170e4485b83c7134aa8f4.

#14 Updated by intrigeri 2019-04-11 13:59:21

  • Status changed from In Progress to Confirmed

#15 Updated by segfault 2019-08-15 18:17:34

  • blocks Bug #16980: Increase size of random seed in the kernel command-line added

#16 Updated by intrigeri 2019-09-13 07:04:30

  • related to Bug #17049: Recent MacBook does not start: no boot menu, black screen added

#17 Updated by intrigeri 2019-09-13 07:14:33

  • Feature Branch changed from wip/feature/6560-secure-boot to feature/6560-secure-boot

#18 Updated by intrigeri 2019-09-13 16:24:27

  • related to deleted (Bug #17049: Recent MacBook does not start: no boot menu, black screen)

#19 Updated by intrigeri 2019-09-13 16:24:34

  • blocks Bug #17049: Recent MacBook does not start: no boot menu, black screen added

#20 Updated by intrigeri 2019-10-01 05:15:15

Possibly relevant although we’re using live-build 2.x and might just do our GRUB config stuff ourselves without live-build’s help: https://bugs.debian.org/940846

#21 Updated by segfault 2019-11-23 20:22:56

> * Find another way to identify the boot device in our partitioning

I can’t find a way to make grub tell us the boot device. If we don’t find an option for that, the only alternative I see is to do the same thing live-boot does to choose the boot device, which is trying to mount all removable devices and choose the first one that contains a squashfs (or other filesystems) in /live/ (see find_livefs() in 9990-misc-helpers.sh.

#22 Updated by intrigeri 2019-11-24 08:36:16

> I can’t find a way to make grub tell us the boot device.

This might help:

probe --fs-uuid ($root)

Documentation: https://www.gnu.org/software/grub/manual/grub/grub.html#probe

But that only works if $root is already set to the partition where GRUB was loaded from, and usually one sets $root in grub.cfg by searching for the drive that has the already-known UUID where GRUB was installed. So perhaps there’s a chicken’n’egg problem here and this suggestion is not helpful. In our 64-bit UEFI + GRUB test images, are $prefix and $root set automatically by GRUB on startup?

> If we don’t find an option for that, the only alternative I see is to do the same thing live-boot does to choose the boot device, which is trying to mount all removable devices and choose the first one that contains a squashfs (or other filesystems) in /live/ (see find_livefs() in 9990-misc-helpers.sh.

It seems that in the end, we decided to repartition only on first boot. Then, what about this: at build time we know what the initial UUID for the FAT filesystem is going to be, so we can hard-code this somewhere in the FAT partition or in the initrd, and then use that in the partitioning script? This can cause trouble if 2 never-booted-yet Tails devices are plugged in, but I believe our current implementation already has this problem.

#23 Updated by segfault 2019-11-24 10:31:04

intrigeri wrote:
> > I can’t find a way to make grub tell us the boot device.
>
> This might help:
>
> […]
>
> Documentation: https://www.gnu.org/software/grub/manual/grub/grub.html#probe

Thanks, I also found that yesterday :)

>
> But that only works if $root is already set to the partition where GRUB was loaded from, and usually one sets $root in grub.cfg by searching for the drive that has the already-known UUID where GRUB was installed. So perhaps there’s a chicken’n’egg problem here and this suggestion is not helpful. In our 64-bit UEFI + GRUB test images, are $prefix and $root set automatically by GRUB on startup?

Yes, $root is set up automatically by GRUB.

#24 Updated by segfault 2019-11-24 10:32:29

  • Status changed from Confirmed to In Progress

Applied in changeset commit:tails|99fd55d467aae03c1c3c67321444af9906324309.

#25 Updated by segfault 2019-11-24 10:35:59

  • Description updated

I pushed a commit which uses GRUB’s probe command to set FSUUID on the kernel command-line. But I had to install grub from sid for that, the grub from buster doesn’t seem to support the probe command (which is strange because the command is older than the 2.02 release that Buster’s grub is based on, and I see grub-core/commands/probe.c in the Debian packaging repo on tag debian/2.02+dfsg1-20).

#26 Updated by intrigeri 2019-11-24 11:12:06

> the grub from buster doesn’t seem to support the probe command (which is strange because the command is older than the 2.02 release that Buster’s grub is based on, and I see grub-core/commands/probe.c in the Debian packaging repo on tag debian/2.02+dfsg1-20).

I think this can be explained by:

grub2 (2.04-3) unstable; urgency=medium

[…]
 * Add probe module to signed UEFI images (closes: #936082).

:)

#27 Updated by intrigeri 2019-12-01 11:24:45

  • Priority changed from Normal to High
  • Target version changed from 2019 to Tails_4.5

Some of this has to be done by May, some of it by July.

#28 Updated by intrigeri 2020-02-16 17:06:34

  • Description updated

#29 Updated by intrigeri 2020-02-21 15:24:48

  • Feature Branch changed from feature/6560-secure-boot to feature/6560-secure-boot+force-all-tests

#30 Updated by intrigeri 2020-02-21 17:35:32

  • Description updated

#31 Updated by intrigeri 2020-02-22 07:04:19

  • Description updated

#32 Updated by intrigeri 2020-02-22 07:04:47

  • Description updated

#33 Updated by intrigeri 2020-02-22 07:15:33

  • Description updated

#34 Updated by intrigeri 2020-02-22 07:22:15

  • Description updated

#35 Updated by intrigeri 2020-02-22 07:24:32

  • Description updated

#36 Updated by intrigeri 2020-02-22 08:17:27

  • Description updated

#37 Updated by intrigeri 2020-02-22 08:48:22

  • Description updated

(Drafted manual testing plan.)

#38 Updated by intrigeri 2020-02-22 08:57:32

  • Description updated

(Add a few manual tests.)

#39 Updated by intrigeri 2020-02-22 10:14:05

  • Description updated

(Describe test protocol more precisely, report some results.)

#40 Updated by intrigeri 2020-02-22 11:15:09

  • Description updated

#41 Updated by intrigeri 2020-02-22 11:17:56

  • Description updated

#42 Updated by intrigeri 2020-02-22 11:20:57

  • Description updated

#43 Updated by intrigeri 2020-02-22 11:59:57

  • Description updated

(Report success for all non-Etcher initial install scenarios.)

#44 Updated by intrigeri 2020-02-22 12:12:02

  • Description updated

(Clarify & fix how upgrade tests are done.)

#45 Updated by intrigeri 2020-02-22 13:30:36

  • Description updated

(Reporting success for all non-Etcher tests.)

#46 Updated by intrigeri 2020-02-23 07:02:36

  • Description updated

#47 Updated by intrigeri 2020-03-15 15:43:45

#48 Updated by intrigeri 2020-03-15 15:43:54

#49 Updated by intrigeri 2020-03-19 09:45:27

  • Status changed from In Progress to Needs Validation
  • Assignee set to sajolida

@sajolida, could you please do the Etcher tests listed as “TODO” in the ticket description? I see no reason why there could be a regression (we’re copying a disk image bit-by-bit after all!) but still, better safe than sorry.

If it works fine, please close this ticket as resolved :)

#50 Updated by intrigeri 2020-03-21 18:28:50

  • Status changed from Needs Validation to Resolved

Applied in changeset commit:tails|7ca99d6f0e1670dc0c5daedf13a449528d5a7783.

#51 Updated by sajolida 2020-03-21 22:07:40

  • Assignee deleted (sajolida)