Feature #6397

Support booting from devices exposed as non-removable

Added by intrigeri 2013-11-02 08:40:08 . Updated 2020-03-09 19:30:38 .

Status:
Confirmed
Priority:
Normal
Assignee:
Category:
Hardware support
Target version:
Start date:
2014-06-30
Due date:
% Done:

10%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
0
Affected tool:
Deliverable for:

Description

Tails refuses to boot from devices that expose themselves as non-removable. Historically, all USB sticks said they were removable, even if this is not correct according to the specification (removable is rather for devices that can be fed with removable medium, such as a floppy drive). But:

  • Some USB sticks (especially Sandisk) expose themselves as non-removable.
  • Some external USB hard drives expose themselves as non-removable.
  • There’s recurring interest in installing Tails on internal hard disks (e.g. for performance reasons, or to dedicate a machine to Tails, or to avoid having a USB stick protruding outside of a laptop, or to install Tails on a tablet’s internal storage): see Feature #8422 for use cases and what it would take to support it.

A plan could be to:

  1. pass the syslinux SYSAPPEND variable a bitmask that enables FSUUID= functionality (bit 0x40000): this feature appends, to the kernel command-line, the UUID of the partition the kernel is booted from (added in 6.03-pre9, see commit 386b59e1 in syslinux Git) DONE in commit:d421f38e22898fc0a6e58b9ae79a25366f0a2dfa
  2. add support to live-boot to search the SquashFS only on the filesystem specified by the FSUUID kernel command-line parameter, when present; when present, FSUUID should override any live-media= found on the command-line, so that we would still pass live-media=removable, that would only be taken into account when booting from DVD.

Subtasks

Feature #7475: Have live-boot honor FSUUID= Confirmed

10


Related issues

Related to Tails - Feature #6641: Allow booting from USB and SDIO, regardless of the removable flag Rejected 2014-02-08
Related to Tails - Bug #6976: Document that Sandisk Cruzer Orbit 32gb stick hangs at installation time Resolved 2014-03-25
Related to Tails - Bug #15989: Update our plans to remove removable flag requirement Resolved 2018-09-28
Related to Tails - Feature #15742: Test running Tails from an external hard disk Resolved 2018-07-19
Blocked by Tails - Feature #7173: Upgrade to syslinux 6.03-pre18 or later Resolved 2014-05-08

History

#1 Updated by sajolida 2013-12-06 01:12:41

If I understand correctly, this plan should also correct some weird behavior that are sometimes experienced while starting Tails with several Tails devices plugged in (DVD + USB, or 2 USB). If not, then this might be worth being taken into consideration.

#2 Updated by intrigeri 2013-12-06 01:27:45

> If I understand correctly, this plan should also correct some weird behavior that are
> sometimes experienced while starting Tails with several Tails
> devices plugged in

This is correct.

#3 Updated by intrigeri 2014-01-12 04:35:22

  • Target version set to Sustainability_M1

#4 Updated by BitingBird 2014-04-07 15:10:38

  • related to Bug #6976: Document that Sandisk Cruzer Orbit 32gb stick hangs at installation time added

#5 Updated by intrigeri 2014-05-08 12:18:58

  • Description updated

#6 Updated by intrigeri 2014-06-28 14:13:10

Is it possible for something running in the initramfs to know for sure what filesystem the initramfs was loaded from (by the bootloader)? If it is, then we don’t need the bootloader to provide us this information in any way.

#7 Updated by intrigeri 2014-06-28 18:20:13

  • Description updated

#8 Updated by intrigeri 2014-06-28 18:34:40

  • Feature Branch set to feature/6397-stop-relying-on-the-removable-bit

#9 Updated by intrigeri 2014-06-29 09:38:08

  • related to Feature #7173: Upgrade to syslinux 6.03-pre18 or later added

#10 Updated by intrigeri 2014-06-30 13:17:22

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

Tested the branch, works fine (I can see FSUUID=1D15-3984 at the end of the kernel command-line). Note that no FSUUID is appended when booting from DVD.

#11 Updated by intrigeri 2014-06-30 13:26:53

  • Description updated

#12 Updated by intrigeri 2014-06-30 13:45:14

  • Assignee set to intrigeri

#13 Updated by BitingBird 2014-06-30 23:38:41

When done, the “known issues” page should be updated to remove all the sandisk sticks, and possibly others :)

#14 Updated by intrigeri 2014-07-01 06:47:20

  • Description updated

#15 Updated by intrigeri 2014-07-28 20:11:08

  • related to deleted (Feature #7173: Upgrade to syslinux 6.03-pre18 or later)

#16 Updated by intrigeri 2014-07-28 20:11:24

  • blocked by Feature #7173: Upgrade to syslinux 6.03-pre18 or later added

#17 Updated by sajolida 2014-07-31 11:46:41

Already two people reported that newer SanDisk might have a different firmware that fixes this issue. But that still need confirmation.

After this, and as SanDisk was the main (if not only) vendor with that behavior, maybe this ticket can be set as low priority or closed…

#18 Updated by intrigeri 2014-08-01 15:07:16

> After this, and as SanDisk was the main (if not only) vendor with that behavior,
> maybe this ticket can be set as low priority or closed…

Agreed. Let’s see what happens :)

#19 Updated by sajolida 2014-08-02 13:34:03

Confirmed, some new SanDisk USB sticks are advertised again as removable:

/sys/devices/pci0000:00/0000:00:12.2/usb1/1-1/1-1:1.0/host2/target2:0:0/2:0:0:0/block/sda/removable
1

#20 Updated by emmapeel 2014-10-20 10:31:16

New user report indicates that SanDisk Extreme 3.0 without the Windows 8 logo, just bought, has removable flag off…

#21 Updated by sajolida 2014-10-20 17:38:23

Actually, this report says that the removable flag is set to “0”
(unset). We want the devices to be announced as removable.

#22 Updated by intrigeri 2014-11-10 16:03:54

  • Assignee changed from intrigeri to sajolida
  • QA Check set to Info Needed

Dear frontdesk, how is it going on your side? Did you notice a substantial change in the rate of bug reports that can be workaround’d by removing live-media=removable, since Sandisk changed their practices?

#23 Updated by sajolida 2014-11-11 11:20:38

Yes, definitely.

Still we got one report from someone who bought a new SanDisk without
the Windows 8 certificate and that wasn’t flagged as removable. We check
this in /sys/devices and all.

Maybe this stick was a mislabeled from factory…

#24 Updated by sajolida 2014-11-11 11:21:41

  • Assignee changed from sajolida to intrigeri
  • QA Check deleted (Info Needed)

#25 Updated by sajolida 2014-11-16 11:17:18

I bought a SanDisk the other day. I was marked as “Software Compatibility: Windows 7, Windows 8” but it didn’t have a Windows 8 logo or certification badge on it. Still it was correctly flagged as removable and boots fine.

#26 Updated by emmapeel 2014-12-14 20:22:40

Today a user reported not being able to boot unless removing the line “live-media=removable” at boot, though maybe External Hard Drives are unsupported. It happenned with a Western Digital Elements USB 3.0 External HDD.

#27 Updated by sajolida 2014-12-16 13:37:55

Emma: I think that this is expected for external hard disk.

#28 Updated by goupille 2015-01-10 19:47:25

maybe that’s a bad news, but we have a report that this issue is also affecting SD-Cards, at least the Sony SDHC 8 GB (Type: SF-8UY/T1)

#29 Updated by BitingBird 2015-01-11 00:33:10

goupille: this should be documented in Known issues, do you care to open a ticket ?

#30 Updated by intrigeri 2015-01-11 09:38:31

> goupille: this should be documented in Known issues, do you care to open a ticket ?

Note that this might be caused to the SD Card reader, rather than to the SD Card itself.

#31 Updated by WhiteWind 2015-01-13 23:42:51

Hello. Just wanted to say, I got a bought-new Sandisk Extreme 64GB USB 3.0 thumb drive (SDCZ80-064G-G46), there is no Windows 8 logo on the box, and I can’t boot from it. BUT that is currently only my first try with something different than Windows, so that could be something else I’ve not done/ I did wrong, on my end. I have followed the instructions given on Tails mainsite, SecureBoot is disabled and the thumb drive is set as the boot drive in the UEFI. When I boot on it, I have a black screen with “No configuration file found. Boot: ”. I would try the live-media= thing, but I’m not too confident, and well I don’t really understand how it is done.

#32 Updated by intrigeri 2015-01-14 09:33:11

redmine@labs.riseup.net wrote (13 Jan 2015 23:42:52 GMT) :
> Hello. Just wanted to say, I got a bought-new Sandisk Extreme 64GB USB 3.0 thumb drive (SDCZ80-064G-G46), there is no Windows 8 logo on the box, and I can’t boot from it.

Please report a bug, including the answers to all questions on https://tails.boum.org/doc/first_steps/bug_reporting/tails_does_not_start/. Thanks!

#33 Updated by sajolida 2015-01-14 09:48:46

The live-media= parameter has to be entered on the boot screen, see:

https://tails.boum.org/doc/first_steps/startup_options#index1h1

So if you don’t get to the boot screen, then I think that you are facing
a different issue. Feel free to contact our support team for more help.

#34 Updated by emmapeel 2015-03-13 08:36:01

New user reports problems with a brand new SanDisk Extreme USB3.0 stick with no Win8 logo.

User needed to remove the boot option to be able to boot.

#35 Updated by BitingBird 2015-03-13 13:12:59

I sent a patch to add SanDisk Extreme and Sony SD Card to the known issues. Further unsupported devices should get a dedicated doc ticket, instead of cluttering this one, which is about the code to solve the issue :)

#36 Updated by BitingBird 2015-03-14 12:57:03

goupille: regarding the SD Card, could you ask this person to run the following command:

sudo find /sys/devices -path “*sd*/removable” -print -exec cat ‘{}’ \;

This returns the status of the removable flag on the device.

#37 Updated by sajolida 2015-03-15 15:56:45

Frontdesk people, please make sure to submit patches to add the relevant hardware on the known issues page. This is definitely on your plate as stated in https://tails.boum.org/contribute/working_together/roles/front_desk.

Also, please make sure to have people run that find command before adding hardware. It’s in the frontdesk repo. As you surely know, people do all kind of crazy stuff to get Tails to run and report them in very funny ways sometimes…

#38 Updated by intrigeri 2015-03-16 22:45:05

It’s getting hard to find the relevant information in here, so let’s make things clear.

Frontdesk people: it’s useful to report general tendencies (or even stats), as I asked on Feature #6397#note-22: this will help us decide how urgent this problem is. (Looking at the history of the known issues section about it may actually be enough to assert its impact, as long as you keep it updated.)

However, this ticket is not about compiling a list of individual devices that can’t boot with live-media=removable, nor about updating the known issues page (there are better places for that anyway). It’s about fixing this problem for good.

Please, let’s stop adding to the (already big) pile of comments anything that doesn’t help fixing the underlying problem. Thanks!

#39 Updated by sajolida 2015-09-22 07:50:14

  • Target version deleted (Sustainability_M1)

#40 Updated by intrigeri 2015-11-01 02:03:59

  • Assignee deleted (intrigeri)

#41 Updated by intrigeri 2017-03-29 12:25:04

  • related to Feature #8422: Support running Tails from internal hard drives added

#42 Updated by intrigeri 2017-04-11 14:53:20

https://sources.open-infrastructure.net/software/system-boot/commit/?id=c6de5da6c4b3e6744cc05601930508472de54549 makes system-boot (a live-boot fork) ignore the “removable” bit, and instead use heuristics based on the bus that’s used (i.e. something relatively similar to what we do in Tails Installer and Tails Persistence Setup, where we treat all deviced plugged via USB or SDIO as OK). Might be good enough for us. I had rejected this idea on Feature #6641 but I think my reasons were too simplistic: as the aforementioned commit shows, there’s a sane way to do it. Now, this addresses the initial goal of this ticket, but it doesn’t address the UX issue one has when starting a computer with 2 Tails devices plugged in — however, Feature #7475#note-7 does, and can nicely combined with the solution described here.

#43 Updated by Anonymous 2017-06-29 10:20:24

  • Subject changed from Support USB devices exposed as non-removable to Support booting from USB devices exposed as non-removable

#44 Updated by Anonymous 2018-08-19 06:55:13

  • related to deleted (Feature #6766: Display an error message when boot fails)

#45 Updated by Anonymous 2018-08-19 06:55:19

  • related to Feature #6766: Display an error message when boot fails added

#46 Updated by Anonymous 2018-08-19 06:55:24

  • related to deleted (Feature #6766: Display an error message when boot fails)

#47 Updated by Anonymous 2018-08-19 06:55:33

  • blocks Feature #6766: Display an error message when boot fails added

#48 Updated by Anonymous 2018-08-19 08:41:40

  • blocked by deleted (Feature #6766: Display an error message when boot fails)

#49 Updated by intrigeri 2018-10-09 09:28:45

#50 Updated by intrigeri 2018-10-09 09:30:42

#51 Updated by intrigeri 2018-10-09 09:30:50

  • related to Bug #15989: Update our plans to remove removable flag requirement added

#52 Updated by intrigeri 2019-06-01 10:42:38

  • related to Feature #15742: Test running Tails from an external hard disk added

#53 Updated by intrigeri 2019-08-30 20:15:06

  • Subject changed from Support booting from USB devices exposed as non-removable to Support booting from devices exposed as non-removable
  • Status changed from In Progress to Confirmed

We don’t care about such USB devices anymore (Feature #16926) but there’s interest in supporting running Tails from an internal hard-drive, which is a similar problem ⇒ repurposing.

#54 Updated by intrigeri 2020-03-09 18:44:17

  • Description updated
  • Feature Branch deleted (feature/6397-stop-relying-on-the-removable-bit)

#55 Updated by intrigeri 2020-03-09 19:30:38

I want to scrap all alternate plans except the initial one, because:

  • The goal of the alternate plans was to avoid having to deal with bootloaders, but we did that already: we now pass FSUUID= on the kernel command line. So, the initial plan is now cheaper than expected initially.
  • At least in some cases, heuristics based on the bus don’t solve the “2 USB sticks plugged into the computer” problem, even combined with the initramfs+SquashFS UUID approach (details: Feature #7475#note-10).
  • Heuristics based on the bus still won’t solve the “booting Tails from an internal drive” scenario.
  • The “backup plan” that I’m deleting from the description assumed that the main way to install Tails was “install from ISO” in Tails Installer. That’s not the case anymore. In our current setup, we cannot rely on the installer to do anything more clever than a bit-by-bit copy.

Note, however, that the initial plan (Feature #7475) has two problems, that both stem from the fact the FSUUID is a publicly known value (currently shared by all versions of Tails: a69020d2), combined with a primary installation method based on bit-by-bit copy:

  • Security regression: when a user starts a newly installed Tails from a USB stick, if an adversary has previously planted a Tails-like-rogue-OS on the internal hard drive of the same computer, there’s a chance we boot from that rogue OS.
    • Once a Tails USB stick has been booted once, and our partitioning script has run, this problem goes away: our partitioning script sets a random FS UUID.
    • We could derive a random FS UUID, at image build time, from SOURCE_DATE_EPOCH. So at least, every Tails release has a different FS UUID. This would make it significantly more expensive, in most cases, to set up this attack successfully. Good enough?
  • If 2 freshly installed, never booted yet, Tails USB sticks are plugged in the computer, then we could partition another USB stick than the one the user elected to boot from, and mount the SquashFS from that other USB stick as well ⇒ confusion. IMO this is too much of a corner case to worry about.