Feature #16926
Forget about USB sticks that used to be configured as fixed disks
100%
Description
We have a section in /support/known_issues about USB sticks requiring live-media=removable to work for Tails:
https://tails.boum.org/support/known_issues/#sandisk
This problem has been reported as fixed by SanDisk mid-2014 (874ab45bdc6) and since then we only added 1 model in 2016 (0d34ce32949).
Several years later, I wonder whether this problem is still reported by users.
If not, we could get rid of this section in /support/known_issues and of the corresponding instructions in “Tails does not start entirely”.
I also don’t mind buying a recent SanDisk to double-check.
Subtasks
Related issues
| Blocks Tails - Feature #14788: Add anchors to the troubleshooting parts of the installation doc | In Progress | 2017-10-04 | |
|
Blocks Tails - |
Resolved | 2016-01-08 |
History
#2 Updated by sajolida 2019-07-30 19:12:47
- blocks Feature #14788: Add anchors to the troubleshooting parts of the installation doc added
#3 Updated by sajolida 2019-07-30 19:13:00
- Subject changed from Forget about USB sticks that are configured as fixed disks to Forget about USB sticks that used to be configured as fixed disks
#4 Updated by sajolida 2019-07-30 19:33:31
- blocks
Feature #16711: Core work 2019Q3 → 2019Q4: Technical writing added
#5 Updated by goupille 2019-07-31 18:46:02
- Assignee changed from goupille to sajolida
I’m using different models of sandisk sticks (including several Cruzer Fit 16Gb that I just saw listed as problematic…) for years without any issue.
I didn’t see that issue reported since a very long time, but sometimes (I think something like twice a year) a user mention using this workaround when they describe the way they are starting tails…
either they are using a problematic USB stick for years, they installed Tails on a hard drive, or they are using a sandisk USB stick, have thoroughly read our documentation and decided to be proactive (despite the warning that it is not a good idea to start tails with this parameters)
in any case I think that it could be a good idea to remove (or rewrite it as a ‘probably solved’ issue, asking people to report any problem instead of giving the workaround directly) this workaround from our known issue page. if the issue is still there then people will report back at some point, if not, yay :)
#6 Updated by sajolida 2019-08-01 19:41:55
- Status changed from New to Confirmed
Thanks for the hints!
After reading your answer I felt like doing some stats on WhisperBack reports and learning about regexp negative lookaheads. I used:
grep -Pl '^BOOT_IMAGE=/live/vmlinuz (?!.+live-media=removable)' *.ascOver the past 12 months, we received 808 reports which included the boot menu options, out of which 12 had no ‘live-media=removable’ in them, including 4 reports by some sajolida dude :)
From the reports without ‘live-media=removable’, none had SanDisk USB sticks. I’m seeing (removing duplicate senders):
| 4dc80ae26f8e3397fbe1437e1e1b232f | idVendor=0bda, idProduct=0316 |
| 872672011f906598b1dc6305a846abe1 | idVendor=058f, idProduct=6362 |
| ce33d6fd3c3c31f0560eedd77200fa24 | idVendor=090c, idProduct=1000 |
| d0d65ad04ab4b16cc6ad9d1bdc619d9 | idVendor=6557, idProduct=5500 |All of these USB devices being also used in other reports that still have ‘live-media=removable’.
In other words, it seems like all these people have removed ‘live-media=removable’ without a real need for it.
From the reports with ‘live-media=removable’, 309 have SanDisk USB sticks. I used:
grep -P 'Direct-Access +(?!ATA)SanDisk' *.ascI’m not sure if all of these are the Tails USB stick itself but it’s still a huge number (38% of reports with ‘live-media=removable’).
So I think that we can safely stop mentioning these in our troubleshooting doc and known issues. The most significant impact will be to hopefully stop people from using this trick when it’s not needed.
#7 Updated by sajolida 2019-08-02 16:17:42
- Status changed from Confirmed to Needs Validation
- Assignee changed from sajolida to intrigeri
- Feature Branch set to doc/16926-always-removable
Checking more carefully (see internal.git:ae1f7cb), SanDisk USB sticks are by far the most popular support for Tails with 247 reports, 30% of all reports including DVDs and none were flagged as removable.
I’m relying on the lines of dmesg where the device is added as a SCSI disk (after checking which $DEVICE is the boot medium):
grep -E '^.+ [0-9]+ [0-9:]+ \S+ kernel: sd [0-9:]+ \[$DEVICE\] Attached SCSI' *.ascHere is a branch.
I’m assigning it to intrigeri because I’m more worried about my correct interpretation of dmesg than my writting.
Do you mind having a look?
#8 Updated by intrigeri 2019-08-04 07:53:54
- Status changed from Needs Validation to In Progress
- Assignee changed from intrigeri to sajolida
> […] and none were flagged as removable.
How did you reach this conclusion?
> I’m relying on the lines of dmesg where the device is added as a SCSI disk (after checking which $DEVICE is the boot medium):
>
> grep -E '^.+ [0-9]+ [0-9:]+ \S+ kernel: sd [0-9:]+ \[$DEVICE\] Attached SCSI' *.asc
> I don’t understand what you think this means nor why: IIRC any USB drive will be added as a SCSI disk.
Can you please clarify?
I’ve skimmed over the discussion above and I see no indication that the many users who pass live-media=removable do not need it. It’s possible that many existing users still use Sandisk sticks that are exposed as non-removable. OTOH these folks already know they need live-media=removable so what matters here is probably new users. So I guess the decision here boils down to how many new users will try installing Tails on an affected USB stick. I wouldn’t mind seeing us make the bet that there are very few of those and thus dropping the doc as proposed. But first I’d like to ensure we’re not making this decision based on incorrect data analysis :)
#9 Updated by sajolida 2019-08-09 14:02:37
>> […] and none were flagged as removable.
>
> How did you reach this conclusion?
Running internal.git:stats/whisperback_scripts/stats-on-reports.rb.
It’s doing:
1. Find out the boot device through the output of mount.
See “# Medium” in the code. Example report line:
/dev/sdd1 vfat 29G 4.4G 25G 16% /lib/live/mount/medium2. When this device was added as a SCSI disk, check whether it was added
as “removable disk” or “disk”.
See “# Medium removable” in the code. Example report line:
amnesia kernel: sd 7:0:0:0: [sdd] Attached SCSI removable diskor
amnesia kernel: sd 7:0:0:0: [sdd] Attached SCSI diskI’d like you to confirm that I can rely on these “Attached SCSI” lines
to determine that the disk is configured as removable or non-removable.
It’s the basis of all my reasoning.
From the same dataset as in Feature #16926#note-6, my script could spot 5
reports with a “Attached SCSI disk” line and none of them had
“live-media=removable” in their boot options. This seems to confirm that
the very few people who have a “Attached SCSI disk” line do need to
remove this option.
None of these were SanDisk USB sticks. I think that these 5 were running
from external hard disks (“WD My Passport” and “Crucial_”). I also found
1 older such report running from an “Apricorn Secure Key” which is a USB
stick with a physical password pad:
https://www.apricorn.com/aegis-secure-key-3
> I don’t understand what you think this means nor why: IIRC any USB
> drive will be added as a SCSI disk.
I hope that I clarified this earlier. Some of these lines are “Attached
SCSI removable disk” and some of these lines are “Attached SCSI disk”.
> I’ve skimmed over the discussion above and I see no indication that
> the many users who pass live-media=removable do not need it.
I’m not sure to understand you here: “live-media=removable” comes by
default. Removing it is something you have to do manually: it always
works but it’s less secure. So there’s no such thing as needing to pass
live-media=removable to be able to start Tails.
> It’s possible that many existing users still use Sandisk sticks that
> are exposed as non-removable.
If we can rely on “Attached SCSI removable disk” to determine that USB
sticks are exposed as non-removable, then I couldn’t find any report
doing so in the 247 reports that my script report as using a SanDisk
boot device.
The number 247 that I’m giving now is smaller than the number 309 that I
gave in Feature #16926#note-6 because my script now only reports the device
which is booted from. The previous number probably included SanDisk USB
sticks that were not booted from as well.
> OTOH these folks already know they need live-media=removable so what
> matters here is probably new users.
If someone had a SanDisk USB stick exposed as non-removable, they would
have to manually REMOVE live-media=removable. They don’t NEED to add
it as your sentence seems to suggest.
#10 Updated by emmapeel 2019-08-11 09:48:26
I haven’t had to say this to users lately (in the last year? or more?). my 2 cents
#11 Updated by intrigeri 2019-08-12 13:26:23
Hi,
sajolida wrote:
> intrigeri wrote:
>> How did you reach this conclusion?
> […]
> I’d like you to confirm that I can rely on these “Attached SCSI” lines to determine that the disk is configured as removable or non-removable. It’s the basis of all my reasoning.
Ah, now I understand!
So, this message comes from:
sd_printk(KERN_NOTICE, sdkp, "Attached SCSI %sdisk\n",
sdp->removable ? "removable " : "");I tried to confirm that sdp->removable reflects the “removable” bit exposed by the USB device but I gave up after 10 minutes: I’m simply not familiar enough with the Linux kernel source tree.
Anyway, let’s hope that’s correct :)
>> I’ve skimmed over the discussion above and I see no indication that the many users who pass live-media=removable do not need it.
> I’m not sure to understand you here: “live-media=removable” comes by default.
I wrote something stupid than what I had in mind. See below:
> If someone had a SanDisk USB stick exposed as non-removable, they would have to manually REMOVE live-media=removable. They don’t NEED to add it as your sentence seems to suggest.
Yes, that’s exactly what I meant: the concern I’ve raised was about new users trying Tails on a USB stick exposed as non-removable. I have no strong opinion wrt. what to do with this; I merely raised it to ensure you factor it into your reasoning & conclusion :)
#12 Updated by sajolida 2019-08-17 10:27:23
Ok! Indeed there might people who still have a SanDisk USB stick from before 2014 and that would be exposed as non-removable. The trade-off that I’m trying to figure out here is whether it’s worth keeping this issue as it’s documented right now.
Each known issues has a cost:
- They make our known issues page longer and harder to parse.
- They also sometimes add yet another thing for people to wonder about
or try out. SanDisk USB sticks are by far the most popular installation
medium and I’m afraid that many people who can’t start Tails for
others reasons (starting with BIOS issues) might think that
it’s because of their SanDisk USB stick and give up.
Seeing that:
- None of the 247 WhisperBack reports that we received from SanDisk USB
sticks in the 3.x series required removinglive-media=removable. - goupille reported not seeing this issue reported since a very long
time; while before it was the case (I was doing help desk myself back then). - Some people are removing
live-media=removablewithout the need for
it.
It means that either:
- These people with old SanDisk USB sticks don’t write WhisperBack
reports and don’t complain to our help desk. - These old SanDisk USB sticks don’t exist anymore.
- Our current documentation is not helpful to solve their problem.
So, in any case the status quo is not good and I really prefer dropping this section instead of trying to improve it for very hypothetical use cases.
#13 Updated by sajolida 2019-08-17 10:31:53
Last tiny argument: when you are tech-savvy enough to remove live-media=removable from your Tails persistenly by editing syslinux/*.cfg (I do that), you transmit this unsafe settings when cloning your Tails. Seeing that some apparently tech-savvy WhisperBack reporters remove this option with no reason, I’m afraid it might impact their less tech-savvy friends who might clone from them.
#14 Updated by sajolida 2019-08-17 10:40:58
- Status changed from In Progress to Resolved
- % Done changed from 0 to 100
Applied in changeset commit:tails|397e427f25c5b08468ba5c53bc58fec2aff588e8.
#15 Updated by sajolida 2019-08-17 10:42:15
- Assignee deleted (
sajolida)
#16 Updated by intrigeri 2019-08-17 15:40:41
> * These old SanDisk USB sticks don’t exist anymore.
I can send you a dozen of them if you’d like.
More seriously:
> So, in any case the status quo is not good and I really prefer dropping this section instead of trying to improve it for very hypothetical use cases.
OK :)
#17 Updated by permezel 2019-12-29 08:09:41
I have a series of SanDisk removable SSD drives, recently purchased. They are not old discontinued crap. Current.
I would like to run Tails on them. All fail to boot. I recently tried again with the most recent Tails (4.1.1) and got thrown into the (initramfs) prompt.
/sys/block/sdb/removable reads 0. All my recent SanDisk Portable SSD T5 in sizes from 250G to 1T have this problem.
This issue is over 6 years old.
Please do something to actually fix it, rather than burying something in a “known issues” file and then deciding to delete it.
It could be as simple as a third boot option (tails // tails non-removable // tails recovery).
#18 Updated by sajolida 2020-01-03 18:39:37
Since you are talking about SSD, I understand that you a referring to
external USB disks and not USB pendrives. Your results match what I
experienced in Feature #15742: Tails does not work out-of-the-box from external
hard disks. I don’t think that this has to do with your disks being
SanDisk or a different brand, what this ticket was originally about.
If you want to run Tails from them, you can remove the
live-media=removable boot option. Either each time or from
syslinux/live.cfg and syslinux/live64.cfg in the Tails partition.