Bug #13206

Install by cloning sometimes silently fails from a stick installed with UUI

Added by goupille 2017-06-28 13:53:19 . Updated 2018-09-10 14:45:00 .

Status:
Rejected
Priority:
Normal
Assignee:
sajolida
Category:
Installation
Target version:
Start date:
2017-09-29
Due date:
% Done:

100%

Feature Branch:
Type of work:
Wait
Blueprint:

Starter:
Affected tool:
Installer
Deliverable for:

Description

some users reported that, after clicking on “install by cloning”, the window just shut down without any error message. It was reported by users trying to clone from a Tails 3.0 installed with UUI and by someone using Tails 3.0 inside virtualbox. sadly there are no logs to provide.

Team: spriver, sajolida


Subtasks

Feature #14734: Test "Clone the current Tails" onto a stick installed by UUI but with an NTFS system partition Resolved

0

Bug #14736: Test that it is still possible to install Tails with UUI Resolved

0

Bug #14907: UUI should preselect "Format in FAT32" when Tails is selected Rejected

0

Feature #15323: Improve visibility of FAT32 formatting in UUI instructions Resolved

0

Bug #15324: Inform the user in case their USB stick is formatted with NTFS Rejected

0


Related issues

Related to Tails - Feature #10984: Test Rufus Rejected 2016-01-22
Related to Tails - Feature #14448: Consider documenting Rufus as a workaround in case UUI doesn't work Rejected 2017-08-24
Related to Tails - Feature #14447: Consider using Win32 Disk Imager to install from Windows In Progress 2017-08-24
Related to Tails - Bug #14959: Test UEFI boot with a Tails installed with UUI Resolved 2017-11-11
Related to Tails - Feature #11679: Rethink the installation process and upgrade process Resolved 2016-08-20
Related to Tails - Bug #15453: Tails Installer 'Install' or 'Upgrade' button greyed out: Permission denied: '/lib/live/mount/medium' Duplicate 2018-03-23
Has duplicate Tails - Bug #11280: Tails Installer fails to mount certain USB sticks in Tails Rejected 2016-03-24

History

#1 Updated by intrigeri 2017-06-28 13:54:11

  • Subject changed from Install by cloning sometimes silently fails to Install by cloning sometimes silently fails from a stick installed with UUI
  • Category set to Installation

#2 Updated by intrigeri 2017-06-28 17:07:57

  • Status changed from New to Confirmed
  • Assignee changed from intrigeri to goupille
  • QA Check set to Info Needed

The info I need is:

  • how exactly was the device installed/upgraded
  • the full output of the tails-installer-launcher command, started in a Terminal

Thanks!

#3 Updated by goupille 2017-07-04 11:41:47

  • Assignee changed from goupille to intrigeri
  • QA Check changed from Info Needed to Dev Needed

I sent the asked informations to intrigeri, re-assigning the ticket

#4 Updated by goupille 2017-07-04 11:46:25

the user reported that “upgrade from iso” “install by cloning” and “upgrade by cloning” had the same result, the window shutdown with no error message.

#5 Updated by sajolida 2017-08-01 15:21:00

Happy to try to test this myself on my Windows machine if needed.

#6 Updated by emmapeel 2017-08-02 11:31:07

On the logs of another affected user, on a Tails installed with UUI, I can read:

ug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]: Traceback (most recent call last):
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/tails_installer/tails-installer", line 105, in <module>
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     main()
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/tails_installer/tails-installer", line 99, in main
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     win = TailsInstallerWindow(opts=opts, args=sys.argv)
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/python2.7/dist-packages/tails_installer/gui.py", line 325, in __init__
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     path=config['running_liveos_mountpoint'])
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/python2.7/dist-packages/tails_installer/source.py", line 67, in __init__
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     self.size = _dir_size(self.path)
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:   File "/usr/lib/python2.7/dist-packages/tails_installer/utils.py", line 57, in _dir_size
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]:     for item in os.listdir(source):
Aug 01 16:42:35 amnesia tails-installer-launcher.desktop[21621]: OSError: [Errno 13] Permission denied: '/lib/live/mount/medium'

The USB stick is easily open with the File manager, and the problem happens with it mounted or not (user tested the USB stick)

#7 Updated by sajolida 2017-08-03 17:42:54

#8 Updated by sajolida 2017-08-03 17:42:59

And once someone managed to use Rufus successfully it would be interesting to see if Tails Installer behaves differently on a USB stick installed using Rufus. Linking this ticket to Feature #10984.

#9 Updated by emmapeel 2017-08-09 09:18:06

intrigeri wrote:
> The info I need is:
> […]
> * the full output of the tails-installer-launcher command, started in a Terminal
>
Another friendly user has sent this output:


(tails-installer-launcher:16536): Gtk-WARNING **: Allocating size to GtkBox
0x5586c31b48b0 without calling gtk_widget_get_preferred_width/height(). How
does the code know the size to allocate?

(tails-installer-launcher:16536): Gtk-WARNING **: Allocating size to GtkBox
0x5586c31b48b0 without calling gtk_widget_get_preferred_width/height(). How
does the code know the size to allocate?

Traceback (most recent call last):

  File "/usr/lib/tails_installer/tails-installer", line 105, in <module>

    main()

  File "/usr/lib/tails_installer/tails-installer", line 99, in main

    win = TailsInstallerWindow(opts=opts, args=sys.argv)

  File "/usr/lib/python2.7/dist-packages/tails_installer/gui.py", line 325,
in __init__

    path=config['running_liveos_mountpoint'])

  File "/usr/lib/python2.7/dist-packages/tails_installer/source.py", line
67, in __init__

    self.size = _dir_size(self.path)

  File "/usr/lib/python2.7/dist-packages/tails_installer/utils.py", line
57, in _dir_size

    for item in os.listdir(source):

OSError: [Errno 13] Permission denied: '/lib/live/mount/medium'

#10 Updated by emmapeel 2017-08-10 16:29:01

A user having this problem with UUI has made a USB stick with Rufus, and the problem was not there anymore.

Ha!

Message-Id: <Um7jEb5fgW1GiTXgg2p0w38sunBVPp2R@tails-bugs.boum.org-schleuder>

#11 Updated by sajolida 2017-08-15 12:12:49

It’s good that you reference the Message-Id!

#12 Updated by sajolida 2017-08-15 12:13:44

If the foundations team thinks that providing an older version of UUI might be an option, I’m happy to dig into my archives.

#13 Updated by BitingBird 2017-08-30 11:11:31

  • Description updated
  • Target version set to 2018

#14 Updated by intrigeri 2017-09-01 13:08:40

  • Assignee changed from intrigeri to emmapeel
  • % Done changed from 0 to 100
  • QA Check changed from Dev Needed to Info Needed

Thanks for the logs, that’s interesting. There seems to be something wrong (early guess: either filesystem type, or filesystem options, or permissions) with the system installed by UUI.

goupille, emmapeel, I need:

  • the full output of the export DEBUG=1 ; tails-installer-launcher command (started in a Terminal) when this happens
  • the output of ls -lAR /lib/live/mount/medium
  • the output of mount -l
  • the output of df -T
  • the output of lsblk

Thanks in advance!

#15 Updated by intrigeri 2017-09-21 12:59:53

In a bug report (96cd2bb82a726017b5e53c70590c1388) I’ve received I see:

/dev/sda1 ntfs 1.9G 1.2G 749M 61% /lib/live/mount/medium

i.e. whatever software was used to install Tails on that medium created, or reused, a NTFS filesystem. Either that software is buggy, or the user didn’t follow our doc that says “Select the Fat32 Format Drive option”, or our doc is outdated wrt. current UUI and the user was confused.

=> Help desk, next time you see people with this problem, ask them the output of mount -l and if /lib/live/mount/medium is NTFS, point them to our installation doc so they get themselves a correct Tails. And if you see this problem again with a proper FAT32 system partition, see my previous comment about what info I need.

sajolida & anonym: I’m not sure if we’re lucky that this drive booted at all: I have no idea if booting from NTFS gives a correctly working Tails or not, and I’m pretty sure that drive won’t boot in UEFI mode. Ideally, we should tell the user earlier (without waiting for something to break in an obvious way like this ticket shows) that their Tails is wrong. Cheap way: add a check + desktop notification pointing to the installation doc. Better, but harder way: add a check run at or before the Greeter, and instead of the regular Greeter window display an error + the installation doc. I don’t know how high priority this would be.

#16 Updated by sajolida 2017-09-29 09:09:21

  • related to Feature #14448: Consider documenting Rufus as a workaround in case UUI doesn't work added

#17 Updated by sajolida 2017-09-29 09:11:11

  • related to Feature #14447: Consider using Win32 Disk Imager to install from Windows added

#18 Updated by rolan 2017-10-13 01:17:02

Programs like Universal-USB-installer will format as NTFS without warning despite the selected “FAT32 option” when installing to a thumbdrives larger than 32GB.

#19 Updated by intrigeri 2017-10-22 09:01:41

  • Assignee changed from emmapeel to sajolida

rolan wrote:
> Programs like Universal-USB-installer will format as NTFS without warning despite the selected “FAT32 option” when installing to a thumbdrives larger than 32GB.

Is it “32GB or larger” or “strictly larger than 32GB”?

Interesting, this could explain a lot, thanks! Next steps could be:

  1. (optional) help desk checks bug reports to see if the affected drives are >=32GB; I’m writing “optional” as I bet that some affected users simply did not check the FAT32 option; anyone up to do that?
  2. someone (sajolida, who’s responsible for this part of our installation proces?) tries to reproduce; next steps assume that rolan is correct;
  3. check with UUI upstream if that behaviour is expected, and if it could be changed; according to https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system a FAT filesystem can be up to 2TB large so I’m curious why UUI would force NTFS. If sajolida doesn’t want to handle this, perhaps I could do it myself (assuming this counts as Foundation Team core work);
  4. if UUI can’t be fixed in this respect, document that drives larger than 32GB are not supported and display a warning when starting Tails from a NTFS system partition (as said earlier on this ticket, this could be done e.g. in live-boot i.e. very early, or after login).

#20 Updated by sajolida 2017-10-24 22:20:27

  • Assignee changed from sajolida to intrigeri
  • QA Check changed from Info Needed to Ready for QA

I’ll report on the test from Feature #14734 here, so that all the information is in a single thread.

Summary: UUI behaves as instructed regarding partition formatting, including on >=32GB USB stick, but Tails Installer doesn’t work when run from an NTFS partition.

Details of the tests and results:

- UUI 1.9.7.7: The version we served from https//tails.boum.org/uui/Universal-USB-Installer.exe until today
  - [X] Install on FAT32 32GB without "Format in FAT32"
    → OK! FAT32 with previous files preserved. I didn't test that Tails.
  - [X] Install on FAT32 32GB with "Format in FAT32"
    → OK! FAT32 with previous files removed. I didn't test that Tails.
  - [X] Install on NTFS 32GB without "Format in FAT32"
    → EXPECTED! NTFS with previous files preserved. I didn't test that Tails.
  - [X] Install on NTFS 32GB with "Format in FAT32"
    → OK! FAT32 with previous files removed. Intermediary Tails boots fine.
  - [X] Install on NTFS 32GB with "Format in FAT32" and clone onto the intermediary Tails
    → OK! Tails Installer runs fine and final Tails boots fine.

- UUI 1.9.7.9: The version we serve starting from today
  - [X] Install on FAT32 32GB without "Format in FAT32"
    → OK! FAT 32 with previous files preserved. I didn't test that Tails.
  - [X] Install on FAT32 32GB with "Format in FAT32"
    → OK! FAT 32 with previous files removed. I didn't test that Tails.
  - [X] Install on NTFS 32GB without "Format in FAT32"
    → EXPECTED! NTFS with previous files preserved.
  - [X] Install on NTFS 32GB without "Format in FAT32" and clone to final Tails
    → REPRODUCED! NTFS with previous files preserved. Tails Installer fails to start on 3.2. I didn't write the error message but it was something about permission denied and /lib/live/mount/medium. I can get the full error message if you want.
  - [X] Install on NTFS 32GB with "Format in FAT32"
    → OK! FAT32 with previous files removed. Intermediary Tails boots fine.
  - [X] Install on NTFS 32GB with "Format in FAT32" and clone to final Tails
    → OK! Tails Installer runs fine and final Tails boots fine.

So I didn’t observed what rolan was describing and, for me, there’s no bug in UUI and our instructions are fine.

Now, what we could ask UUI is whether they can preselect “Format in FAT32” when Tails is selected, explaining that installing on NTFS leads to broken stuff (I would need more help on describing why the solution to this problem cannot be to make Tails work properly from an NTFS partition). UUI might be reluctant to do that because it’s erasing people’s file on the device but, on the other hand, it’s what we’re advertising on our official instructions anyway.

If you think that’s a good idea, maybe create another subticket to contact UUI and I can do it (unless the Foundations Team wants to take over).

#21 Updated by sajolida 2017-10-25 09:00:05

If UUI doesn’t do the change, for whatever reason, we could also:

  • (a) Improve the documentation to insist more on “Format in FAT32”. But this won’t catch everybody.
  • (b) Check how hard it would be to make it possible to clone from an intermediary Tails on NTFS. I’m not saying that we should fix Tails on NTFS, but make it possible to go on with the rest of the instructions even if the intermediary Tails is on NTFS prevent a bunch of failures.
  • © Add a check early like intrigeri suggested in Bug #13206#note-15. Note that if we put this check before or instead of Tails Greeter, then people won’t be able to see it in their favorite language so, especially if we can do (b), I would rather be in favor of something on the desktop inviting them to clone to a final Tails. If we can’t do (b), the check should be after Tails Greeter possibly instead of the desktop.

#22 Updated by intrigeri 2017-10-29 09:02:33

  • Assignee changed from intrigeri to sajolida
  • QA Check changed from Ready for QA to Dev Needed

> I’ll report on the test from Feature #14734 here, so that all the information is in a single thread.
> […]
> Details of the tests and results:

Amazing, thanks!

> So I didn’t observed what rolan was describing and, for me, there’s no bug in UUI and our instructions are fine.

OK. So the engineer’s point of view is that everything is fine. But in the real world, the users’ & help desk’s point of view is that very often “it doesn’t work”. So I’m glad you’re proposing improvements despite these test results :)

> Now, what we could ask UUI is whether they can preselect “Format in FAT32” when Tails is selected, explaining that installing on NTFS leads to broken stuff (I would need more help on describing why the solution to this problem cannot be to make Tails work properly from an NTFS partition). UUI might be reluctant to do that because it’s erasing people’s file on the device but, on the other hand, it’s what we’re advertising on our official instructions anyway.

Good idea. Here’s the info you requested:

  • We don’t want to support NTFS because it supports ownership & permissions, and Tails requires the desktop user (amnesia, 1000:1000) to have full read access to the files we install on the system partition (otherwise Tails Installer would need even more admin credentials, which is bad from both a security and UX perspective). Given we have no control over what user UUI is run as on Windows, this implies we have no guarantee that the installed system files will be readable by the desktop user on Tails. So we need a filesystem that doesn’t support ownership & permissions. To the best of my knowledge, the only such filesystem that works with UEFI is FAT.
  • Regardless of NTFS vs. FAT32, we don’t want to support installing on an existing filesystem that might already have data and it’s impossible to guarantee anything wrt. how pre-existing files may affect Tails’ behavior. So we need “Format”.

It logically follows that we need “Format in FAT32”.

(As said above, EFI support might be another argument against NTFS, but I didn’t actually test EFI on NTFS so let’s not mention it.)

> If you think that’s a good idea, maybe create another subticket to contact UUI and I can do it (unless the Foundations Team wants to take over).

Bug #14907

I’ll let you adjust metadata (e.g. call it UX core work or not, set a target version or not, priority etc.) as you wish. I bet our help desk would call it high priority given the amount of pain they hear about (confirmed by what I see on XMPP).

#23 Updated by intrigeri 2017-10-29 09:16:35

> If UUI doesn’t do the change, for whatever reason, we could also:

> * (a) Improve the documentation to insist more on “Format in FAT32”. But this won’t catch everybody.

Yes.

> * (b) Check how hard it would be to make it possible to clone from an intermediary Tails on NTFS. I’m not saying that we should fix Tails on NTFS, but make it possible to go on with the rest of the instructions even if the intermediary Tails is on NTFS prevent a bunch of failures.

I think the only way to do this (without having to require an admin password being set and prompt for it) is to delegate the actual files copy to some privileged helper program (run e.g. via sudo). I don’t know how hard it would be to do it safely, hopefully not more than 2 days of work. It increases the Tails attack surface though.

Regardless of how much work it requires, I’m not convinced it’s a good idea: people do use Tails installed via UUI in production, and as long as it boots, some will keep doing that no matter what, and it’s hard to prevent them from doing so (both technically and in terms of UX, e.g. there are people who can run Tails installed by UUI but not the final one installed with Tails Installer). So I think that “I’m not saying that we should fix Tails on NTFS” is highly theoretical and hard to make real. IMO either we clearly make “Tails on NTFS” unsupported (e.g. via © below) or we start supporting it for real, which requires more work than just (b).

> * © Add a check early like intrigeri suggested in Bug #13206#note-15. Note that if we put this check before or instead of Tails Greeter, then people won’t be able to see it in their favorite language

Good catch!

> the check should be after Tails Greeter possibly instead of the desktop.

I don’t think that’s technically required (but it might be the best option in terms of UX, you tell me). We need to display the error/warning message in the chosen language once a language has been chosen. This could happen in the Greeter, e.g.:

When I start Tails installed on NTFS
Then I see an error message about it in the Greeter
And the Login button is disabled
When I select my preferred language in the Greeter
Then the error message is localized (like the rest of the Greeter's UI)
And the Login button remains disabled

Anyway, let’s not put too much time into this until Bug #14907 reaches some conclusion :)

#24 Updated by sajolida 2017-11-06 18:58:54

Quick note to myself: What about having an animated GIF on the selection of “Format FAT32” in the instructions :)

#25 Updated by intrigeri 2017-11-11 12:38:36

intrigeri wrote:
> I bet our help desk would call it high priority given the amount of pain they hear about (confirmed by what I see on XMPP).

This made it into the hot topics for help desk last month so I’ll bump priority.

#26 Updated by intrigeri 2017-11-11 14:38:05

  • related to Bug #14959: Test UEFI boot with a Tails installed with UUI added

#27 Updated by Anonymous 2018-01-17 13:11:34

@sajolida, afaik, the target version “2018” refers to our roadmap. Therefore, it would be nice to use another target version instead, 3.x - for example.
Also, at this point it’s not clear to me what you plan to do, will you simply improve our doc or contact the UUI developers, too?

#28 Updated by Anonymous 2018-01-17 13:34:59

Oops, let me make it clearer: this ticket is indeed on our roadmap, but as it is written in #note25 that this seems to be a hot topic on helpdesk, I think it deserves a more precise target version.

#29 Updated by sajolida 2018-02-01 21:07:26

  • blocks Feature #14761: Core work 2017Q4 → 2018Q1: User experience added

#30 Updated by sajolida 2018-02-01 21:07:42

I marked this as a block on UX2018Q1.

#31 Updated by intrigeri 2018-02-03 16:17:32

  • related to Feature #11679: Rethink the installation process and upgrade process added

#32 Updated by intrigeri 2018-02-06 16:04:46

#33 Updated by sajolida 2018-02-18 12:37:19

  • QA Check deleted (Dev Needed)

I tried to reproduce this with Tails 3.5 now that Tails Installer has no splash screen. The behavior changed and now Tails Installer starts but the Install button remains unavailable. Which is not ideal but not that bad either :)

#34 Updated by intrigeri 2018-02-18 12:50:04

> The behavior changed and now Tails Installer starts but the Install button remains unavailable. Which is not ideal but not that bad either :)

I suspect that might be Bug #15254 or Bug #15233 i.e. unintended behavior that will be fixed in Tails 3.6.
In the meantime you might want to test with en_US locales (to workaround these bugs).

#35 Updated by sajolida 2018-02-19 16:05:40

I thought about that as well but I always use en_US. Well, LANG=en_US.UTF-8 of course :)

#36 Updated by sajolida 2018-02-19 16:06:17

  • Target version changed from 2018 to Tails_3.7
  • Type of work changed from Research to Test

But I’ll test again after 3.6.

#37 Updated by sajolida 2018-02-23 12:50:29

  • Target version deleted (Tails_3.7)
  • Type of work changed from Test to Wait

The last open subtask is Bug #14907 with is a “Wait” and with little hope to get done any time soon.

So I’m removing the target version. Worse case, we can close this ticket once Feature #15292 is done.

#38 Updated by emmapeel 2018-03-23 11:06:07

  • related to Bug #15453: Tails Installer 'Install' or 'Upgrade' button greyed out: Permission denied: '/lib/live/mount/medium' added

#39 Updated by emmapeel 2018-03-23 11:09:51

  • has duplicate Bug #11280: Tails Installer fails to mount certain USB sticks in Tails added

#40 Updated by sajolida 2018-03-27 17:51:39

  • blocked by deleted (Feature #14761: Core work 2017Q4 → 2018Q1: User experience)

#41 Updated by sajolida 2018-09-10 14:44:33

#42 Updated by sajolida 2018-09-10 14:45:00

  • Status changed from Confirmed to Rejected

We now have a contract for Feature #15292 that will get rid of UUI, so I’m rejecting this.