Bug #17634

Etcher checksum verification mismatch

Added by numbat 2020-04-17 16:02:26 . Updated 2020-04-23 15:57:25 .

Status:
Confirmed
Priority:
Normal
Assignee:
Category:
Installation
Target version:
Start date:
Due date:
% Done:

0%

Feature Branch:
Type of work:
Research
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Multiple users have been reporting that Etcher claims that the copied Tails image doesn’t match the image file.

(:sajolida: is watching this ticket)


Files


Subtasks


History

#1 Updated by intrigeri 2020-04-19 15:35:19

  • Subject changed from Etcher checksum verification mismactch to Etcher checksum verification mismatch

#2 Updated by intrigeri 2020-04-19 15:40:10

Hi,

> Multiple users have been reporting that Etcher claims that the copied Tails image doesn’t match the image file.

Interesting!
As I understand it, Etcher’s job is to blindly copy bits as-is from the image file to the USB stick, so I have a hard time figuring out how our image file could trigger this behavior. This smells like a buggy USB stick.
Just curious: did you get any such report with Tails << 4.5~rc1 ?
I can’t think of anything else at the moment.

I’ll leave reproducing attempts to others, who have access to Windows/macOS, such as sajolida.

#3 Updated by BlueberryMuffin 2020-04-20 19:25:42

intrigeri wrote:
As I understand it, Etcher’s job is to blindly copy bits as-is from the image file to the USB stick, so I have a hard time figuring out how our image file could trigger this behavior. This smells like a buggy USB stick.
Just curious: did you get any such report with Tails << 4.5~rc1 ?
I can’t think of anything else at the moment.

#4 Updated by BlueberryMuffin 2020-04-20 19:51:57

BlueberryMuffin wrote:
> intrigeri wrote:
> As I understand it, Etcher’s job is to blindly copy bits as-is from the image file to the USB stick, so I have a hard time figuring out how our image file could trigger this behavior. This smells like a buggy USB stick.
> Just curious: did you get any such report with Tails << 4.5~rc1 ?
> I can’t think of anything else at the moment.

I am one of those who reported this bug. And no, it’s definitely not a problem with the usb-sticks, and it’s definitely not a problem with the computers. It’s simply a problem with Etcher/Windows/Linux (Tails).

I’ve tried 3 completely new, 16GB usb-sticks, from 2 different brands. I’ve also tried downloading and installing Tails multiple times on 2 different Windows 10 computers, from 2 different brands. I’ve also tried downloading Etcher directly from their own website, where I tried both the portable version and the standard version of Etcher (for Windows). I’ve also tried formatting the usb-sticks as both fat32 and NTFS before installing Tails onto them, to check if the different types of format would make any difference. But none of these things made any difference. The end-result is still the same every time. (“1 failed device” —> checksum verification mismatch.)

So as you can see, I’ve tried changing every possible variable in an attempt to discover what the problem is, but nothing has helped. And this problem was equally present with the previous version of Tails, before 4.5 .

However, I did read something online about how “Windows is unable to read Linux-software”, so that might be what’s causing it… I’m not a computer-technician, so I have no idea, but maybe Windows simply can’t read Tails after it’s been installed onto the usb-stick - because Tails is Linux. So maybe that’s why the verification of the checksum always fails after the successful installation. It would kind of make sense, because Tails itself seems to work perfectly when I try it out (after having installed it onto the usb-stick by using Etcher). It’s only the verification that fails; Tails itself seems to run just fine.

#5 Updated by BlueberryMuffin 2020-04-20 20:02:02

Just to add: I’ve tried downloading the Tails-image multiple times as well, and I verified the image successfully via the browser-extension every time. So the problem is not a corrupt Tails-image.

#6 Updated by sajolida 2020-04-21 15:55:10

Hi @BlueberryMuffin, thanks for the detailed report and all the tests!

I did some tests myself and couldn’t get Etcher to fail on my very minimal Windows 10 install. I tried:

  • Etcher 1.5.79, Tails 4.5 USB image (.img)
  • Etcher 1.5.79, Tails 4.5 ISO image (.iso)
  • Etcher 1.5.81, Tails 4.5 USB image (.img)
  • Etcher 1.5.81, Tails 4.5 ISO image (.iso)
  • Etcher 1.5.79, Tails 4.4 USB image (.img)

I also couldn’t find anything relevant in the Etcher bug tracker: https://github.com/balena-io/etcher/issues.

BUT then I found this thread in the Etcher forum: https://forums.balena.io/t/checksums-do-not-match/36537.

As far as I understand, on some Windows systems and depending on the other software that is installed, the system partition is accessed automatically in the background by some other application and Windows creates the “System Volume Information” folder in it. The verification fails because the content on the USB stick is different than than what Etcher wrote.

@BlueberryMuffin, would you be up for testing thes fix mentioned in this thread in order to disable the creation of the “System Volume Information” folder?

https://superuser.com/questions/1199823/how-to-prevent-creation-of-system-volume-information-folder-in-windows-10-for/1199824#1199824

Otherwise, maybe you could:

  • Try installing an ISO image: https://tails.boum.org/install/download-iso/. I bet that Windows would have a much harder time doing any change to its content. Using an ISO image has downsides (no automatic upgrades or Persistent Storage) but testing this could help us understand better this problem.
  • Try to open a failed installation and analyze its content. Try to open it from another Tails (maybe the ISO image?) using GNOME Disks. See screenshot in attachment. You should only see the following folders: EFI, live, syslinux, and utils. I can give you more detailed instructions if needed.

If @BlueberryMuffin confirm that this is the issue, then we could:

  • Create an issue on Etcher to implement locking the device. This is suggested in the thread but I haven’t found a GitHub issue to track this idea. GitHub is unreliable right now so I couldn’t do it already.
  • Make it harder for Windows to open and manipulate the content of the system partition. I thought that it was hidden already but it’s not enough. For example, it’s in FAT right now. What if it was in Ext4?
  • Document the superuser.com hack in our doc. If I understand correctly, users of Windows 10 Home would have to edit the registry by hand, which is probably out of reach of most of our target user base and could be dangerous.

#7 Updated by sajolida 2020-04-21 16:04:50

  • Description updated

#8 Updated by BlueberryMuffin 2020-04-21 18:30:48

Thanks for the help. But I’d rather not fool around with the “System Volume Information” and things like that, as I’m not a computer guy, so I don’t want to risk ruining something on my computer.

By the way, can I install the ISO-image on a usb-stick, or does it have to be installed on a DVD? And are there any risks involved with using the ISO-version of Tails, regarding my online-anonymity, as opposed to using the IMG-version of Tails?

And if I were to use the successfully installed IMG-version of Tails instead (which failed the verification), could that somehow be dangerous to my online-anonymity? Because Tails itself seems to run just fine, in spite of the fact that it failed the verification.

#9 Updated by sajolida 2020-04-22 21:21:25

> But I’d rather not fool around with the “System Volume Information” and things like that, as I’m not a computer guy, so I don’t want to risk ruining something on my computer.

@BlueberryMuffin: Understood!

So skip trying out this tutorial from superuser.com and don’t mess with
your Windows :)

As instructed in my previous note, we would still be very interested if
you could:

  • Test installing an ISO image.
  • Try to see if the USB stick that fails the verification has a “System
    Volume Information” in its system partition.

To do so:

1. Reinstall a fresh USB stick with an Etcher that fails verification.
2. Start from this USB stick. I understand from your last note that you
can still start from it.
3. Open the Files browser.
4. Click on “Other Locations” and “Computer” and navigate to the folder
/lib/live/mount/medium.
5. Tell us if you see anything else than: EFI, live, syslinux and utils.

> By the way, can I install the ISO-image on a usb-stick, or does it have to be installed on a DVD?

You should be able to install the ISO image on a USB stick and start
from it. At least it was possible some years ago. UEFI and Secure Boot
might not work, though.

> And are there any risks involved with using the ISO-version of Tails, regarding my online-anonymity, as opposed to using the IMG-version of Tails?

If it works, the only downside is that you won’t be able to have
automatic upgrades and a Persistent Storage. Other than that the
security properties are the same.

If you manage to start from the ISO image, you can also then installed a
second USB stick by cloning it from Tails Installer. This second USB
stick can have automatic upgrades and a Persistent Storage.

See https://tails.boum.org/install/clone

What I’m really interested in testing here is whether Etcher will behave
differently when installing an ISO image instead of an USB image. If it
works for you to install an ISO image, then it would confirm that we are
facing the same problem as these people in the forum.

Once we confirm this hypothesis, we will be able to work on solving this
problem for everybody else who faces this issue.

> And if I were to use the successfully installed IMG-version of Tails instead (which failed the verification), could that somehow be dangerous to my online-anonymity?

If we are facing the same problem as these people in the forum, I would
say that it’s safe to use it anyway.

> Because Tails itself seems to run just fine, in spite of the fact that it failed the verification.

Right.

#10 Updated by BlueberryMuffin 2020-04-23 13:30:34

I’ve started up an IMG-version of Tails which had failed the Etcher-verification. I then located the folder /lib/live/mount/medium.
Within this folder (medium), I can see the following folders appear: EFI, live, syslinux, System Volume Information, utils.

As for the ISO-version of Tails, Etcher installed it flawlessly onto a brand new USB-stick. (With this ISO-version of Tails, Etcher says “Flash Complete! 1 successful device”. But with the IMG-version of Tails, Etcher would always say “Flash Complete! 1 failed device”.)

#11 Updated by sajolida 2020-04-23 15:57:25

  • Status changed from New to Confirmed

Wow, these are excellent test results! Thanks A LOT for going these extra steps.

Both results confirm that we are facing https://forums.balena.io/t/checksums-do-not-match/36537 and that Windows has been accessing the Tails system partition in the background:

  • The “System Volume Information” is created by Windows when installing an USB image.
  • Windows cannot modify the file system of the ISO image because of how ISO images are structured.

I’ve been debugging this issue with Etcher until now on my volunteer time because I have a Windows 10 to do the tests but I don’t think it’s part of my responsibilities to work on technical solutions. So I’m going back to my questions to the Foundations Team:

  • Shall we create an issue on Etcher to implement locking the device? Whose responsibility should it be? This is suggested in the thread but I haven’t found a GitHub issue to track this idea. Worse, I don’t mind doing this on my volunteer time.
  • Could we make it harder for to open and manipulate the content of the system partition? I thought that it was hidden already but apparently it’s not enough and some Windows installations already do manipulate its content. For example, it’s in FAT right now. Could it be Ext4?