Make it possible to verify the integrity of a Tails USB device
It would be useful to be able to verify the integrity of the system created by Tails Installer.
This is not straightforward because it implies verifying not only the content of the Tails system partition but also the MBR and partition tables.
team: segfault, anonym+intrigeri, jvoisin, sycamoreone
Related to Tails -
Related to Tails -
Related to Tails -
Related to Tails -
Related to Tails -
Blocked by Tails -
Blocked by Tails -
#2 Updated by alant 2014-07-06 15:46:51
From what system do we want to verify the installed device? If it is Tails, then why not to clone and upgrade? So want to verify it from another OS (typically Windows).
There are then several set of data that should be checked:
- most of them can be generated during build and should be retrived securely somehow
- there a syslinux autogenerated file in the filesystem (
ldlinux.sys) that is not possible to compare to a trusted source
#3 Updated by sajolida 2014-07-07 11:01:22
>>From what system do we want to verify the installed device? If it is Tails, then why not to clone and upgrade? So want to verify it from another OS (typically Windows).
Other usecases would be:
Verifying that you are using a genuine Tails key:
- You start using Tails by getting a USB key from an organization or
- After some time using it, liking it, and getting to know other
Tails users you want to make sure it was genuine.
Checking the where people distributing Tails keys are trustworthy. The
same technique could be used to verify the keys sold or made by other
organization and checking whether they are trustworthy.
#5 Updated by ioerror 2014-12-08 17:17:20
I’ve created a start to this for a Debian system - this will hopefully, eventually, run on Tails too:
I need help though - I need a collection of every GnuPG signature for every release, as well as a copy of every ISO or every Torrent file.
Could someone please share these with me?
#7 Updated by intrigeri 2014-12-11 17:23:49
> I need help though - I need a collection of every GnuPG signature for every release, as well as a copy of every ISO or every Torrent file.
> Could someone please share these with me?
For the ISO and detached signatures, it’s WIP:
Feature #8389. For the torrent files, you can find them in Git history.
I’m curious how you’re dealing with
ldlinux.sys, but no time to check the code right now.
#8 Updated by intrigeri 2014-12-19 18:04:30
- Status changed from Confirmed to In Progress
- Assignee set to ioerror
> ioerror wrote:
> > I need help though - I need a collection of every GnuPG signature for every release, as well as a copy of every ISO or every Torrent file.
> > Could someone please share these with me?
> For the ISO and detached signatures, it’s WIP:
Done, sent credentials in a private email. Enjoy!
#14 Updated by jelhan 2015-09-23 17:24:34
As integrity mentioned verifying device integrity seems to be easy since a hybrid ISO image is skipped. I got correct sha256 of a Tails 1.6 which was manually installed on an USB stick.
$ sudo dd if=tails-i386-1.6.iso of=/dev/sdc && sync
$ stat tails-i386-1.6.iso
Size: 987033600 Blocks: 1927824 IO Block: 4096 regular file
$ sudo dd if=/dev/sdc bs=1 count=987033600 | sha256sum
$ sha256sum tails-i386-1.6.iso
Just reading in the device with a block size of 1 bit really takes it time. But I think there should be a better solution than
#16 Updated by anonym 2015-11-03 10:01:32
I’m wondering if we can rethink incremental upgrades and leverage Grub2’s ISOBoot functionality to get this and a lot of other very nice things. Please bear with me and imagine something like this:
- We kill the IUK concept. We may still be able to support incremental upgrades (see below) but for the time being, think that full upgrades are the only way to upgrade Tails. Note that we definitely sill will support automatic upgrades (full upgrades,
- The partition table is made very simple to ease verification. The only variable should be the size (and perhaps some uuid?) of the TailsData partition, and/or whether these is one at all; everything else on the partition table level is static and identical for all Tails installations. I’m no expert at these things, but this is possible, right? Any way, we publish some description of this + sign it.
- We have a tiny
$BOOT_PARTITIONfor Grub2. We generate a blob that has it installed at build time + sign it, and ship it together with the Installer. It can be installed and extracted (for verification) easily with
ddor whatever. Grub2 is configured to boot some specific ISO from…
- … another partition,
$SYSTEM_PARTITION, storing ISOs (note: plural; see below).
- On the
$SYSTEM_PARTITIONwe store everything needed to verify all pieces:
- the description of the partition table + signature
- the signature of the
- the signatures of all ISOs in
Given that, each installation includes signatures, descriptions and hashes so all bits of the installation can be verified by an external tool with access to our signing key (and, I guess, the blob for the
$BOOT_PARTITION; we could publish these so the verification tool can download them automatically).
Bonus: now we can support upgrading a USB/SD device while it is running; we just need the ISO some how. Alternatives:
- The Upgrader could download the ISO for us (
Feature #7499). It would also download the signature (or UDF) for us and verify it. So we’d still have automatic upgrades, like promised above.
- Users could manually get hold of the ISO (like our Installer currently supports). I suppose we could offer to download the signature (or UDF) and verify it, like above.
- The ISO can be extracted from a DVD with Tails burned to it. We just need to know its exact length which we can encode in the ISO metadata, or possibly even check in the UDF, which we can download. We can also offer to verify it like above, or using the UDF.
- The ISO copied from another USB installation. We would also extract the signatures, descriptors etc so it’s verified.
Now we just install the new ISO (and/or IUKs) into
$SYSTEM_PARTITION, leaving the old ISO (and/or IUKs) there, since it’s mounted and in use. But only for the time being, cause we add a hook that deletes old ISOs (and old IUKs, signatures etc etc) during boot to recover the space.
So far we’ve only talked about the scenario where we boot from a simple “ISO”, whether it’s burned to a DVD, or loop-mounted (by Grub2). There are no incremental upgrades. This is a good thing:
- Code and design complexity would go down in several areas (in particular, no aufs/overlaysfs stacking is involved)
- Easier and faster release process since there are no IUKs
- It’s just easier and more intuitive to reason about Tails’ filesystem
So what about incremental upgrades? Well, we could get them, and also support them endlessly (
Feature #8534) if we manage to figure out how to use binary diffs. We’d probably want a format that is optimized for ISOs to make them more compact, so our experience with the IUK format will probably help. Note that if we want to keep the “always boot from ISO” concept, then this is the only way we can do incremental upgrades.
The big problem with binary diffs of large files (ISOs) are the space requirements when applying them (we have access to monster machines to generate them, of course). That may make the above approach effectively impossible unless we can use a lot of space temporarily, either in RAM, in
$SYSTEM_PARTITION or on TailsData (if none is used, we could temporarily create a partition there and use the space). Making
$SYSTEM_PARTITION large just for this is sad, since it eats from TailsData’s space. This may be hard, and introduce a lot of “ifs”, complexity and potential user confusion.
Actually, since we have
Feature #7499, the only advantage of incremental upgrades is that it requires downloading less when upgrading. Most of the time IUKs are ~250 MiB, and ISOs are ~1 GiB, so there’s a 300% increase. Also note that we probably can optimize our IUKs more, especially after we have deterministic builds, so the loss of efficiency of doing this might be even larger. Still, this simplification (only support full upgrades) might be worth considering.
To end with, I know I have been digressing far and wide, but I think it’d be nice if we designed something that will solve Feature #7496 and
Feature #8534 at the same time, or if we cannot have Feature #8534, that it solves Feature #7499. So there is a broader discussion for us to have, which I hope that I now have started a bit.
#18 Updated by sajolida 2015-11-04 07:49:30
Thanks for brainstorming on this. I find it useful when people have “crazy” ideas that look a bit further than the next release sometimes :)
I’m not against the idea of killing IUKs if the cost/benefit is good enough. Right now the benefits you’re outlining are Feature #7496 (this ticket), a bit less RM work, a simplier file system (cf. recent issues while upgrading, etc) and I’m not sure they are worth it at the moment. But that’s not a good reason to not investigate your idea a bit further!
A few more related comments:
- Downloading through Tor is on average not that much slower than outside of Tor. Let’s say less than twice slower in most cases (see
- I’d love to have stats about what upgrade paths our users follow. For example, we’re only providing IUKs from the last version. This assumes that our users are all regular users that update every 6 weeks. Confronting this with real data might change our vision a lot. Maybe the answer would be to ship IUK for n-1 and n-2, or to go for your solution. I’ll try to get such stats, see
#19 Updated by segfault 2015-12-07 19:35:09
- File <del>missing: tails-verifier.tar.gz</del> added
I wrote a script to verify a Tails device using a Tails ISO.
I’m using the hashrat package from Debian testing (https://packages.debian.org/stretch/hashrat) to create hashes of all the files in the ISO and compare them to the files on the device.
I tested it with tails-i386-1.5.1.iso and tails-i386-1.7.iso. I used the Tails installer’s “Upgrade from ISO” method to install the ISO. It verifies correctly after excluding “squashfs/var/lib/urandom/random-seed” and replacing “boot/isolinux” with “boot/syslinux” in the hashfile.
I also tested if it still verifies correctly after booting the device and it does.
It is currently not checked if there are any new files on the device, which are not in the ISO.
Please take a look at the code and tell me what you think about it.
Should I put this in a fork of Tails or somewhere else?
#21 Updated by sajolida 2015-12-08 04:32:06
ioerror: are you volunteering to test and review this script or shall we look for somebody else? If so, shall we set the target version to something sooner than “2017”. See https://tails.boum.org/contribute/calendar/ for time estimates of the upcoming versions.
#22 Updated by segfault 2015-12-08 07:45:22
- File <del>missing: tails-verifier.tar.gz</del> added
I fixed two minor issues (printing a warning if hashrat is not found and counting ‘failures’ in the hashrat output as well as ‘errors’. The second could have resulted in the output “Successfully verified” while the verification actually failed).
I also took a look at the code of hashrat to inspect what would have to be done to check for additional files. I found it very bloated with 22000 lines of C code. It doesn’t seem to have many users, the only issue on github was created by me. And it is not very well maintained, the very simple issue is still unfixed after almost 3 months.
So I think about writing a python script to replace hashrat. But I know you prefer including Packages from upstream rather than writing code for Tails.
What do you think about this?
#23 Updated by segfault 2015-12-08 13:19:43
- File tails-verifier.tar.gz added
I added functionality to check for additional files on the device and to check the MBR.
Now the only remaining problem I see is verifying ldlinux.sys and verifying the VBR. The ldlinux.sys files on my devices differ only in a few bytes, but I was not (yet) able to find out why they differ.
#24 Updated by Anonymous 2015-12-31 03:57:10
- Assignee changed from ioerror to segfault
Assigning this ticket to segfault who has been working on it and has a working PoC.
Segfault, you can ask on tails-dev for reviewing this as is or redo it in Python as you planned before of you wish to do so.
#25 Updated by segfault 2016-01-31 22:24:39
- % Done changed from 0 to 80
- Type of work changed from Research to Security Audit
I put my work on the Tails Verifier on gitlab:
I realized half way through that this project was getting far too complex and it would be better to focus on changing the Tails installation and upgrade process in a way that makes verification easier (which I intend to work on). But I finished it anyway and it’s working for me.
I set the type of work to security audit, although currently I don’t think we should ship this, because it will be much easier, safer and more stable if we manage to simplify the installation and upgrade process.
#26 Updated by sajolida 2016-02-01 13:15:43
- Assignee deleted (
- Target version changed from 2017 to Tails_2.2
Congrats! Since you’re saying that your work is finished I’m marking this as “QA Check: Ready for QA”. I’m also unassigning it from you since someone else should do the review. I’ll also set it for 2.2, just to make sure we identify someone to review this during the next monthly meeting. But this doesn’t mean this will be in 2.2 :)
#32 Updated by anonym 2016-02-10 22:51:42
- % Done changed from 80 to 50
> I put my work on the Tails Verifier on gitlab:
Impressive! This looks like the largest, by far, code contribution made to Tails! :) My initial look is very positive (code looks clean and nicely modularized). Yay! This makes me excited, but also sad since I won’t have time to look at it until after Tails 2.2 is out. Don’t hesitate to ping me in mid-March if you haven’t seen any activity from me on this ticket.
> I realized half way through that this project was getting far too complex and it would be better to focus on changing the Tails installation and upgrade process in a way that makes verification easier (which I intend to work on).
I’m very much interested in what your thoughts are on this, and if you have any opinions of my ideas in Feature #7496#note-16 above. Perhaps you have some wild ideas of your own?
> But I finished it anyway and it’s working for me.
> I set the type of work to security audit, although currently I don’t think we should ship this, because it will be much easier, safer and more stable if we manage to simplify the installation and upgrade process.
Still, it seems quite a bit of the complexity stems from the vbr and ldlinux verification, which we might remain relevant even with a new installation paradigm, and the same applies to some other parts of the code.
[Also, I’ve set the progress to 50% to leave some room for back-and-forth:s later.]
#34 Updated by segfault 2016-02-10 23:11:17
> Impressive! This looks like the largest, by far, code contribution made to Tails! :) My initial look is very positive (code looks clean and nicely modularized). Yay!
> I’m very much interested in what your thoughts are on this, and if you have any opinions of my ideas in Feature #7496#note-16 above. Perhaps you have some wild ideas of your own?
I have opinions on your ideas and some ideas of my own, but nothing perfect yet. I think I will create a blueprint about this, where we can balance pros and cons of the ideas.
> Still, it seems quite a bit of the complexity stems from the vbr and ldlinux verification
> which we might remain relevant even with a new installation paradigm
I think that these parts could actually become irrelevant if we would use only grub as the bootloader instead of syslinux (which could be done easier and faster than reworking the whole installation paradigm).
#35 Updated by intrigeri 2016-02-11 01:33:38
> I think that these parts could actually become irrelevant if we would use only grub as the bootloader instead of syslinux (which could be done easier and faster than reworking the whole installation paradigm).
… and is something I’ve been forcing myself to ignore quite a few times already, but I must say I’m very much tempted! (for USB installations, I mean; on DVD I see no reason to move away from isolinux)
#36 Updated by anonym 2016-02-16 14:50:47
I’ve opened a ticket about “Endless automatic upgrades”,
Feature #11131, and a blueprint (which can be edited by anyone using the web interface) that includes installation verification as part of the things to consider. I guess this will be a better place to continue the discussion.