Tails

#2 Updated by intrigeri 2018-06-25 18:26:49

FWIW that’s what I do (only the persistent volume though).

#6 Updated by blue9 2018-07-27 10:28:32

I have been using a Clonezilla (https://clonezilla.org/) liveUSB to backup my Tails disk. I store the backups in a separate volume on the Clonezilla LiveUSB. Clonezilla is oldschool but seems to be well maintained and frequently updated. It works for the persistence volume alone or for a full disk image. (I’ve been using the latter technique to simplify the restoration process.) Backing up the persistence volume when it is “locked” means that I don’t have to worry quite as much about the trustworthiness of the backup utility or the strength of its crypto. (Which is not to imply that I see a problem with either; they both look fine to me.)

FYI, after backing up with Clonezilla’s GUI, it displays the ‘partclone’ commandline that it uses behind the curtain. That might be useful if someone wants to integrate a backup GUI into Tails.

Clonezilla is in Debian stable, so an alternative workflow might look something like:

- Boot without persistence but with an admin password,

- Install clonezilla with apt,

- Mount an external storage device, and
- Use clonezilla (or the corresponding ‘partclone’ commandline) to make a backup of the (locked) persistence volume.

The above would be more work to restore, unless someone builds a Tails GUI for it, but would obviate the need for users to plug their Tails USB sticks into their devices while those devices are running from a non-Tails LiveUSB.

Why Clonezilla rather than Gnome Disks?

Because at least one of the following things are true:

1. I do not understand how LUKS works and have observed a supernatural phenomenon; and/or
2. Encrypted LUKS volumes expose information about used vs unused space, and ‘partclone’ is smart enough to backup only the used space.

In other words (assuming the latter is true), Clonezilla can make “sparse” disk image backups of encrypted volumes. Say, for example, I have a 64GB USB stick with an 8GB Tails volume and a 56GB persistence volume. And suppose that persistence volume contains 10 GB of content. With Clonezilla, a full-disk backup image requires at most 18 GB, whereas (in my limited experience) ‘dd’ and Gnome Disks backups require 64 GB. (Maybe Gnome Disks can do this, as well, and I just don’t know how? Assuming of course it is theoretically possible and I’m not just confused.) Because I am backing up my images to a second volume on my (256 GB) Clonezilla LiveUSB, this feature is quite valuable for me.

FWIW, a colleague and I have both tested this, and the restored images appear to work fine.

How I created my Clonezilla LiveUSB on Tails

Warning: requires the installation of ’libc6-i386’

1. Install dependencies: 
```
sudo apt-get install libc6-i386
```
2. [Download the .zip archive](https://clonezilla.org/downloads/download.php?branch=stable)
    - File type: zip
    - Repository: Sourceforge
    - Click *Problems downloading?*
    - Click the *direct link*
3. [Setup the LiveUSB](https://clonezilla.org/liveusb.php) on a 256GB USB stick
    - Use the *Disks* application to create two partitions:
        1. *clonezilla*: A 250MB FAT partition for Clonezilla (the instructions say 200MB, but that is too small)
        2. *backups*: Use the remaining space for a second (EXT4) partition to hold backups
    - Mount the *clonezilla* partition using *Disks*
    - Extract the `.zip` and copy contents to the root of the *clonezilla* partition. 
        - *Extract to* in the file browser would create an extra folder
        - The archive includes a hidden `.disk` folder that must be included
    - Use *Disks* to find the address of the *clonezilla* partition. (We use `/dev/<partition1>` below.)
    - Make it bootable:
    ```
    cd /media/amnesia/clonezilla/utils/linux
    sudo bash makeboot.sh /dev/<partition1>
    ```
    - Answer `y` to all prompts
    - Unmount the *clonezilla* volume
    - Remove the USB stick

#9 Updated by blue9 2018-07-30 10:03:19

Thanks Sajolida!

> I found the interface and interactions of Clonezilla completely obscure. It took me ~30 min

Fair point. The ncurses interface is quite awful. It doesn’t bother me as a special-purpose LiveOS, but it’s a bit shocking when you launch it as an application. I apologise. You will never get those 30 minutes of your life back, and I blame myself. :)

For a quick-and-easy graphical walkthrough (that doesn’t require developing your own “Tails Backup” GUI), Gnome Disks is probably the right answer. But I still think the commandline has potential, so…

> I tested backing up a 3.5 GB persistence.

How much data was in it? And did you see any difference in the size of the resulting backup? The difference is obviously more prominent with larger (and mostly empty) persistence volumes. Just curious if you saw any difference at all.

> Do you mind having a closer look at the partclone backend used in your Clonezilla command?

I will do this and send in the exact commandline later on today.

> Is it using something else than partclone.dd and doing smart things to avoid copying the
> whole partition in the first place? Or is the resulting image only smaller because of
> compression?

I do not think this is just compression. Or, if it is, then it’s compression that takes advantage of Clonezilla’s ability to “do…smart things.” In other words, pigz might be responsible for truncating a bunch of zeros, but partclone is responsible for giving zeros to pigz rather than giving it a bunch of random-looking, uncompressable, encrypted zeros.

I say this because:

1. Encrypted data cannot be compressed; and
2. The difference is not just a few GB (a lightly used 64GB Tails disk will end up being, like, 4 GB; with Gnome Disks, I assume it will be 64 GB).

After a year of using Tails to store data sets, some large git repos, lots of screenshots, a few .ISOs and other assorted files, my 50GB persistence folder is filling up quickly. This means my backups are getting bigger, but it was nice, in the early days, that I could store many small, sub-10GB backups rather than filling up my entire backup USB with a single image.

#11 Updated by segfault 2018-07-30 12:44:46

I just want to make sure that you are aware that we have a documentation draft for manually backing up persistent data (it waits for review since 6 months, see Feature #12214): https://labs.riseup.net/code/attachments/2096/Backup%20Documentation.png

It does not document creating a disk image, but copying the persistent data to a LUKS encrypted device, since we already have documentation for creating and using LUKS encrypted devices: https://tails.boum.org/doc/encryption_and_privacy/encrypted_volumes/index.en.html

#13 Updated by sajolida 2018-08-03 13:49:27

> You will never get those 30 minutes of your life back, and I blame myself. :)

He he, don’t worry. I also learn from bad designs :)

>> I tested backing up a 3.5 GB persistence.
>
> How much data was in it?

2.7 GB.

> And did you see any difference in the size of the resulting backup?

Seeing that dd and the partclone.dd command under Clonezilla produce
exactly the same binary (same checksum), I conclude that the content of
the persistence doesn’t impact the size of the uncompressed disk image.

Clonezilla does some compression and I guess that’s where you gain in
size on partition that are not full. See below.

>> Do you mind having a closer look at the partclone backend used in your Clonezilla command?
>
> I will do this and send in the exact commandline later on today.

Ok.

> I do not think this is just compression. Or, if it is, then it’s compression that takes advantage of Clonezilla’s ability to “do…smart things.” In other words, pigz might be responsible for truncating a bunch of zeros, but partclone is responsible for giving zeros to pigz rather than giving it a bunch of random-looking, uncompressable, encrypted zeros.

Remember that I go the exact same binary from dd and partclone.dd,
so least my partclone is not doing smart things. But maybe your
partclone command is different.

> I say this because:
>
> # Encrypted data cannot be compressed; and
> # The difference is not just a few GB (a lightly used 64GB Tails disk will end up being, like, 4 GB; with Gnome Disks, I assume it will be 64 GB).

My guess is that the difference comes from zeroes that remain at the end
of the partition on the space that has never been used yet. I also guess
that once all the space has been used and freeded, it doesn’t have
zeroes anymore and won’t compress well.

> After a year of using Tails to store data sets, some large git repos, lots of screenshots, a few .ISOs and other assorted files, my 50GB persistence folder is filling up quickly. This means my backups are getting bigger, but it was nice, in the early days, that I could store many small, sub-10GB backups rather than filling up my entire backup USB with a single image.

I can definitely understand that!

I guess that would be the challenge for “Tails Backup” GUI: do backups
and recovery faster, save storage space, allow inspection of increments,
etc. But I don’t think we can get that with disk images.

#14 Updated by sajolida 2018-08-03 13:52:14

> I just want to make sure that you are aware that we have a documentation draft for manually backing up persistent data (it waits for review since 4 months, see Feature #12214): https://labs.riseup.net/code/attachments/2096/Backup%20Documentation.png

I’m aware for that. Sorry for the huge delay. The Technical Writing team
hasn’t been super productive since April but it’s on our radar.

#15 Updated by sajolida 2018-08-03 15:56:00

> But I don’t think we have any guarantee that all system partitions our tools/doc have ever created satisfy this so there’s probably Tails sticks with some free space between the 2 partitions in the wild.

Ok.

> […] The good news is that it means we’re creating a slightly smaller persistent volume than we could, which increases the odds that it can be restored on another USB stick that in theory has the same size but can very well be a bit smaller.

This matches what I’ve experienced.

> Note that I don’t understand “If we didn’t had this space people could create an empty partition instead of having to create a dummy partition so that it ends up at the right place”. What’s “the right place”? As long as the partition is created with udisks2 (e.g. GNOME Disks) — which will align stuff correctly I think — and is large enough so that the user can dump their backup on it, what’s the problem?

D’oh! I don’t know why I thought that not having the restored partition at the same exact place than the original one would be a problem. Since the user will first create a dummy partition the partition table will know where to look for it. Thanks for enlightening me!

I think that now the only thing stopping us from documenting this is to check whether the Clonezilla of blue9 is also using partclone.dd or something else that does smarter things.

So I’m reassigning to him.

#17 Updated by blue9 2018-08-04 00:13:19

>> Do you mind having a closer look at the partclone backend used in your Clonezilla command?
> I will do this and send in the exact commandline later on today.

Sorry for the delay. Travel… Here is the command that Clonezilla stitches together (and displays while waiting for confirmation to start the backup):

/usr/sbin/ocs-sr -q2 -c -j2 -z1p -i 4096 -sfsck -enc -p choose savedisk 2018-08-03-img-t-3.9-testing sdc

Breaking that down…

• /usr/sbin/ocs-sr, Commandline tool installed along with Clonezilla (seems to rely on partclone, partimage, dd and perhaps other utilities)
• -q2, Use partclone to save partition(s) (i.e. partclone > partimage > dd).
• -c, Wait for confirmation before saving or restoring.
• -j2, Use dd to clone the image of the data between MBR (1st sector, i.e. 512 bytes) and 1st partition, which might be useful for some recovery tool.
• -z1p, Compress using parallel gzip program (pigz) when saving: fast and small image file, good for multi-core or multi-CPU machine.
• -i 4096, Set the size in MB to split the partition image file into multiple volumes files.
• -sfsck, Presumably: do not run fsck before backing up the disk.
• -enc, Encrypt the image.
• -p choose, When save/restoration finishs, choose action in the client, poweroff, reboot (default), in command prompt or run CMD
• savedisk, Backup the full disk rather than (a) specific partition(s).
• 2018-08-03-img-t-3.9-testing, Destination file.
• sdc, Source disk.

(The ocs-sr manpage is broken, but you can see all options for saving and restoring with sudo /usr/sbin/ocs-sr —help.)

If I understand correcty, the command you tested was:

>> * partclone.dd -z 10485760 -N -L /var/log/partclone.log s /dev/sdc2 —output | pigz -c —fast -b 1024 -p 16 —rsyncable | split -a 2 b 4096MB /home/partimag/2018-07-29-12-img/sdc2.dd-img. 2> /tmp/split_error.rc2Gz1

Did you extract this commandline from Clonezilla? (I thought it displayed the underlying partclone command, but now I’m only seeing the ocs-sr line above.) partclone.dd is Clonezilla’s fallback option if it can’t figure out what filesystem to use:

>>> …you can clone GNU/Linux, MS windows, Intel-based Mac OS, FreeBSD, NetBSD, OpenBSD, Minix, VMWare ESX and Chrome OS/Chromium OS, no matter it’s 32-bit (x86) or 64-bit (x86-64) OS. For these file systems, only used blocks in partition are saved and restored by Partclone. For unsupported file system, sector-to-sector copy is done by dd in Clonezilla.

I thought perhaps it was able to use something smarter for LUKS volumes, but if you got that commandline from Clonezilla, then I guess not. Maybe the difference was the fact that you only had 800MB free, so no empty “chunks” were sent to pigz?

>> My guess is that the difference comes from zeroes that remain at the end of the partition on the space that has never been used yet. I also guess that once all the space has been used and freeded, it doesn’t have zeroes anymore and won’t compress well.

Seems likely. I’ve been meaning to test a backup of a restored image. Given that restoring an empty persistence volume seems to take just as long as restoring a full one, I’m guessing it fills in the (formerly) unused space on restore. Which would make subsequent backups full-sized.

In other words, this is kind of a best-case optimisation. But, like I say, it served me well for a year, so it may be a relatively common best-case…