Tails

#7 Updated by anonym 2016-12-13 12:37:53

anonym wrote:
> [Right before I was gonna post this I realized that my data isn’t perfect since 2.0 was released 6+ months into the Jessie cycle, so probably the number will be higher than 20. But we could always have a release in the middle of the Debian cycle were we reset what is considered as the first release. Or we just decide to support IUK upgrades only for the 20 last releases, which may mean that those that installed Tails early in a series might have to do a full installation closer to the end of it — that’s still a lot better than what we have now.]

Another strategy would be to “synchronize” each n (e.g. ~10) releases. Example: let’s say 2.0 is the “first” release. Let’s say the next n releases puts us at 2.5 — at that time we’ll have IUKs 2.0_to_${i} for each of them. From then on, the next n releases will provide IUKs from 2.5, i.e. 2.5_to_${i}. And so on. So, to support users installing an early 2.x, like 2.0, all the way until 2.x’s EOL, which is the worst case vs system partition disk needs, we would need to stack a few IUKs, i.e. floor(N/10) + 1 where N is the total number of releases during 2.x.

So, this way we’d never need to generate more than n IUKs per release. And I think we can assume that the average IUK size will go down quite a bit. On the mirrors we’ll have a choice: when we release a new version that is the new IUK synchronization point we would keep all IUKs upgrading to the new release for a while. The choice I’m talking about is in how long we keep them. Either we keep them until the EOL, so we always support even the oldest version (but this will require a lot of space on the mirrors); or we optimize for users that upgrade semi-regularly, so we kill all those old IUKs after a few releases. At the very least I think we can kill IUKs for synchronization point number i when we release synchronization point number i + 2. That way the mirrors will only ever store up to 2 * n IUKs.

So, yeah, we have a lot of parameters to play with. :)

#8 Updated by anonym 2016-12-13 12:43:21

anonym wrote:
> Another strategy would be to “synchronize” each n (e.g. ~10) releases.

Or when the IUK grows too big. The point is the concept of the synchronization points.

> So, yeah, we have a lot of parameters to play with. :)

Indeed!

#9 Updated by intrigeri 2016-12-27 08:50:00

> While working on Bug #11857 it occurred to me that we can provide more or less endless upgrades until Tails is rebased on a new version of Debian (for IUK size reasons) simply by generating IUKs for each release back to the first one, and extending Tails Upgrader to be able to replace IUKs so that we never stack IUKs again (so we should probably drop the “I” in “IUK” :)), but always install one that upgrades to the current version.

Sound like a good idea, that’s worth thinking about!

> Example: let’s say I once installed Tails 2.2 on a USB stick, and at some point updated it to 2.3 with the 2.2_to_2.3 IUK. Let’s say I am thrown in prison for 6 months and when I get out the current version is 2.7.1. Then Tails Upgrader will download the 2.2_to_2.7.1 IUK and store it in addition to 2.2_to_2.3, and update Tails.modules to only list 2.2_to_2.7.1. The next time we want to upgrade (= next time we will remount the drive in read-write mode), the 2.2_to_2.3 IUK is cleaned up to make space for the new IUK (therefore the Upgrader can discount the space occupied by old IUKs when doing the disk space check).

Tails Upgrader could actually delete (unlink) the 2.2_to_2.3 IUK while installing the 2.2_to_2.7.1 one: it’s in use so disk space won’t be freed (and the system will keep working), but it might make it easier to decide which IUK can be deleted (i.e. exactly the ones we’re going to remove from Tails.modules).

> Unexpectedly they grow significantly throught the Debian release cycle, but I have a hunch we can expect 2.0_to_XX size to not increase much beyond the current value — after all, it generally is the same suspect packages that are upgraded throughout a Debian release.

I suspect that a substantial part of the growth is caused by our own changes (e.g. adding packages, moving stuff around) rather than by package upgrades, but it’s not important at this point.

> The IUK size is involved in at least three concerns:

[…]

> * Disk space needs on the Tails system partition: we need to be able to store two large IUKs. The worst case would be when e.g. 2.0 was the the first ISO we installed from, and that we have upgraded to 3.0-2 and want to upgrade to 3.0-1, because those two IUKs are reasonably those that will occupy the most space. Tails 2.0 uses about 1.1 GiB of the 2.5 GiB system partition. That means we’d be ok if those two IUKs are about (2.5 - 1.1)/2 = 0.7 GiB. That seems quite plausible for the 2.x series, so we could try it. And for 3.0 we can increase the system partitions size so we have a better marigin.

Tails Upgrader needs more free space than that: it requires 3 times the size of the IUK it’s going to install to be free on the system partition before it can download and install it. This might be fixable given enough time+energy, but let’s start by reasoning about the cheapest option, that requires the smallest amount of changes to Tails Upgrader ⇒ time to redo the maths.

#15 Updated by intrigeri 2018-02-03 15:49:10

Added to the blueprint my research about non-NIH endless automatic upgrades solutions. They have plenty of advantages over the one proposed on this ticket but are not fully ready yet and will require lots of work on our side => I’ll come back to it on Feature #15277. Meanwhile, let’s see if the solution proposed on this ticket can be implemented quickly and cheaply: if it can, it could be worth it; if it can’t, then perhaps we should skip this temporary solution and instead focus our efforts directly on the nicer, long-term one.

#16 Updated by anonym 2018-02-03 16:22:47

intrigeri wrote:
> >> Another strategy would be to “synchronize” each n (e.g. ~10) releases.
>
> > Or when the IUK grows too big. The point is the concept of the synchronization points.
>
> I like it. I’d like to see the maths wrt. disk space and memory requirements, computed on the actual 2.x cycle until now.

So in the end there was 14 releases for 2.x. I make IUKs from 2.0 to the last few releases:

678M Tails_i386_2.0_to_2.10.iuk
682M Tails_i386_2.0_to_2.11.iuk
634M Tails_i386_2.0_to_2.12.iuk

So, actually the last IUK is the smallest, and the most space consuming upgrade will be 2.10 to 2.11. So:

• The 2.0 installation itself uses 1082M, so upgraded to 2.10 the total usage is 1082M + 678M = 1760M.
• Due to the Upgrader requiring 3x the size of the IUK on the Tails system partition, the actual requirement is 1760M + 3*682M = 3806M of free disk space. That wouldn’t have worked during the 2.x times, when the system partition was 2.5 GB, and it barely would work with our current 4.0 GB partition.
• When both IUKs are present at the same time (i.e. before rebooting after having applied the upgrade) the disk usage is 1760M + 682M = 2442M.
• After the old IUK is purged the disk usage is down to 2442M - 678M = 1764M.

> Also, Tails Upgrader requires twice the size of the to-be-installed IUK as free memory before it downloads it. It would be good to take this into account and do the maths for that too.

So 2*682M = 1364M of RAM.

#17 Updated by anonym 2018-02-03 17:00:16

intrigeri also pointed out that we probably do not want to support upgrading very old Tails installations because our signing key probably will have been updated in the meantime, and/or some critical update to Tails Upgrader (e.g. UDF version). For instance, in the example in Feature #11131#note-6, our Tails 2.2 (with no upgrades, so it was manually installed as 2.2) has a too old key (commit:098f578586f1e49b6cc63431f661b9c20151916c) so it would not be able to verify the UDF letting us know about the existence of Tails 2.7.1 and its IUK. So while we provide a 2.2_to_2.7.1 IUK, it can actually only be used by those that update regularly, e.g. someone that installed Tails 2.2, and installed each automatic upgrade.

Conclusion: let’s stick with out current “you can at most skip one planned release” so it in effect means “you must upgrade once every three months”.

#19 Updated by intrigeri 2018-02-03 17:04:10

We have to implement a part of this idea in order to migrate from aufs to overlayfs; see Feature #8415#note-35 for details.

#20 Updated by intrigeri 2018-02-03 17:11:26

anonym wrote:
> Conclusion: let’s stick with out current “you can at most skip one planned release” so it in effect means “you must upgrade once every three months”.

I think we could actually do a little bit better without making the cost noticeably higher: we could support upgrading for any older releases as long as it has the current signing key and a version of the Upgrader that is compatible (i.e. there’s been no major change or bugfix in it since).

#31 Updated by intrigeri 2018-02-06 15:32:47

Deprecated for now in favour of Feature #15281. We’ll reconsider after Feature #15277.