Tails

#5 Updated by intrigeri 2016-09-21 07:36:20

I won’t be able to attend the October monthly meeting, so it would be nice if a couple other people checked beforehand if some more input from me is required.

#6 Updated by sajolida 2016-09-21 11:13:58

I’m sending you in private some stats on the size of storage devices gathered from WhisperBack (over the last year). They are quite dirty because they would require rounding up the sizes and filtering out which device are actually used to start Tails to get something more relevant.

It’s the output of internal:stats/whisperback_scripts/storage_size.sh. Feel free to write something better and I’ll run it.

But for the time being, I summed up in a spreadsheet the number of occurences of something close to usual USB sizes and the results sounds plausible:

128     151     11%
64       78      6%
32      151     11%
16      352     25%
8       416     30%
4       193     14%
2        48      3%
1        13      1%
       1402 

The most popular size is 8 GB, which is, in my experience, the smallest and cheapest USB sticks you can get nowadays in western Europe. Then 16 GB for people who want a bit more space. And the 4 GB for people still using sticks from 1-2 years ago.

Other than that, how complicated would it be to create system partitions of 4 GB on sticks of 8 GB or more, and system partitions of 2.5 on sticks of 4 GB?

Or to have the size of the system partition calculated based on the size of the USB stick, so it automatically gets bigger by default as USB stick get bigger on the market?

#7 Updated by emmapeel 2016-09-21 11:30:34

sajolida wrote:
>
> Or to have the size of the system partition calculated based on the size of the USB stick, so it automatically gets bigger by default as USB stick get bigger on the market?

I like this!

#8 Updated by intrigeri 2016-09-22 02:38:11

> I’m sending you in private some stats on the size of storage devices […]

Thanks!

> Other than that, how complicated would it be to create system partitions of 4 GB on sticks of 8 GB or more, and system partitions of 2.5 on sticks of 4 GB?

It should be pretty easy (not to say I think it’s a good idea: I did not think about it yet).

> Or to have the size of the system partition calculated based on the size of the USB stick, so it automatically gets bigger by default as USB stick get bigger on the market?

Same (once one has a good algorithm).

#9 Updated by intrigeri 2016-09-22 08:14:26

> Other than that, how complicated would it be to create system partitions of 4 GB on sticks of 8 GB or more, and system partitions of 2.5 on sticks of 4 GB?
> Or to have the size of the system partition calculated based on the size of the USB stick, so it automatically gets bigger by default as USB stick get bigger on the market?

I see one problem with this approach: “Upgrade by cloning” would not be guaranteed to work anymore, since one could not e.g. upgrade a stick with a 2.5GB system partition, using one with a system partition that has more than 2.5GB used. Granted, my proposal has the same problem, at least during some transition period, if we start mixing old sticks (2.5GB system partition) with newer ones (e.g. 4GB system partition). I don’t remember us handling this in any way back when we bumped the size to 2.5GB, by the way. In all cases, this can be addressed by adding a check in Tails Installer that compares the size of the destination partition with the size of the data to copy. And in the end, the only difference between all these approaches is how common it would be for users to hit this painful situation, and for how long (transition period with a potentially long tail vs. forever).

All this reminds me of some ideas that have been suggested in the past, like letting the user choose how they want to allocate storage, between having more space for incremental updates, and having more space for persistence. Historically, I’ve been reluctant at the idea of adding one more “what should I do?” step to the installation process (even with sane defaults, users will wonder if they need to change the default settings, and this will cause confusion and more user support work). FTR I still am reluctant but I guess I could live with it; now, this requires quite some UX design work to find a good way to expose this choice to the user, based e.g. on IUK size estimates and release frequency.

#10 Updated by sajolida 2016-09-24 05:57:07

Good catch.

And you’re right, the difference is about how common the problem would
be but also maybe how easy it would be to transmit to the user. Let’s
say we have three different cases:

• All new sticks get 4GB: then trying to upgrade an old stick with a
  new one could give an error saying “*Due to changes in Tails 2.7, it’s
  impossible to clone and upgrade the USB stick TOSHIBA TransMemory (4.0
  GB). You can upgrade from ISO instead.*”.

• Sticks of 4GB get 2.5GB and all others get 8GB: then trying to
  upgrade a 4GB stick with a 8GB one could give an error saying “**The USB
  stick TOSHIBA TransMemory (4 GB) is too small to be upgraded from the
  USB stick UFD 2.0 Silicon-Power8G (8 GB). You can upgrade from ISO
  instead.”. But the problem would also apply to “Install by cloning”
  which is probably much more frequent but here we should probably not let
  people install on 4GB sticks anymore.

• **Sticks of 4GB get 2.5GB, sticks of 8GB get 4GB, sticks of 16GB get
  6GB: then it’s like in the previous case except that the problem with
  “Install by cloning” might happen for example when installing a 16GB
  stick from a 32GB stick and this should be possible.

All-in-all, this doesn’t seem crazy to explain to people.

Also, I don’t remember how things went the last time we bumped the
system partition (and thus the requirements for USB sticks) but if we
want to say that 4GB sticks are not supported anymore, then it would be
good to give some extra love to people who will have to move their data
from a 4GB stick to another one. This might actually be a reason for
people to stick with smaller sticks for long: it’s a pain to migrate all
your data and setup to a different stick. I’m not sure what this could
imply but /doc/first_steps/persistence/copy is bit hardcore…

Regarding letting the user choose, I’m confident that we could find a
way of:

• Explaining this option in a understandable way. For example as a
  thread off between how long you want to be able to do automatic upgrades
  (“during 6 months…”) and how big you want your persistent volume to be.
• Moving this option a bit out of the way and relying on a sane
  default for people who won’t bother.

I’m more concerned about the fact that this would be more work put into
Tails Installer while we still haven’t clarified its future. Still, if
someone is interested in coding this without waiting for Feature #11679, I don’t
mind helping with the UX. But if we were to improve a bit Tails
Installer as part of this transition maybe I’d find it more interesting
to work on an option to copy all the data of the persistent volume;
which would solve the transition issues I raised in my previous point
and be a very basic backup solution that could allow us to take it
easier on Feature #5301.

#13 Updated by intrigeri 2016-09-25 03:05:40

> And you’re right, the difference is about how common the problem would be but also maybe how easy it would be to transmit to the user. Let’s say we have three different cases:

Just to make it explicit: in all the following scenarios, the “it’s impossible to […]” errors would only be needed when the source system partition is already full enough of upgrades to not fit on the destination system partition, and not in the general case.

> All-in-all, this doesn’t seem crazy to explain to people.

Cool :)

> Also, I don’t remember how things went the last time we bumped the system partition (and thus the requirements for USB sticks) but if we want to say that 4GB sticks are not supported anymore, then it would be good to give some extra love to people who will have to move their data from a 4GB stick to another one. This might actually be a reason for people to stick with smaller sticks for long: it’s a pain to migrate all your data and setup to a different stick. I’m not sure what this could imply but /doc/first_steps/persistence/copy is bit hardcore…

Agreed.

> Regarding letting the user choose, I’m confident that we could find a way of:

Cool as well! This is reassuring.

> I’m more concerned about the fact that this would be more work put into Tails Installer while we still haven’t clarified its future.

I don’t really follow you on this one, so probably I’m missing something: the way I get it, the part of Tails Installer’s future that is unclear is its life on non-Tails platforms; but regardless of if/how we change the installation process on non-Tails platforms, and if/how we change the partition table layout, we should keep the “Upgrade by cloning” option available in Tails itself, no? And then, basically all the reasoning that we are doing on this ticket should remain valid for the foreseeable time, no?

> Still, if someone is interested in coding this without waiting for Feature #11679, I don’t mind helping with the UX. But if we were to improve a bit Tails Installer as part of this transition maybe I’d find it more interesting to work on an option to copy all the data of the persistent volume; which would solve the transition issues I raised in my previous point and be a very basic backup solution that could allow us to take it easier on Feature #5301.

I agree!

There’s just a part that I don’t understand: more interesting than what would it be?

#14 Updated by sajolida 2016-09-25 10:41:55

> Just to make it explicit: in all the following scenarios, the “it’s
> impossible to […]” errors would only be needed when the source
> system partition is already full enough of upgrades to not fit on the
> destination system partition, and not in the general case.

Yes, I had this in mind as well.

>> I’m more concerned about the fact that this would be more work put
>> into Tails Installer while we still haven’t clarified its future.
>
> I don’t really follow you on this one, so probably I’m missing
> something: the way I get it, the part of Tails Installer’s future
> that is unclear is its life on non-Tails platforms; but regardless of
> if/how we change the installation process on non-Tails platforms, and
> if/how we change the partition table layout, we should keep the
> “Upgrade by cloning” option available in Tails itself, no? And then,
> basically all the reasoning that we are doing on this ticket should
> remain valid for the foreseeable time, no?

I didn’t consider this.

First, if we decide to replace Tails Installer by something else on
every other platform, and if this “something else” allows for automatic
upgrades and persistence, then we might ask ourselves whether it’s worth
maintaining Tails Installer inside Tails if it’s “only” for cloning. Or
at least that’s not something I remember we decided.

Second, I thought that when we decided to pause our future plans for
Tails Installer, this implied pretty much everything that we could feel
as “our duty” to fix (to me, regardless of whether the feature is useful
inside or outside Tails because I admit that I didn’t consider that we
could keep Tails Installer inside Tails).

Third, the feature that you are proposing to add here (“being able to
choose the portion of the USB stick allocated to the system partition”)
is used at installation time (and not upgrade time). That’s why I
associated it with the future of Tails installation.

But if you were referring only to “bumping the system partitions size”
and “providing useful messages when upgrade by cloning is impossible”,
then I agree that this work could be done in Tails Installer even if its
future is uncertain (as part of its maintenance if we decide that the
current system partition size is really too small nowadays).

>> Still, if someone is interested in coding this without waiting for
>> Feature #11679, I don’t mind helping with the UX. But if we were to improve
>> a bit Tails Installer as part of this transition maybe I’d find it
>> more interesting to work on an option to copy all the data of the
>> persistent volume; which would solve the transition issues I raised
>> in my previous point and be a very basic backup solution that could
>> allow us to take it easier on Feature #5301.
>
> I agree!
>
> There’s just a part that I don’t understand: more interesting _than
> what_ would it be?

… “an option to copy all the data of the persistent volume” more
interesting than “being able to choose the portion of the USB stick
allocated to the system partition”. More interesting to me at least
because probably helpful to more people and in more use cases and easier
to design in terms of UX (and Gtk code). Now I guess that it would be
more work on the back end :)

#15 Updated by intrigeri 2016-09-25 11:10:09

> But if you were referring only to “bumping the system partitions size” and “providing useful messages when upgrade by cloning is impossible”, then I agree that this work could be done in Tails Installer even if its future is uncertain (as part of its maintenance if we decide that the current system partition size is really too small nowadays).

We can definitely agree on this! :)

(So I won’t debate the details of your argumentation even though I’m not exactly on the same page.)

(For the record I personally never proposed making the size of the system partition configurable or automatically adjusted to the device size. I was merely trying to factor in proposals and ideas I’ve heard in the wild, even if I had my doubts about them.)

>> There’s just a part that I don’t understand: more interesting than what would it be?

> … “an option to copy all the data of the persistent volume” more interesting than “being able to choose the portion of the USB stick allocated to the system partition”.

Fully agreed.

#22 Updated by intrigeri 2017-05-20 06:31:04

It’s now too late to implement this in time for 3.0, but still: it would be nice to make a decision during this time frame, as the current UX is really problematic.

#23 Updated by sajolida 2017-05-23 22:19:55

Tonight I did more stats. See [internal]stats/whisperback_scripts/usb_size.rb.

In 2016 we had 10% of 4 GB USB sticks:

    528 Not running from USB
    264 8
    262 16
    135 4
    115 32
     37 64
     18 128
      6 60
      6 124
      2 36
      2 120
      1 68
      1 536
      1 320
      1 136

In 2017 we have so far we have 8% of 4 GB USB sticks:

    136 Not running from USB
    107 16
    105 8
     87 32
     55 64
     48 4
     17 128
      7 124
      2 196
      1 60
      1 252
      1 240

#27 Updated by intrigeri 2017-06-05 13:20:27

My goal for 3.1 is limited to filing the corresponding tickets and ensure we know who’s going to do what and when.