Bug #15987

Check the system partition on every boot and grow it if needed

Added by Anonymous 2018-09-28 10:20:53 . Updated 2019-07-19 21:25:01 .

Status:
Rejected
Priority:
Normal
Assignee:
Category:
Target version:
Start date:
2018-09-28
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description


Subtasks


Related issues

Related to Tails - Feature #15319: Grow system partition during boot when started from USB Resolved 2018-10-16
Related to Tails - Feature #15293: Creating & preparing the disk image Resolved 2018-02-17
Related to Tails - Bug #16389: Some USB sticks become unbootable in legacy BIOS mode after first boot Resolved 2019-01-25

History

#1 Updated by Anonymous 2018-09-28 10:22:15

  • Target version set to Tails_3.11

#2 Updated by Anonymous 2018-09-28 10:35:52

  • blocked by Bug #15991: Code review & rubber-duck for USB Image added

#3 Updated by intrigeri 2018-10-09 09:23:21

  • blocks deleted (Bug #15991: Code review & rubber-duck for USB Image)

#4 Updated by intrigeri 2018-10-09 12:04:17

AFAICT the current, WIP code only does this on first boot, so my understanding is that this is not a duplicate of Feature #15319.

#5 Updated by segfault 2018-10-18 19:45:55

What is the reasoning for doing this on every boot? Currently, the code checks if the partitioning script has been executed before (by comparing the GUID with the default value) and exits in that case.

#6 Updated by intrigeri 2018-11-02 15:46:43

  • Assignee changed from segfault to intrigeri
  • QA Check set to Info Needed

#7 Updated by intrigeri 2018-11-02 19:01:02

  • Assignee changed from intrigeri to segfault
  • QA Check deleted (Info Needed)

segfault wrote:
> What is the reasoning for doing this on every boot?

Sadly, I don’t remember and it does not seem to be explained anywhere. I’ve checked the Git history of fundraising.git but that line was in the initial version we’ve imported from the pad. So we’re left with reverse engineering why we wrote that in the first place. I’m not sure whose job that is (probably not mine) but I fancied brainstorming it a bit and the only reasons I can easily think of are these ones:

  • The magic numbers (size) we’re hardcoding in the partitioning script will probably change some day; that’s simply how it is. Growing if needed on every boot gives us and the users who have no persistent volume an already existing migration path to the new settings.
  • Say I have installed Tails on a small USB stick and I’m not using persistence. After a while, I can’t apply an automatic upgrade anymore because my system partition is too small, so I decide to migrate to a larger stick. For some reason, instead of installing a new Tails from scratch, I Google around and copy’n’paste a dd command line from Stack Overflow which creates a binary copy of my old Tails on the new stick. My new stick will presumably boot just fine, but it won’t solve my problem at all, unless we do grow the system partition on every boot.

TBH none of those seem to be strong reasons to bother to me, which makes me wonder why we wrote this bullet point in the first place: at least 3-4 people have ACK’ed it and we must have had a good reason. Hopefully.

Note that this is not an explicit sponsor deliverable, so we’re not tight by a contract and can use our best judgement without having to explain anyone why the plan changed. If we decide it’s not worth the effort, fine, let’s skip this. I’d like to make this decision with our team-mates though, they might remember or think about a good reason to do it :) Sadly, sajolida won’t attend the next meeting so you’ll need to get his input via different means. While doing so, please ensure he’s aware of your time constraints.

#8 Updated by intrigeri 2018-11-02 19:01:18

  • related to Feature #15319: Grow system partition during boot when started from USB added

#9 Updated by segfault 2018-11-02 23:12:29

  • Assignee changed from segfault to sajolida
  • QA Check set to Info Needed

sajolida, do you remember why this was added to the pad? The deadline for implementing this is Nov 27, and we will talk about it during our next team meeting (Wednesday Nov 7).

#10 Updated by intrigeri 2018-11-03 06:34:26

> sajolida, do you remember why this was added to the pad? The deadline for implementing this is Nov 27, and we will talk about it during our next team meeting (Wednesday Nov 7).

Other time constraints: my next round of code review is scheduled on Nov 12 and the hopefully final one is scheduled on Nov 22-23, so we need to make a decision sufficiently early so that segfault has time to implement whatever it is in time, ideally before the Nov 12 review.

#11 Updated by sajolida 2018-11-05 02:35:34

  • Assignee changed from sajolida to segfault

I’m not sure either why we wrote this.

intrigeri’s first point seems relevant: it sounds interesting to automatically grow the size of the system partition whenever we decide to use a bigger one. But I understand that this can’t work for people with a persistence.

So maybe it depends on how much extra work it would be to also check for the size of the partition (in addition to checking the UUID as you do already) before deciding to enlarge it in the partition script.

From your #note-5 I understand that you are already checking the system partition on every boot and maybe that’s what we meant here. The title of this ticket could then be: “Check the system partition on every boot and grow it if needed” which is vaguely similar and looks like what you’re already doing.

#12 Updated by segfault 2018-11-07 09:27:39

  • Target version changed from Tails_3.11 to Tails_3.12

#13 Updated by intrigeri 2018-12-07 13:30:40

#14 Updated by Anonymous 2018-12-17 14:21:54

  • Subject changed from Check & grow partition size on every boot to Check the system partition on every boot and grow it if needed

#15 Updated by Anonymous 2018-12-17 14:33:21

> intrigeri’s first point seems relevant: it sounds interesting to automatically grow the size of the system partition whenever we decide to use a bigger one. But I understand that this can’t work for people with a persistence.

I agree with both your reasoning.

But if this does not work for people using persistence, the cost-benefit ratio might be quite low. Sadly, we don’t have statistics about using persistence.

> So maybe it depends on how much extra work it would be to also check for the size of the partition (in addition to checking the UUID as you do already) before deciding to enlarge it in the partition script.

> From your #note-5 I understand that you are already checking the system partition on every boot and maybe that’s what we meant here. The title of this ticket could then be: “Check the system partition on every boot and grow it if needed” which is vaguely similar and looks like what you’re already doing.

I’ve renamed the ticket.

@segfault: do you think this is something

- worth doing

- that you can do
- and within which time frame do you see yourself doing this?

As this is not a sponsor deliverable, we could re-schedule it.
And if we decide not to do that, we could reject this ticket.

What do you think?

#16 Updated by Anonymous 2018-12-27 09:46:28

ping? your input is needed, see my last comment.

#17 Updated by Anonymous 2018-12-31 15:16:48

ping?

#18 Updated by intrigeri 2019-01-02 06:30:10

#19 Updated by segfault 2019-01-03 15:53:46

Sorry for the late response when it was apparently wanted urgently - my understanding is that this is not a sponsor deliverable and not a blocker for the beta, so I didn’t handle it has high priority.

I’m really not sure if the two use cases are worth the effort, so I would prefer to wait until late in the project and see if this still fits in the budget. On the other hand, a change like this could cause bugs which we might not want to introduce after the beta and extensive testing.

#20 Updated by Anonymous 2019-01-03 16:19:37

  • QA Check deleted (Info Needed)

Agreed. Let’s reevaluate this during a meeting.

#21 Updated by intrigeri 2019-01-04 07:38:48

> Agreed.

Same. I’ll adjust the metadata accordingly.

#22 Updated by intrigeri 2019-01-04 07:39:10

  • Target version deleted (Tails_3.12)
  • Deliverable for deleted (316)

#23 Updated by Anonymous 2019-01-29 10:47:34

#24 Updated by Anonymous 2019-01-29 10:47:42

  • related to Feature #15293: Creating & preparing the disk image added

#25 Updated by adamantium 2019-02-14 09:28:01

I see adjustment of partition sizes as a potentially problematic issue. If persistence exists, perhaps a nearly full volume, then an expansion of the system partition might be impossible or at least unwanted by the user. Automatic expansion just sounds like a bad idea in every case aside from a new USB stick install from a USB .img file using Disks.

Case in point: Had 3.11 with a persistent volume, tried to auto-upgrade to 3.12, but got a message that it failed and would need to be manually upgraded to repair, as the USB stick probably wouldn’t boot and needed repair (it did require repair).

Relevant Backstory:Modified things awhile ago and did some manual backing up related to Bug #10976 recovery before I completely trusted just reconfiguring persistence with the same password. I had about 12-13GB of persistent data. New policy with tails released Juneish 2018 changed the default system partition on a new usb stick to 8GB, but my stick was only 16GB, and an 8GB system partition prevented me from restoring a 12-13GB persistent volume. I scratched my head awhile, and ended up booting from a different Tails, and using GPartEd to resize the 8GB tails system partition on my stick of interest down to about 4GB. No problem. Problem was solved, I created a persistent volume on the remainder of the stick, and migrated my old persistence volume to it. The stick has worked great ever since…. until yesterday.

During an attempted 3.11 to 3.12 automatic upgrade, it failed as described 2 paragraphs ago. I booted from another (older) tails, and tried using tails installer and the 3.12 .iso to repair the Tails that broke. It didn’t work, and acted like it would boot, but stalled on the bootloader countdown (UEFI grey background I believe). Never booted. After much research and head scratching, I eventually decided to install to a 3rd USB an intermediary Tails from the (now just released) 3.12.1 .img usb image using Disks. It completely wiped the usb, and thank you for clearly putting the warning that this would happen, as tails installer only wipes the tails system partition, not the persistence volume.

Finally I booted the intermediary 3rd USB stick (Tails 3.12.1), and using Tails installer, I cloned the 3.12.1 to the original stick that broke during the automatic upgrade. Success! Though it took me half a day to achieve it.

Though my efforts described in this message perhaps qualify me as a power user, I still am not a linux guru. My understanding is ambitious but partial. There seem to be some hiccups in the transition from the legacy .iso based upgrades to the .img USB based. I still don’t completely see how this switch will eliminate the eventual periodic need for manual upgrades. I have a genuine curiousity for how all these things work together.

Additionally, THANK YOU tails developers for your work on making the most principled OS in existence IMHO. Tails makes more sense, and is more adaptive for most people (whether they are aware of it yetor not). Built on a solid ideological foundation, Tails remains both libre & gratis, and worthy of donations.

#26 Updated by Anonymous 2019-03-06 11:52:39

  • related to Bug #16389: Some USB sticks become unbootable in legacy BIOS mode after first boot added

#27 Updated by segfault 2019-07-18 09:38:29

Do we still want to do this? We didn’t note why we wanted this and I don’t remember. Was this about the (unsupported) way to clone a Tails device by dd’ing it to another device, on which we could grow the system partition if it’s bigger than the old device?

#28 Updated by intrigeri 2019-07-18 14:32:50

> Do we still want to do this?

Earlier here, we’ve reached a consensus on “wait until late in the project and see if this still fits in the budget”.
Does it?

> We didn’t note why we wanted this and I don’t remember.

We’ve discussed it earlier here, e.g. Bug #15987#note-7. To consensus seemed to be that it would be nice to have but we were not sure about the cost/benefit, because we don’t know about how hard it is to implement.

#29 Updated by segfault 2019-07-18 15:57:52

intrigeri wrote:
> > Do we still want to do this?
>
> Earlier here, we’ve reached a consensus on “wait until late in the project and see if this still fits in the budget”.
> Does it?

I’m (slightly) over budget for the USB image project.

> > We didn’t note why we wanted this and I don’t remember.
>
> We’ve discussed it earlier here, e.g. Bug #15987#note-7. To consensus seemed to be that it would be nice to have but we were not sure about the cost/benefit, because we don’t know about how hard it is to implement.

Oh yeah, thanks, I was looking for that in the meeting logs but couldn’t find it.

#30 Updated by Anonymous 2019-07-19 12:14:04

segfault wrote:
> intrigeri wrote:
> > > Do we still want to do this?
> >
> > Earlier here, we’ve reached a consensus on “wait until late in the project and see if this still fits in the budget”.
> > Does it?
>
> I’m (slightly) over budget for the USB image project.

So let’s drop it.

> > > We didn’t note why we wanted this and I don’t remember.
> >
> > We’ve discussed it earlier here, e.g. Bug #15987#note-7. To consensus seemed to be that it would be nice to have but we were not sure about the cost/benefit, because we don’t know about how hard it is to implement.
>
> Oh yeah, thanks, I was looking for that in the meeting logs but couldn’t find it.

I think we need a better place to store decisions in the future, than storing them on Redmine, blueprints and meeting logs.

That said: I propose that we reject this ticket and we can reopen it if necessary.

#31 Updated by segfault 2019-07-19 21:25:01

  • Status changed from Confirmed to Rejected
  • Assignee deleted (segfault)

u wrote:
> segfault wrote:
> > intrigeri wrote:
> > > > We didn’t note why we wanted this and I don’t remember.
> > >
> > > We’ve discussed it earlier here, e.g. Bug #15987#note-7. To consensus seemed to be that it would be nice to have but we were not sure about the cost/benefit, because we don’t know about how hard it is to implement.
> >
> > Oh yeah, thanks, I was looking for that in the meeting logs but couldn’t find it.
>
> I think we need a better place to store decisions in the future, than storing them on Redmine, blueprints and meeting logs.
>
> That said: I propose that we reject this ticket and we can reopen it if necessary.

Agreed