Bug #16003

Write automated tests for growing system partition

Added by Anonymous 2018-09-28 10:49:38 . Updated 2019-01-09 18:29:07 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
2018-09-28
Due date:
% Done:

100%

Feature Branch:
test/16003-growing-system-partition
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:
316

Description

The time spent on this is to be equally shared between Cyril and intrigeri (mentoring).

The new code that requires additional testing is https://git.tails.boum.org/tails/tree/config/chroot_local-includes/usr/share/initramfs-tools/scripts/init-premount/partitioning?h=feature/15292-repartition

New tests that might be worth writing:

  • the FAT filesystem on the system partition is at least 4096M large
  • the label of the FAT filesystem on the system partition is “Tails”
  • the system partition is an ESP
  • the FS UUID for the system partition is not a69020d2 (i.e. /dev/disk/by-uuid/A690-20D2 does not exist)
  • the system partition has the expected GPT attributes/flags (regression test for Bug #16178)

Expected GPT attributes:

SYSTEM_PARTITION_FLAGS = (
    1 << 0 |   # system partition
    1 << 2 |   # legacy BIOS bootable
    1 << 60 |  # read-only
    1 << 62 |  # hidden
    1 << 63    # do not automount
)

Subtasks


Related issues

Blocked by Tails - Feature #15319: Grow system partition during boot when started from USB Resolved 2018-10-16

History

#1 Updated by Anonymous 2018-09-28 10:50:30

  • blocked by Feature #15319: Grow system partition during boot when started from USB added

#2 Updated by Anonymous 2018-09-28 10:54:58

  • blocks Bug #16005: Code review & rubber-duck added

#3 Updated by intrigeri 2018-10-12 11:58:21

  • blocked by deleted (Bug #16005: Code review & rubber-duck)

#4 Updated by intrigeri 2018-11-27 15:50:04

  • Description updated

#5 Updated by CyrilBrulebois 2018-11-28 15:26:58

  • Assignee changed from CyrilBrulebois to intrigeri
  • QA Check set to Ready for QA
  • Feature Branch set to test/16003-growing-system-partition

Here’s the new scenario:

Scenario: Starting Tails from an USB image, and making sure it is correctly set up and resized
  Given a computer
  And I temporarily create a 7200 MiB disk named "usbimage"
  And I write the Tails USB image to disk "usbimage"
  And I start Tails from USB drive "usbimage" with network unplugged and I login
  Then Tails is running from USB drive "usbimage"
  And the label of the FAT filesystem on the system partition on "usbimage" is "Tails"
  And the system partition on "usbimage" is an ESP
  And the FAT filesystem on the system partition on "usbimage" is at least 4096M large
  And the FS UUID for the system partition on "usbimage" was randomized

And I’ve managed to test both positive and negative results with an IMG file built from the 3.10.1 ISO (so without any repartition code), from the current feature branches for Feature #15292, with or without the prospective workaround mentioned in Bug #16153.

#6 Updated by intrigeri 2018-11-28 16:26:07

  • Status changed from Confirmed to In Progress

Applied in changeset commit:tails|c81ef61ea9a852f5d6559cea83cf87443ae4dbbc.

#7 Updated by intrigeri 2018-11-28 17:39:40

  • Assignee changed from intrigeri to CyrilBrulebois
  • % Done changed from 0 to 30
  • QA Check changed from Ready for QA to Dev Needed

Great job!

  • I think we need to adjust a few things now that we support TAILS_IMG: look for TAILS_ISO in features/support/hooks.rb.
  • In the “the FAT filesystem on the system partition” step, I’m not sure about using guestfs on the device while the system is booted. I trust you it works right now but it feels risky to rely on it. I’d rather query this info from the system under test (as you do for the filesystem size). Alternatively, let the system boot, then shut it down, then do all the partition and FS checks offline with guestfs.
  • I’ve pushed a few commits where I explain why I’m changing things, as agreed elsewhere.

#8 Updated by intrigeri 2018-11-29 12:22:12

https://jenkins.tails.boum.org/view/Tails_ISO/job/test_Tails_ISO_test-16003-growing-system-partition/3/cucumberTestReport/ failed with FAT filesystem is too small: 4286566400 is less than 4294967296. Pushed a fix for it so we benefit from this test and the following steps.

#9 Updated by intrigeri 2018-12-02 14:56:56

  • Description updated

Given Bug #16178 got reported I’m adding a new test to the list in the ticket description.

#10 Updated by intrigeri 2018-12-02 19:59:06

Also, “the label of the FAT filesystem on the system partition” checks the label of the partition (which is indeed “Tails”), not the label of the filesystem. Otherwise the test would fail: the label is actually “TAILS”. I think you can extract it from the IdLabel field in the org.freedesktop.UDisks2.Block section of the udisksctl info --block-device /dev/sda1 output.

#11 Updated by intrigeri 2018-12-07 13:29:41

#12 Updated by intrigeri 2018-12-16 11:43:06

  • Target version changed from Tails_3.11 to Tails_3.12

#13 Updated by Anonymous 2018-12-27 09:49:06

Hi Cyril! I think we initially wanted to have this before the end of the year, or at least for 3.12. Are you available to work on this in time? If not: please send me an email and let’s discuss. Thanks!

#14 Updated by CyrilBrulebois 2018-12-27 14:51:23

Hi u, I’ve been dealing with administrativia and therefore a bit away from hacking lately; I’ve resumed actual work a few days ago, and this is next on my list. If that’s fine with you: Let’s discuss further in one week if progress hasn’t happened by then?

#15 Updated by Anonymous 2018-12-27 14:56:15

yay! thank you for your fast reply :)

#16 Updated by Anonymous 2019-01-03 16:21:31

That was now a week ago. There are a few remaining details to fix. Can you give us an ETA?

#17 Updated by intrigeri 2019-01-04 14:01:39

#18 Updated by CyrilBrulebois 2019-01-04 16:59:55

After discussion with intrigeri, we decided to have both label checks: one for the partition (Tails) and one for the FS (TAILS).

Just pushed this accordingly:

    c8fcdafd5d..48bd529ad0  HEAD -> test/16003-growing-system-partition

Will look at the GPT flags in a moment.

#19 Updated by CyrilBrulebois 2019-01-05 10:28:30

  • QA Check changed from Dev Needed to Ready for QA

System partition flags check just pushed:

   48bd529ad0..8beba325ad  HEAD -> test/16003-growing-system-partition

which I think was the last missing piece for this bug report? Switching to Ready for QA.

#20 Updated by intrigeri 2019-01-05 11:31:50

  • QA Check changed from Ready for QA to Dev Needed
  • Nitpicking: to be more consistent with how we write things elsewhere in our test suite, please replace "(.+)" with "([^"]+)" for matching captured bits of step titles.
  • API clarity: at least one test takes a on "$image" argument but does not use it: /dev/sda1 is hard-coded in the test implementation. I think it’s fine to hard-code /dev/sda1 there (assuming we don’t have code yet to find this out dynamically, or we do the same elsewhere already) but then let’s not pretend we’re supporting checking an arbitrary drive, and instead drop that argument.
  • Maybe you missed the first part of my review i.e. Bug #16003#note-7?

#21 Updated by CyrilBrulebois 2019-01-05 15:35:18

#20: points 1 & 2 addressed.

#7: point 1 addressed I think (looping and working on ISO/IMG in the same way, except for OLD_TAILS_* where factorizing wasn’t really worth it), but 2 still needs addressing. (And yeah, it worked reliably for me until now.)

I’ll get back to that during next time I’m waiting on other feature/buster things; keeping this Dev Needed for now, but feel free to discover other issues.

#22 Updated by CyrilBrulebois 2019-01-05 21:28:32

  • Assignee deleted (CyrilBrulebois)
  • QA Check changed from Dev Needed to Ready for QA

Just pushed a few more commits:

   cbbaf4dad1..eef12f0fb5  HEAD -> test/16003-growing-system-partition

I’ve fixed the guestfs issue for all tests I worked on, introduced a wrapper as discussed elsewhere to organize udisks-provided info into a little tree (I almost stripped the common prefix for the keys for brevity, to get Partition, Block, etc. keys, but finally decided against it). Note the XXX, as we have entries spanning across several lines. I haven’t seen enough examples of udisksctl info output to conclude it’s safe enough for a default case… We should probably match for leading spaces, strip all leading spaces, and concat that to the previous value with an extra \n character, and leave the default character as a catch-all which would log/generate an error when encountering an unknown pattern?

Some other tests might benefit from this wrapper, instead of just trying to match the whole output of this command, but that looks like out of scope for this specific task.

#23 Updated by intrigeri 2019-01-06 07:07:29

  • Assignee set to intrigeri

(As your designated reviewer / rubber-duck for this task :)

#24 Updated by Anonymous 2019-01-08 10:58:33

Hi Cyril & intrigeri, could you try to review this for January 21st latest delay please?
We need this to be done on February 1st and I would like to leave some time for intrigeri|kibi in case something needs fixing after the review. Thanks!

#25 Updated by intrigeri 2019-01-08 11:07:20

> Hi Cyril & intrigeri, could you try to review this for January 21st latest delay please?

Sure, I’ll review this really soon now that the Buster sprint is over!

#26 Updated by intrigeri 2019-01-09 18:27:49

  • Status changed from In Progress to Resolved
  • % Done changed from 30 to 100

Applied in changeset commit:tails|64d58e436c958d28de92c1acfb7240cbb9ccdcf2.

#27 Updated by intrigeri 2019-01-09 18:29:07

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass