Bug #7945

boot-profile generates 3x larger than normal squashfs sort file

Added by anonym 2014-09-24 15:05:59 . Updated 2014-12-03 19:41:42 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Build system
Target version:
Tails_1.2.1
Start date:
2014-09-24
Due date:
% Done:

100%

Feature Branch:
bugfix/7945-directories-in-boot-profile
Type of work:
Research
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

When preparing Tails 1.1.2 I followed the usual instructions for generating the squashfs sort file but the resulting file became three times larger (in terms of entries) than the previous one. A quick look reveals that files related to e.g. scribus and libreoffice are touched, which doesn’t make sense. See attached file.

For what it’s worth, for an overview of the additional files touched, I did this:

diff -NaurB \
    <( cut -d' ' -f1 config/binary_rootfs/squashfs.sort | sort ) \
    <( cut -d' ' -f1 /tmp/boot-profile | sort )  \
    | less

Files

boot-profile (990074 B) anonym, 2014-09-24 14:59:54
boot-profile.filtered (327191 B) anonym, 2014-10-15 16:45:16

Subtasks

History

#1 Updated by intrigeri 2014-09-25 22:39:50

  • Assignee deleted (intrigeri)
  • Type of work changed from Code to Research

Most of the added paths are directories. Our previous boot-profile has no directory. So, I see two possibilities:

  1. either we were already accessing these directories, and a change in the kernel (in aufs or inotify, I would guess) makes /usr/local/sbin/boot-profile’s mechanism to ignore directory access ineffective; we filter directories out in process_IN_OPEN only, we could try doing the same i.e. in process_IN_ACCESS;
  2. or, something new or changed in our own source code (or installed packages, whatever) accesses way more directories than previously.

The way I would deal with it is:

  1. first, quickly try what I suggest in the first case above, to ignore directory access
  2. if it fails: have a look at the kernel and aufs changelog for changes that may affect this kind of stuff
  3. if nothing rings a bell: makes our boot-profile script output the name of the program that accesses a file, as well as its path, if the inotify interface allows it
  4. if still nothing found: compare our list of packages and look at the new ones, and at the modified packages’ changelog for hints (especially those that we pull from testing/sid).

#2 Updated by intrigeri 2014-10-13 02:41:41

  • Target version changed from Tails_1.2 to Tails_1.2.1

It seems we won’t tackle that in time for 1.2 => postponing.

#3 Updated by anonym 2014-10-15 16:49:00

As a workaround, what about running the something like the following after the profiling has stopped (i.e. still inside the profiled Tails session):

cat /var/log/boot-profile | while read line; do
  f="$(echo "${line}" | cut -d' ' -f1)"
  if [ ! -d "/${f}" ]; then
      echo "${line}"
  fi
done > boot-profile.filtered

I know it’s not perfect, e.g. perhaps it skips symlinked directories, and any directories deleted will be skipped too. Also, it will require root access, and then I think the rootpw= boot parameter is less intruding than setting a sudo password in Tails Greeter.

Any way, see attached file for the result. It looks pretty reasonable. Do the diff described above and you’ll see that there’s only 680 removals and 501 additions compared to the one in Git. Sounds like something quite reasonable given the switch of kernel and Iceweasel to TBB migration.

#4 Updated by intrigeri 2014-10-16 01:29:46

Yep, we still can filter the data after collecting it. And then, the RM can do it on their own system, rather than inside of Tails, when updating the boot profile.

#5 Updated by intrigeri 2014-11-05 13:55:06

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 50
  • QA Check set to Ready for QA
  • Feature Branch set to bugfix/7945-directories-in-boot-profile

#6 Updated by anonym 2014-11-13 13:19:26

  • Status changed from In Progress to Fix committed
  • % Done changed from 50 to 100
  • QA Check changed from Ready for QA to Pass

#7 Updated by BitingBird 2014-12-03 19:41:42

  • Status changed from Fix committed to Resolved