Tails

#1 Updated by intrigeri 2015-05-17 10:46:51

• Status changed from Confirmed to In Progress
• % Done changed from 0 to 10

Initial test results: our build system seems to work just fine on Jessie, which is no big surprise given that most of the build process happens in a Wheezy chroot anyway.

Next step: compare the resulting ISO with one built on Wheezy. The preliminary work made on Feature #5630, and the debbindiff tool, should help a lot.

#2 Updated by intrigeri 2015-06-05 21:54:12

• Assignee changed from intrigeri to anonym
• Target version changed from Tails_1.4.1 to Tails_1.5
• QA Check set to Info Needed

I doubt I’ll have time to work on this during the 1.4.1 cycle (and same the next one, possibly).

anonym, assuming ISO images built on Wheezy and on Jessie don’t essentially differ, do you think that we should block this task on the corresponding upgrade of our Vagrant basebox? If yes, then we’ll need to find out who/when does that (or completes the migration to Docker :)

#3 Updated by anonym 2015-06-10 10:01:46

• Assignee changed from anonym to intrigeri
• % Done changed from 10 to 30
• Feature Branch set to feature/9262-update-vagrant-to-jessie

intrigeri wrote:
> I doubt I’ll have time to work on this during the 1.4.1 cycle (and same the next one, possibly).
>
> anonym, assuming ISO images built on Wheezy and on Jessie don’t essentially differ, do you think that we should block this task on the corresponding upgrade of our Vagrant basebox? If yes, then we’ll need to find out who/when does that (or completes the migration to Docker :)

I did the upgrade (incl. building and pushing a new basebox) while procrastinating today. However, there are some issues:

* eatmydata doesn’t work in the chroot and we get spammed with:

ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded: ignored.

Seems related to Bug #9523, right?

• Builds fail due to monkeysphere not installing properly. It looks exactly like Debian bug #778833 which apparently wasn’t fixed in Wheezy. I wonder what’s different compared to the Wheezy-based builder, since this error occurs in the Wheezy chroot used by live-build. Any ideas?

It should be said that I managed to build a seemingly working image by omitting monkeysphere, so it’s quite close to being useful.

#5 Updated by intrigeri 2015-06-10 12:29:47

• blocked by Bug #9523: Discrepancy between eatmydata versions breaks Jessie build added

#6 Updated by intrigeri 2015-06-10 12:43:31

• QA Check deleted (Info Needed)

#8 Updated by anonym 2015-06-10 21:27:20

• QA Check set to Info Needed

Ok, so this branch works well for me now. I’ve managed to build both stable and feature/jessie seemingly with great success. Is anything else required from me for this ticket?

#9 Updated by intrigeri 2015-06-11 07:48:30

• Assignee changed from intrigeri to anonym

> Ok, so this branch works well for me now.

Woohoo!

> Is anything else required from me for this ticket?

Let me have a look at the branch before you disappear in vacation, then. It looks very good overall, here are a few nitpicks (nothing that would really block me from merging TBH, but still):

> d-i passwd/root-login boolean false
> +d-i passwd/root-password password vagrant
> +d-i passwd/root-password-again password vagrant

Assuming this results in a working root account, can it be ssh’ed into from the host system with that password? Or does the new “no password auth for root over SSH” default Jessie restriction apply?

> +d-i passwd/user-default-groups string audio cdrom video admin

Do we really have an admin group on Jessie? I don’t have it on my system, so this looks suspiciously like something that was copied’n’pasted from Ubuntu-specific docs ;)

Also, why does the default user account need to be in the audio, cdrom and video groups?

> +# Add jessie-backports for apt-cacher-ng. The version in jessie is older
> +# than the one in wheezy-backports that we used to install

That comment seems incorrect to me: wheezy-backports has 0.8.0-3~bpo70+1, and Jessie has 0.8.0-3.
(And in general, this is always incorrect because it would break upgrades; that’s why Debian also has sloppy backports.)
Is there any other reason why we really need acng 0.8.3-1, or?

Otherwise, looks good!

#10 Updated by anonym 2015-06-11 10:12:23

• Assignee changed from anonym to intrigeri

intrigeri wrote:
> > d-i passwd/root-login boolean false
> > +d-i passwd/root-password password vagrant
> > +d-i passwd/root-password-again password vagrant
>
> Assuming this results in a working root account, can it be ssh’ed into from the host system with that password? Or does the new “no password auth for root over SSH” default Jessie restriction apply?

No, the root account is disabled so the added lines are NOOP:s…

> > +d-i passwd/user-default-groups string audio cdrom video admin
>
> Do we really have an admin group on Jessie? I don’t have it on my system, so this looks suspiciously like something that was copied’n’pasted from Ubuntu-specific docs ;)

Almost. :)

> Also, why does the default user account need to be in the audio, cdrom and video groups?

Obviously we do not.

I just made our template conform exactly with veewee’s jessie template modulo comments and the lines we actually want to diverge on. I had issues with the d-i/preseeding stuff failing at some unspecified step, and I have no idea how to debug it properly, so this seemed like the thing to do that would waste the least amount of time.

> > +# Add jessie-backports for apt-cacher-ng. The version in jessie is older
> > +# than the one in wheezy-backports that we used to install
>
> That comment seems incorrect to me: wheezy-backports has 0.8.0-3~bpo70+1, and Jessie has 0.8.0-3.
> (And in general, this is always incorrect because it would break upgrades; that’s why Debian also has sloppy backports.)
> Is there any other reason why we really need acng 0.8.3-1, or?

Ah, I must have looked at wheezy-backports-sloppy for whatever reason. :S Fixed in commit:9c8776a.

Anything else?

#11 Updated by intrigeri 2015-06-11 10:32:45

• QA Check changed from Info Needed to Dev Needed

> Anything else?

No, that’s great. Thanks!

#12 Updated by intrigeri 2015-07-13 05:48:52

• Feature Branch changed from feature/9262-update-vagrant-to-jessie to feature/9262-build-on-jessie

(Let’s share a single branch, whose name is not Vagrant-specific.)

#13 Updated by intrigeri 2015-07-13 06:00:30

• Feature Branch changed from feature/9262-build-on-jessie to feature/9262-build-on-jessie, puppet-tails:feature/9262-build-on-jessie

#15 Updated by intrigeri 2015-07-18 06:26:33

• % Done changed from 30 to 40

I’ve built two ISO images from the topic branch, one in a Wheezy VM, the other one in a Jessie VM, and I’ve looked at the output of debbindiff + diff’ed the initramfs and SquashFS manually:

• package list is the same
• differing initrd’s size (a bit smaller on Jessie), but only difference in content is mtimes
• Number ​of inodes in SquashFS: 141721 vs. 141718, explained since three files are in the SquashFS only when built on Wheezy:
  • /dev/xconsole: created at runtime if missing
  • /var/lib/urandom/random-seed: created at runtime if missing (and Feature #7642 seems to indicate that we should not ship it in the ISO anyway)
  • /run/sendsigs.​omit.​d/rsyslog: created at runtime if missing
• /run/motd.​dynamic content differs because it embeds the kernel version of the build system

I saw no other relevant non-mtime-related differences, so IMO next steps are:

• run the automated test suite on an ISO built on Jessie;
• make a decision on Feature #7642, so we can be comfortable not shipping that random seed in the ISO.

#16 Updated by intrigeri 2015-07-18 06:29:44

• blocked by Feature #7642: Investigate whether we should resume shipping a static random seed added

#18 Updated by intrigeri 2015-07-19 03:36:12

• % Done changed from 40 to 50

intrigeri wrote:
> I saw no other relevant non-mtime-related differences, so IMO next steps are:
>
> * run the automated test suite on an ISO built on Jessie;

Done, so the only blocker left is:

> * make a decision on Feature #7642, so we can be comfortable not shipping that random seed in the ISO.

#19 Updated by intrigeri 2015-07-29 00:34:38

• Target version changed from Tails_1.5 to Tails_1.6

Too late to reach a conclusion on Feature #7642 before the freeze => postponing this one.

#20 Updated by intrigeri 2015-09-16 14:24:23

• Target version changed from Tails_1.6 to Tails_1.7

I don’t think it’s realistic to expect bertagaz to have time to resolve the remaining blocker (Feature #7642) within a week => let’s postpone.

#21 Updated by intrigeri 2015-10-05 13:28:45

• Target version changed from Tails_1.7 to Tails_1.8

intrigeri wrote:
> I don’t think it’s realistic to expect bertagaz to have time to resolve the remaining blocker (Feature #7642) within a week => let’s postpone.

This is again the case.

#22 Updated by intrigeri 2015-11-06 11:44:07

• Target version changed from Tails_1.8 to 246

#23 Updated by intrigeri 2015-11-20 02:21:32

• blocks Bug #10592: Upgrade i18nspector to 0.20+ on the Jenkins slave that runs check_PO_master added

#24 Updated by sajolida 2015-11-27 04:46:16

• Target version changed from 246 to Tails_2.0

#25 Updated by intrigeri 2015-12-13 12:17:42

• blocks Feature #6297: Save list of packages used at ISO build time added

#27 Updated by intrigeri 2015-12-14 02:45:18

• blocked by deleted (Feature #6297: Save list of packages used at ISO build time)

#28 Updated by intrigeri 2015-12-14 13:08:41

• Target version changed from Tails_2.0 to Tails_2.2

#29 Updated by bertagaz 2016-01-27 13:02:20

• blocks Bug #11006: Reinstall Lizard's isobuilders from scratch added

#30 Updated by intrigeri 2016-02-12 23:55:17

• Target version changed from Tails_2.2 to Tails_2.3

This is blocked by Feature #7642, that I’m told should move forward by early March, which will likely be too late for me to complete this in time for Tails 2.2.

#31 Updated by intrigeri 2016-02-21 15:02:46

• blocks deleted (Feature #7642: Investigate whether we should resume shipping a static random seed)

#32 Updated by intrigeri 2016-02-21 15:06:59

• Target version changed from Tails_2.3 to Tails_2.2

We’re done on Feature #7642 as far as this ticket is concerned, and anonym and I will try to get this merged today.

#33 Updated by intrigeri 2016-02-21 16:50:14

• % Done changed from 50 to 60

anonym did most of the work, I reviewed the code at commit:01e930550598ba70a2bae7e12d336e400d0bcff4 and updated the Puppet stuff. Next steps are:

• to compare two ISO images (one built on Wheezy, the other built on Jessie)
• to run some automated tests on an ISO built on Jessie

… and then we can merge in Git, and deploy on our infra + upgrade our isobuilders on lizard.

#34 Updated by intrigeri 2016-02-22 11:52:00

• File diffoscope-wheezy_to_jessie-no-timestamps.html.xz added

Here’s a diffoscope report with two ISO images built respectively on a Wheezy and a Jessie VM, from the same source, that has a hack to find / -exec touch --no-create -t 197001010000 {} \; as a chroot hook, to eliminate file timestamp variations. I’ll provide another one, with --no-dereference passed to touch as well so we also eliminate these variations for symlinks.

#35 Updated by intrigeri 2016-02-22 12:53:23

• Assignee changed from intrigeri to anonym
• % Done changed from 60 to 70
• QA Check changed from Dev Needed to Ready for QA

I say let’s merge.

#36 Updated by anonym 2016-02-22 13:02:41

• Status changed from In Progress to Fix committed
• Assignee deleted (anonym)
• % Done changed from 70 to 100
• QA Check changed from Ready for QA to Pass

Merged! A full automated test suite run of an image built from this branch has passed!

#37 Updated by intrigeri 2016-02-28 11:15:00

• Status changed from Fix committed to Resolved

(This was merged into master already.)