Feature #10298

Upgrade to Linux 4.x

Added by intrigeri 2015-09-28 06:51:06 . Updated 2016-09-20 16:47:23 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Hardware support
Target version:
Tails_2.6
Start date:
2015-08-11
Due date:
% Done:

100%

Feature Branch:
feature/10298-linux-4.x-aufs
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

Tails should work on recent hardware. If we don’t do anything special, we’re stuck with Linux 3.16, missing out on the Linux kernel’s hardware support improvements.

Our preferred way to upgrade Tails to Linux 4.x would be to migrate from aufs to overlayfs (Feature #8415). We have done a lot of the needed work on our side, but before we go on we are blocked by work that must be done upstream (AppArmor support for overlayfs in the Linux kernel, aka. Bug #9045).

If this plan works in time for Tails 3.0 (Debian Stretch), fine: we’ll then have to finish porting Debian Live and Tails -specific bits to overlayfs.

If it does not, then we will have to stick to aufs, which implies building out-of-tree aufs modules ourselves (note that the support patches have been re-introduced in Debian’s Linux kernel, so that is possible now.

In any case, we will also have to deal with other new issues we’ll discover with recent kernels (see subtasks).

Subtasks

Bug #9969: Re-enable dkms modules building with Linux 4.x Resolved

100
Feature #11511: Drop custom value for kernel.perf_event_paranoid sysctl Resolved

100

Related issues

Related to Tails - Bug #11441: Stretch's X.Org cannot start with Linux 3.16's qxl Resolved 2016-05-18
Related to Tails - Bug #11518: Linux 4.x QXL 64-bit kernel modesetting breaks 32-bit X.Org Resolved 2016-06-08
Blocks Tails - Feature #7649: Include a grsecurity-patched kernel Rejected 2015-01-07
Blocks Tails - Bug #10289: Tails based on Debian Stretch Resolved 2015-09-27
Blocks Tails - Feature #11281: Enable kASLR Resolved 2016-03-24
Blocks Tails - Bug #8485: Consider including an x86 kernel with SMP support Resolved 2014-12-25

History

#1 Updated by intrigeri 2015-09-28 06:51:33

#2 Updated by intrigeri 2015-09-28 06:52:53

  • Description updated

#3 Updated by intrigeri 2016-01-05 16:59:38

  • blocks Feature #7649: Include a grsecurity-patched kernel added

#4 Updated by intrigeri 2016-01-17 04:00:07

  • blocks Bug #10289: Tails based on Debian Stretch added

#5 Updated by intrigeri 2016-03-24 20:12:57

#6 Updated by intrigeri 2016-05-18 08:29:55

  • related to Bug #11441: Stretch's X.Org cannot start with Linux 3.16's qxl added

#7 Updated by intrigeri 2016-05-23 12:01:33

  • related to Feature #11421: Change kernel.perf_event_paranoid sysctl to 2 added

#8 Updated by intrigeri 2016-06-03 23:21:12

  • Description updated

#9 Updated by Dr_Whax 2016-06-04 21:27:27

  • blocks Feature #11511: Drop custom value for kernel.perf_event_paranoid sysctl added

#10 Updated by intrigeri 2016-06-06 04:59:00

  • blocked by deleted (Feature #11511: Drop custom value for kernel.perf_event_paranoid sysctl)

#11 Updated by intrigeri 2016-06-06 05:00:55

  • related to deleted (Feature #11421: Change kernel.perf_event_paranoid sysctl to 2)

#12 Updated by intrigeri 2016-06-07 09:15:09

  • Description updated

#13 Updated by intrigeri 2016-06-08 03:44:21

  • Feature Branch set to feature/10298-linux-4.x-aufs

Here’s a branch that builds and includes the out-of-tree aufs modules. It seems to boot fine (except with QXL) both on 32-bit and 64-bit (virtual) CPUs but I didn’t test any further. It shares some of the issues of feature/8415-overlayfs but at least it supports AppArmor, and should require little additional porting work, that also would be needed with overlayfs anyway (see subtasks). Let’s see what Jenkins thinks of it.

#14 Updated by intrigeri 2016-06-08 08:53:18

  • blocks Bug #8485: Consider including an x86 kernel with SMP support added

#15 Updated by intrigeri 2016-06-08 09:04:01

  • Description updated

#16 Updated by intrigeri 2016-06-09 08:13:20

  • Status changed from Confirmed to In Progress

#17 Updated by intrigeri 2016-06-10 11:58:13

  • related to Bug #11518: Linux 4.x QXL 64-bit kernel modesetting breaks 32-bit X.Org added

#18 Updated by intrigeri 2016-06-10 12:13:17

  • Target version changed from 2016 to Tails_2.6
  • QA Check set to Ready for QA

I’ve seen all test suite scenarios pass, Jenkins doesn’t particularly dislike it, and I see no blockers left. anonym, how about merging this for 2.6?

#19 Updated by anonym 2016-06-14 05:54:14

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Ready for QA to Dev Needed

When I built this branch (merged into devel) I ran out of space (RAM) when creating the ISO image (at 97.5%) and genisoimage was OOM killed. I guess you used defaultcomp, which still should work, as opposed to gzipcomp, which I used. IMHO branches should remain buildable with gzipcomp since it makes the review’n’merge work less tedious. Bumping BUILD_SPACE_REQUIREMENT by 128 fixes it, which I was ready to do myself so I could merge this now…

… which I’d like to do and actually was ready to, because this branch is otherwise great and I have no remarks, except that it breaks vboxvideo in a 32-bit VirtualBox VM (64-bit is still ok). Boot stops at:

[  OK  ] Started GNOME Display Manager.


If I login as root in a different VT I can see this in the Xorg log after the vboxvideo X driver is ~loaded:

[...]
(II) vboxvideo: kernel driver found, not loading.
(EE) No devices detected.
[...]


If I remove the /etc/X11/xorg.conf.d file created by live-{boot,config} (or whatever) which has a section that (force) loads the vboxvideo Xorg driver, and restart Xorg, then everything works as expected (e.g. VirtualBox’s auto-resolution feature). Xorg will then use the fbdevhw Xorg driver, which is the correct thing when the vboxvideo kernel driver is loaded.

So, it would seem to me that we could fix this if we prevent the /etc/X11/xorg.conf.d file from being created by live-{boot,config} (or whatever).

(Note: on 64-bit we still want the vboxbideo Xorg driver since we don’t have the kernel driver, but it will be properly autodetected by Xorg without the /etc/X11/xorg.conf.d file, so good riddance. :))

So, in conclusion, please:

  • bump Vagrant’s BUILD_SPACE_REQUIREMENT by 128 or so, and make sure that this branch builds with gzipcomp, and
  • prevent the /etc/X11/xorg.conf.d file (that (force) loads the vboxvideo driver) from being created by live-{boot,config} (or whatever).

#20 Updated by cypherpunks 2016-06-25 05:51:26

When we get 4.x, we’ll also be stuck with unprivileged USERNS (which cannot be disabled with anything other than a custom kernel). Are we going to do anything about that? Since we’re not using grsecurity yet, we’re going to be stuck with all the horrible bugs that unprivileged USERNS pumps out on a regular basis, like these two from just today: http://www.openwall.com/lists/oss-security/2016/06/24/5

#21 Updated by intrigeri 2016-06-25 07:05:39

> When we get 4.x, we’ll also be stuck with unprivileged USERNS (which cannot be disabled with anything other than a custom kernel).

Indeed, these have a long history of security issues. https://anonscm.debian.org/cgit/kernel/linux.git/tree/debian/patches/debian/add-sysctl-to-disallow-unprivileged-CLONE_NEWUSER-by-default.patch is applied to the Debian 4.x kernel. If we used this sysctl to disable unprivileged CLONE_NEWUSER, would this address your concern?

#22 Updated by cypherpunks 2016-07-09 01:24:43

intrigeri wrote:
> Indeed, these have a long history of security issues. https://anonscm.debian.org/cgit/kernel/linux.git/tree/debian/patches/debian/add-sysctl-to-disallow-unprivileged-CLONE_NEWUSER-by-default.patch is applied to the Debian 4.x kernel. If we used this sysctl to disable unprivileged CLONE_NEWUSER, would this address your concern?

Yeah, a lot better than nothing!

#23 Updated by intrigeri 2016-07-15 05:06:54

> When I built this branch (merged into devel) I ran out of space (RAM) when creating the ISO image (at 97.5%) and genisoimage was OOM killed. I guess you used defaultcomp, which still should work, as opposed to gzipcomp, which I used. IMHO branches should remain buildable with gzipcomp since it makes the review’n’merge work less tedious.

Sorry about that. I used gzipcomp as usual, and it works fine for me here, so I’m not quite sure why it doesn’t for you. I’ll bump BUILD_SPACE_REQUIREMENT anyway :)

> … which I’d like to do and actually was ready to, because this branch is otherwise great and I have no remarks, except that it breaks vboxvideo in a 32-bit VirtualBox VM (64-bit is still ok). Boot stops at:

Thanks for the detailed analysis! It seems to me that you found a bug in live-config, that should be fixed in there instead of merely workaround’ed on our side, so I’ve reported it. Meanwhile, I’ll patch out the faulty code via config/chroot_local-patches/. In doubt, I’ll only drop the bits that force the video driver, but I’ll leave the ones about vboxmouse in place. I’d appreciate if you could test it again (both 32-bit and 64-bit) and check if there’s anything wrong with the input devices. Then I’ll upstream the patch :)

> So, in conclusion, please:

Sure! I’m on it.

#24 Updated by intrigeri 2016-07-16 05:01:20

Status: update the code according to anonym’s review, running the full test suite. Then we’ll need another review and more testing on VirtualBox as requested above.

#25 Updated by intrigeri 2016-07-16 08:45:23

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Dev Needed to Ready for QA

I’ve seen all tests pass (except Bug #11571) so please review, re-do the VirtualBox tests, and merge :)

#26 Updated by intrigeri 2016-08-01 07:35:55

I’d like to ease reviewing for the 2.6 RM, and to get automated tests running about the combination of all these changes ASAP in the 2.6 dev cycle. So, I’ve merged this work, along with the other major branches I’m proposing for 2.6, into the feature/from-intrigeri-for-2.6 integration branch (Jenkins builds and tests.

#28 Updated by anonym 2016-08-23 08:28:07

  • Status changed from In Progress to Fix committed
  • Assignee deleted (anonym)
  • QA Check changed from Ready for QA to Pass

#29 Updated by itiamo 2016-08-28 04:10:05

This build confirmed working on my GeForce 940MX GPU, ThinkPad T460p laptop.

#30 Updated by anonym 2016-09-20 16:47:23

  • Status changed from Fix committed to Resolved

#31 Updated by intrigeri 2017-01-01 11:27:20

  • related to deleted (Feature #8415: Migrate from aufs to overlayfs)