Feature #12025

Default panic_on_warn to 1

Added by cypherpunks 2016-12-09 10:07:39 . Updated 2019-03-09 09:23:53 .

Status:
Confirmed
Priority:
Normal
Assignee:
anonym
Category:
Target version:
Start date:
2016-12-09
Due date:
% Done:

0%

Feature Branch:
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

This replaces WARN () with panic(), which is useful because many security features like CONFIG_DEBUG_CREDENTIALS and CONFIG_DEBUG_LIST trigger a WARN () upon violation. These are enabled by default in Debian’s kernel config, IIRC, but are not particularly useful for security unless the system panics when violations are detected. Unlike panic_on_oops=1, this probably won’t cause lots of issues for people with flaky hardware. Additionally, many miscellaneous cases which should never be hit in the kernel have WARN ()s, even when they can lead to degraded system security.

This can be enabled either with the boot parameter “panic_on_warn”, or by setting the sysctl “kernel.panic_on_warn=1”.


Subtasks


Related issues

Related to Tails - Feature #11886: Upgrade Linux to 4.8 and adjust our kernel tweaks accordingly Resolved 2016-10-25

History

#1 Updated by mercedes508 2016-12-11 15:54:35

  • Assignee set to anonym

#2 Updated by anonym 2016-12-11 16:51:34

  • Status changed from New to Confirmed

Note that this option originally was added as a kernel debugging convenience option: https://lkml.org/lkml/2014/10/28/536. I just have a bad feeling about enabling a non-security feature for security purposes, even if I get your point. I mean, in case of a compromise which would generate such a WARN() it will currently go completely unnoticed by users.

> Unlike panic_on_oops=1, this probably won’t cause lots of issues for people with flaky hardware.

I fear I’d like some data to back this up before testing it live on our user base releasing it. Tails just freezing without feedback about what happened (which this would result in, right?) is pretty bad UX, and will generally encourage them to move to worse solutions.

So I feel there’s some more convincing needed about this point before this can be considered further.

#3 Updated by anonym 2016-12-11 17:47:11

  • Assignee changed from anonym to sajolida
  • QA Check set to Info Needed

anonym wrote:
> I fear I’d like some data to back this up before testing it live on our user base releasing it.

sajolida, could you run:

grep --recursive --count --fixed-strings "WARNING: CPU:" ...


where ... is the path(s) to some large amount of whisperback logs?

#4 Updated by cypherpunks 2016-12-14 05:51:50

anonym wrote:
> Note that this option originally was added as a kernel debugging convenience option: https://lkml.org/lkml/2014/10/28/536. I just have a bad feeling about enabling a non-security feature for security purposes, even if I get your point. I mean, in case of a compromise which would generate such a WARN() it will currently go completely unnoticed by users.

A lot of “debugging” features are actually security features, which is a common problem in the kernel. For example, the kernel hacking section in the kernel config contains a large amount of security features. CONFIG_DEBUG_* options are often sanity checks which are highly security-relevant, and Kees from the KSPP recommends most of these options be enabled (with the exception of CONFIG_DEBUG_PI_LIST, which he said would probably have too much of a performance hit for such low coverage). CONFIG_DEBUG_RODATA makes vital sections of the kernel read-only and is perhaps the most controversially named kernel option, as it is a fundamental security primitive in the kernel. Kees has written a lot about this (as he was the one who made it mandatory to use on x86 hardware, even though it is named a “debug” option).

Note that SLUB redzoning and sanity checks, and disabled merging of slabs are all also debugging features, but they were added to Tails. Likewise, slab poisoning and page poisoning will be added to Tails when supported (if I understand correctly).

anonym wrote:
> anonym wrote:
> > I fear I’d like some data to back this up before testing it live on our user base releasing it.
>
> sajolida, could you run:
> […]
> where ... is the path(s) to some large amount of whisperback logs?

You could also check when the WARN()s occurred. If they occur sporadically, then it may be a problem, but if on problematic hardware, they consistently occur very early at boot, perhaps as soon as the network is enabled or the user logs in, then it would provide a reliable way for the system to know if it is on hardware where setting kernel.panic_on_warn=1 is safe. Example behavior would be to check the system log upon starting the display manager, and if no warnings have occurred, toggle the sysctl, otherwise keep the default off. This would provide a significant opportunistic security improvement for supported hardware (i.e. the majority), without causing unnecessary crashes on buggy hardware. Of course, that would only be necessary if WARN()s occur in appreciable levels in the first place.

#5 Updated by sajolida 2016-12-26 11:06:49

  • Assignee changed from sajolida to anonym
  • QA Check deleted (Info Needed)

I have no clue how sensitive this data would be so I’m sending the output to you by email.
Aggregated stats:

  • 1311 logs
  • 53 with matches
  • Matches count passed through uniq -c:
# of logs   # of matches
        1   40
        1   24
        1   15
        1   12
        1   7
        2   5
        2   2
       44   1

Tell me if you want to see the logs with most matches.

#6 Updated by anonym 2016-12-26 23:25:03

  • Assignee changed from anonym to sajolida
  • QA Check set to Info Needed

sajolida wrote:
> I have no clue how sensitive this data would be so I’m sending the output to you by email.

Thanks!

> Aggregated stats:
>
> * 1311 logs
> * 53 with matches

Ok, so around 4%.

> Tell me if you want to see the logs with most matches.

I’d like to have 2-3 randomly selected ones. And the one with 40 cause I’m curious wtf is going on. :)

#7 Updated by intrigeri 2016-12-27 09:44:34

  • related to Feature #11886: Upgrade Linux to 4.8 and adjust our kernel tweaks accordingly added

#8 Updated by cypherpunks 2016-12-29 04:09:24

anonym wrote:

> Ok, so around 4%.

But only 0.7% occur more than once. It would be useful to know if the ones that don’t occur once occur at the same time (e.g. early at boot, only after networking is enabled, etc), or at random times during runtime.

> I’d like to have 2-3 randomly selected ones. And the one with 40 cause I’m curious wtf is going on. :)

What percentage of those occur after /etc/sysctl.d/ and /etc/sysctl.conf are processed? The one with 40 probably has buggy drivers causing problems that occurs throughout uptime (or bad RAM, or even exploit attempts), but the others could very well occur only early at boot, so would not trigger if the setting is enabled via a sysctl instead of a boot parameter.

#9 Updated by sajolida 2017-01-16 19:37:13

  • Assignee changed from sajolida to anonym
  • QA Check deleted (Info Needed)

Sent.

#10 Updated by Anonymous 2018-01-16 13:55:04

  • QA Check set to Info Needed

cypherpunks wrote:
> anonym wrote:

> > I’d like to have 2-3 randomly selected ones. And the one with 40 cause I’m curious wtf is going on. :)
>
> What percentage of those occur after /etc/sysctl.d/ and /etc/sysctl.conf are processed?

Looks like this question was not answered.

Even though I’m unsure about the UX of this proposal.

#11 Updated by cypherpunks 2018-06-05 01:09:12

u wrote:
> cypherpunks wrote:
> > anonym wrote:
>
> > > I’d like to have 2-3 randomly selected ones. And the one with 40 cause I’m curious wtf is going on. :)
> >
> > What percentage of those occur after /etc/sysctl.d/ and /etc/sysctl.conf are processed?
>
> Looks like this question was not answered.
>
> Even though I’m unsure about the UX of this proposal.

The UX of this proposal depends, in part, on the answer to that question. If none of the WARN() events occurred sporadically during runtime and only popped up early on, it would be quite trivial for a script to use that to determine the safe default panic_on_warn value to maximize security.

#12 Updated by Anonymous 2019-03-08 14:20:36

  • QA Check deleted (Info Needed)

#13 Updated by intrigeri 2019-03-08 14:23:57

@anonym, do you still have the files sajolida sent you before writing Feature #12025#note-9? If yes, can you please send them to me?

#14 Updated by sajolida 2019-03-09 09:23:53

I sent them to you.