Feature #6918

Track hardening status of the binaries shipped in Tails

Added by intrigeri 2014-03-12 10:03:57 . Updated 2019-08-04 14:50:58 .

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
Start date:
2014-03-12
Due date:
% Done:

10%

Feature Branch:
Type of work:
Sysadmin
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

One area where Tails security could be improved is hardening of binaries. Debian has made great progress on this for the Wheezy release, and Jessie will be even better. To help improve this in areas that matter to Tails (read: in packages that are shipped in Tails), we need:

  1. statistics about the current status, to track progress of the proportion of the binaries shipped in Tails are hardened. That is, something like the Debian Security Hardening Statistics, but limited to the binaries shipped in Tails;
  2. the list of packages that still lack hardening, to know where we should focus our efforts.

Note that we need statistics about the current state of binaries in Debian unstable (as opposed to the binaries shipped in current Tails), because this is where improvements can be made.

One should use the list of binary packages shipped in ISO images built from the feature/stretch branch (see the .build-manifest file there) as input data.

There is no need to reinvent the wheel: the Debian Security Hardening Statistics link to the source code used to generate it. We should improve these scripts to take any parameter we need (likely: our list of packages as input, and an option to output the list of packages that lack hardening), and contribute the changes upstream. Note that Kees Cook <kees@debian.org> has published hardening statistics limited to the packages present in a default Debian installation, so probably most of the functionality we need is already there.

Hopefully, Kees will happily run the resulting scripts for us, and add a “Tails Security Hardening Statistics” page to his website. Worst case, we’ll run it and host the resulting web pages on our own infrastructure.

Files

packages-missing-hardening (1242 B) intrigeri, 2016-07-05 08:14:07
output.yml (14685 B) intrigeri, 2016-07-05 08:28:18
packages-missing-hardening-udd.py (643 B) intrigeri, 2019-08-04 14:42:30
packages-missing-hardening-udd.output (7705 B) intrigeri, 2019-08-04 14:46:36

Subtasks

Related issues

Related to Tails - Feature #6919: Harden the binaries shipped in Tails Duplicate 2014-03-12

History

#1 Updated by intrigeri 2014-03-12 10:09:01

  • Description updated

#2 Updated by intrigeri 2014-03-12 10:13:16

  • related to Feature #6919: Harden the binaries shipped in Tails added

#3 Updated by BitingBird 2014-04-09 15:35:58

  • Target version set to Tails_2.0

#4 Updated by intrigeri 2014-04-09 15:56:40

  • Target version deleted (Tails_2.0)

I really don’t understand what this has to do with milestone 4.0 (porting to Jessie), so I’m reverting the last change. This instead could be seen as part of our hardening goal (3.0), which I’m going to propose at the summit.

#5 Updated by BitingBird 2014-06-22 15:35:04

  • related to Feature #7424: Have Tails based on Jessie building and starting added

#6 Updated by eillip 2014-07-21 20:48:18

Hi,

The script linked on Debian Hardening Statistics pagge run against a full mirror of Debian archive. It only outputs the final count of hardened packages for each hardening function. Either it should run on a Tails mirror but it will not catch the newest packages from Debian, or it should be changed to keep a list of tested packages (which would make other graphs creation very easy).

For the second goal (for me it looks like the most important) I don’t think this script can be used without a lot of modifications but the hardening status of a package should be extractable from the lintian tests and, except for the fact that lintian.debian.org is down right now, I think it should be a more up to date and easier to set up source of information.

#7 Updated by intrigeri 2014-07-21 21:37:39

  • Description updated

#8 Updated by intrigeri 2014-07-21 21:49:44

Glad someone is interested! :)

> Either it should run on a Tails mirror but it will not catch the newest packages from
> Debian, or it should be changed to keep a list of tested packages (which would make
> other graphs creation very easy).

There’s no such thing as a Tails mirror yet (Feature #5926). If Kees doesn’t mind generating the graphs himself, and we provide a patch so that his scripts take a list of packages as input, then problem solved, no? In any case, right, the solution you’re describing should work too, and indeed allows to reach the second goal quite easily.

I like both solutions, but I strongly prefer the one that someone (you?) will implement :)

> For the second goal (for me it looks like the most important) I don’t think this
> script can be used without a lot of modifications but the hardening status of
> a package should be extractable from the lintian tests and, except for the fact that
> lintian.debian.org is down right now, I think it should be a more up to date and
> easier to set up source of information.

Ah, interesting idea. I didn’t think of it. I see no Lintian test for PIE and bindnow, though: it seems that it only tests for the default set of hardening flags. If it did test PIE and bindnow too, then indeed, it would be cheaper to do that, compared to doing the same job again from the .deb’s found on a Debian mirror.

Does it test libraries too, or only executable binaries?

#9 Updated by intrigeri 2014-08-26 18:17:12

intrigeri wrote:
> I see no Lintian test for PIE and bindnow, though: it seems that it only tests for the default set of hardening flags. If it did test PIE and bindnow too, then indeed, it would be cheaper to do that, compared to doing the same job again from the .deb’s found on a Debian mirror.

Actually, Lintian does have checks for PIE and bindnow too, internally. But it has no corresponding “tag”, which means that these checks don’t translate into anything user-visible, if I understand it clearly. Reported as Debian bug #759363.

> Does it test libraries too, or only executable binaries?

https://lintian.debian.org/tags/hardening-no-relro.html includes .so files, so this should be good enough.

#10 Updated by intrigeri 2014-08-27 00:43:34

Actually, lintian.debian.org doesn’t provide the machine-readable output we would need: https://bugs.debian.org/759403

#11 Updated by intrigeri 2014-08-27 00:45:00

  • related to deleted (Feature #7424: Have Tails based on Jessie building and starting)

#12 Updated by sajolida 2014-11-03 20:57:50

  • Target version set to Hardening_M1

#13 Updated by sajolida 2015-09-22 07:53:18

  • Target version deleted (Hardening_M1)

#14 Updated by intrigeri 2016-01-23 12:18:18

intrigeri wrote:
> Actually, lintian.debian.org doesn’t provide the machine-readable output we would need: https://bugs.debian.org/759403

Lintian 2.5.40 added hardening-no-bindnow and hardening-no-pie tags, so https://lintian.debian.org/ should now have all the info we need. Next step is probably to find out how we can get the information we need from that website without scraping web pages.

#15 Updated by intrigeri 2016-07-05 08:11:24

  • Description updated

#16 Updated by intrigeri 2016-07-05 08:22:05

  • File packages-missing-hardening added
  • Status changed from Confirmed to In Progress
  • Assignee set to intrigeri
  • % Done changed from 0 to 10

The attached script implements the 2nd part (“list of packages that still lack hardening”), building on top of the machine-readable Lintian reports (draft code submitted for comments on https://bugs.debian.org/759403). Usage info:

# Usage: packages-missing-hardening BUILD_MANIFEST LINTIAN_YAML_REPORT
#
# Prints to stdout a YAML-encoded list of binary packages listed
# in a Tails build manifest, that lack some hardening build flags
# according to a Lintian report.

Once lintian.debian.org exports the YAML report file we need, on a rainy day I’ll have this script run daily on our infra and publish the output file over HTTP.

But it doesn’t address the 1st bullet point (“statistics about the current status, to track progress of the proportion of the binaries shipped in Tails are hardened”): the only stat we get is the number of packages that ship at least one not-100%-hardened file, without detailed info about each hardening flag. But at this point I don’t care anymore about the historical perspective, and just want to know what still needs to be fixed, so I’ll repurpose this ticket to skip the stats part.

#17 Updated by intrigeri 2016-07-05 08:31:00

intrigeri wrote:
> The attached script implements the 2nd part (“list of packages that still lack hardening”), […]

… and here’s a current output file (I’ve used the build manifest from the last successful feature/stretch build), that can already be used to fix stuff in Debian!

#18 Updated by intrigeri 2016-07-05 08:33:18

  • Type of work changed from Code to Sysadmin

#19 Updated by intrigeri 2016-08-18 08:47:31

  • Starter deleted (Yes)

#20 Updated by intrigeri 2019-03-07 15:50:46

  • Assignee deleted (intrigeri)

It would be really nice if someone worked on this. All the work can be done in the open and no need for specific Tails credentials. Happy to help someone get started :)

#21 Updated by intrigeri 2019-08-03 17:04:51

Update: looks like we can now query the info we need directly from UDD (https://bugs.debian.org/759403#60).

#22 Updated by intrigeri 2019-08-04 14:50:58

intrigeri wrote:
> Update: looks like we can now query the info we need directly from UDD (https://bugs.debian.org/759403#60).

Attaching PoC, that must be run on a host that has access to UDD such as coccia.debian.org:

  • input: a Tails build manifest named latest.build-manifest
  • outputs the list of binary packages (one per line) with a Lintian tag whose name starts with “hardening”

I’m also attaching the output generated using a recent feature/buster build manifest.

For now, one can then use e.g. https://udd.debian.org/lintian/?email1=&email2=&email3=&packages=evince&ignpackages=&format=html#all to find out which exact hardening flag is missing on package X. It would be nicer if the script output this info itself.