Feature #15052
Adding pdf-redact-tools
100%
Description
In the [context of SecureDrop](https://github.com/freedomofpress/securedrop/issues/2643) there is a need for tools to handle PDF files, in addition to [MAT](https://0xacab.org/mat/mat).
I’m willing to add [pdf-redact-tools](https://github.com/firstlookmedia/pdf-redact-tools/graphs/contributors) to Debian GNU/Linux and to the backports so it can be included in tails eventually.
However, if adding pdf-redact-tools is something you do not want to ship by default for some reason, I will find another way to make it available for SecureDrop.
What do you think ?
Files
Subtasks
Related issues
|
Related to Tails - |
Rejected | 2018-03-12 | |
|
Blocked by Tails - |
Resolved | 2017-12-29 |
History
#1 Updated by intrigeri 2017-12-13 05:40:40
- Assignee set to dachary
- QA Check set to Info Needed
> In the [context of SecureDrop](https://github.com/freedomofpress/securedrop/issues/2643) there is a need for tools to handle PDF files, in addition to [MAT](https://0xacab.org/mat/mat).
I had a quick look. While there’s definitely some overlap between pdf-redact-tools and MAT’s mission, the UX and functionality pdf-redact-tools provides would not really fit in MAT, so it seems to be a valid candidate and not a duplicate tool.
> I’m willing to add [pdf-redact-tools](https://github.com/firstlookmedia/pdf-redact-tools/graphs/contributors) to Debian GNU/Linux and to the backports so it can be included in tails eventually.
Much appreciated.
> However, if adding pdf-redact-tools is something you do not want to ship by default for some reason, I will find another way to make it available for SecureDrop.
OK, let’s see:
- I’m assuming you wrote “by default” because you’re talking about Tails devices that are used offline, in an air-gapped environment. Correct? FYI in the next major Tails release (3.6, 2018-03-13), our Additional software packages feature will work fully offline, and then the SecureDrop’s air-gapped Tails device provisioning could prepare a persistent volume, pre-configured to install pdf-redact-tools, and with a persistent APT cache seeded to include pdf-redact-tools and its missing dependencies. Actually one of our major motivations to make this feature work offline was precisely to better support air-gapped usage scenarios. For this to work, pdf-redact-tools needs to be available in Debian (or in an additional APT repository configured in your persistent volume. Would this help you drop your “by default” requirement? If not, independently from pdf-redact-tools, I would personally love to better understand why this would not help in the general case :)
- The PPA’s dependencies are:
Depends: python:any (<< 2.8), python, python:any (>= 2.7~), imagemagick, libimage-exiftool-perl. Are they correct? The easiest way to test this is in a minimal Debian chroot, e.g. with pbuilder or using debootstrap manually. If they are, cool: we have them all in Tails currently (until we manage to drop Python 2.x from the ISO, but we’re not there yet). So adding pdf-redact-tools would basically have zero impact on the ISO size.
#2 Updated by dachary 2017-12-13 07:55:36
> so it seems to be a valid candidate and not a duplicate tool.
Thanks for the quick assessment, this is encouraging :-)
#3 Updated by intrigeri 2017-12-13 08:10:49
> Thanks for the quick assessment, this is encouraging :-)
:)
Meta: I’ve heard that SecureDrop is considering moving from Tails to QubesOS. I want to make sure this is done for good reasons (there are surely plenty of good reasons to use QubesOS!) and not primarily because our two teams failed at communicating wrt. your needs and how we should prioritize them. Let’s not discuss this right here, but I’d love to read a summary on tails-dev@boum.org about the major issues SecureDrop developers and operators are experiencing with Tails.
#4 Updated by dachary 2017-12-13 08:33:26
> I’m assuming you wrote “by default” because you’re talking about Tails devices that are used offline, in an air-gapped environment. Correct?
I should have detailed a little more the SecureDrop use case, let me try to fix that. We are indeed talking about the installation and upgrade of an airgap machine based on tails. The journalists using SecureDrop and the SecureDrop admin helping them also use tails but it is connected to the net and therefore less challenging .
The journalists uses the airgap tails to work on documents they received and they need tools to do so, pdf-redact-tools is one of them but not the only one. There are dozens of SecureDrop installations, all using tails, and the vast majority use the tails with no additional software, even when tails is not airgap . Although they could install additional packages /#additional_software it turns out they do not. They are aware it is possible but maybe it is too complicated for them to feel comfortable ? Maybe they should be trained but the lack of resources prevents that from happening ? Whatever the reason the fact remains: it does not happen.
IMHO we (SecureDrop developers) need to make an effort upstream (i.e. tails + Debian GNU/Linux) to provide additional software without requiring SecureDrop admins to add them themselves to the vanilla tails key. This is why I filed this issue and this is what I mean by default. So the existing SecureDrop users will benefit from the new tool the next time they upgrade their tails key (airgap or not).
#5 Updated by intrigeri 2017-12-13 08:47:34
> Although they could install additional packages /#additional_software it turns out they do not. They are aware it is possible but maybe it is too complicated for them to feel comfortable ? Maybe they should be trained but the lack of resources prevents that from happening ? Whatever the reason the fact remains: it does not happen.
I have good news for you: we have identified this usability problem a while ago (currently one needs to do stuff as root on the command line to use Additional Software Packages) and our plans for 2018 include fixing it. We have funding for this project so it’ll definitely happen. See Feature #14568 and subtasks + blueprints if you want to follow along :)
#6 Updated by dachary 2017-12-13 09:22:19
Thanks for the nice discussion :-)
#7 Updated by dachary 2017-12-28 20:33:41
Packaging of pdf-redact-tools is tracked upstream at https://github.com/firstlookmedia/pdf-redact-tools/issues/26
#8 Updated by dachary 2017-12-30 19:30:20
Now that pdf-redact-tools is in sid, I suppose the next step is to submit a patch to the devel branch, like so:
From de8dae347bbd49b241606cad9d83060ee33d246e Mon Sep 17 00:00:00 2001
From: Loic Dachary <loic@dachary.org>
Date: Sat, 30 Dec 2017 20:13:23 +0100
Subject: [PATCH] Add pdf-redact-tools to tails pre-installed software
While there's definitely some overlap between pdf-redact-tools and
MAT's mission, the UX and functionality pdf-redact-tools provides
would not really fit in MAT.
It complements MAT functions and is be useful to journalists working
with sensitive documents.
refs: <del><a class='issue tracker-2 status-3 priority-4 priority-default closed child' href='/code/issues/15052' title='Adding pdf-redact-tools'>Feature #15052</a></del>
---
config/chroot_local-packageslists/tails-common.list | 1 +
1 file changed, 1 insertion(+)
diff --git a/config/chroot_local-packageslists/tails-common.list b/config/chroot_local-packageslists/tails-common.list
index 98f8fe15cf..e6e654fad4 100644
--- a/config/chroot_local-packageslists/tails-common.list
+++ b/config/chroot_local-packageslists/tails-common.list
@@ -149,6 +149,7 @@ live-tools
lvm2
macchanger
mat
+pdf-redact-tools
mesa-utils
monkeysign
monkeysphere
--
2.11.0
Reading the contributor guide I found the merge policy from which I (hopefully ;-) got the branch name right. However it seems to be a guide for people who are trusted to push to the tails repository. Would you be so kind as to point me to the URL that explains the process for first time contributors ?
Cheers
#9 Updated by intrigeri 2018-01-02 08:24:04
> Now that pdf-redact-tools is in sid, I suppose the next step is to submit a patch to the devel branch, like so:
Thanks! I’ll take a first quick look (as part of my “Welcome new contributors” job) before I hand it over to the release manager for our next major release
> Reading the contributor guide I found the merge policy from which I (hopefully ;-) got the branch name right. However it seems to be a guide for people who are trusted to push to the tails repository. Would you be so kind as to point me to the URL that explains the process for first time contributors ?
#10 Updated by intrigeri 2018-01-02 08:28:35
- Status changed from New to In Progress
- Assignee changed from dachary to intrigeri
- Target version set to Tails_3.6
- % Done changed from 0 to 20
- QA Check changed from Info Needed to Ready for QA
- Type of work changed from Debian to Code
#11 Updated by dachary 2018-01-02 08:59:30
> Sure: https://tails.boum.org/contribute/how/code/#submit
Oh… I wonder how I missed it. Thanks for the kind reminder.
#12 Updated by intrigeri 2018-01-02 10:16:19
- blocked by
Bug #15132: devel branch FTBFS since aufs-dkms 4.14 is in sid added
#13 Updated by intrigeri 2018-01-02 12:15:06
- Assignee changed from intrigeri to dachary
- QA Check changed from Ready for QA to Dev Needed
- Feature Branch set to feature/15052-pdf-redact-tools
The branch FTBFS because our APT pinning prevents installing packages from sid unless an exception is explicitly added, see config/chroot_apt/preferences.
#14 Updated by dachary 2018-01-02 14:06:20
Updated the feature branch and included an exception to config/chroot_apt/preferences does that look ok ?
#15 Updated by intrigeri 2018-01-02 15:16:31
> Updated the feature branch and included an exception to config/chroot_apt/preferences does that look ok ?
Yes, this does look better! Please follow the “submit your work” doc once you’ve successfully built an ISO and tested pdf-redact-tools in there. You’ll probably have to merge the branch for Bug #15132 in order to build an ISO because branches based on devel currently FTBFS.
Also, I think this deserves to be mentioned next to MAT in wiki/src/doc/about/features.mdwn.
#16 Updated by dachary 2018-01-03 20:38:39
- File screenshot.png added
built the image, ran it and verified it contains a working pdf-redact-tools
Also the feature branch was updated with a commit that includes a mention of pdf-redact-tools next to MAT.
What should I do next ?
#17 Updated by dachary 2018-01-03 21:20:42
For the record the pdf-redact-tools dependencies are imagemagick and libimage-exiftool-perl and python, all of which are already in tail. The pdf-readact-tools .deb is ~5kb.
#18 Updated by intrigeri 2018-01-04 07:24:40
> built the image, ran it and verified it contains a working pdf-redact-tools
\o/
> Also the feature branch was updated with a commit that includes a mention of pdf-redact-tools next to MAT.
> What should I do next ?
Follow https://tails.boum.org/contribute/merge_policy/#submit :)
I think Target version should be the next major release (3.6).
#19 Updated by dachary 2018-01-04 10:51:44
- Assignee deleted (
dachary) - QA Check changed from Dev Needed to Ready for QA
- Feature Branch changed from feature/15052-pdf-redact-tools to https://gitlab.com/dachary/tails/commits/feature/15052-pdf-redact-tools
#20 Updated by bertagaz 2018-01-08 14:26:13
- Status changed from In Progress to Fix committed
- % Done changed from 20 to 100
- QA Check changed from Ready for QA to Pass
Merged into devel, so it will be shiped in 3.6, congrats! I’ve taken the liberty to update the URL of the homepage of the project, which seems to have changed, and document a little bit more its usage.
#21 Updated by dachary 2018-01-08 15:13:18
Thanks a lot, much appreciated :-)
#22 Updated by dachary 2018-01-08 15:18:30
- Status changed from Fix committed to In Progress
Applied in changeset commit:520acf9101f1e5a7411ce6cf44c4a95accc5e281.
#23 Updated by bertagaz 2018-01-08 15:18:30
- Status changed from In Progress to Fix committed
Applied in changeset commit:79dc57024d4391103abd28127cb5d67a5cdf8bd2.
#24 Updated by sajolida 2018-03-12 16:13:09
- related to
Bug #15398: Document pdf-redact-tools added
#25 Updated by bertagaz 2018-03-14 11:09:25
- Status changed from Fix committed to Resolved