Feature #10543

Document how to add APT sources using Dotfiles

Added by sajolida 2015-11-13 13:25:05 . Updated 2016-09-20 16:50:04 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Target version:
Tails_2.6
Start date:
2015-11-13
Due date:
% Done:

100%

Feature Branch:
doc/10543-dotfiles-for-apt
Type of work:
End-user documentation
Blueprint:

Starter:
Affected tool:
Deliverable for:

Description

It works, see Bug #10143. We could document this as an advanced topics and they disable them.

Subtasks

Related issues

Blocked by Tails - Bug #10143: Test whether Dotfile allow adding additional APT repositories Resolved 2015-09-01
Blocked by Tails - Bug #11635: Include additional APT sources in WhisperBack reports Resolved 2016-08-12

History

#1 Updated by sajolida 2015-11-13 13:25:41

  • Status changed from New to Confirmed
  • Assignee set to sajolida

Assigning this to me as a start but anybody should feel free to steal it from me.

#2 Updated by sajolida 2015-11-14 01:50:39

  • Subject changed from Document how to enable non-free using Dotfiles to Document how to add APT sources using Dotfiles

This could be make a general HOWTO on adding additional APT sources actually.

#3 Updated by intrigeri 2015-11-15 09:49:44

> This could be make a general HOWTO on adding additional APT sources actually.

This is quite a change of scope that is not obviously a good idea to me, so let’s document a bit why we would want to do that (I’m mostly trying to convince myself here).

I know why we want to support enabling Debian non-free APT sources (and in turn, document the process): once we disable non-free by default, this preserves backward compatibility for those of our users who need software from Debian non-free.

I can see that various organizations may want to document such things on their own (and perhaps are already doing it), so strategically it may be worth documenting it ourselves, to get a better chance to let users know about the consequences of creating Frankenstein Tails-based systems with pieces from third-party repositories added to the mix. I’d love to hear about other good reasons we may have to document this, but don’t spend hours on it.

I trust you folks will keep these concerns in mind when phrasing this HOWTO: as usual, let’s not advertise Tails as a more flexible system than what we’re willing or able to support :)

#4 Updated by sajolida 2015-11-17 03:33:53

  • blocked by Bug #10143: Test whether Dotfile allow adding additional APT repositories added

#5 Updated by sajolida 2015-11-17 03:38:32

Right, I had in mind the use case of possible derivatives. I also through that documenting how to add back the non-free section was basically explaining how to add any APT source. So I thought it was better to acknowledge this scary fact and wrap it with adequate warnings instead of pretending that we’re only documenting how to add back slightly more innocuous non-free.

Anyway, I’ll keep this in mind and we can discuss this again based on a draft.

#6 Updated by sajolida 2015-11-17 03:40:12

My current snippet for that:

chmod 755 /live/persistence/TailsData_unlocked/apt
mkdir /live/persistence/TailsData_unlocked/apt/sources
grep '/etc/apt/sources.list.d\ssource=apt/sources,link' persistence.conf || \
    echo '/etc/apt sources.list.d source=apt/sources,link' >> persistence.conf

#7 Updated by intrigeri 2015-11-17 06:50:57

> My current snippet for that:

Shout when you want someone to make the shell/regexp bits more robust.

#8 Updated by sajolida 2016-07-17 11:05:31

  • Assignee changed from sajolida to intrigeri
  • QA Check set to Ready for QA
  • Feature Branch set to doc/10543-dotfiles-for-apt

So here is the doc. Please have a look and fix my shell.

Also, you’ll see that I mention non-free sections as an example of why using this, so maybe we want to remove the non-free sections in the same branch or at least merge everything at the same time.

#9 Updated by sajolida 2016-07-17 13:46:36

  • Status changed from Confirmed to In Progress

Applied in changeset commit:af19e7cc459443c74b16b96d2f7988d60414fb1e.

#10 Updated by intrigeri 2016-07-18 05:50:10

> So here is the doc. Please have a look and fix my shell.

Sure! Thanks for working on this. Timing will depend on:

> Also, you’ll see that I mention non-free sections as an example of why using this, so maybe we want to remove the non-free sections in the same branch or at least merge everything at the same time.

So if I got it right, merging this is blocked by some code work on the parent ticket (Bug #10130) that has no assignee yet, and that you suggest I handle. I’m totally fine with taking it, but best case that’ll be for 2.6 (not suitable for 2.5), and sadly I can’t promise any such thing, and 2.8 might be more likely: I already have quite a few WIP things I’d like to complete in time for 2.6. So I suggest you try and find someone else to do the code bits earlier than what I would be able to (I’m thinking of muri, segfault, hybridwipe for example). If that fails, then feel free to assign the parent ticket to me with Target version = 2.6. Fair enough?

#11 Updated by intrigeri 2016-07-18 09:11:48

  • Assignee changed from intrigeri to sajolida
  • % Done changed from 0 to 60
  • QA Check changed from Ready for QA to Dev Needed

“Create files for your additional `sources.list` entries […]” doesn’t tell anything about ownership and permissions. Worst case, this can either harm usability (e.g. apt-cache search as non root), if the permissions are too strict; or harm security (e.g. local priv. escalation to root), if the permissions are too loose. I think we need to document the exact ownership and permissions one must use for those files. That’s what I was referring to in Bug #10143#note-6.

Other than that, all this looks good, just some questions / nitpicking:

  • Files in /etc/apt/sources.list.d can’t have arbitrary names. Do we want to rely purely on the example you provide to communicate this fact, or should we point to the sources.list(5) manpage that has the details, or duplicate parts of it?
  • On this ticket we agreed previously that we needed scary warnings. I assume you relied on the ones on top of the page to fit the bill. This currently works for me, but I’m concerned that sooner or later, we’ll add a HTML anchor, point directly to the new “Configuring additional APT repositories”, and lose the warnings. This is probably not worth moving the new content to a new page, but perhaps add a comment stating that why adding an anchor would be a bad idea? (Of course people will point to the anchors generated by the TOC, but well, I don’t think that we’re trying to protect against that elsewhere in our doc so I’ll disregard that other problem. One can always be partially quoted as soon as one writes something.)

#12 Updated by sajolida 2016-07-21 08:55:29

+1 on the permissions. Fixed in commit:2fe344e.

So far I put these files under apt/sources which seems super logical but
/lib/live/mount/persistence/TailsData_unlocked/apt itself is chmod 600
(which everything in it is world readable). So unless we change this,
the amnesia user won’t be able to read these sources.list and we’ll get
Ignoring 'non-free.list' in directory '/etc/apt/sources.list.d/' as it is not a regular file on apt-cache search.

For the moment, I’ll move apt/sources outside of apt in d134ddb.

But we could also change the ownership of
/lib/live/mount/persistence/TailsData_unlocked/apt to add at least +x.
Then we would have to document how to do this change on this page to
cope with people who already have a persistence will stricter permissions.

Would you prefer that?

> * Files in /etc/apt/sources.list.d can’t have arbitrary names. Do we want to rely purely on the example you provide to communicate this fact, or should we point to the sources.list(5) manpage that has the details, or duplicate parts of it?

Fixed in commit:b5d2f8c.

> * On this ticket we agreed previously that we needed scary warnings. I assume you relied on the ones on top of the page to fit the bill. This currently works for me, but I’m concerned that sooner or later, we’ll add a HTML anchor, point directly to the new “Configuring additional APT repositories”, and lose the warnings. This is probably not worth moving the new content to a new page, but perhaps add a comment stating that why adding an anchor would be a bad idea? (Of course people will point to the anchors generated by the TOC, but well, I don’t think that we’re trying to protect against that elsewhere in our doc so I’ll disregard that other problem. One can always be partially quoted as soon as one writes something.)

Fixed in commit:9ca333f.

#13 Updated by sajolida 2016-07-21 08:57:58

  • Assignee changed from sajolida to intrigeri
  • QA Check changed from Dev Needed to Ready for QA

#14 Updated by intrigeri 2016-07-23 06:51:20

  • QA Check changed from Ready for QA to Pass

> +1 on the permissions. Fixed in 2fe344e.

Good!

> For the moment, I’ll move apt/sources outside of apt in d134ddb.

Better. IMO source=sources.list.d is too generic a name. How about source=apt-sources.list.d instead? Now, I won’t argue about it any further: don’t bother if you don’t like it :)

> But we could also change the ownership of /lib/live/mount/persistence/TailsData_unlocked/apt to add at least +x. Then we would have to document how to do this change on this page to cope with people who already have a persistence will stricter permissions.

> Would you prefer that?

I think I’d rather not open that can of worms, so let’s stick to the option you’ve implemented already.

> Fixed in b5d2f8c.

Good.

JFTR I’m not convinced by the terminology (“`sources.list` entries”, “your additional `sources.list`”) for various reasons, including the fact that the entries are inside these files, not the files themselves, not but surely it’s not worth putting any more time into this, so let’s keep it as-is.

> Fixed in 9ca333f.

All right!

#15 Updated by intrigeri 2016-07-23 07:12:39

  • Assignee changed from intrigeri to sajolida
  • % Done changed from 60 to 70
  • QA Check deleted (Pass)

sajolida, I’m reassigning to you so that you deal with the next step: Feature #10543#note-10.

Now, one potential blocker I hadn’t thought about earlier (I’ve just re-read the concerns I had expressed about this idea initially, since back then we had postponed the discussion until there was a draft): I want to see the content of /etc/apt/sources.list.d/* included in WhisperBack reports. This would allow frontdesk to triage more easily reports coming from Frankenstein systems that we can’t realistically support (e.g. I expect that reddit or whatnot will soon enough document how to add deb-multimedia.org and friends using dotfiles). As a developer I would appreciate if reports coming from such systems were filtered out before they’re sent onto my plate (or worst case, if frontdesk did not realize it themselves, then at least I can judge myself whether the system has been modified too much for me to support it). I realize that we are already documenting a few other ways to turn a pristine Tails into something different and potentially hard to support (e.g. installing additional packages), and that I’m setting my limit to a somewhat arbitrary place — if I had to argue about it, I would make the point that installing random software from Debian is one thing, but installing random software from non-Debian sources goes to a substantially different level, in terms of potential for breaking our stuff.

I don’t want to generalize this discussion to the broader “what exactly are we willing to support, and how do we handle the boundary in terms of UX and for frontdesk/developers” right now. I don’t want the work done on this ticket (and Bug #10130) to be blocked by our usual “oh but if we want to fix this, then we also need to fix that, and the world as a whole!” tendency. That’s why I’m proposing a very simple ad-hoc, and hopefully consensual, solution to the problem I have with this branch.

What do you think?

#16 Updated by sajolida 2016-08-03 14:13:36

  • Assignee changed from sajolida to intrigeri
  • QA Check set to Ready for QA

I changed the name of the folder and the “entry” terminology. Please review again.

I’m fine with adding the content of these extra files to WhisperBack reports. I had a very quick look to the WhisperBack config but couldn’t spot how to include a whole folder (and not a set of files). Also, it might be better to include only the additional APT sources and not the default APT sources.

#17 Updated by sajolida 2016-08-08 08:04:53

> Better. IMO source=sources.list.d is too generic a name. How about
> source=apt-sources.list.d instead? Now, I won’t argue about it any
> further: don’t bother if you don’t like it :)

Ok. `apt-sources.list` is more ugly but less ambiguous, so let’s go for
this version. See 4c1d379.

> JFTR I’m not convinced by the terminology (“`sources.list` entries”,
> “your additional `sources.list`”) for various reasons, including the
> fact that the entries are inside these files, not the files
> themselves, not but surely it’s not worth putting any more time into
> this, so let’s keep it as-is.

You’re right. I initially used “entry” because I saw it in `man
sources.lists` but actually I misread and entries are written to
files. So I tried to fix this in 39e3c73.

#18 Updated by intrigeri 2016-08-11 12:40:26

> See commit:4c1d379.
> […]
> So I tried to fix this in 39e3c73.

Both look good!

#19 Updated by intrigeri 2016-08-11 12:49:42

  • Assignee changed from intrigeri to sajolida
  • QA Check changed from Ready for QA to Dev Needed

> I changed the name of the folder and the “entry” terminology. Please review again.

ACK.

> I’m fine with adding the content of these extra files to WhisperBack reports. I had a very quick look to the WhisperBack config but couldn’t spot how to include a whole folder (and not a set of files).

At first glance, one way to do it would be to add a debug_directory function in tails-debugging-info, that would call list all files in a directory, non-recursively, with a long listing (ls -l) to get metadata, and call debug_file on each of them.

> Also, it might be better to include only the additional APT sources and not the default APT sources.

Excellent idea! I believe that we can achieve this by calling debug_directory (as defined above) on /live/persistence/TailsData_unlocked/apt-sources.list.d.

I could do that myself, under the conditions I’ve stated on Feature #10543#note-10 for other “Code” work that goes together with the great documentation work you’ve done. So I’m reassigning to you so that you deal with next Redmine & coder-finding steps, and come back to me later if needed.

#20 Updated by sajolida 2016-08-12 06:10:57

  • Assignee changed from sajolida to intrigeri
  • QA Check changed from Dev Needed to Info Needed

Ok, I wrote them for Bug #10130 (except hybridwipe because I’m not sure what email to use). I created Bug #11635 about the WhisperBack improvement. I didn’t put blocking or parenthood relationships between Bug #10130 and Bug #11635 because you said “potential blocker”. I’ll let you adjust to your taste.

#21 Updated by intrigeri 2016-08-12 07:37:41

  • blocked by Bug #11635: Include additional APT sources in WhisperBack reports added

#22 Updated by intrigeri 2016-08-12 07:41:44

> Ok, I wrote them for Bug #10130 (except hybridwipe because I’m not sure what email to use). I created Bug #11635 about the WhisperBack improvement.

Excellent, thanks!

> I didn’t put blocking or parenthood relationships between Bug #10130 and Bug #11635 because you said “potential blocker”. I’ll let you adjust to your taste.

IMO Bug #11635 blocks Feature #10543 (that is otherwise basically done), which itself blocks Bug #10130 ⇒ I’ve expressed that via Redmine.

#23 Updated by intrigeri 2016-08-14 08:09:04

  • Assignee changed from intrigeri to sajolida
  • QA Check deleted (Info Needed)

#24 Updated by intrigeri 2016-08-27 02:21:52

  • Assignee changed from sajolida to anonym
  • Target version set to Tails_2.6
  • QA Check set to Ready for QA

#25 Updated by anonym 2016-08-28 11:36:03

  • Status changed from In Progress to Fix committed
  • Assignee deleted (anonym)
  • % Done changed from 70 to 100
  • QA Check changed from Ready for QA to Pass

#26 Updated by anonym 2016-09-20 16:50:04

  • Status changed from Fix committed to Resolved