Bug #12169

Tails ISO has $stable-proposed-updates enabled when Debian/Tails release timing don't play nicely together

Added by intrigeri 2017-01-24 15:40:38 . Updated 2017-03-09 13:59:02 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Target version:
Start date:
2017-01-24
Due date:
% Done:

100%

Feature Branch:
bugfix/12169-disable-proposed-updates
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Additional Software Packages
Deliverable for:

Description

We see this in Tails 2.10. That’s fine at ISO build time (we were using an APT snapshot that gave us essentially the latest Jessie point-release as a result) but it’s not OK at runtime: it means that Tails users who install additional packages will get it before Debian users would.

If we don’t do anything about it, this will be the case until next Tails major release (2.12). It’s too late to fix that in 2.10, but IMO we should mitigate it in 2.11.

Maybe we need to make config/chroot_local-includes/lib/live/config/1500-reconfigure-APT disable proposed-updates sources?


Subtasks


History

#1 Updated by intrigeri 2017-01-24 15:40:51

anonym, what do you think?

#2 Updated by anonym 2017-01-24 15:53:52

  • QA Check deleted (Info Needed)

intrigeri wrote:
> We see this in Tails 2.10. That’s fine at ISO build time (we were using an APT snapshot that gave us essentially the latest Jessie point-release as a result) but it’s not OK at runtime: it means that Tails users who install additional packages will get it before Debian users would.

My bad, sorry! I should have filed a follow-up ticket once I merged the branch for Feature #12124. I’ll take care of the cleanup!

> If we don’t do anything about it, this will be the case until next Tails major release (2.12). It’s too late to fix that in 2.10, but IMO we should mitigate it in 2.11.

Let’s remove it ASAP!

> Maybe we need to make config/chroot_local-includes/lib/live/config/1500-reconfigure-APT disable proposed-updates sources?

This sounds like an excellent suggestion! In fact, let’s put as many sanity checks (with sane resolutions!) as possible there, so screw-ups like this can be tolerated.

#3 Updated by intrigeri 2017-01-24 16:27:31

> I should have filed a follow-up ticket once I merged the branch for Feature #12124.

The thing is, we couldn’t simply revert “add -stable-updates” because 2.10 uses a snapshot of Debian that predates the Debian point release.
So I’m not quite sure how we could have dealt with this better, at least as long as one looks at the ISO build time topic only (which is what I did :)

>> Maybe we need to make config/chroot_local-includes/lib/live/config/1500-reconfigure-APT disable proposed-updates sources?

> This sounds like an excellent suggestion! In fact, let’s put as many sanity checks (with sane resolutions!) as possible there, so screw-ups like this can be tolerated.

Adding an automated test should be mostly trivial.

#4 Updated by anonym 2017-01-25 10:29:01

  • Status changed from Confirmed to In Progress
  • Assignee changed from anonym to intrigeri
  • % Done changed from 0 to 40
  • QA Check set to Ready for QA
  • Feature Branch set to bugfix/12169-disable-proposed-updates

Ok, I think I got it! Please review’n’merge into stable and devel!

#5 Updated by intrigeri 2017-01-27 14:23:31

I find it a bit sad to rely on a not very accurate regexp (i.e. it matches a superset of what we really want), especially in a file that has examples of more precise ones, but in practice it’s very unlikely that this is ever problematic, so I’ll cross fingers and won’t nitpick.

#6 Updated by intrigeri 2017-01-27 14:30:40

  • Assignee changed from intrigeri to anonym
  • % Done changed from 40 to 50

I figured it would take me less time to add an automated test than to test this branch by hand, so here’s the deal: feel free to merge yourself after adding / running / debugging the following (untested!) automated test:

diff --git a/features/apt.feature b/features/apt.feature
index c835d27681..70c2413719 100644
--- a/features/apt.feature
+++ b/features/apt.feature
@@ -10,6 +10,7 @@ Feature: Installing packages through APT

   Scenario: APT sources are configured correctly
     Then the only hosts in APT sources are "vwakviie2ienjx6t.onion,sgvtcaew4bxjd7ln.onion,jenw7xbd6tf7vfhp.onion,sdscoq7snqtznauu.onion"
+    And no proposed-updates APT suite is enabled

   @check_tor_leaks
   Scenario: Install packages using apt
diff --git a/features/step_definitions/apt.rb b/features/step_definitions/apt.rb
index 86f878d4a3..654d773914 100644
--- a/features/step_definitions/apt.rb
+++ b/features/step_definitions/apt.rb
@@ -11,6 +11,13 @@ Given /^the only hosts in APT sources are "([^"]*)"$/ do |hosts_str|
   }
 end

+Given /^no proposed-updates APT suite is enabled$/ do
+  $vm.file_content("/etc/apt/sources.list /etc/apt/sources.list.d/*").chomp.each_line { |line|
+    assert(not(line.match("-proposed-updates")),
+           "proposed-updates APT source found: #{line}")
+  }
+end
+
 When /^I configure APT to use non-onion sources$/ do
   script = <<-EOF
   use strict;

Fair enough? :)

#7 Updated by anonym 2017-02-17 13:26:34

  • Assignee changed from anonym to intrigeri
  • QA Check changed from Ready for QA to Info Needed

intrigeri wrote:
> I find it a bit sad to rely on a not very accurate regexp (i.e. it matches a superset of what we really want), especially in a file that has examples of more precise ones, but in practice it’s very unlikely that this is ever problematic, so I’ll cross fingers and won’t nitpick.

I want to get it right, but I fail to see what is wrong. Please clarify!

I have pushed your test too, and I’d like to see it pass once on Jenkins before considering this branch for merging.

#8 Updated by intrigeri 2017-02-17 16:11:25

  • Assignee changed from intrigeri to anonym
  • QA Check changed from Info Needed to Ready for QA

> intrigeri wrote:
>> I find it a bit sad to rely on a not very accurate regexp (i.e. it matches a superset of what we really want), especially in a file that has examples of more precise ones, but in practice it’s very unlikely that this is ever problematic, so I’ll cross fingers and won’t nitpick.

> I want to get it right, but I fail to see what is wrong. Please clarify!

/-proposed-updates/d applies to any part of an APT source line, e.g. the hostname, and not only to what this sed regexp is meant to match (i.e. the suite component).

(BTW, some day I want to switch to deb822 style sources, that are a bit more structured and nicer to parse with higher-level tools. See sources.list(5) for details if curious :)

> I have pushed your test too, and I’d like to see it pass once on Jenkins before considering this branch for merging.

Thanks!

#9 Updated by anonym 2017-02-17 17:22:42

  • Assignee changed from anonym to intrigeri

intrigeri wrote:
> > intrigeri wrote:
> >> I find it a bit sad to rely on a not very accurate regexp (i.e. it matches a superset of what we really want), especially in a file that has examples of more precise ones, but in practice it’s very unlikely that this is ever problematic, so I’ll cross fingers and won’t nitpick.
>
> > I want to get it right, but I fail to see what is wrong. Please clarify!
>
> /-proposed-updates/d applies to any part of an APT source line, e.g. the hostname, and not only to what this sed regexp is meant to match (i.e. the suite component).

Ah, sure. Fixed now.

> (BTW, some day I want to switch to deb822 style sources, that are a bit more structured and nicer to parse with higher-level tools. See sources.list(5) for details if curious :)

Looks cool, but I don’t really think it will be easier to parse since you need to keep state across read lines => pretty much rules out grep, and requires sed that most people don’t know about.

#10 Updated by intrigeri 2017-02-17 17:30:54

> Looks cool, but I don’t really think it will be easier to parse since you need to keep state across read lines => pretty much rules out grep, and requires sed that most people don’t know about.

Hence “with higher-level tools” :)

#11 Updated by anonym 2017-02-18 10:54:21

intrigeri wrote:
> > Looks cool, but I don’t really think it will be easier to parse since you need to keep state across read lines => pretty much rules out grep, and requires sed that most people don’t know about.
>
> Hence “with higher-level tools” :)

What, sed is not higher-level? :) Ok, then we’ll need pretty powerful (arbitrary queries and manipulation) libraries with nice interfaces for at least shell, Ruby (test suite) and Python (shell replacement), but really for all languages where we interact with the APT lists.
</off-topic>

Also, I had to fixup your test since the semantics of VM.file_content() has changed in the base branch (stable) which eluded both of us since we didn’t merge it in when running your test. :) Let’s see what Jenkins thinks now.

#12 Updated by intrigeri 2017-03-02 08:43:50

  • Status changed from In Progress to Fix committed
  • % Done changed from 50 to 100

Applied in changeset commit:ee16b49eb15b62277755c5b90e0561a29c9ab29c.

#13 Updated by intrigeri 2017-03-02 08:46:42

  • Assignee deleted (intrigeri)
  • QA Check changed from Ready for QA to Pass

#14 Updated by intrigeri 2017-03-02 08:46:57

  • Affected tool set to Additional Software Packages

#15 Updated by anonym 2017-03-09 13:59:02

  • Status changed from Fix committed to Resolved