Feature #8702

Write regression test for Unsafe Browser update checks

Added by anonym 2015-01-14 16:47:21 . Updated 2015-02-24 22:52:46 .

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Test suite
Target version:
Start date:
2015-01-14
Due date:
% Done:

100%

Feature Branch:
kytv:test/8702-unsafe-browser-upgrade-checks
Type of work:
Code
Blueprint:

Starter:
Affected tool:
Unsafe Browser
Deliverable for:

Description

We should monitor all network connections when starting the Unsafe Browser and make sure no hosts are connected to that are in control of the Tor Project (e.g. check.torproject.org so we don’t reintroduce Bug #8694). Reverse DNS of the hosts’ IP addresses might be helpful here. We probably have to wait a few minutes so an update check actually is sent so it should be researched how long we minimally have to wait.

The best would be if the Unsafe Browser doesn’t connect to any hosts at all on normal startup (its start page is a local html file) but who knows what craziness a Firefox-based browser may be up to. As a bonus, this would also be way easier to write an automated test for since we can use the existing “all Internet traffic has only flowed through Tor” step, so this approach should probably be investigated first.


Subtasks


Related issues

Related to Tails - Bug #8694: Upgrade notification to Firefox 4.0.3 in the Unsafe Browser Resolved 2015-01-14
Blocked by Tails - Feature #7821: Automatic tests for Tor Resolved 2014-08-26

History

#1 Updated by anonym 2015-01-14 16:47:35

  • related to Bug #8694: Upgrade notification to Firefox 4.0.3 in the Unsafe Browser added

#2 Updated by intrigeri 2015-01-14 17:07:08

  • blocks #8538 added

#3 Updated by intrigeri 2015-01-14 17:08:00

I would check reverse DNS for {.,}torproject.org and {.,}boum.org.

#4 Updated by kytv 2015-02-10 02:19:14

  • Status changed from Confirmed to In Progress

#5 Updated by kytv 2015-02-15 18:18:22

  • Target version changed from Tails_1.3 to Tails_1.3.2

#6 Updated by kytv 2015-02-15 20:15:17

  • Target version changed from Tails_1.3.2 to Tails_1.3

#7 Updated by kytv 2015-02-18 22:54:35

  • Assignee changed from kytv to anonym
  • % Done changed from 0 to 30
  • QA Check set to Ready for QA
  • Feature Branch set to kytv:test/8702-unsafe-browser-upgrade-checks

Perhaps this is a decent way to check this…

#8 Updated by anonym 2015-02-20 11:02:50

  • Assignee changed from anonym to kytv
  • QA Check changed from Ready for QA to Dev Needed

kytv wrote:
> Perhaps this is a decent way to check this…

> + cmd = execute("echo '#{line}' | tee -a '#{file}'", user)

I’m curious if there’s a benefit with this approach over “echo ‘#{line}’ >> ‘#{file}’”.

> +But /^update checking is disabled in the Unsafe Browser$/ do

I think I’d prefer “… in the Unsafe Browser’s configuration”. To distinguish it from actually the application and verifying that it is disabled, which is what this whole scenario does, in total.

> + Scenario: The Unsafe Browser will not make any connections to the Internet which are not user initiated
> […]
> + And all Internet traffic has only flowed through Tor

This last step is what verifies that no connections that aren’t user initiated are made. However, that step is a bit simplistic: it checks that we only contact hosts that are either a Tor relay (from the consensus) or a Tor authority. So if check.torproject.org or whatever server that need to be contacted for browser updates runs on a host that also run a Tor relay/authority, the test is incorrect. My bad since I didn’t think about this when writing my suggestion in the ticket description.

However, we have a pretty simple way to count packets by looking at the clearnet user’s exception rule in the iptables -L -n -v output. Even better: in test/7821-tor (Feature #7821) I’ve added an iptables output parser that easily can be extended to also get the packet count. With that we can create a the clearnet user has sent no packets in the clear to the Internet step (an improvement of the step name is welcome!) which we run before starting the Unsafe Browser, and as the last step, and then we’re sure the scenario is correct. I guess we can still keep “all Internet traffic has only flowed through Tor”.

So, yeah, I think I’ll propose that you rebase your branch on test/7821-tor (or just merge it) and do this. Then Feature #7821 will block this ticket, which potentially could block it from getting into 1.3, but I think that’s the smartest option. How does that sound?

One more thing about this scenario:
> + Given a computer

This will make us drop the background snapshot, and do a full reboot. I don’t see why we need this, but perhaps you though it necessary for the “I capture all network traffic” step? Any way, it isn’t, we can just start sniffing the traffic before we start the Unsafe Browser and that should be fine, since we in this scenario only care about the traffic it generates.

#9 Updated by kytv 2015-02-20 20:14:58

anonym wrote:
> kytv wrote:
> > Perhaps this is a decent way to check this…
>
> > + cmd = execute("echo '#{line}' | tee -a '#{file}'", user)
>
> I’m curious if there’s a benefit with this approach over “echo ‘#{line}’ >> ‘#{file}’”.

I don’t know, I’ve just seen a few user-facing instructions that use tee so I tried it for a bit. echo "blah" >> somefile is more of my “style”. I’ll revert back to >>.

> > +But /^update checking is disabled in the Unsafe Browser$/ do
>
> I think I’d prefer “… in the Unsafe Browser’s configuration”. To distinguish it from actually the application and verifying that it is disabled, which is what this whole scenario does, in total.

Yes, that’s nicer.

> > + Scenario: The Unsafe Browser will not make any connections to the Internet which are not user initiated

Rebasing on test/7821-tor and parsing iptables seems like a good plan.

> So, yeah, I think I’ll propose that you rebase your branch on test/7821-tor (or just merge it) and do this. Then Feature #7821 will block this ticket, which potentially could block it from getting into 1.3, but I think that’s the smartest option. How does that sound?

Perfect. :)

> One more thing about this scenario:
> > + Given a computer
>
> This will make us drop the background snapshot, and do a full reboot. I don’t see why we need this, but perhaps you though it necessary for the “I capture all network traffic” step?

I did.

> Any way, it isn’t

Excellent, I’ll remove.

> , we can just start sniffing the traffic before we start the Unsafe Browser and that should be fine, since we in this scenario only care about the traffic it generates.

As always, your guidance is appreciated. :)

#10 Updated by kytv 2015-02-20 20:17:04

#11 Updated by intrigeri 2015-02-20 20:28:49

> Rebasing on test/7821-tor and parsing iptables seems like a good plan.

Not sure, given it’s unclear when we can merge that branch due to what looks like a regression in the remote shell handling. (Let’s not discuss that here, though :)

#12 Updated by kytv 2015-02-21 00:08:54

intrigeri wrote:
> > Rebasing on test/7821-tor and parsing iptables seems like a good plan.
>
> Not sure, given it’s unclear when we can merge that branch due to what looks like a regression in the remote shell handling. (Let’s not discuss that here, though :)

Hmm. How about this then?

Two branches:

  • one rebased on test/7821-tor, to be merged when test/7821-tor is ready to be merged
  • one based on testing

In the one based on the testing branch I can do quick’n’dirty iptables parsing, perhaps something like this? (not yet tested)

Then /^the clearnet user has (|not)sent packets to the Internet$/ do |sent|
  next if @skip_steps_while_restoring_background
  pkts = 0 
  uid = @vm.execute_successfully('id -u clearnet').stdout.chomp.to_i
  iptables_out = @vm.execute_successfully("iptables -vnL OUTPUT").stdout.chomp
  iptables_out.split("\n").each do |line|
    m = /(\d+).+owner UID match '#{uid}'.+/.match(line)
    if !m.nil?
      pkts = pkts + m[1].to_i
    end 
  end 

  case sent
  when ''
    assert(pkts > 0, "Packets have not gone out to the internet.")
  when 'not'
    assert_equal(pkts, 0, "Packets have gone out to the internet.")
  end
end

I’ll proceed with both ways. If nothing else this will be an opportunity to learn. :)

#13 Updated by kytv 2015-02-21 02:36:35

  • Assignee changed from kytv to anonym
  • QA Check changed from Dev Needed to Ready for QA

This is what I came up with since the review:

diff --git a/features/step_definitions/unsafe_browser.rb b/features/step_definitions/unsafe_browser.rb
index bcccafa..665c5f3 100644
--- a/features/step_definitions/unsafe_browser.rb
+++ b/features/step_definitions/unsafe_browser.rb
@@ -207,8 +207,28 @@ Then /^I configure the Unsafe Browser to check for updates more frequently$/ do
   @vm.file_append(prefs, 'pref("app.update.interval", 5);')
 end

-But /^update checking is disabled in the Unsafe Browser$/ do
+But /^checking for updates is disabled in the Unsafe Browser's configuration$/ do
   next if @skip_steps_while_restoring_background
   prefs = '/usr/share/tails/unsafe-browser/prefs.js'
   assert(@vm.file_content(prefs).include?('pref("app.update.enabled", false)'))
 end
+
+Then /^the clearnet user has (|not )sent packets out to the Internet$/ do |sent|
+  next if @skip_steps_while_restoring_background
+  pkts = 0
+  uid = @vm.execute_successfully("id -u clearnet").stdout.chomp.to_i
+  iptables_out = @vm.execute_successfully("iptables -vnL OUTPUT").stdout.chomp
+
+  iptables_out.split("\n").each do |line|
+    m = /(\d+).+owner UID match \b#{uid}\b/.match(line)
+    if !m.nil?
+      pkts = pkts + m[1].to_i
+    end
+  end
+  case sent
+  when ''
+    assert(pkts > 0, "Packets have not gone out to the internet.")
+  when 'not'
+    assert_equal(pkts, 0, "Packets have gone out to the internet.")
+  end
+end
diff --git a/features/unsafe_browser.feature b/features/unsafe_browser.feature
index 4fc3345..ac7c095 100644
--- a/features/unsafe_browser.feature
+++ b/features/unsafe_browser.feature
@@ -97,23 +97,22 @@ Feature: Browsing the web using the Unsafe Browser
     When I successfully start the Unsafe Browser
     And I open the address "https://check.torproject.org" in the Unsafe Browser
     Then I see "UnsafeBrowserTorCheckFail.png" after at most 60 seconds
+    And the clearnet user has sent packets out to the Internet

   Scenario: The Unsafe Browser cannot be configured to use Tor and other local proxies.
     When I successfully start the Unsafe Browser
     Then I cannot configure the Unsafe Browser to use any local proxies

   Scenario: The Unsafe Browser will not make any connections to the Internet which are not user initiated
-    Given a computer
-    And I capture all network traffic
-    And I start Tails from DVD and I login
-    And GNOME has started
+    Given I capture all network traffic
     And Tor is ready
     And I configure the Unsafe Browser to check for updates more frequently
-    But update checking is disabled in the Unsafe Browser
+    But checking for updates is disabled in the Unsafe Browser's configuration
     When I successfully start the Unsafe Browser
     Then the Unsafe Browser has started
     And I wait between 60 and 120 seconds
     And all Internet traffic has only flowed through Tor
+    And the clearnet user has not sent packets out to the Internet

   Scenario: Starting the Unsafe Browser without a network connection results in a complaint about no DNS server being configured
     Given a computer

It seems to do what it’s supposed to as demonstrated with the ‘anti-test’ when going to https://check.torproject.org.

If this is acceptable and merge-worthy, when test/7821-tor is merged I can modify these steps to go use its (test/7821-tor’s) iptables_parse.

#14 Updated by kytv 2015-02-21 11:24:52

…and now I’ve also got the version rebased on test/7821-tor using iptables_parse at git://repo.or.cz/tails/kytv.git in the test/8702-unsafe-browser-upgrade-checks-rebased-7821-tor branch

#15 Updated by Tails 2015-02-23 09:50:11

  • Status changed from In Progress to Fix committed
  • % Done changed from 30 to 100

Applied in changeset commit:1b6386c5b1aadb8eaf58f974aa69654b14b07d53.

#16 Updated by anonym 2015-02-23 09:54:48

  • Assignee deleted (anonym)
  • QA Check changed from Ready for QA to Pass

kytv wrote:
> …and now I’ve also got the version rebased on test/7821-tor using iptables_parse at git://repo.or.cz/tails/kytv.git in the test/8702-unsafe-browser-upgrade-checks-rebased-7821-tor branch

This is the one I merged, since test/7821-tor was merged too. Woo!

I also pushed a tiny fix with commit 6bae7df. The !VAR.nil? construction isn’t needed (and is pretty hard to read) since nil is treated as false and everything nonnil, nonfalse is treated as true (cf. None in python).

#17 Updated by BitingBird 2015-02-24 22:52:47

  • Status changed from Fix committed to Resolved