Tails

#1 Updated by intrigeri 2015-01-06 13:39:35

• blocks #8538 added

#2 Updated by intrigeri 2015-03-07 10:01:45

• related to Feature #9031: Upgrade to Tor Browser 4.5 added

#4 Updated by intrigeri 2015-03-09 15:57:31

• Description updated

#5 Updated by anonym 2015-04-29 12:10:27

• related to deleted (Feature #9031: Upgrade to Tor Browser 4.5)

#7 Updated by anonym 2015-06-27 06:11:43

• % Done changed from 0 to 10
• Feature Branch set to feature/8548-tor-browser-5.0

In anticipation of Tor Browser 5.0a3, which will be the first esr38-based build, I’ve tested 5.0a2 in Tails. An image built from the feature/8548-tor-browser-5.0 branch passes all our browser-related features (git grep -l -i browser -- features/*.feature). That’s encouraging, but 5.0 so far haven’t introduced much that should affect what we test, so it’s also fairly expected.

Should this ticket perhaps have its subject changed to “Migrate to Tor Browser 5.0”? We’ll want a separate ticket for that any way, and I see little point in having two tickets.

#9 Updated by anonym 2015-06-29 04:27:43

• Subject changed from Migrate to Firefox 38 ESR to Install Tor Browser 5.0
• Description updated

#10 Updated by anonym 2015-06-29 07:52:22

• % Done changed from 10 to 20

I have force pushed the branch, now installing Tor Browser 5.0a3-build4, which is based on esr38. I’ve done some manual testing (not exhaustive!) and run a full test suite run. Most tests passed, below I’ll list all errors.

All-in-all it looks very promising.

Issues in all browsers (due to the esr38 migration)

• There is a “Share this page” button (that seemingly always is greyed-out/disabled) right next to the search bar. It doesn’t seem like something we want, or that the normal Tor Browser wants, IMHO.
• We may want to disable the menu item Tools -> Apps which is a link to https://marketplace.firefox.com. We (and the normal Tor Browser) still has “Set up sync” which arguably is a lot worse, so perhaps this is moot.
• The “New Tab” page is enhanced with “Tiles”, and soon we can expected ads, yay! We probably want to switch to the “classic” version (which I assume is the same as in Firefox 31), which for some reason currently can’t be enabled. In the “worst case” we can switch to the “blank” version, which does work.

Issues in Tor Browser

• Youtube is the default search engine. Perhaps the amnesia branding extension has to be updated again.

Automated test suite issues:

  Scenario: I can view a file stored in "~/Tor Browser" but not in ~/.gnupg                                           # features/torified_browsing.feature:56       
[...]
    When I open the address "file:///home/amnesia/.gnupg/synaptic.html" in the Tor Browser                            # features/step_definitions/common_steps.rb:645                                                                                 
    Then I see "TorBrowserUnableToOpen.png" after at most 10 seconds                                                  # features/step_definitions/common_steps.rb:445                                                                                 
      FindFailed: can not find TorBrowserUnableToOpen.png on the screen.
      Line ?, in File ? (RuntimeError)
      features/torified_browsing.feature:64:in `Then I see "TorBrowserUnableToOpen.png" after at most 10 seconds'

There is no such error message any more. If a page is already open, the page remains and the URL bar even reverts to the previous entry, so there’s nothing new to look for for sikuli. In a new tab the URL bar remains, though. I’m not sure of the best way to fix the test.

Issues with the Unsafe Browser

Automated test suite failures:

  Scenario: Starting the Unsafe Browser works as it should.                # features/unsafe_browser.feature:18                                                     
    When I successfully start the Unsafe Browser                           # features/step_definitions/unsafe_browser.rb:140                                        
    Then the Unsafe Browser runs as the expected user                      # features/step_definitions/common_steps.rb:708                                          
    And the Unsafe Browser has a red theme                                 # features/step_definitions/unsafe_browser.rb:125                                        
      FindFailed: can not find UnsafeBrowserRedTheme.png on the screen.
      Line ?, in File ? (RuntimeError)
      features/unsafe_browser.feature:21:in `And the Unsafe Browser has a red theme'

Fails due to the “Share this page” button (mentioned above) being in the way. The theme is still red. I’ll wait with updating the image until we’ve decided if we want the “Share this page” button at all.

  Scenario: The Unsafe Browser cannot be configured to use Tor and other local proxies. # features/unsafe_browser.feature:49
    When I successfully start the Unsafe Browser                                        # features/step_definitions/unsafe_browser.rb:140                           
    Then I cannot configure the Unsafe Browser to use any local proxies                 # features/step_definitions/unsafe_browser.rb:169                           
      FindFailed: can not find UnsafeBrowserPreferencesWindow.png on the screen.
      Line ?, in File ? (RuntimeError)
      features/unsafe_browser.feature:51:in `Then I cannot configure the Unsafe Browser to use any local proxies'

Indeed, the preferences window is no more — the preferences are instead opened in a browser tab and looks quite different than. The test must be updated (lots of images, perhaps more, perhaps some scrolling… yay…).

  Scenario: The Unsafe Browser will not make any connections to the Internet which are not user initiated # features/unsafe_browser.feature:53
[...]
    Then the Unsafe Browser has started                                                                   # features/step_definitions/unsafe_browser.rb:58          
    And I wait between 60 and 120 seconds                                                                 # features/step_definitions/checks.rb:77                  
      Slept for 120 seconds
    And the clearnet user has not sent packets out to the Internet                                        # features/step_definitions/unsafe_browser.rb:262         
Full network capture available at: /home/tailstester-rjb/TailsToaster/sniffer.pcap-2015-06-29T14:59:59+02:00
    And all Internet traffic has only flowed through Tor                                                  # features/step_definitions/common_steps.rb:450           
      The following IPv4 TCP non-Tor Internet hosts were contacted:
      52.24.151.177
      93.184.220.29
      68.232.34.191 (RuntimeError)
      ./features/support/helpers/firewall_helper.rb:115:in `assert_no_leaks'
      ./features/step_definitions/common_steps.rb:453:in `/^all Internet traffic has only flowed through Tor$/'                                                     
      features/unsafe_browser.feature:62:in `And all Internet traffic has only flowed through Tor'

Looks like a call home to some Mozilla service:

• https://www.robtex.net/#!dns=52.24.151.177&rev=1 (tiles.r53-2.services.mozilla.com and tiles.services.mozilla.com)
• https://www.robtex.net/#!dns=68.232.34.191&rev=1 (various mozilla.net domains)
• https://www.robtex.net/#!dns=93.184.220.29&rev=1 (ocsp.digicert.com so probably HTTPS is used to access some of the above Mozilla services)

What “feature” is this? Updates are still disabled AFAICT, altough there are seemingly new timesstamps for various app.update.LastUpdateTime.* prefs.

#25 Updated by intrigeri 2015-07-19 07:30:28

• Status changed from Confirmed to In Progress

#26 Updated by anonym 2015-07-23 08:48:28

• Assignee changed from anonym to intrigeri
• QA Check set to Info Needed

I’ve updated the branch so it installs 5.0a3 and I’ve adapted new tests. However, your commit:0e1b6ae causes issues. My adaptation is (since we don’t get any good graphical feedback — see Feature #8548#note-10) to grep for an appropriate audit line in syslog, which should say that apparmor blocked access to “/tmp/synaptic.html”, but there is none. Any idea why files in /tmp are treated differently by apparmor and doesn’t generate such messages? Can it be fixed?

#27 Updated by intrigeri 2015-07-29 00:27:27

• Assignee changed from intrigeri to anonym
• QA Check changed from Info Needed to Dev Needed
• Affected tool deleted (Browser)

> Any idea why files in /tmp are treated differently by apparmor and doesn’t generate such messages?

That’s because of commit:51c0060 (Bug #9558).

> Can it be fixed?

Not in the current state of AppArmor parser, unless we want to duplicate the gnome abstraction, drop the user-tmp abstraction from that copy, and use that modified copy just for the torbrowser profile. It’s ugly, but I believe this could be done relatively easily at build time.

#29 Updated by anonym 2015-08-03 16:40:54

• Assignee changed from anonym to intrigeri
• QA Check changed from Dev Needed to Info Needed

intrigeri wrote:
> > Can it be fixed?
>
> Not in the current state of AppArmor parser, unless we want to duplicate the gnome abstraction, drop the user-tmp abstraction from that copy, and use that modified copy just for the torbrowser profile. It’s ugly, but I believe this could be done relatively easily at build time.

Adding such complexity in Tails itself only for making a test easier seems like a bad idea. I guess I’ll have to change that part of the scenario to “try opening it, wait a bit, make sure we don’t see the image”. I don’t like that approach, though: it adds a delay, and prone to false-negatives. At least I only have to do it for the /tmp case.

> First very quick look:
>
> * the duplication of stuff between Tor Browser’s prefs.js + userChrome.css, and the I2P browser’s and Unsafe browser’s one, is starting to be worrying; not a blocker, but we should find a more clever way, e.g. storing common stuff in a shared place and generating the final files out of it at ISO build time;

Is this something that can be postponed? If so I’ll open a ticket.

> * not sure why the I2P Browser lacks the “Don’t use geographically specific search prefs” bits

It doesn’t have a search bar.

> * there’s trailing whitespace introduced somewhere;

Whitespace shame! :S Fixed.

> please configure your Git to forbid that.

Which method do you recommend?

Also, from IRC:
> (21:00:21) intrigeri: anonym: it seems unclear (see #tor-dev) what exactly will be done in TB 5.0 wrt. the fonts, though.

Indeed. However, do you feel comfortable merging this branch when all scenarios using the any of the browsers will fail? It’s painful, since we likely will have to repeat the process (on the image building day, no less) but I see no way around it.

#30 Updated by intrigeri 2015-08-04 01:59:12

• Assignee changed from intrigeri to anonym
• QA Check changed from Info Needed to Dev Needed

> Adding such complexity in Tails itself only for making a test easier seems like a bad idea. I guess I’ll have to change that part of the scenario to “try opening it, wait a bit, make sure we don’t see the image”. I don’t like that approach, though: it adds a delay, and prone to false-negatives. At least I only have to do it for the /tmp case.

OK.

>> * the duplication of stuff between Tor Browser’s prefs.js + userChrome.css, and the I2P browser’s and Unsafe browser’s one, is starting to be worrying; not a blocker, but we should find a more clever way, e.g. storing common stuff in a shared place and generating the final files out of it at ISO build time;

> Is this something that can be postponed?

Yes, as I wrote: “not a blocker”.

> If so I’ll open a ticket.

Please do so :)

>> please configure your Git to forbid that.

> Which method do you recommend?

I use that:

git config --global apply.whitespace error
git config --global color.diff.whitespace "red reverse"

> Also, from IRC:
>> (21:00:21) intrigeri: anonym: it seems unclear (see #tor-dev) what exactly will be done in TB 5.0 wrt. the fonts, though.

> However, do you feel comfortable merging this branch when all scenarios using the any of the browsers will fail?

Of course not :)

> It’s painful, since we likely will have to repeat the process (on the image building day, no less) but I see no way around it.

Right. Just make sure such test suite image adjustments live in a single commit that can be easily reverted in case the TB developers rollback all fonts-related changes?

#31 Updated by anonym 2015-08-04 10:25:44

• Assignee changed from anonym to intrigeri
• QA Check changed from Dev Needed to Ready for QA

intrigeri wrote:
> > Adding such complexity in Tails itself only for making a test easier seems like a bad idea. I guess I’ll have to change that part of the scenario to “try opening it, wait a bit, make sure we don’t see the image”. I don’t like that approach, though: it adds a delay, and prone to false-negatives. At least I only have to do it for the /tmp case.
>
> OK.

That is commit:949b003. I’ve opened Bug #9895 for reproducing this in FF38.

> >> * the duplication of stuff between Tor Browser’s prefs.js + userChrome.css, and the I2P browser’s and Unsafe browser’s one, is starting to be worrying; not a blocker, but we should find a more clever way, e.g. storing common stuff in a shared place and generating the final files out of it at ISO build time;
>
> > Is this something that can be postponed?
>
> Yes, as I wrote: “not a blocker”.

Whoops. Opened as Bug #9896.

> > If so I’ll open a ticket.
>
> Please do so :)
>
> >> please configure your Git to forbid that.
>
> > Which method do you recommend?
>
> I use that:
>
> […]

I already had that set, so I guess apply.* doesn’t apply when committing “normally”? I suppose I should get more careful about git diff:ing before committing…

> > Also, from IRC:
> >> (21:00:21) intrigeri: anonym: it seems unclear (see #tor-dev) what exactly will be done in TB 5.0 wrt. the fonts, though.
>
> > However, do you feel comfortable merging this branch when all scenarios using the any of the browsers will fail?
>
> Of course not :)

All images should be updated now. At least a full run passes for me.

> > It’s painful, since we likely will have to repeat the process (on the image building day, no less) but I see no way around it.
>
> Right. Just make sure such test suite image adjustments live in a single commit that can be easily reverted in case the TB developers rollback all fonts-related changes?

Done in commit:2f02f5e.

I believe this branch is ready for merging so we’ll get it in the 1.5 RC.

#32 Updated by intrigeri 2015-08-04 10:58:18

• Assignee changed from intrigeri to anonym
• QA Check changed from Ready for QA to Dev Needed

I did a more thorough code review up to commit:97f58fa. I only have some nitpicking:

• “Disable Tiles in the Unsafe Browser’s new tab page”: IIRC given the corresponding design doc says that all our prefs are explained with inline comments, it would be good to explain why we do so there. Same for “Don’t use geo-specific search engine prefs in our browsers”, “Hide the Tools -> Apps link to the Firefox Marketplace”, “Also disable Tiles in the I2P Browser”, commit:abe9635 (that has a buggy message, hence I’m referring to it by ID), and I’m stopping the list here => please look at the diff from devel and adjust.
• s/apparmor has/AppArmor has/, please :)
• def scroll_page_to_extreme_right seems to be Unsafe Browser -specific, so IMO its name should indicate it (putting stuff in a specific step_definitions/*.rb does not magically add namespacing, I think)
• the message for commit:949b003 is wrong: the user-tmp abstraction has nothing to do with it; our own overriding of things allowed in user-tmp (introduced in Bug #9558), is why it’s not logged; I think we could make this overriding log, but then it might become an issue with Feature #9337; too late to change the commit message, but please fix the inline comment introduced by that commit (and the “defnied” typo in it, while you’re at it :)

Congrats!

#33 Updated by intrigeri 2015-08-04 11:00:38

• % Done changed from 20 to 30

Forgot to mention: I’ll do some manual + automated testing later today, or tomorrow.

#35 Updated by anonym 2015-08-05 03:53:33

• Assignee changed from anonym to intrigeri
• QA Check changed from Dev Needed to Info Needed

intrigeri wrote:
> I did a more thorough code review up to commit:97f58fa. I only have some nitpicking:

Thanks!

> * “Disable Tiles in the Unsafe Browser’s new tab page”: IIRC given the corresponding design doc says that all our prefs are explained with inline comments, it would be good to explain why we do so there. Same for “Don’t use geo-specific search engine prefs in our browsers”, “Hide the Tools -> Apps link to the Firefox Marketplace”, “Also disable Tiles in the I2P Browser”, commit:abe9635 (that has a buggy message, hence I’m referring to it by ID), and I’m stopping the list here => please look at the diff from devel and adjust.
> * s/apparmor has/AppArmor has/, please :)
> * def scroll_page_to_extreme_right seems to be Unsafe Browser -specific, so IMO its name should indicate it (putting stuff in a specific step_definitions/*.rb does not magically add namespacing, I think)

All these should be fixed now in the top three commits.

> * the message for commit:949b003 is wrong: the user-tmp abstraction has nothing to do with it; our own overriding of things allowed in user-tmp (introduced in Bug #9558), is why it’s not logged; I think we could make this overriding log, but then it might become an issue with Feature #9337; too late to change the commit message, but please fix the inline comment introduced by that commit (and the “defnied” typo in it, while you’re at it :)

Could you please propose something so it gets correct for sure this time?

#36 Updated by intrigeri 2015-08-05 04:28:47

• Assignee changed from intrigeri to anonym

> Could you please propose something so it gets correct for sure this time?

Done in commit:8d3aa84 :)

=> I guess I could merge the branch as-is now. Any reason why I should not?

#37 Updated by intrigeri 2015-08-05 05:19:11

• QA Check changed from Info Needed to Dev Needed

Merged into devel, but it’s of course not “fix committed” since that’s not the final Tor Browser 5.0.

#38 Updated by anonym 2015-08-10 10:08:23

• Status changed from In Progress to Fix committed
• % Done changed from 30 to 100

Applied in changeset commit:ed475f9a0bae77687d8ff4217fd04bcd935d356b.

#39 Updated by anonym 2015-08-10 10:19:41

• Assignee deleted (anonym)
• QA Check changed from Dev Needed to Pass

#40 Updated by BitingBird 2015-08-11 10:42:59

• Status changed from Fix committed to Resolved