#1 Updated by intrigeri 2017-12-21 08:38:22

• blocks Feature #13245: Core work 2018Q1: Foundations Team added

#2 Updated by intrigeri 2017-12-21 08:39:31

• Assignee set to anonym
• Target version set to Tails_3.6

anonym, do you think you can handle the initial assessment (what needs to be done and when, which probably requires testing the add-ons we ship on the latest Thunderbird non-ESR/beta releases) in 2018Q1?

#4 Updated by anonym 2017-12-21 12:12:04

• Assignee changed from anonym to intrigeri
• QA Check set to Info Needed

• I’ve asked Sukhbir what his plans for TorBirdy are
• Upstream Engimail has a ticket to make Enigmail into an embedded WebExtension

intrigeri wrote:
> I don’t know what the timeline for this is. Likely we need to be ready for the next ESR. Assuming it’s Thunderbird 60, it depends how much overlap there is (if any) in support for the old (52) and new (60) ESR branches.

Since the stable Thunderbird releases all are ESR, we (and Debian) won’t get any of versions 57-59, but only 60. And when that one is released and imported into Debian I suppose the Debian package maintainer for enigmail would be forced to import a compatible version (i.e. a version after the conversion to an embedded WebExtension). What can we expect on the Debian side in this situation?

#5 Updated by intrigeri 2017-12-22 07:02:09

• Assignee deleted (intrigeri)

> Since the stable Thunderbird releases all are ESR, we (and Debian) won’t get any of versions 57-59, but only 60. And when that one is released and imported into Debian I suppose the Debian package maintainer for enigmail would be forced to import a compatible version (i.e. a version after the conversion to an embedded WebExtension). What can we expect on the Debian side in this situation?

Good question! Better ask it to the maintainer of torbirdy in Debian ⇒ reassigning.

u, if you need help (or a rubber-duck) for this, I’m here :)

#6 Updated by intrigeri 2018-01-26 16:13:50

• Assignee set to anonym
• QA Check deleted (Info Needed)

anonym: looks like I got confused and misread Torbirdy when you wrote Enigmail.

So yeah, on the Enigmail side I’m not very concerned because it’s actively developped and I’m confident dkg will import whatever upstream release is needed to keep it working. There’s already been an update of Enigmail in Stretch (or was it Jessie?) to keep it working with the Thunderbird security update.

I’m much more concerned about Torbirdy upstream. In Debian we can’t do much better than importing upstream stuff. And same as Enigmail, if it’s broken in stable due to a Thunderbird security update, we will need to either update Torbirdy there too, or drop it from stable.

#9 Updated by intrigeri 2018-01-29 10:25:07

• Target version changed from Tails_3.6 to Tails_3.7
• Type of work changed from Research to Communicate

#10 Updated by anonym 2018-01-30 20:31:51

• Target version changed from Tails_3.7 to Tails_3.6
• Type of work changed from Communicate to Research

Now that Thunderbird 57 is out, I’ve asked Sukhbir for a more concrete plan/schedule.

#11 Updated by anonym 2018-01-30 20:32:17

• Target version changed from Tails_3.6 to Tails_3.7
• Type of work changed from Research to Communicate

#13 Updated by intrigeri 2018-02-20 08:42:22

• blocks Bug #11082: Replace Liferea added

#14 Updated by intrigeri 2018-02-20 08:43:29

• blocks Feature #15139: Core work 2018Q2: Foundations Team added

#15 Updated by intrigeri 2018-02-20 08:43:58

• blocked by deleted (Feature #13245: Core work 2018Q1: Foundations Team)

#16 Updated by intrigeri 2018-03-28 10:02:24

• Subject changed from Get ready for Thunderbird 57+ to Get ready for Thunderbird 60+

#19 Updated by intrigeri 2018-04-08 14:35:29

• Description updated

#20 Updated by intrigeri 2018-04-08 14:37:12

• Type of work changed from Communicate to Code

#22 Updated by intrigeri 2018-04-11 07:07:05

• Description updated

#23 Updated by intrigeri 2018-04-13 12:50:44

• Description updated

#25 Updated by intrigeri 2018-04-13 15:07:56

• Assignee changed from anonym to CyrilBrulebois
• Estimated time set to 6 h

#26 Updated by intrigeri 2018-04-13 15:09:44

• Due date set to 2018-05-15

#27 Updated by bertagaz 2018-05-10 11:09:31

• Target version changed from Tails_3.7 to Tails_3.8

#28 Updated by intrigeri 2018-05-16 14:08:54

• Description updated

(added upstream timeline for 32.x and 60.x releases)

#29 Updated by intrigeri 2018-06-24 07:24:13

• Description updated

#30 Updated by intrigeri 2018-06-26 16:28:04

• Target version changed from Tails_3.8 to Tails_3.9

#33 Updated by intrigeri 2018-06-28 14:22:26

• Due date changed from 2018-05-15 to 2018-07-03

#34 Updated by intrigeri 2018-06-29 06:30:54

• Description updated

#35 Updated by intrigeri 2018-06-29 06:33:24

• Description updated

#36 Updated by intrigeri 2018-07-09 08:17:28

• Description updated

#37 Updated by intrigeri 2018-07-10 22:40:10

• Description updated

#39 Updated by intrigeri 2018-07-10 22:48:24

• Due date changed from 2018-07-03 to 2018-07-21

#41 Updated by intrigeri 2018-07-10 23:12:30

• related to Bug #15602: Fix EFAIL added

#42 Updated by intrigeri 2018-07-10 23:18:08

• Estimated time changed from 6 h to 10 h

I suspect 6 hours of focussed work (that excludes ISO build & automatic test time) might not be enough to do Feature #15091#note-24. So bumping that number in case if you need more and I’m not around to validate it.

#43 Updated by intrigeri 2018-07-22 01:21:30

• blocks Feature #15334: Core work 2018Q3: Foundations Team added

#44 Updated by intrigeri 2018-07-22 01:21:36

• blocked by deleted (Feature #15139: Core work 2018Q2: Foundations Team)

#45 Updated by intrigeri 2018-07-22 01:50:16

• blocks Feature #7625: Persistence preset: RSS feeds added

#46 Updated by intrigeri 2018-07-22 01:54:05

• blocks Feature #15745: Update doc wrt. Liferea → Thunderbird as the default feed reader added

#52 Updated by CyrilBrulebois 2018-08-05 04:19:03

• Feature Branch set to https://salsa.debian.org/kibi/thunderbird/tree/feature-15091-get-ready-for-thunderbird-60-plus

(Sorry, I should have read the whole ticket before asking…)

Here’s a report after roughly 6 hours spent on TB.

Building

I’ve opened a bug report against the experimental version of src:thunderbird → https://bugs.debian.org/905465 with my findings regarding build dependencies and how to backport thunderbird to stretch.

I’ve also rebased the 18 Tails-specific patches from b2 on top of b10. Note: files were moved around between two beta releases, so it’s really important to use git rebase with appropriate parameters instead of exporting the patches and applying them (filenames would need extensive tweaking).

Only two patches needed manual updates, because the context changed a bit:

• 0005 = Document that mailnews.{auto_config,mx_service}_url can be disabled.
• 0013 = Add pref to force SSL for ISP fetch during autoconfiguration.

As documented in the Debian bug report, I’ve built the packages in a stretch chroot, with stretch-proposed-updates enabled for some build dependencies (cargo etc.).

Since the patches touch upstream files directly, dpkg-source refuses to create a source package, so I’ve built the binaries by running dpkg-buildpackage -b manually, in an sbuild-createchroot’d chroot, with build dependencies installed manually; note: /dev/shm is required (to avoid EPERM in the Python multiprocessing module), so I went for a bind-mount.

Binaries and git branch

The resulting binaries are available in the following repository, signed with my “corporate” key:

• deb https://apt.debamax.com/tails/debian stretch-tails main
• https://apt.debamax.com/tails/0x27869B61.asc

The git branch is available on salsa (thunderbird.git is a bit heavy so I didn’t publish that one on my own server) → https://salsa.debian.org/kibi/thunderbird/tree/feature-15091-get-ready-for-thunderbird-60-plus

Running

Installing these binaries on Tails 3.8, three l10n packages go away (-bn-bd, -pa-in, -ta-lk), while two get installed (-kk, -ms), along with a new dependency: libjsoncpp1.

Upon start-up, the account creation screen shows up, and an attempt is made to derive the right settings from my mail address.

On the addons side, 2 are installed:

• enigmail: 2.0.7, last updated 2018-06-14 → no worries
• torbirdy: 0.2.1, last updated 2016-11-24; disabled, incompatible with TB 60.0

The addons screen doesn’t make it possible to perform an online search, so I had to download the .xpi manually to load it from a local file; after a restart, torbirdy is successfully enabled (as reported with green indication at the bottom: “TorBirdy Enabled: tor”).

Trying to create an account results in a different screen: no autodetection is performed this time, and I have to pick between IMAP and POP protocols. After setting it up, I’ve been able to download mails from my server using IMAP. I’ve also managed to configure an SMTP server and to send mail from there.

Message seen in the terminal thunderbird was started from: it tries to read from /etc/dconf/db/local and this isn’t possible; a warning regarding performance issues is printed accordingly; in dmesg, the same issue can be seen, through DENIED log lines.

Note: I cannot seem to be able to easily extract the apt log, but I chose to install the new profile (/etc/apparmor.d/usr.bin/thunderbird) instead of keeping the old one. I suppose an update is in order?

#53 Updated by CyrilBrulebois 2018-08-07 09:21:09

• Assignee deleted (CyrilBrulebois)
• QA Check set to Info Needed

I’d be happy get some feedback on my findings. :)

FWIW, TB60 (final) was just released → https://www.thunderbird.net/en-US/thunderbird/60.0/releasenotes/

#54 Updated by intrigeri 2018-08-07 10:12:07

• Status changed from Confirmed to In Progress
• Assignee set to intrigeri

(Better assign to the person you’re requesting info from :)

#55 Updated by Anonymous 2018-08-07 12:53:48

• related to Bug #15600: Upgrade to Torbirdy 0.2.5 added

#56 Updated by intrigeri 2018-08-07 13:18:26

• Assignee changed from intrigeri to CyrilBrulebois
• QA Check changed from Info Needed to Dev Needed

Woohoo, awesome progress! \o/

> Here’s a report after roughly 6 hours spent on TB.

Glad there’s some time left. We’ll update the budget dynamically in the next days, keep talking to me along the way :)

> I’ve also rebased the 18 Tails-specific patches from b2 on top of b10. Note: files were moved around between two beta releases, so it’s really important to use git rebase with appropriate parameters instead of exporting the patches and applying them (filenames would need extensive tweaking).

Congrats! I wonder if it’s worth also preparing a secure_account_creation-60.0_b10 branch forked off the upstream/60.0_b10 tag with these patches. Probably not very useful since the patch series is tracked via quilt in the branch you’ve prepared already + I doubt anonym will work on upstreaming it any time soon. Now, if you think it’s a matter of 15 min of work, please do it.

I’ll take a look at the updated patch series later this week.

> Since the patches touch upstream files directly, dpkg-source refuses to create a source package, so I’ve built the binaries by running dpkg-buildpackage -b manually, in an sbuild-createchroot’d chroot, with build dependencies installed manually; note: /dev/shm is required (to avoid EPERM in the Python multiprocessing module), so I went for a bind-mount.

I’m not sure I understand. I’ll try building in the way we document and will see.

> Upon start-up, the account creation screen shows up, and an attempt is made to derive the right settings from my mail address.

Sounds good! Does it show a “only use secure protocols” (or similar) checkbox? (i.e. are our patches still effective?)

> On the addons side, 2 are installed:

> * enigmail: 2.0.7, last updated 2018-06-14 → no worries
> * torbirdy: 0.2.1, last updated 2016-11-24; disabled, incompatible with TB 60.0

> The addons screen doesn’t make it possible to perform an online search, so I had to download the .xpi manually to load it from a local file; after a restart, torbirdy is successfully enabled (as reported with green indication at the bottom: “TorBirdy Enabled: tor”).

Good! Next step: install the version from Debian. To do so, please create a feature/15091-thunderbird-60 branch (forked off tails.git’s devel branch) that installs torbirdy 0.2.5. It’s been in testing/sid for some time and Ulrike uploaded it to stretch-backports today.

> Message seen in the terminal thunderbird was started from: it tries to read from /etc/dconf/db/local and this isn’t possible; a warning regarding performance issues is printed accordingly; in dmesg, the same issue can be seen, through DENIED log lines.

Looks like an AppArmor problem. Let’s take a closer look once the profile is updated i.e.:

> Note: I cannot seem to be able to easily extract the apt log, but I chose to install the new profile (/etc/apparmor.d/usr.bin/thunderbird) instead of keeping the old one. I suppose an update is in order?

So far we’ve adjusted the AppArmor profile for our needs via config/chroot_local-patches/apparmor-adjust-thunderbird-profile.diff (tails.git). This avoids having to build & upload src:thunderbird every time we want to patch it. So we need to update that patch on the aforementioned feature/15091-thunderbird-60 branch. Then that branch will FTBFS (the updated patch won’t apply) until it installs Thunderbird 60, so I’ll try to build & upload a package to an overlay APT suite ASAP, which should unblock your next steps :)

Meta: I think we’ll need to start filing subtasks soonish, before this ticket becomes too complex and makes it impossible to track what’s done vs. TBD.

#58 Updated by intrigeri 2018-08-07 14:41:38

• Feature Branch changed from https://salsa.debian.org/kibi/thunderbird/tree/feature-15091-get-ready-for-thunderbird-60-plus to feature/15091-thunderbird-60, icedove:feature/15091-thunderbird-60

> Now I understood: you’ve imported the patch series directly on your Debian packaging branch instead of doing so as quilt patches (which we’ve done so far, using the secure-account-creation “topic”). I’ll fix that so I can build the package without doing crazy stuff.

I’ve:

1. enabled the feature-15091-thunderbird-60 APT overlay suite on a new feature/15091-thunderbird-60 branch forked off devel (tails.git)
2. imported the patch series into debian/patches/secure-account-creation (gbp pq FTW)
3. rewrote Git history to avoid reverting 18 commits and the confusion it would cause
4. created a fresh feature/15091-thunderbird-60 branch in icedove.git
5. merged my rewritten feature-15091-get-ready-for-thunderbird-60-plus into feature/15091-thunderbird-60; while resolving conflicts I’ve dropped most of the previous tails/stretch delta (that comes from Debian’s debian/stretch branch) to remain as close as possible to the branch you’ve tested; that delta wll come back once Thunderbird 60 is in Stretch and we resume merging from Debian’s debian/stretch branch, though.
6. created a sbuild chroot with pu enabled (sudo sbuild-createchroot --arch=amd64 --include=debhelper,eatmydata --components=main,contrib,non-free --extra-repository='deb http://deb.debian.org/debian stretch-proposed-updates main' --chroot-prefix=stretch-pu stretch /var/cache/sbuild/stretch-pu-amd64-sbuild http://localhost:3142/debian and then tweaked the new config file in /etc/schroot/chroot.d/ to suit my taste)

Next steps on my side:

1. build a package with gbp buildpackage --git-debian-branch=feature/15091-thunderbird-60 --git-export=WC --git-builder='sbuild --chroot=stretch-pu-amd64-sbuild' or similar
2. upload to the package to the feature-15091-thunderbird-60 APT overlay suite

I’ll keep you updated.

And then Cyril can (some of that stuff can be done already though):

1. have the branch install a compatible Torbirdy (now tracked on Bug #15600)
2. update the AppArmor patch (shout if you need help understanding/rebasing our diff)
3. do the remaining parts of Feature #15091#note-24
4. check if we’re using the updated patch series correctly (IIRC I’ve noticed a bunch of prefs and I think some of them are new, possibly off by default, and meant to be enabled or tweaked in Tails)

#59 Updated by intrigeri 2018-08-07 14:45:43

• Subject changed from Get ready for Thunderbird 60+ to Upgrade to Thunderbird 60

(“Get ready” made sense back in May but we’re not there anymore.)

#60 Updated by intrigeri 2018-08-07 14:47:45

• related to deleted (Bug #15602: Fix EFAIL)

#61 Updated by intrigeri 2018-08-07 14:48:00

• blocks Bug #15602: Fix EFAIL added

#63 Updated by intrigeri 2018-08-08 15:05:07

• blocks Bug #15551: Generating a revocation certificate with Enigmail fails added

#64 Updated by intrigeri 2018-08-09 05:59:15

• blocked by deleted (Feature #7625: Persistence preset: RSS feeds)

#65 Updated by intrigeri 2018-08-09 15:18:06

• blocks Bug #15775: Adjust persistence setup to Thunderbird being recommended for feeds added

#69 Updated by CyrilBrulebois 2018-08-14 14:00:45

• Feature Branch changed from feature/15091-thunderbird-60, icedove:feature/15091-thunderbird-60 to kibi/feature/15091-thunderbird-60, icedove:feature/15091-thunderbird-60

Thanks for the sysctl setting, using 0 instead of 2 indeed lets me check TB’s behaviour. :)

So: even if both configuration files are indeed read successfully, only the setting in /usr/share/xul-ext/torbirdy/defaults/preferences/prefs.js seems to matter. So I’ve added a local patch as discussed.

Confirmed to work fine with this commit in my feature/15091-thunderbird-60 branch (https://mraw.org/git/?p=tails.git;a=shortlog;h=refs/heads/feature/15091-thunderbird-60):

commit 7f1fb753251b352a7af0eae52bc865c899846632
Author: Cyril Brulebois <cyril@debamax.com>
Date:   Tue Aug 14 12:57:10 2018 +0200

    Torbirdy: Enable the email wizard in the system configuration.

    There's:
      /usr/share/xul-ext/torbirdy/defaults/preferences/000system.js → /etc/xul-ext/torbirdy.js

    which is read after:
      /usr/share/xul-ext/torbirdy/defaults/preferences/prefs.js

    but setting extensions.torbirdy.emailwizard to true in the former
    doesn't seem sufficient, even though strace confirms it's being read.

#70 Updated by CyrilBrulebois 2018-08-14 19:29:05

• Feature Branch changed from kibi/feature/15091-thunderbird-60, icedove:feature/15091-thunderbird-60 to kibi:feature/15091-thunderbird-60, icedove:feature/15091-thunderbird-60

Just pushed a refresh branch which includes borrowing code from Tor Browser to set the locale according to $LANG. That seems to work fine for a few languages, like French, German, Chinese, and Japanese.

Latest commit:

commit b6e401f9b5a843410500fff07330abe8fa583f24
Author: Cyril Brulebois <cyril@debamax.com>
Date:   Tue Aug 14 17:17:40 2018 +0200

    Thunderbird: Fix regression in defaultprotocol handling (refs: <del><a class='issue tracker-2 status-3 priority-4 priority-default closed child' href='/code/issues/15091' title='Upgrade to Thunderbird 60'>Feature #15091</a></del>).

    Preferences handling seems to have changed heavily in TB60, so switch to a
    user_pref in prefs.js instead of a regular pref in preferences/0000tails.js

    Another possibility would be to use /etc/thunderbird/pref/thunderbird.js but
    that would need to happen in gdm's PostLogin (root privileges would be
    required).

It would be nice to test what happens when persistency is enabled, as I haven’t exactly reproduced the issue as it is. I’ve just inferred it from what was happening with language settings.

For reference, mailnews.auto_config.fetchFromISP.ssl_only is set to false, and I’m not sure I’ve successfully tested what’s happening there. I’ll follow up later on that.

#81 Updated by intrigeri 2018-08-15 17:59:47

• related to Bug #15791: Update test suite for Thunderbird 60 added

#82 Updated by intrigeri 2018-08-15 17:59:57

• related to Feature #15792: Upgrade to Thunderbird 60 final added

#83 Updated by intrigeri 2018-08-15 18:03:20

• Status changed from In Progress to Fix committed
• Assignee deleted (CyrilBrulebois)
• QA Check changed from Dev Needed to Pass

intrigeri wrote:
> I’ll do the automated tests (and relevant parts of the manual tests) by hand and will then merge if happy!

Merged! See you on Bug #15791 and Feature #15792 (that I’ve unparented to be able to mark this one as fix committed).

#84 Updated by intrigeri 2018-08-16 09:14:29

• Status changed from Fix committed to In Progress

Applied in changeset commit:8921d19b2be25173b3bc344e9d717034786d8828.

#85 Updated by intrigeri 2018-08-16 10:15:31

• Status changed from In Progress to Fix committed

#86 Updated by intrigeri 2018-09-05 16:11:00

• Status changed from Fix committed to Resolved

#87 Updated by intrigeri 2019-11-30 19:21:33

• related to Bug #17277: Make the Thunderbird autoconfig wizard not trust the result of DNS requests added