Feature #10453: Provide a script to migrate from the default MH mailbox to Icedove

Feature #10453

Provide a script to migrate from the default MH mailbox to Icedove

Added by sajolida 2015-10-29 09:55:25 . Updated 2015-11-03 11:28:08 .

Status:

Resolved

Priority:

Normal

Assignee:

Category:

Target version:

Tails_1.7

Start date:

2015-10-29

Due date:

% Done:

100%

Feature Branch:

doc/7158-icedove

Type of work:

Code

Blueprint:

Starter:

Affected tool:

Email Client

Deliverable for:

268

Description

The script is here:

https://git-tails.immerda.ch/tails/plain/wiki/src/misc/claws_to_icedove.sh?h=doc/7158-icedove

Files

output.txt (18393 B)

sajolida, 2015-10-29 09:59:20

Subtasks

Feature #10454: Improve the user-facing strings of the migration script

Resolved

100

Related issues

Related to Tails - ~~Feature #10457~~: Consider shipping the nmh package for some time

Resolved

2015-10-30

History

#1 Updated by sajolida 2015-10-29 10:00:46

File output.txt added
QA Check set to Dev Needed

I tested this script in the context of Tails and it didn’t work. Here is what I did:

1. Configure Claws Mail to fetch a Riseup account with POP. By default this goes into ~/.claws-mail/Mail/.
2. Downloaded and run the script.

You can find the output in attachment. It seems to parse the whole home directory and then creates lots of crap in .icedove/profile.default/Mail/Local Folders/. I tried to fix that myself quickly in the script but couldn’t make it.

#2 Updated by sajolida 2015-10-29 10:01:02

blocks #8668 added

#3 Updated by sajolida 2015-10-29 10:03:02

Parent task set to ~~Feature #7773~~

#4 Updated by kytv 2015-10-29 12:32:10

sajolida wrote:
> I tested this script in the context of Tails and it didn’t work. Here is what I did:
>
> 1. Configure Claws Mail to fetch a Riseup account with POP. By default this goes into ~/.claws-mail/Mail/.
> 2. Downloaded and run the script.
>
> You can find the output in attachment. It seems to parse the whole home directory and then creates lots of crap in .icedove/profile.default/Mail/Local Folders/. I tried to fix that myself quickly in the script but couldn’t make it.

Confirmed. One of the problems is this change from what I “published”.

@@ -15,9 +14,18 @@
 packf="/usr/bin/mh/packf"
 ICEDOVE="$HOME/.icedove/profile.default"
 SAVEPATH="${ICEDOVE}/Mail/Local Folders"
+BACKUPPATH="${ICEDOVE}/Mail/Local Folders Backup"
+FULLPATH="$HOME/.claws-mail/Mail/"
+

But this $FULLPATH variable is never used. It’s reassigned on line 54, so the script is indeed running from $HOME.

Untested, but this will probably do what’s intended.

diff --git a/wiki/src/misc/claws_to_icedove.sh b/wiki/src/misc/claws_to_icedove.sh
index 47095f0..5a81a74 100644
--- a/wiki/src/misc/claws_to_icedove.sh
+++ b/wiki/src/misc/claws_to_icedove.sh
@@ -15,7 +15,7 @@ packf="/usr/bin/mh/packf"
 ICEDOVE="$HOME/.icedove/profile.default"
 SAVEPATH="${ICEDOVE}/Mail/Local Folders"
 BACKUPPATH="${ICEDOVE}/Mail/Local Folders Backup"
-FULLPATH="$HOME/Mail"
+MAILPATH="$HOME/.claws-mail/Mail"

 # Check if Icedove is running
 if [ "$(pidof icedove)" ]; then
@@ -51,6 +51,7 @@ fi
 [ -d "$SAVEPATH" ] || mkdir -p "$SAVEPATH"
 echo "Saving Mailboxes to $SAVEPATH"

+cd "$MAILPATH"
 for FULLPATH in $(find . -type d)
 do
     FOLDER="$(basename ${FULLPATH})"

#5 Updated by Anonymous 2015-10-29 12:50:10

Hi,

indeed, FULLPATH is the path from which you execute the script from.

I’ve hardcoded that because i want the script the be executed from whereever the user saves it to.
So i’d suggest to simply change it from FULLPATH=“$HOME/Mail” to FULLPATH=“$HOME/.claws-mail/Mail”

In my Tails install this script worked fine though, but I trust sajolida’s findings.

kytvs’s modification boils down to the exact same result. adding another var the cd-ing into.

Cheers!

#6 Updated by sajolida 2015-10-30 04:41:47

Assignee set to kytv

Thanks for the quick feedback. I tried again and transfer from MH to “Local Folders” worked. But I’m now facing another issue as Icedove does not pick up our “Local Folders” but creates a different one “Local Folders-1”.

But if I change the local folders settings to go to .icedove/profile.default/Mail/Local Folders and restart then I’ve got my data, so the migrate itself works fine. Maybe Icedove is missing the .msf files to pick up the one we’ve created.

What shall we do?

And please, try to test the full scenario yourself from a fresh Tails 1.7 to make sure it “works on your machine”.

amnesia@amnesia:~$ find .icedove/
.icedove/
.icedove/profile.default
.icedove/profile.default/Mail
.icedove/profile.default/Mail/Local Folders
.icedove/profile.default/Mail/Local Folders/Trash
.icedove/profile.default/Mail/Local Folders/Inbox
.icedove/profile.default/Mail/Local Folders/Drafts
.icedove/profile.default/Mail/Local Folders/Sent
.icedove/profile.default/Mail/Local Folders/Unsent Messages

amnesia@amnesia:~$ find .icedove/profile.default/Mail/
.icedove/profile.default/Mail/
.icedove/profile.default/Mail/Local Folders
.icedove/profile.default/Mail/Local Folders/Trash
.icedove/profile.default/Mail/Local Folders/Inbox
.icedove/profile.default/Mail/Local Folders/Drafts
.icedove/profile.default/Mail/Local Folders/Sent
.icedove/profile.default/Mail/Local Folders/Unsent Messages
.icedove/profile.default/Mail/pop.riseup.net
.icedove/profile.default/Mail/pop.riseup.net/msgFilterRules.dat
.icedove/profile.default/Mail/pop.riseup.net/Trash
.icedove/profile.default/Mail/pop.riseup.net/Trash.msf
.icedove/profile.default/Mail/pop.riseup.net/Inbox.msf
.icedove/profile.default/Mail/pop.riseup.net/Inbox
.icedove/profile.default/Mail/Local Folders-1
.icedove/profile.default/Mail/Local Folders-1/Trash
.icedove/profile.default/Mail/Local Folders-1/Trash.msf
.icedove/profile.default/Mail/Local Folders-1/Unsent Messages.msf
.icedove/profile.default/Mail/Local Folders-1/Unsent Messages

#7 Updated by bertagaz 2015-10-30 04:43:37

Had a quick look into it, two remarks pop up:

If the nmh package is needed for this migration I think we should ship it in Tails rather than asking for users to install it. That’s bad UX to me if users that run this script and learn they have to reboot their Tails because they didn’t know they need a sudo password. And I don’t think a word in the documentation about that is a good fix to this problem.

I see:

66     set +u
67     if [ "$DIR" != "." ]; then
68         [ ! -f "$SAVEPATH/$DIR" ] || touch "$SAVEPATH/$DIR"
69         set -u

This set +u set -u dance seems a bit hackish to me. I think this would be unnecessary if $DIR was set to something different than . in the else condition above, or if another variable was responsible to track the need to execute this code block.

I’ll try to have another look at it later.

#8 Updated by sajolida 2015-10-30 10:41:47

I did two other small improvements on the script, please review:

971e4e3 to timestamp the backups and allow unlimited run of the script
db5dddf to fix the backup triggering

#9 Updated by sajolida 2015-10-30 10:49:12

related to ~~Feature #10457~~: Consider shipping the nmh package for some time added

#10 Updated by Anonymous 2015-11-02 00:40:07

Assignee deleted (~~kytv~~)

Currently reviewing this one on 1.7

#11 Updated by Anonymous 2015-11-02 02:22:07

Feature Branch changed from doc/7158-icedove to 451f:doc/7158-icedove

Actually, nmh assumes that our mails lie in $HOME/Mail. I’ve added a tweak in b9922b598d69fda8cacad54b25aa7d7c7e3c1db7 so that this does not happen.

#12 Updated by Anonymous 2015-11-02 02:23:39

There is one last little thing which I do not like so much: when running the packf command, the script might say “packf: no messages in /home/amnesia/.claws-mail/Mail” - because in my case it took care of subfolders only. there is no option to let packf be less verbose. Any ideas?

#13 Updated by Anonymous 2015-11-02 06:01:44

Assignee set to sajolida
QA Check changed from Dev Needed to Ready for QA

With anonym’s insight, i’ve managed to silencs packf’s output. The script now workd perfectly for me in 1.7rc.

I’ve also added a message at the end, indication that the script is done and that Icedove can now be started.

Please retest one last time.

#14 Updated by bertagaz 2015-11-02 06:44:01

I know it’s short in time, but my comment didn’t had much answer: the hackish “set +u” is still there, and we still have to know what will be decided in ~~Feature #10457~~.

#15 Updated by Anonymous 2015-11-02 11:45:10

Status changed from Confirmed to In Progress

#16 Updated by sajolida 2015-11-02 15:07:25

Assignee changed from sajolida to anonym
Feature Branch changed from 451f:doc/7158-icedove to doc/7158-icedove

I had to fix two more bugs introduced by your last commit, see 28277ea and 1e6fa1d. But now my test pass. Thanks also for documenting the work around for Local Folders-1! Assigning to anonym for the final merge.

#17 Updated by anonym 2015-11-02 15:40:39

Status changed from In Progress to Fix committed
% Done changed from 20 to 100

Applied in changeset commit:133fbcddfa024d4e182174dd356dd2be0db5a5f2.

#18 Updated by anonym 2015-11-02 16:01:20

Assignee deleted (~~anonym~~)
QA Check changed from Ready for QA to Pass

#19 Updated by anonym 2015-11-03 11:28:08

Status changed from Fix committed to Resolved