Search Support

Avoid support scams. We will never ask you to call or text a phone number or share personal information. Please report suspicious activity using the “Report Abuse” option.

Learn More

Some long-overdue enhancements: synchronize tags & filters (long)

  • 1 reply
  • 2 have this problem
  • 2 views
  • Last reply by Matt

more options

This past week I migrated my email from an old, white MacBook to a (relatively) new MacBook Air. The old computer had been upgraded to a 500 MB SSD, but the new one came with only 256MB. Also, I use TB on two iMacs, each with over a TB of storage. For this reason settings on the MBA have to be different from those on the other machines, so sharing a single profile on a cloud drive is out of the question.

Instead, I had to set up TB on the new computer, install necessary plugins, and then migrate tags & filters. In the second half of the second decade of the 21st century, synchronizing such things across different platforms has become quite common. So one would expect migrating such settings to take at most a few minutes. But somehow TB has stayed behind the times. So the process took me more than a day. In light of this pain, here are some suggestions for moving into the present.

1. Tags. Why the heck are they integrated into prefs.js files? These files also contain device-specific settings, so migrating the file itself would break other settings. Instead, one has to first learn what file holds the tags, edit the file, locate the tags (by the mailnews settings), copy-and-paste them somewhere, repeat the drill on the target machine, and then replace the tags already existing on the second device with the saved ones. Much of this could be avoided simply by using a separate file, tags.js, for tags only. Prefs.js could be modified to just include this file, or perhaps TB could read it directly. (I know TB stores tags for iMap messages as keywords on the iMap server, so in theory machines accessing an iMap account all have access to its tags. But tags also involve colors, which are stored in prefs.js. And surprisingly, newly created iMap accounts pointing to existing iMap stores do not even automatically import the tags from the store. So tag configuration is left entirely as an exercise for the user.)

2. Filters are trickier because they are account-specific. In my case, I had to go into the iMap section of the profile and then into the different accounts. The first problem is that even though I use only 4 different email accounts, the old profile had about 6-7 because some accounts became inactive. Even worse, among the active accounts there were two gmails (gmail and gmail-1). I suspect this is partly because my employer switched from having its own email to Google Apps for Education, so under the covers my account became a gmail account. The problem is I only know the account by its email domain, which doesn't mention gmail at all. So it was difficult to tell which gmail account does what on both machines, creating 2x2 confusion. Eventually I sorted things out, and I'm sure the email address is stored in one of the other files under the account's root directory. But how hard would it be to include a "accountname.txt" file in each account section of the profile to help the end user easily identify the account?

3. After migration, somehow several filters that moved messages to different folders broke. Fixing this was not too hard, but since the move was from one iMap folder to another, and both devices used the same iMap accounts, I don't understand how it is possible that a filter moving a message from one iMap inbox to another iMap folder works on one machine but not another. Perhaps if I understood it better, I could suggest a fix or improvement. Can someone please explain this?

4. Filters are account-specific, but some users have asked for filters shared by all accounts. This makes me wonder how a single msgFilterRules.dat file stored in a profile's root directory and then replicated in each account's root directory via symbolic links would work. I don't have time to experiment, but this seems worthwhile to try. Also, I realize this would break the account-specific nature of filters, but would it really be that hard to implement TB so it has both? Perhaps this has something to do with #3 and the way paths are handled internally. If so, then it seems some work on path-definition, taking account that many paths today are not local (esp. on iMap), is in order -- with something akin to absolute paths based on the address of the server's home directory (and not what TB decides to call the account based on its email address) -- may be in order. (Also see #6 below.)

5. Hoping to keep my filters (and tags) in sync in the future, I experimented with some network-capable synchronization software (mainly Synkron and rsync through various GUI front ends). But the naming problem described in #2 made this difficult, and eventually I opted for a 1-shot, manual migration. I think rsync might eventually be the solution, but I use it so infrequently that it will take a week or more to set up even one good synchronization. Then, because TB makes up the account names used in the profile, there would have to be separate sync scripts for each pair of devices. (Again, a directory named with the email address (e.g., JohnDoe@anywhere.USA) and perhaps with symbolic links to the directory as TB currently names them (e.g., imap.gmail-1.com) might solve this for the short term.) [Aside: When Apple tightened security on OS X by prohibiting apps from installing directly in /usr, the MacTeX group installed TeX in /usr/local instead; to allow updating without wiping out earlier work, the calling scripts were reconfigured to use /Library/TeX/texbin instead of the previous /usr/texbin. Both of the latter are symbolic links, so that changing versions is done simply by changing the link.] Nonetheless, some samples of working rsync commands for cross-machine synchronization would be immensely helpful.

6. Ultimately, the entire tag and filter mechanisms need to be rethought. Tags seem easier, but there are some obvious issues. For example, unlike filters, tag information is stored in a single file even though the tags themselves are stored in the iMap stores of different accounts or in the headers of downloaded, and therefore local, POP messages. The design criteria for tags need to be respecified, with portability and synchronization included in the list. Similarly, for filters. For filters, and perhaps tags, a more complex vision may be in order: some filters might apply only to a specific email account; others to all accounts; and there probably should be both machine-specific and cross-machine, synchronized versions of these. Moreover, synchronization probably should allow merging filters from different machines, so that someone using multiple devices can add or modify a filter at any time and have it propagate. Dealing with such things would make for some great "Summer of Code" projects.

This past week I migrated my email from an old, white MacBook to a (relatively) new MacBook Air. The old computer had been upgraded to a 500 MB SSD, but the new one came with only 256MB. Also, I use TB on two iMacs, each with over a TB of storage. For this reason settings on the MBA have to be different from those on the other machines, so sharing a single profile on a cloud drive is out of the question. Instead, I had to set up TB on the new computer, install necessary plugins, and then migrate tags & filters. In the second half of the second decade of the 21st century, synchronizing such things across different platforms has become quite common. So one would expect migrating such settings to take at most a few minutes. But somehow TB has stayed behind the times. So the process took me more than a day. In light of this pain, here are some suggestions for moving into the present. 1. Tags. Why the heck are they integrated into prefs.js files? These files also contain device-specific settings, so migrating the file itself would break other settings. Instead, one has to first learn what file holds the tags, edit the file, locate the tags (by the mailnews settings), copy-and-paste them somewhere, repeat the drill on the target machine, and then replace the tags already existing on the second device with the saved ones. Much of this could be avoided simply by using a separate file, tags.js, for tags only. Prefs.js could be modified to just include this file, or perhaps TB could read it directly. (I know TB stores tags for iMap messages as keywords on the iMap server, so in theory machines accessing an iMap account all have access to its tags. But tags also involve colors, which are stored in prefs.js. And surprisingly, newly created iMap accounts pointing to existing iMap stores do not even automatically import the tags from the store. So tag configuration is left entirely as an exercise for the user.) 2. Filters are trickier because they are account-specific. In my case, I had to go into the iMap section of the profile and then into the different accounts. The first problem is that even though I use only 4 different email accounts, the old profile had about 6-7 because some accounts became inactive. Even worse, among the active accounts there were two gmails (gmail and gmail-1). I suspect this is partly because my employer switched from having its own email to Google Apps for Education, so under the covers my account became a gmail account. The problem is I only know the account by its email domain, which doesn't mention gmail at all. So it was difficult to tell which gmail account does what on both machines, creating 2x2 confusion. Eventually I sorted things out, and I'm sure the email address is stored in one of the other files under the account's root directory. But how hard would it be to include a "accountname.txt" file in each account section of the profile to help the end user easily identify the account? 3. After migration, somehow several filters that moved messages to different folders broke. Fixing this was not too hard, but since the move was from one iMap folder to another, and both devices used the same iMap accounts, I don't understand how it is possible that a filter moving a message from one iMap inbox to another iMap folder works on one machine but not another. Perhaps if I understood it better, I could suggest a fix or improvement. Can someone please explain this? 4. Filters are account-specific, but some users have asked for filters shared by all accounts. This makes me wonder how a single msgFilterRules.dat file stored in a profile's root directory and then replicated in each account's root directory via symbolic links would work. I don't have time to experiment, but this seems worthwhile to try. Also, I realize this would break the account-specific nature of filters, but would it really be that hard to implement TB so it has both? Perhaps this has something to do with #3 and the way paths are handled internally. If so, then it seems some work on path-definition, taking account that many paths today are not local (esp. on iMap), is in order -- with something akin to absolute paths based on the address of the server's home directory (and not what TB decides to call the account based on its email address) -- may be in order. (Also see #6 below.) 5. Hoping to keep my filters (and tags) in sync in the future, I experimented with some network-capable synchronization software (mainly Synkron and rsync through various GUI front ends). But the naming problem described in #2 made this difficult, and eventually I opted for a 1-shot, manual migration. I think rsync might eventually be the solution, but I use it so infrequently that it will take a week or more to set up even one good synchronization. Then, because TB makes up the account names used in the profile, there would have to be separate sync scripts for each pair of devices. (Again, a directory named with the email address (e.g., JohnDoe@anywhere.USA) and perhaps with symbolic links to the directory as TB currently names them (e.g., imap.gmail-1.com) might solve this for the short term.) [Aside: When Apple tightened security on OS X by prohibiting apps from installing directly in /usr, the MacTeX group installed TeX in /usr/local instead; to allow updating without wiping out earlier work, the calling scripts were reconfigured to use /Library/TeX/texbin instead of the previous /usr/texbin. Both of the latter are symbolic links, so that changing versions is done simply by changing the link.] Nonetheless, some samples of working rsync commands for cross-machine synchronization would be immensely helpful. 6. Ultimately, the entire tag and filter mechanisms need to be rethought. Tags seem easier, but there are some obvious issues. For example, unlike filters, tag information is stored in a single file even though the tags themselves are stored in the iMap stores of different accounts or in the headers of downloaded, and therefore local, POP messages. The design criteria for tags need to be respecified, with portability and synchronization included in the list. Similarly, for filters. For filters, and perhaps tags, a more complex vision may be in order: some filters might apply only to a specific email account; others to all accounts; and there probably should be both machine-specific and cross-machine, synchronized versions of these. Moreover, synchronization probably should allow merging filters from different machines, so that someone using multiple devices can add or modify a filter at any time and have it propagate. Dealing with such things would make for some great "Summer of Code" projects.

Modified by Gnosos

All Replies (1)

more options

The recommended approach is to copy the profile folder from the old device to the new. It does not take hours and migrates everything.

Not to address your other statements.

1. Tags are in the prefs.JS file because Thunderbird allows you to define lots. IMAP server often restrict you to about 5.

2. Not an issue if you move the profile folder.

3. You are moving from one disk location to another. The IMAP move is not done by the filer as such. the move for most IMAP server has to be presented as a delete and an add, as they do not support move.

4. Poorly. Just about everything in windows using symbolic links is fragile to say the least. OSX has not done anything to enhance their implementation for years. move the file the symlink points to an it just breaks.

In the end, copying the profile folder ends almost all of the content of your message.