Thunderbird -- Changes to files (size increase) on 11/30/2023
My backup tool blew up due to 1000's of Thunderbird files being modified.
I can't tell if this was due to some kind of malware, or if Version 115.5.1 of Thunderbird went through the database/file structure and wholesale modified everything. If the former, I don't want to backup the files (yes, I have run BitDefender of the profile structure: all 125K files and it comes up clean). If the latter, and everything is operating as normal/expected, then I do. I have seen no issues with the operation of Thunderbird (which I love, BTW).
So, did Thunderbird change 1000's files that were otherwise not accessed by me? (see example in attachment)
I am currently running the Version 115.5.2 (64-bit) / December 11, 2023. I presume this event followed shortly after the installation of "Version 115.5.1, first offered to channel users on November 27, 2023."
All Replies (9)
If relevant, running Surface Pro 6 with Edition Windows 11 Home Version 22H2 Installed on 2022-11-24 OS build 22621.2861 Serial number 0709XXXXXXXX Experience Windows Feature Experience Pack 1000.22681.1000.0
I do not run this in conjunction with any cloud-ware (i.e., no synchronization with other devices). The vast majority of these files are local backup/archive, and are not shared with the email server / ISP.
Athraithe ag jonathan67 ar
--removed--
Athraithe ag jonathan67 ar
I really have no idea what your issue is. I do not think that there has been any wholesale changes. Without having some idea what the files were I can not even begin to guess.
I have some maildir enabled accounts (one file per email) and they frequently see files stamps change as the imap sync works with them. Most of my local folders (which a rather large files) are opened at least daily, either to view or update them.
Matt said
I really have no idea what your issue is. I do not think that there has been any wholesale changes. Without having some idea what the files were I can not even begin to guess. I have some maildir enabled accounts (one file per email) and they frequently see files stamps change as the imap sync works with them. Most of my local folders (which a rather large files) are opened at least daily, either to view or update them.
@Matt, thanks for responding.
The vast majority are folders. Those should not have been touched by any software between backups, including TB (I avoided using any folder names with identifying metadata). These subset-of-examples (the majority of changed folders as similar in the application) are strictly local and not shared with IMAP or other SW. All archiving is done locally and manually. Only active mail is left on the server. For example:
"C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\Mail\Local Folders\PEOPLE.sbd\_History" "C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\Mail\Local Folders\PEOPLE.sbd\_History.msf" "C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\Mail\Local Folders\PEOPLE.sbd\Inbox" "C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\Mail\Local Folders\PEOPLE.sbd\Inbox.msf" "C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\Mail\Local Folders\PERSONAL.sbd\Finance.sbd\Bank of America" "C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\Mail\Local Folders\PERSONAL.sbd\Automotive.sbd\4runner"
Sync'd folders/files also changed, but I can't be assured that they weren't touched by the mail server. These change frequently and constantly. There are less than 20 of these folders. Example:
"C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\ImapMail\hostxxxx.hostmonster.com\INBOX" "C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\ImapMail\hostxxxxx.hostmonster.com\INBOX.msf" ...Sent & Temp & Draft & Spam & Trash & Archive....
As a side thought: I'm used to various programs like music players leaving crumbs within files after opening them. But, I've never seen this done in TB before. I might convince myself that MS might do the same when indexing, but those indexes shouldn't be changing -- they never have before. Similarly, I suppose BitDefender could be making changes when scrubbing the system. But, again, why? In the end, the close time association between the prior TB version upgrade and the change in the timestamps of the files makes it look like TB might have been messing about. Or, I picked up some malware about the same time. Or, ???
Athraithe ag jonathan67 ar
example :"C:\Users\***me***\AppData\Roaming\Thunderbird\Profiles\b51any78.default-release\Mail\Local Folders\PEOPLE.sbd\_History"
I presume the image showing Size and date info relates to the file eg: 'History'
The 'History' is an mbox text file which contains a load of emails appended to the file in the order of when put into that file. You will see the History file contents in the Folder Pane as a folder called 'History'.
The 'History' file can get decreased in size if you have deleted/moved emails out of it and then compacted the folder. But it will not show a decrease in size if you have only deleted/moved emails and Not compacted. So, if a compact on folder occured maybe because you ran it manually or an auto compact on all folders occured then suddenly you may see a decrease. Obviously, an increase occurs when additional emails are added to the file.
@Todd,
Thanks for responding.
I clearly haven't been sufficiently clear. Sorry.
It is possible that 10's of folders, both sync'd and local-only, were due to adds/deletes/moves (or maybe even access to the contents thereof?). B U T, we are talking about many 100's of folders (well over 1000 files in the profile folder) the vast majority of which have not been touched by me nor should have been touched by any synchronization/archiving/optimization process in years (archived files).
While I would not put it past MS Windows to do something like this, I would not expect MS Windows to do it only to the TB files. Similarly, I can't think why any software on my machine would target only those files (most don't even have access).
Regarding "history" files, let's make sure that your are referring to the "Mail\Local Folders\PEOPLE.sbd\_History" files I showed in the prior example list (probably a poor choice to put in the example list). That folder is my folder within my "people" folder and archives emails related to a human history project that has nothing to do with TB. I note that the last new email added to that folder was in July 2021 (see attachment; I note that the "Local Folders" directory shown in the attachment is not synchronized anywhere; it is 7.5 GB is size). There is no reason to believe any changes should have been made to it since then.
Regarding the prior, mentioned attachment, that simply shows the timestamps of the last changes on a variety of folders=files within the aforementioned directory. If we were to scan through that directory, we would see that all the 1000+ files in question were changed within seconds of each other, further indicating that this was not done by directly or indirectly by me. That would literally impossible for me to do.
So, the original question remains. Before I go though the painstaking process of restoring everything back to the state prior to the TB Version 115.5.1, how can I know if TB did some kind of housekeeping (indexing, metadata update, restructuring, whatever) that changed and increased the size of 1000+ files in the profile?
Athraithe ag jonathan67 ar
The simple act of compacting, which occurs automatically by default rewrites all of the folders it replaces. Generally this does not affect folders that only grow, and compacting is about rewriting the files that have deleted massages stored in them. But having said that I have never checked post compact if it is every file or just the new ones that have changed attributes (compact writes new files so they get new attributes and deletes the old)
The update to 115 will almost certainly have touched all the MSF files as the new UI will have seen additional information stored in them. They are a MORK formatted file and one of the objectives of the development team is to remove all instances of MORK files from the code base. As they are a self describing file format, adding stuff is almost always backward compatible. It is removing stuff or changing it that causes issues. The MBOX files, I can see no reason for them to change size other than adds or deletes, the underlying mbox format has remained unchanged for decades. Having said that I am aware that a lot of work is going on in the storage space to try and untangle 20+ years of patches and modification that have become so intertwined that change is often closely linked to regressions.
Fundamentally you will not get a concrete answer about what if anything Thunderbird actually did. Update logging is turned off by default, so you can not even check the logs. Personally I really do not understand what you are talking about restoring files for. All you will do is blow up your backup tool again if it was a result of the update, and it probably was. All I can say is looking in the IMAP mail folders of my profile I have files and folders with years old file dates.
If you have backups as you say you do, compare one file and see what has changed in it. Just don't look at MSF files as they should update if the underlying MBOX updates.
Thanks again for the feedback. Looks like I don't have the right tools to compare files and get definitive/quality answers.
For example, using MS Word to compare two files (i.e., folder) with one mail item, MS showed the "revision" in the first attached screen capture. I could find associated change in the two files by comparing side-by-side.
Again, these are additions to the original file, hence the apparent increase in file size.
The second capture is from the associated MSF file comparison, there the difference was more obvious, but that doesn't mean the Word isn't hiding something.
These files where chose to show here because it is simple, and doesn't contain any personal information. And, no, there is no reason to believe that any editing tool was used to change formatting in the first.
Happy to forward the four files under separate cover if that helps.
I do not think having the tools are the problem. Every copy of windows since 1885 has had a copy of notepad. But using a word processor to compare text files is almost guaranteed to fail. Like I look at those images and get told something about formatting of the document and bullets . Text files just don't have bullets to reformat.
There are far more capable free text editors out there than notepad, like notepad++ that copes well with large files and can also compare files.
Try it. There is a tutorial here https://www.makeuseof.com/tag/notepad-compare-two-files-plugin/ It looks like it would work for you much better than word.
I have never used notepad++ for comparison, hence the link to a tutorial. I use the tools for it's built in MIME tools and it's ability to color code various file formats.