Compacting request repeats endlessly - something is corrupted
TB 102.8.0.
Since December 2022, compacting is not working properly. I have set TB to ask me whether to compact when 100 MB would be saved. This has worked fine for years. Since December, I receive a pop-up advising that 237 MB or slightly more can be saved. After compacting, the message repeats every hour or two.
Compacting is actually, working except for the 237 MB chunk of data that either can't be compacted or doesn't really exist and fools TB into thinking compaction is needed. For example, if I set the compaction threshold to 500 MB then the mailbox will grow and I will eventually receive a reminder to compact when more than 500 MB can be saved. Compacting at that point does indeed shrink the mailbox. But, immediately thereafter, if I set the threshold back to 100 MB I get a message that 237 MB can be saved by compacting. Thus, the compact request repeats endlessly, all day long.
Technical details can be found here: https://bugzilla.mozilla.org/show_bug.cgi?id=1791354
I can't take it any more. Does anyone have a suggestion for forcing TB to re-examine or re-index my inbox in a manner that correctly determines its size and ends this nuisance?
Thanks.
Gekose oplossing
This problem appears to be solved, using the method on Toad-Hall's post, but I had to track down the correct folder to copy / delete / restore.
Despite the e-mails identified in the gloda error messages, the phantom message(s) were different and were located somewhere else. Once I found out where, copied that folder in TB to a new folder, and deleted + renamed the copy in my profile, as described above, I stopped getting compaction requests. Since then, compaction has worked as expected.
One way to figure out which TB folder is bloated is to (1) compact (which makes the next step cleaner) and then (2) make a new folder in TB and copy all messages into it, as described above, one folder at a time. Then, look in your TB profile in the place described by Toad-Hall. The old and new folder sizes should match. If they don't, that's where the problem is.
Thanks again for the instructions!
Lees dié antwoord in konteks 👍 0All Replies (13)
Check your folders for corruption:
- Right click on folder and select 'Properties'
- Click on 'Repair folder' button
- click on 'OK'
Then force a fresh global database to be created.
- Menu icon > Help > More TRoubleshooting Information
- Under 'Application Basics' half way - Profile Folders - click on 'Open Folder'
A new window opens displaying the contents of your profile name folder
- Exit Thunderbird now
Delete the following file:
- global-messages-db.sqlite
Restart Thunderbird.
Empty the 'Junk' folder. Empty the Trash. Do a manual compact on each folder in turn:
- Right click on folder and select 'Compact'
Once you have done all of the above, wait and see if you still get same problem.
Thank you, but no change. (I do not have a junk folder, and I did not empty the trash because I use it as storage).
After deleting the sqlite database, I watched TB go through my folders and claim to compact each one, and in the end TB claimed to save saved about 240 MB of space. But once again, I am getting a message that 238 MB can be saved.
FWIW, I know exactly which folder in my inbox has this "space" to be compacted. If I compact the folder, TB tells me that it saved 235 MB. If I compact it again right away, I again see that TB saved 235 MB. If I restart TB immediately afterwards, I get the same warning message that I need to compact and that I can save 238 MB....
One thing: global-messages-db.sqlite, when it regenerated, dropped from 1.46 GB to a tidy 9 MB or so. But it soon grows back to 375 MB.
Sorry to report this. Would welcome other ideas.
The deleted folder is not storage. The number of folk I have seen in th9is forum lamenting the sudden and inexplicable loss of their trash because they used it for storage is quite high over the years. Particu8larly in IMAP account were providers implement their own retention strategies that do not include the Trash/Deleted, which is replicated to Thunderbird.
I suggest you think of another strategy. Like having a storage folder in your account to avoid disappointment.
Do you have a third party product like an anti virus scanning in your Thunderbird profile? Is the profile on a local disk, or stored in some more volatile location like a network drive or the cloud or a USB device. Is the profile being backed up or streamed to somewhere like the cloud. Do you have nstmp folders appearing in your folder pane? When you compact that folder open the error console (ctrl+ shift+J). Clear it using the trash icon and try the compact. What messages appear in the error console about the failure to complete the action? (Right click to copy them)
I'm not lamenting the loss of my trash folder. But I've emptied it as you requested.
No antivirus. Profile is in the standard Windows location (.../AppData/Roaming/.....)
The profile is backed up when I back up my computer. It is not "synced" with anything and not being over-written by a backup copy, if that's what you were thinking.
No ntmsp folders.
The latest error log contains no messages about TB's inability to complete anything. It doesn't like my window layout and can't find "eastern time" but that's it. If you look at this bug report, however, you can see an example of the "sketch_key" messages that accompany this problem.
https://bugzilla.mozilla.org/show_bug.cgi?id=1791354
I'll paste those messages here when they reappear.
The 'sketchy key' error mentions a Pop mail account and the 'Sent' folder.
Please do the following:
- Create a folder called 'OldSent'
- Get copies of all your emails currently in 'Sent' and put them in 'OldSent'. I suggest you do it in batches as there may be a lot of emails.
When you have all the copies you require in the 'OldSent' folder do the following:
- Menu icon > Help > More TRoubleshooting Information
- Under 'Application Basics' half way - Profile Folders - click on 'Open Folder'
A new window opens displaying the contents of your profile name folder
- Exit Thunderbird now
Delete the following file again because it will contain data about the Sent folder which is about to get a real sort out.
- global-messages-db.sqlite
- Click on 'Mail' folder
- Click on the pop mail account name folder
Delete the following files to completely clear the 'Sent' folder.
- Sent (no extension)
- Sent.msf
- If you also have a 'Sent.sbd' folder - delete that folder as well.
Start Thunderbird.
When you next send an email, Thunderbird should auto create a new 'Sent' folder because you are using a Pop mail account.
Please wait and report on results. Do you still get the compact will save message or has it now been fixed?
Do you have any 'sketchy key' errors that mention another folder ?
So we understanbd how you manage your pop account can you tell us a bit more about the Inbox. Do you keep your 'Inbox' low in numbers of emails - meaning you have created other folders to organise and store your emails. eg: folders called 'Bills', 'Online Orders', 'Friends', 'Family' etc ?
My inbox is as you describe, with subfolders. There are about 6200 messages in my inbox proper, and about 50 subfolders and most likely 25,000+ messages in there. The Inbox.sbd folder takes up 27 GB.
I have 32 sketchy_key messages in the error console, and all of them point to a subfolder of the inbox. Here is the first of them:
gloda.index_msg: Observed header that claims to be gloda indexed but that gloda has never heard of during compaction. In folder: mailbox://[user]%40[domain].net@pop.ionos.com/Inbox/clients/[CLIENT_NAME] sketchy key: 1178473992 subject:[SUBJECT IS HERE] IndexMsg.jsm:1356:21
This message exists and was received in December 2015.
I have not followed the instructions above for the /clients/[CLIENT NAME] folder in my inbox because it's not a standard POP3 folder and I don't think it would be re-created for a fresh start.
Appreciate your time and expertise here, thank you!
As you "think" you know what folder is the root cause, please have a look in the file system that the file and MSF for that folder and ensure that neither are marked as read only. I would expect there to be an error is that were the case. But I have learned to no assume anything.
Nothing in the mail directory is read only:
C:\Users\[NAME]\AppData\Roaming\Thunderbird\Profiles\[profile].default\Mail\[foldername]>dir /S /A:R
Volume in drive C has no label. Volume Serial Number is 661D-ADB9
File Not Found
re: pop.ionos.com/Inbox/clients/[CLIENT_NAME] folder - I have not followed the instructions above for the /clients/[CLIENT NAME] folder in my inbox because it's not a standard POP3 folder and I don't think it would be re-created for a fresh start.
Create a new subfolder in the 'clients' subfolder and call it the same name but add a number on the end - 'CLIENT_NAME2' Select the problem CLIENT_NAME folder Highlight batches of emails - right click on highlighted emails > select 'Copy to' > select the 'CLIENT_NAME2' folder. Repeat until you have got copies of all those emails into the 'CLIENT_NAME2' folder.
Now Access the error console and empty it.
Then do the following:
- Menu icon > Help > More TRoubleshooting Information
- Under 'Application Basics' half way - Profile Folders - click on 'Open Folder'
A new window opens displaying the contents of your profile name folder
- Exit Thunderbird now
Delete the following file again because it will contain data about the Sent folder which is about to get a real sort out.
- global-messages-db.sqlite
- Click on 'Mail' folder
- Click on the pop mail account name folder
- Click on 'Inbox.sbd' folder
- Click on 'clients.sbd' folder
Delete the following files:
- CLIENT_NAME (no extension)
- CLIENT_NAME.msf
Rename the following file:
- CLIENT_NAME2 (no extension) rename as CLIENT_NAME
Delete the .msf file as mentioned below - it will get recreated when you restart.
- CLIENT_NAME2.msf
Start Thunderbird
Wait and see if you get the compact message. Check error console. Report on results.
Gewysig op
Steps completed. Compact message appeared on startup. 257 MB to be saved. Error console empty.
I manually compacted the Inbox and it removed 26 MB of stale data. This seems like a real compaction. I then compacted the folder that I suspect has the phantom messages in it -- which is not the CLIENTS folder -- and got a message that 235 MB had been saved.
No error messages in the console.
Restarted TB. This time, no message about needing to compact. I'll report back when TB next asks me to compact. It is set to ask when 100 MB has accumulated.
The cat came back.... 247 MB to save, this time. (I've been cleaning house a little)
Clicking Compact generated about 75 of these, right after the folder that I am suspicious of was compacted:
gloda.index_msg: Exception while attempting to mark message with gloda state afterdb commit Exception { name: "NS_ERROR_ILLEGAL_VALUE", message: "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIMsgDBHdr.getUint32Property]", result: 2147942487, filename: "resource:///modules/gloda/IndexMsg.jsm", lineNumber: 154, columnNumber: 0, data: null, stack: "_commitCallback@resource:///modules/gloda/IndexMsg.jsm:154:33\nhandleCompletion@resource:///modules/gloda/GlodaDatastore.jsm:64:11\n", location: XPCWrappedNative_NoHelper }
Gekose oplossing
This problem appears to be solved, using the method on Toad-Hall's post, but I had to track down the correct folder to copy / delete / restore.
Despite the e-mails identified in the gloda error messages, the phantom message(s) were different and were located somewhere else. Once I found out where, copied that folder in TB to a new folder, and deleted + renamed the copy in my profile, as described above, I stopped getting compaction requests. Since then, compaction has worked as expected.
One way to figure out which TB folder is bloated is to (1) compact (which makes the next step cleaner) and then (2) make a new folder in TB and copy all messages into it, as described above, one folder at a time. Then, look in your TB profile in the place described by Toad-Hall. The old and new folder sizes should match. If they don't, that's where the problem is.
Thanks again for the instructions!