Intermittent appearance of two copies of a "random" old e-mail in my in box, as if the message had just been received.
Here are details from the most recent 'zombies' to pop up in my in box. It appears that random 'old, already received messages' appear at random, occasionally. EACH time this occurs TWO copies of the message, with identical data, appear in my in box as if the message had just been received, but with the original data time showing. SUMMMARY of important date time info. TIME APPEARING IN THE INBOX ORIGINALLY RECEIVED From - Fri Nov 11 06:51:19 2016 for <my_id@cox.net>; Thu, 23 Jul 2015 00:11:03 -0400 From - Fri Nov 11 06:51:19 2016 for <my_id@cox.net>; Fri, 28 Oct 2016 15:21:57 -0400 From - Fri Nov 11 00:10:49 2016 for <my_id@cox.net>; Sun, 3 Jul 2016 19:08:55 -0400 From - Wed Nov 09 14:11:19 2016 for <my_id@cox.net>; Wed, 18 May 2016 07:23:05 -0400 From - Sat Nov 05 01:30:11 2016 for <my_id@cox.net>; Fri, 12 Aug 2016 07:21:51 -0400 From - Sun Nov 06 21:50:13 2016 for <my_id@cox.net>; Thu, 9 Jun 2016 14:42:50 -0400 From - Sun Nov 06 09:40:12 2016 for <my_id@cox.net>; Sat, 30 Apr 2016 17:41:27 -0400 From - Wed Nov 02 16:58:35 2016 for <my_id@cox.net>; Fri, 12 Aug 2016 07:21:51 -0400 From - Wed Nov 02 16:58:36 2016 for <my_id@cox.net>; Fri, 12 Aug 2016 08:25:46 -0400 From - Wed Nov 02 05:44:42 2016 for <my_id@cox.net>; Sat, 4 Jun 2016 10:12:33 -0400
The first one I noticed and collected. I may have deleted others earlier before I realized what was going on: From - Sun Sep 04 15:16:09 2016 for <my_id@cox.net>; Mon, 2 May 2016 13:10:32 -0400
Today, I ran a packet capture program (Wireshark) and found that Thunderbird is REQUESTING that the cox server send the messages. Here is a RETR request:
No. Time Source Destination Protocol Info
27098 15437.547114 192.168.0.5 pop.east.cox.net POP Request: RETR 11027
The data sent by the mail server in response was the latest 'zombie' that appeared.
Why is thunderbird asking for a re=transmission of some random message from my inbox?
What needs to be done to stop this?
All Replies (7)
Is it set to leave messages on the server? If not, then these messages shouldn't still be on the server, right? Maybe the server is at fault for not deleting them?
Do you habitually leave messages in the Inbox?
Maybe your popstate.dat file is scrambled. Try deleting it. Yes, you'll probably get one more set of duplicates.
I have always left message on the server, for over 20 years and have used thunderbird and kept it up to date. Never had this problem before sometime this year.
I did rebuild my thunderbird profile back on 3/6/2016, when my computer crashed. Maybe that broke things.
But thunderbird 'usually' works fine, just these rare but annoying zombies that pop up.
I would say this is some indexing issue. Did you try "repairing" the affected folder by right-clicking it and selecting Properties > Repair Folder ?
Leaving messages on a POP server should be fine as long as you don’t run out of space there (check occasionally). However I have experienced a similar issue where hundreds of existing messages were retrieved as new as a result of the ISP doing some "rebuilding", causing a number of them to get new POP IDs (not message IDs, but what follows after the LIST command) so they appeared as new, but identical. Something similar could have happened for you, OR it’s a local glitch for 2 of them you could repair, either as above (I would try first) or via popstate.dat.
You could of course set up the account a second time for testing and let it fetch the headers only without removal to see how many of them really exist (I wouldn’t rely on webmail showing proper content.) Since ISPs probably don’t duplicate messages for you, either popstate.dat didn’t catch up properly (but should still do so at another retrieval if so), or the POP’s ID changed. If the zombies are really random however, an indexing issue might be more obvious.
I 'captured' a copy of my popstate.dat (actually saved it with the name popstate.dat.bad). Currently have it open and have the 'active' popstate.dat also open in notepad+. Occasionally notepad+ notes that the active version has changed and asks me if I want to update the copy I am looking at. So this gives me a very good way to know when popstate.dat has changed.
I then do a 'copy all' and paste to one pane in WinMerge. My 'bad' copy is in the other pane of WinMerge.
I then do a 'refresh' and look at the differences between the two versions.
So far, the updated version has just occasionally ADDED a new line of data , in the midst of the other data.
No zombies have shown up in thunderbird yet. Should be interesting to see what shows up in the active popstate.dat when that happens.
Still no zombies. In studying the popstate.dat file, it seems like the format is rather simple with each data line something like this: k <75su1r00c45oxQM015sv4W> 1473151001 I am guessing the 'k' is some kind of flag. The next term is the message ID and appears in the message header in normal messages in at least two places: near the top of the header in an X-UIDL line X-UIDL: <75su1r00c45oxQM015sv4W> and later in the header as a message ID line Message-Id: <75su1r00c45oxQM015sv4W> The "1473151001" appears to be a date time code of some sort. Not yet sure how it relates to the From - Tue Sep 06 03:26:41 2016 at the top of the header, but I believe it does. I notice that the headers of some of the 'zombies' do seem to be rather mangled, missing some parts of the fields, but I don't see any corruption in my popstate.dat file.
I am noticing that thunderbird often rewrites the popstate.dat file without making any changes to it.
Athraithe ag bz_tb ar
There just appeared in the popstate.dat file, a line:
k <z 1479150334
This clearly does NOT match the normal format!
AND a zombie just popped up in the inbox!
From - Mon Nov 14 13:05:34 2016 X-Account-Key: account1 X-UIDL: <Z
A few moments later the OTHER copy of the zombie appeared, with this header:
From - Mon Nov 14 13:15:34 2016
X-Account-Key: account1
X-UIDL: <5q5Z1r00a10koAM01q5alu>
So, the problem is something that thunderbird is doing that is 'corrupting the popstate.dat file!
Thunderbird has removed the
k <z 1479150334
line, probably when it created the 2nd copy of the zombie message.
These clues should tell the thunderbird programmers that the problem is something in their code failing to handle something and ADDing a spurious line of garbage to the popstate.dat file and then attempting to fix this and messing things up even more.
Further clue. This may have happened right after I had a message I transmitted 'bounced' because it called for the wrong server in the settings.
If so, the problem is that thunderbird isn't handling such errors correctly and it is corrupting its own database when the occur.
Athraithe ag bz_tb ar
There is another possibility. Assuming you are talking about the 'Inbox'. If the Inbox file becomes corrupted, one signal that something is wrong is reappearing mail. One reason that can cause this is a lack of regular compacting of the Inbox.
Corrupt folders tend to occur when you have lots of messages in a folder, many of them are deleted, and you don't compact often enough. This is usually only a problem for the inbox folder as while other folders may get very large you typically don't delete messages in them often. It's recommended that you keep the number of messages in your inbox small, and move any you want to permanently keep to other folders/child folders.
I would suggest you do the following:
- Move all good wanted emails out of the 'Inbox' into other suitably named folders.
- Delete all unwanted emails.
- When 'Inbox' is empty: right click on 'Inbox' folder and select 'Compact'.
Suggest you also: Right click on 'Junk' folder and select 'Empty Junk' Right click on 'Junk' folder and select 'Compact'.
Thanks for the suggestion. I have moved the ~15,000 messages from 'inbox' to 'old_received_messages' and will see if that stops the problem.
November 30. Still no more new zombies seen. Up to 608 lines in the popstate.dat file
Athraithe ag bz_tb ar