Firefox does not handle large bookmark "files" gracefully, it HANGS
Essentially large #'s ( massive ) cause Firefox to go off the deep end and never shut down in any reasonable time. I tried backups in JSON and HTML, files produced took ~2-3 days, 50mbyte and ~90mbytes each. Essentially when you exceed past some indeterminate bookmarks "size" Firefox never gracefully in some useful time frame shut down, folks here have said when firefox is attempted shutdown and if it does not, then go to task manager and shut the Firefox process. THat is less desireable. Question is when firefox closes, ( request to shutdown ) is it trying to do a whole host of task cleaning ./ bookmarks backup? I tried cutting back my books marks slightly, now FF can be closed without Taskmanager shutdown, but now none of my addons ( that I disabled and deleted some ) none seem to restart ( Firefox seems to thrash CPU utilization from near zero, to ~100% via trying to restart NoScript, and pounds away for many minutes, until I say this is not good ).
I have Vista Business 32, configured to work well latest FF 26.0.
Chrome ( SR Ironware Iron browser ) that I have zero bookmarks runs fine, I need FF to work reconfigured without a reinstall due to all the passwords I have loaded into the FF over time.
GIven that a minor ?10-20% reduction in Bookmarks that took me a few hours to do, returns the ability to not have to go to taskmanager to brutally shut the FF task when an exit from FF does not remove it and it s large memory footprint, and now mostly does, with minor reduction of a massive bookmarks set, this is weird. FF never exceeds my win 32 memory, largest footprint I had I think was ~1gb in memory from lots of web pages.
But presently I cannot seem to restart Noscript... which is troubling
All Replies (5)
Over how many bookmarks are you talking?
Over 50,0000 or 100,000 or more?
Note that current Firefox releases have the number of bookmarks in the name of the automatic JSON backup in the bookmarkbackups folder (xxxx in bookmarks-####-##-##_xxxx.json).
Did you check for a possible corrupted file places.sqlite?
Hi anonmice,
Sorry you are having problems. It may have been easier had you asked earlier on as the problem developed rather than waiting, but it still will be fixable, or have a workaround. It is just takes more effort to fix.
I have hundreds of pages of bookmarks in one Library (profile), but it backs up near enough instantly to a file of about 5MB. I know at one time at least some operations on the bookmarks could reportedly cause hangs although I have not seen much mention of that recently. (Someone with 170k bookmarks did have some problem).
Maybe if you have massive mountains of bookmarks you should use multiple profiles and so split bookmarks up. Not all tasks will need all bookmarks. Have a profile for different tasks or subjects.
If you have duplicated bookmarks there may be addons to help once the bookmarks are behaving better. In the medium/long term I suspect you are going to need to
- Set up multiple Firefox profiles, with the intention of splitting tasks and bookmarks between profiles.
- Also or alternatively manage the massive bookmarks collection externally. Setup folders of HTML bookmarks. Use a Word Processor to sort and arrange the bookmarks and folders of bookmarks.
In the short term and to get Firefox running without losing passwords AND to preserve bookmarks data
- Back up the complete Firefox Profile, and put it somewhere safe outside the Profile and program paths, the Desktop is ok, but a documents folder may be a better location.
- Create a new profile, probably a good name is Main_2014 with the intention of moving on later to modify and use that one and another Test_2014a
- Good practice at least for now, use suggested locations.
Do not:- Nest profiles or use anything but empty folders for the location.
- Rename or delete profiles
- See Profile Manager - Create, remove or switch Firefox profiles
- Good practice at least for now, use suggested locations.
- After first opening the new main profile disable
it'sits existing database file places.sqlite maybe call it places.sqlite.oldM01- Opening Firefox from that profile is the easiest method of locating and opening the correct profile folder (So safer than deleting that database file in advance of opening the profile)
- Open the profile folder by using the option in the Troubleshooting Information page. Then close down Firefox and rename the places.sqlite file AND rename the bookmarks backup folder
see Use the Troubleshooting Information page to help fix Firefox issues
& Profiles - Where Firefox stores your bookmarks, passwords and other user data - That profile will now have all passwords etc but is a clean slate as far as bookmarks go. This should behave as normal and have no problems or hangs.
- Do something similar with the test profile, but this time disable the database by renaming it BUT do not rename the bookmarks backup folder.
- This will have a recreated bookmarks database. It will recreate the database from a bookmarks backup It may behave better (Or it may have used another problematic file). There are other things we can try later.
You may be able to do all that and then report back or you may have further questions before attempting this.
Modified
I am not a software engineer, merely a user, I am not asking for help for me to personally fix what is a fundamentally a product software problem.
I'd also point out that in addition to Hanging in backups, 3 days actually of FF running the bookmarks backup that produced 50mb JSON file and 90 mb HTML backup file, after 1/2 day respite of actually being able to have firefox close on user exiting after i completed the backup and reduced bookmarks slightly ( ~10% deleted ), now it is back to its usual self, taking forever to shut down, looking like a memory hog monster with >0.5 GB ram footprint, and often grows upon exit that the program persists in Task Manager Process List ad nauseum, till a manual task manager forced shutdown.
A user ( not a software coder ) should not have to twiddle with arcane settings buried in some obscure file undocumented, to avoid having a program hog their machine for 3 days just to backup Bookmarks manually ( meaning the code in the bookmarks backup procedure is coded rather inefficiently when scaled to large # of bookmarks ),
and I suspect FF shutdown internal procedure similarly does some rather inefficient logging to needlessly find grained writing to files on disk upon request by user to EXIT ( hopefully shut in reasonable time to free the dual core CPUs from ?50-100% utilization for days to backup / save stuff I could care not enough to wait for days for backup completion.
If RAM was used to assemble the backup in memory entirely, then to enable doing a small series of long serial block writes to disk, I suspect this all might go smoother than what appear to be a huge series of small writes to the disk as present it seems.
Possibly less sensible at present, and I can tell this by watching the disk activity light slowly flashing and Task Manager CPU activity spiking 100% and dropping regularly ( at high frequency ) for hours or days in the case of manual backup.
Load ALL the data to RAM instead and do one large sequential write ( sort of obvious for performance of the code and avoiding days to backup bookmarks ? ) There is plenty of room to load the JSON or HTML directly all to ram, and massage, and dump in one long faster sequential write to disk? Or am I missing something?
The justifications explained are almost silly, as I merely want to use FF, not make a career of learning undocumented buried feature settings. I have better things to do with my time, and I will remark the issues I describe might not be unique to FF's browser, it might also be common on other broswers ( I do not know, but I will be gracious enough to say so ).
Granted the program is free, but memory and task management efficiency is mediocre for truly heavy users. I read a lot and have sometimes 100 tabs open, the manner this is handled with FF sometimes brings a dual core CPU in Win Vista ( optimized via reg settings recommended in deeper articles about VIsta to be more XP like in operation ) to its knees.
This reminds me of the creeping featurization of GMAIL where originally GMAIL was fast and efficient, impressively so, FF might not properly vet ADDONS for good memory and shut down behavior and might consider doing so, to block those which affect stable operation, from being downloaded, as might actually testing / validating the safety & security & privacy of some addons ( rather than presently leaving to unsophisticated users to do so haphazardly ).
just some obvious observations.
It should not take common users to fiddle with undocumented obscure features ( hidden settings in undocumented files ) to make a quality product ( FF is good ) run properly. There is something to think about here.
I am not a software engineer either, I am merely a Firefox user like you.
Your Firefox install is obviously broken at present. It is easy to do a clean install and get it working again. I am trying to give you a fix that preserves as much data as possible, and a method of moving forward with use of Firefox.
You did not answer the question asking
- how many bookmarks you have ?
It may be that you have more bookmarks than Firefox is normally expected to be used with. As I pointed out I have hundreds of library pages of bookmarks without having problems. I am guessing you are using hundreds of thousands of bookmarks. You may have problems in addition to; or in part because; of your large number of bookmarks.
Not sure what you are saying the software fault is. I am however thinking that if you have made all these bookmarks they could be important to you. You have already said your passwords are important to you.
As for the memory use that is a slightly different subject. 1/2 GB may not seem excessive to some users and on some machines will not cause any problems at all.
- I wonder how much RAM you have installed on the machine that is in use ?
NoScript and Skype may possibly be part of the reason your memory use could be higher than may be the case compared to a basic install of Firefox.
Your hangs again may or may not be related to other issues you have.
Please remember Firefox is highly customisable and runs on many platforms. Problems you see may not affect others.
We are fellow Firefox users. We can help you fix your problems, if you wish but if you just need to make requests for Firefox enhancements you need to do that elsewhere. (I am able to make suggestions on that if it would help.)
- Running Firefox in a new profile may well get it working properly again. Have you tried that yet ?
Alternatively just try the Firefox Reset.
That option is available from the Troubleshooting page, and is designed to be quick and easy to do by ordinary users.
The problem with the reset, and the reason I did not suggest it earlier is that it may not solve all your issues, and it probably will leave you with the bookmarks problem.
TAKE CARE
If you do try any uninstall, do NOT use any option that removes personal data or you will lose passwords and bookmarks.
P.S. If I knew how many bookmarks you have I could see what happens, but I doubt it would take three days to back them up. I would however point out
- The maximum size of the places.sqlite is probably related to the amount of Ram available. So my test may not be comparative.
- IIRC It is likely that History may be lost if places maxes out.