TextDecoder.decode() allocation size overflow for session restore
2019.04.11, 7600+ pages(tabs), session restore tab show up,click restore button, hanging on for a day, restore failed Firefox take up 5G memory, os System disk with low capacity, total memory used up.
2019.04.12 Add 8G memory up to 16G mem, session restore again, firefox used 9G memory, 4G mem free, Firefox hanging on for another day. Kill process. Reduce memory back to 8G
2019.04.12 ---04.16 Give up session restore, leave the session restore tab for future . Open new tabs for surfing. At 04.16 ,a sudden firefox update, firefox opened up 3 windows to prompt update notification, Close the 2 windows, from the main window, the session restore tab gone, the history box not loaded(in the start tab), The firefox response slowly, firefox may have some problems. Kill process and open again, not getting better .
From that time point , session restore tabs never shows up. Close firefox by kill process, the size of Recovery.jsonlz4 increased accordingly,
Recovery.jsonlz4 size 57M Pages/Tabs:7600+ 2019/4/21 Try firefox version 52 , 57, 62,64 respectively to restore session , all failed. Try mozlz4-edit-master https://www.jeffersonscher.com/ffu/scrounger.html
Scrounger have no response, Mozlz4 failed with error:
TextDecoder.decode(file) allocation size overflow
I will not give up rescuing the sessions. I want all the tabs back, they are so lovely and useful.
Thanks for your help,
Yours
Tutte le risposte (6)
It's possible the file has become corrupted -- 57 megabytes for a compressed file is sort of crazy. But let's try at least one more thing.
Instead of having the Scrounger do the decoding, try having Firefox do it using the Browser Console.
Firefox 56-65: https://gist.github.com/jscher2000/07f94249b0a5f6d565fb20d88b73bb91
Firefox 66: https://gist.github.com/jscher2000/4403507e33df0918289619edb83f8193 (this might work in earlier versions but I haven't tested it)
Can you get a valid JSON file?
If so, you could try to scrounge that to create an enormous HTML page of clickable links and see whether that works. It's probably too big.
Whether Firefox will ever be able to restore it using the "swap" method, I don't know. You may already be familiar with that from other threads.
This code uses the internal Firefox code to decompress LZ4 files. You can use the Browser Console to run the code. You need to enable the command line
If Firefox can't handle a compressed file of the size (i.e. isn't able to decompress) you have then you are probably out of luck with recovering this file.
async function decompressFile(oFilePath,nFilePath){ let jsonString = await OS.File.read(oFilePath, { compression: "lz4" } ); await OS.File.writeAtomic(nFilePath, jsonString ); } var fu = Cu.import("resource://gre/modules/FileUtils.jsm").FileUtils; var fp = Cc["@mozilla.org/filepicker;1"].createInstance(Ci.nsIFilePicker); fp.init(window, "Open File", Ci.nsIFilePicker.modeOpen); fp.appendFilter(".*LZ4* Compressed Files","*.*lz4*"); fp.appendFilter(".LZ4 Compressed Files","*.lz4"); fp.displayDirectory = fu.File(OS.Path.join(OS.Constants.Path.profileDir, "")); fp.open((aResult) => { if (aResult == Ci.nsIFilePicker.returnOK) { if (fp.file.exists() && fp.file.isFile() && fp.file.isReadable()) { var oldfile = fp.file.path; var newfile = oldfile + ".json"; try { decompressFile(oldfile, newfile); console.log("Saved as: \"" + newfile + "\""); if(confirm("Open JSON file in a Firefox tab?")){ var uri="file:///"+newfile.replace(/\\/g, "/"); window.open(uri, "_blank"); } } catch (err) {console.log(err);} } } else {console.log("<canceled>");} });
jscher2000 said
It's possible the file has become corrupted -- 57 megabytes for a compressed file is sort of crazy. But let's try at least one more thing. Instead of having the Scrounger do the decoding, try having Firefox do it using the Browser Console. Firefox 56-65: https://gist.github.com/jscher2000/07f94249b0a5f6d565fb20d88b73bb91 Firefox 66: https://gist.github.com/jscher2000/4403507e33df0918289619edb83f8193 (this might work in earlier versions but I haven't tested it) Can you get a valid JSON file? If so, you could try to scrounge that to create an enormous HTML page of clickable links and see whether that works. It's probably too big. Whether Firefox will ever be able to restore it using the "swap" method, I don't know. You may already be familiar with that from other threads.
Thanks for your help. that works for me, and It successfully created recovery.jsonlz4_converted.json file which is 300M+ that's a little shocked.
but I hope, if any possible, should I just change the extension to .js, try to make firefox restore from it, it seems it have the same format as the js restoration file.
I get 2 ideas to do this elegantly: 1.is there a way to fix the jsonlz4 file to make the firefox recognize it again(I am not sure whether the file is corrupted) 2. extend the firefox session restore tab embedded utility, add some functionality, such as : A. edit the item, delete or remove the item B. could save the edited file, through remove action make the compressed file smaller for firefox use. currently, the firefox restoration tab only have check and uncheck options for recovery the tabs.
I am not sure whether the jsonlz4 file have some problems, but you method could decompress it. I still want the firefox could restore it since the Firefox have an open sequence, which recorded the time and place and event sequence happens. I use this to track down what I have done in the past time. the past 3 years, I have learned too much, I jumped into a new territory to learns some new techs(AI etc..),
so please tell me, how can I implement such tool to use in firefox. I have an idea to implement it not running on firefox but running standalone using C# , based on the jsonlz4 algorithm(which I have not beginning searching them.)
cor-el said
This code uses the internal Firefox code to decompress LZ4 files. You can use the Browser Console to run the code. You need to enable the command line If Firefox can't handle a compressed file of the size (i.e. isn't able to decompress) you have then you are probably out of luck with recovering this file.
async function decompressFile(oFilePath,nFilePath){ let jsonString = await OS.File.read(oFilePath, { compression: "lz4" } ); await OS.File.writeAtomic(nFilePath, jsonString ); } var fu = Cu.import("resource://gre/modules/FileUtils.jsm").FileUtils; var fp = Cc["@mozilla.org/filepicker;1"].createInstance(Ci.nsIFilePicker); fp.init(window, "Open File", Ci.nsIFilePicker.modeOpen); fp.appendFilter(".*LZ4* Compressed Files","*.*lz4*"); fp.appendFilter(".LZ4 Compressed Files","*.lz4"); fp.displayDirectory = fu.File(OS.Path.join(OS.Constants.Path.profileDir, "")); fp.open((aResult) => { if (aResult == Ci.nsIFilePicker.returnOK) { if (fp.file.exists() && fp.file.isFile() && fp.file.isReadable()) { var oldfile = fp.file.path; var newfile = oldfile + ".json"; try { decompressFile(oldfile, newfile); console.log("Saved as: \"" + newfile + "\""); if(confirm("Open JSON file in a Firefox tab?")){ var uri="file:///"+newfile.replace(/\\/g, "/"); window.open(uri, "_blank"); } } catch (err) {console.log(err);} } } else {console.log("<canceled>");} });
Thanks for your help, it successfully created a json restoration file. and the method you mentioned here have some in common with @jscher2000's. and I hope you can also consider my propose on advance the session restore tab.
Thanks.
Considering the size of the file, it may contain some garbage, but to try to feed it to Firefox, check the "manual swap" procedure listed in this thread: https://support.mozilla.org/questions/1240688
You might need to hide sessionstore-backups (e.g., rename to sessionstore-backupsOLD) to get Firefox to look for a sessionstore.js file.
Note that the code I posted should work in all Firefox 56+ versions (Task.jsm was removed in 66).
It is possible that tabs have a large tab history. I don't know if you could be easy to cleanup the file and only keep the main tab and not not possible tab history. Closed tabs and windows and cookies can also take up space. I don't know whether the builtin JSON viewer can handle a file of this large size (it will take quite some time to open and parse this file).