How do you log/save the actual pages which are part of a session?
based on the information given:
https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging
you may thoroughly log the inner workings of HTTP sessions as handled by the browser.
I also need to save and keep the actual pages that are being downloaded by the browser (including the supporting css, javascript, images, ...)
How do you achieve that?
lbrtchx
Wubrane rozwězanje
OK, I was finally able to do what I wanted.
Here is an outline for anyone to benefit from, improve it:
// __ latest firefox development version (aurora)
https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-aurora/firefox-52.0a2.en-US.linux-i686.tar.bz2
$ ls -l firefox-52.0a2.en-US.linux-i686.tar.bz2 -rw-r--r-- 1 knoppix knoppix 65124511 Dec 27 05:31 firefox-52.0a2.en-US.linux-i686.tar.bz2
$ md5sum -b firefox-52.0a2.en-US.linux-i686.tar.bz2 0877480001d91ebdb180011db8f84df5 *firefox-52.0a2.en-US.linux-i686.tar.bz2
// __ untar/install it
$ _LOG="tar_-xjvf_firefox-52.0a2.log"
$ date; time tar -xjvf firefox-52.0a2.en-US.linux-i686.tar.bz2 > "${_LOG}" Wed Dec 28 04:09:02 EST 2016
$ wc -l "${_LOG}" 78 tar_-xjvf_firefox-52.0a2.log
$ ls -l "${_LOG}"
// __ run/test it
$ cd firefox
$ pwd .../firefox
$ ./firefox --version Mozilla Firefox 52.0a2
// __ While developing, I find useful running tcpdump, wireshark and firefox in logging mode
// __ run firefox and set up logging via about:config
_DT=$(date +%Y%m%d_%H%M%S)
_LOGFL="${_ODIR}/${_DT}_firefox-52.0a2_MOZ.log" echo "// __ \$_LOGFL: |$_LOGFL|"
export MOZ_LOG=timestamp,rotate:200,nsHttp:5,nsSocketTransport:5,nsStreamPump:5,nsHostResolver:5
export MOZ_LOG_FILE="${_LOGFL}"
./firefox &
Toś to wótegrono w konteksće cytaś 👍 0Wšykne wótegrona (12)
You can try Wireshark.
- Wireshark: http://www.wireshark.org/download.html
- Wireshark - Wikipedia: https://en.wikipedia.org/wiki/Wireshark
Try wireshark for that? Firefox is already nicely logging the HTTP sessions
What I need is for firefox not to delete the data feeds of html pages, js, ... as it finishes to process, show them.
There should be some sort of "about:cache_duration" or "about:cache_length" option,
What is it?
Wót Albretch
Probably, another MOZ_LOG parameter would do, but I don't know what would it be.
MOZ_LOG parameters don't look like about:config parameters
to me.
See:
MOZ_LOG_FILE and MOZ_LOG_MODULES are now the environment variables for logging:
Wót cor-el
See:
yes, but in that document it says:
Using about:networking This is available starting with Firefox 52. Navigate to about:networking and go to the Logging section. ...
and the most up to date version of firefox is 50.1 right now
and the beta release is 51.0beta:
https://www.mozilla.org/en-US/firefox/50.1.0/releasenotes/ https://www.mozilla.org/en-US/firefox/51.0beta/releasenotes/
so I am not getting something right or you guys are
not being clear enough to me.
There must be a way to do such a thing, which you
are using while debugging ;-)
Wót Albretch
You would have to install the Firefox Developer version (52.0a2) to set all parameters from within Firefox via about:networking. In the current release you only can do that via environment variables.
#!/bin/sh # https://developer.mozilla.org/en/HTTP_Logging # *https://developer.mozilla.org/en/Mozilla/Developer_guide/Gecko_Logging # try one of these (remove the #) # export MOZ_LOG=nsHttp:3 # export MOZ_LOG=nsHttp:5 export NSPR_LOG_FILE=~/loghttp.txt # set the correct path to Firefox cd /usr/local/firefox-RE # start Firefox ./firefox -no-remote -P
Albretch said
I also need to save and keep the actual pages that are being downloaded by the browser (including the supporting css, javascript, images, ...)
I doubt that logging will give you a convenient format. I once turned on logging for something. After 25 seconds, I had a text file of nearly 392,000 lines consuming over 42 megabytes. The responses were encoded (sent with gzip encoding) so they are indecipherable -- if they are included in the file, which I can't confirm. Every so often I find decoded response headers but I don't see the actual response content (text or image, as the case may be).
Hopefully there is another way, such as an add-on, that is simpler than fishing around in the cache.
Wubrane rozwězanje
OK, I was finally able to do what I wanted.
Here is an outline for anyone to benefit from, improve it:
// __ latest firefox development version (aurora)
https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-aurora/firefox-52.0a2.en-US.linux-i686.tar.bz2
$ ls -l firefox-52.0a2.en-US.linux-i686.tar.bz2 -rw-r--r-- 1 knoppix knoppix 65124511 Dec 27 05:31 firefox-52.0a2.en-US.linux-i686.tar.bz2
$ md5sum -b firefox-52.0a2.en-US.linux-i686.tar.bz2 0877480001d91ebdb180011db8f84df5 *firefox-52.0a2.en-US.linux-i686.tar.bz2
// __ untar/install it
$ _LOG="tar_-xjvf_firefox-52.0a2.log"
$ date; time tar -xjvf firefox-52.0a2.en-US.linux-i686.tar.bz2 > "${_LOG}" Wed Dec 28 04:09:02 EST 2016
$ wc -l "${_LOG}" 78 tar_-xjvf_firefox-52.0a2.log
$ ls -l "${_LOG}"
// __ run/test it
$ cd firefox
$ pwd .../firefox
$ ./firefox --version Mozilla Firefox 52.0a2
// __ While developing, I find useful running tcpdump, wireshark and firefox in logging mode
// __ run firefox and set up logging via about:config
_DT=$(date +%Y%m%d_%H%M%S)
_LOGFL="${_ODIR}/${_DT}_firefox-52.0a2_MOZ.log" echo "// __ \$_LOGFL: |$_LOGFL|"
export MOZ_LOG=timestamp,rotate:200,nsHttp:5,nsSocketTransport:5,nsStreamPump:5,nsHostResolver:5
export MOZ_LOG_FILE="${_LOGFL}"
./firefox &
Wót Albretch
then the wireshark and tcpdump part
// __ You don't want to run wireshark as root!
$ sudo dpkg-reconfigure wireshark-common
$ sudo adduser -a $USER wireshark
// __ Set up logging file names for easy management
_NM="..."
_DATA_DIR=".../data"
_ODIR="${_DATA_DIR}/${_NM}"
mkdir "${_ODIR}"
_DT=$(date +%Y%m%d_%H%M%S)
_WSLOG="${_ODIR}/wireshark-gtk_${_DT}.log" echo "// __ \$_WSLOG: |$_WSLOG|"
// __ running wireshark
sudo wireshark-gtk -i eth0 -k -w "${_WSLOG}" &
// __ running tcpdump
sudo tcpdump -pni eth0 -s65535 -G 3600 -w "${_ODIR}/tcpdump_%Y%m%d_%H%M%S.pcap" &
jscher2000 said
Albretch saidI also need to save and keep the actual pages that are being downloaded by the browser (including the supporting css, javascript, images, ...)I doubt that logging will give you a convenient format ...
That is why I am a bit more comfortable using more than
one tool and keeping an eye on all of them. Also, gzip you can easily unzip
Wót Albretch
Using only "nsHttp:3" (LogLevel::Info) in the DE version with about:networking seems to give a log with the HTTP request and response headers and not that much extra data.