Firefox tends to evict from cache JavaScript with size over 15MB
We are using web application that uses JavaScript batch files, where one of them exceeded size of 15MB. Since then Firefox seems to evict the file from cache and always load it from the server also excluding If-Modified-Since header. When the file size is decreased to former size ( around 12MB ), caching works again as expected: file is either served from Firefox cache or If-Modified-Since header is sent in request to verify if server file changed. When I checked difference between files server, there was nothing changed except file size and content ( i.e. there is no dynamic changing of file, cache headers stay the same, ... ).
Is there some configuration setting or hard-coded limit which forces Firefox to evict larger files from cache and enforce their reloading?
Here is the list of selected response headers on mentioned file:
X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN Content-Security-Policy: frame-ancestors 'self' Expires: Tue, 22 Sep 2020 07:43:13 GMT Cache-Control: max-age=31536000 Cache-Control: public Last-Modified: Thu, 01 Jan 1970 00:00:00 GMT ETag: "0" Content-Type: application/javascript;charset=UTF-8 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Transfer-Encoding: chunked
Thanks for any hints.
Chosen solution
In the end it seemed that issue has been present only in Firefox 60 ESR branch. Newest Firefox releases do not have this issue.
Read this answer in context 👍 0All Replies (6)
Firefox can send both a request to load content from the cache and also send a request to the server. Which response arrives first is used. This is called a 'racing' feature (RCWN: Race Cache With Network). If you have a slower hard drive then in some cases the server might respond quicker and send a partial file. The about:networking#rcwn page shows the current RCWN stats.
The Network Monitor shows a 'raced' value in the transfer column. There is a network.http.rcwn pref (network.http.rcwn.enabled) that controls this feature, so you can test whether your issue is about RCWN or something different.
Modified by cor-el
You may have found this preference --
browser.cache.disk.max_entry_size
-- with a default value of 50MB (51,200 KB). That seems large enough for your application's needs.
You can review the comments in the source code to learn more about the user-configurable parameters and some of Firefox's behaviors. Other behaviors might not be as clearly documented:
https://dxr.mozilla.org/mozilla-release/source/modules/libpref/init/all.js#31
cor-el said
... The Network Monitor shows a 'raced' value in the transfer column. There is a network.http.rcwn pref (network.http.rcwn.enabled) that controls this feature, so you can test whether your issue is about RCWN or something different.
Thank you cor-el for info about this feature, but unfortunately it seems not to be the cause. When I've disabled rcwn, caching was still not used for 15MB file.
jscher2000 said
You may have found this preference -- browser.cache.disk.max_entry_size -- with a default value of 50MB (51,200 KB). That seems large enough for your application's needs. You can review the comments in the source code to learn more about the user-configurable parameters and some of Firefox's behaviors. Other behaviors might not be as clearly documented: https://dxr.mozilla.org/mozilla-release/source/modules/libpref/init/all.js#31
Yes, I've checked those. Just to be sure, I've tried to increase the caching limit preferences to 10 times its default size, also disabled disk smart cache size and extended its limit to a few gigs, but it still did not help.
What I noticed also is that, when I completely clear Firefox cache, the 15MB file is cached and is being used from cache for a while. It is also shown in about:cache disk stats that the file is there and usage count increases on page reloads. But after a short time period ( few page reloads ) the file is being transferred from network again, despite I can still see it in about:cache listing - just usage count is not increasing of course. When hitting CTRL+F5 on the page, I can see new refreshed entry in about:config with usage count one or a few times, but then again after a while file is being requested from network.
Did you check in the Network Monitor whether the RCWN feature might be causing Firefox not to use the disk cache like I posted above?
I checked the Network Monitor regularly, but never seen RCWN feature kick in. Is it marked by 'raced' string in Transfer column? In Transfer I always saw either file size or 'cached' string.
Chosen Solution
In the end it seemed that issue has been present only in Firefox 60 ESR branch. Newest Firefox releases do not have this issue.