Firefox 39 does not respect hotlink protection in Apache servers
An ISP (VentraIP) has confirmed my observations. Apparently, if you enable hotlink protection on selected files such as style.css, you can use Firefox 39 to type a URL like http://www.[webname].com/style.css and viola! You sudden;y see the the contents of style.css. Try it with Google Chrome or Safari 8.x and hot linking works by denying access to style.css (or gets re-directed to a 404.shtml or some other web page).
The ISP is certain it is a bug in Mozilla's flagship software for web browsing. I originally thought is was a bug in the Apache server, but not so according to the ISP.
Is this really a bug in Firefox 39?
Mafitar da aka zaɓa
The referrer is only used to verify that you clicked a link on a specific page (i.e. a link on a page from your domain).
You can see the referrer (Referring URL) in "Tools > Page Info > General". In a lot of cases only a wrong referrer might prevent a server from sending protected content and not in case the referrer is blocked (i.e. no referrer is received).
Karanta wannan amsa a matsayinta 👍 0All Replies (11)
AFAIK, that type of protection is to stop websites from hot linking to your website - effectively stopping other websites from displaying your hosted images on their webpages and using your bandwidth, rather just out-and-out "stealing" your images and hosting them on "their own dime". I am not familiar with setting up the necessary .htaccess file or if it works for .css files and most examples I have seen have dealt with image files.
If you could provide a "real" example (vs. a theoretical example for discussion), I'm sure one of the users around here can look into this further - latter in the day when they are usually here.
edit This is far beyond a Firefox end-user support issue. You might be better off asking in a Mozilla development type fora, "group", or news-list. http://www.mozilla.org/about/forums/#general-development
An gyara
Hi there
For people who are unfamiliar with hotlink protection, everyone who has a web site online (hosted by an ISP) using a server (usually Apache) to deliver the web pages will always contain some invisible files at the root level of your web site folder (normally called "public_html"). One of these invisible files is called .htaccess. This file is very important as it allows you to control and tell the Apache server who can access your web site (via IP address blocking), how to speed up web page loading for online users (e.g., GZIP compression, cache control and leverage etc), any tweaks to PHP to allow you to send messages from online PHP scripts in your web pages, how to automatically redirect users to other web pages etc.
Now there is a thing called Hotlink Protection, which basically means you can configure the server to allow any file to be served online to users but prevent users from directly accessing those files via a web address straight to those files. It is used, as "the-edmeister" correctly stated, to prevent online users from directly linking to, say, images on your web site in their own web sites, thereby you getting all the extra bandwidth from people viewing your images loading up elsewhere (but not exactly through your own web site).
To enable hotlink protection, all you do is find (or if it doesn't exist, you create the text file) .htaccess. Then you add the following code at the end:
RewriteCond %{HTTP_REFERER} !^http://[yourwebname].com/.*$ [NC] RewriteCond %{HTTP_REFERER} !^http://[yourwebname].com$ [NC] RewriteCond %{HTTP_REFERER} !^http://www.[yourwebname].com/.*$ [NC] RewriteCond %{HTTP_REFERER} !^http://www.[yourwebname].com$ [NC] RewriteRule.*\.(jpg|jpeg|gif|png|bmp|css|ini|mp3|swf)$ http://www.[yourwebname].com/404.shtml [R,NC]
where [yourwebname] is, as the name implies, the name of your web site. Of course, this assumes your web site has ".com" at the end. Change this too if it is something different.
Now the part that tells you which files get hotlink protected is where it says "jpg|jpeg|gif|png|bmp|css|ini|mp3|swf". In other words, this is nothing more than file extensions. For the server to know which files are to be hotlink protected, you can tell it to assume that all files whose file extension is part of this list are the ones to be hotlink protected. So yes, you can hotlink protect a file called style.css if you want to (the one that contains your stylesheet for presenting your web pages in a graphically pretty way). In fact, any file extension can be used here.
Now once you do this, there is the curious case of Firefox 39 (not sure if it happens in earlier versions) where if you type http://www.[yourwebname].com/style.css (assuming style.css is located at the root level of your web site) and press the Return key, you are suddenly presented with the actual stylesheet css code in the web browser's window. Huh? Haven't we just hotlink protected this file? How is it possible that Firefox 39 has managed to bypass the server instructions to not display the file's contents when we have told the server it should be hotlink protected?
Well, certainly the browsers do need to read this file in order to present the web pages nicely. So it must be stored in the browser's memory somewhere. But instead of grabbing it from the server and seeing what the server says, Firefox 39 has decided it would use what's in memory and present it to the user (probably to save on bandwidth for the user in not telling the server to re-download it assuming the server would allow it).
Okay. So maybe hotlink protection doesn't work per se, and the code above is all hogwash.
So we open up Safari 8.x (the latest), or even Google Chrome (just to make it a slightly more statistically valid test). Type the above URL and guess what? The browsers get correctly told by the server to wander off and be redirected to the web site's 404.shtml page. And as any web site owner knows, 404.shtml tells the user that the file he tried to directly link to does not exist (well, we are obviously fudging it a bit here since the browser and the server knows it is there, but we don't want the user to know that too, now do we?).
At first I thought it was a bug with the Apache server from which the files are being served to the user. After a little discussion with the ISP, they can confidently state it is not an Apache issue. It would appear that there is a bug in Firefox 39 (the latest one). Of course, I don't know how long this bug has existed, but this is fairly serious in my mind. Well, if people can link directly to any file on the web site, including PHP files and all the rest, eventually people can download the entire web site, including SQL databases or any file that contains personal information about users, right? I don't know about you, but I think that's a bit of a security concern.
Now if anyone can reproduce this bug at their end, then I reckon we should let the Mozilla team know about it, kind of like sooner rather than later, because I'm sure lots of people out there would not like to know that their web sites are being compromised because of this bug (or maybe some people do if they want to steal entire web sites with ease).
Just something I wanted to let Mozilla know in case it really is a bug and it needs to be fixed fairly soon.
Thanks.
An gyara
Hotlink protection depends on the referrer that is received by the server, so apparently a correct referrer is send when Firefox gets the file(s).
Note that it wouldn't be that difficult anyway to force a specific referrer to be send with a request if you really want to see a file that is protected this way. In the past we have seen sites that depended on hotlink protection for content served via a plugin like Flash and Firefox doesn't pass the referrer to the flash plugin, so this content fails to load. I don't know if there is still no correct referrer for requests send by the flash plugin.
Let me see if I have this right.
Basically how a server delivers web pages on a browser is depended on the type of referrer that is sent by a browser such as FireFox, Safari or Google Chrome to the server. In other words, if the "referrer" (whatever it is and whoever sent it) is not correct for whatever reason, the server gets all muddled up and does not know how to deliver the information online. So either the server chooses not to deliver content (the safest and most secure way) to web browsers (as in the Flash plug-in, which we all know has its own security issues) or, in my case, will deliver the content anyway irrespective of whether hotlink protection is enabled for the files containing the delivered content
I can see what you are saying, and it kind of makes sense. However, if hotlink protection is meant to be enabled, technically a server should never serve any content from the hotlink protected file when accessed directly even if the server is confused over some incorrect referrer.
So, essentially, the behaviour of FireFox 39, no matter what the referrer is, should be the same as with Safari and Google Chrome even if the referrer is unknown or just not working right.
This is looking more like a bug in FireFox 39 by the day. I will escalate this to Mozilla's development team to see if they can do their magic to ensure consistency in web browser behaviour (and a bit of security too, even if I've let this one out of the bag).
An gyara
Actually, I'm not sure how to escalate this issue to the Mozilla development team. Perhaps someone more experienced with the layout of this web site can do it.
Thanks
Zaɓi Mafita
The referrer is only used to verify that you clicked a link on a specific page (i.e. a link on a page from your domain).
You can see the referrer (Referring URL) in "Tools > Page Info > General". In a lot of cases only a wrong referrer might prevent a server from sending protected content and not in case the referrer is blocked (i.e. no referrer is received).
For example, in the case of this page, the referring URL is https://support.mozilla.org/en-US/users/auth. So if Firefox sees this referral and Mozilla has, for argument sake, hotlink protected a file like style.css, then technically Mozilla's own server, in receiving this referral should know the file is protected and perform whatever action the people at Mozilla wants. So a referral like https://support.mozilla.org/en-US/style.css (assuming the file exists in this location, which incidentally it doesn't but let us entertain ourselves on the thought) should tell the server that:
1. The referral is from the domain "mozilla"; and 2. The file to serve is style.css
However, since a file with the file extension css is already hotlink protected, the server should know straightaway not to serve it, and instead just generate an error or re-direct the online user to another page.
In that case, observations by me and VentraIP of at least two browsers re-directing the online user correctly to an error page for a properly protected file and only FireFox 39 not doing the same thing would suggest we have a bug in FireFox 39.
Correct me if I am wrong, but I think what's happening is that the server is not doing anything wrong. It is doing exactly what it is intended. Thus in order to display a web page with a bit of styling through a css file, if the user legitimately is in the domain looking at stuff and not trying to directly link to a specific web site resource from outside, the stylesheet information and the web page, together with any images, have to be served by the server, and hence "downloaded" by the browser. Now here is where things get really interesting,. Once the information is downloaded, what happens to this information afterwards? Certainly the browser has to do its magic of displaying the information in a fancy way in order to thrill the online user to no end. But once it is served and displayed, does the information get cleared (other than what's displayed on the screen)? Even if the information is cleared, the problem seems to be that FireFox does not re-verify with the server each time a web page or a particular file is given a new display based on the referral given by the online user. It looks to me like FireFox just assumes that because the information has already been downloaded and kept in some cache or wherever, that it is okay to display it again no matter if the information is protected.
If I am right on this aspect, then Firefox is a really great product in terms of speed and even being consistent in how web pages should look, which is a far cry from the idiot Microsoft Internet Explorer that still exists today, but securitywise it is a pretty lousy app. And all it only takes is just one silly browser to flout the security settings on web sites to make all the efforts of webmasters and their companies look like a waste of time in protecting their assets.
Well, that's my theory anyway.
Does anyone have a better theory of why FireFox39 behaves differently from Safari, Google Chrome etc when it comes to hotlink protected files?
And perhaps the biggest $64,000 question on my mind is, will Mozilla be able to fix the security problem?
An gyara
Can you post a direct link to a protected file or a page that contains protected data that requires a referrer?
If you open a protected link directly in the location/address bar then you should either get a 404 not found error or possibly get redirected if the server is set up that way. Note that Firefox will load such a file from the cache if it is already stored there, in that case reload the file and bypass the cache (Ctrl+Shift+R) or clear the cache.
Sure, I love to. Only problem is, I don't want to lead other people to the web site in case some do actually want to grab the style.css (not that I care about it that much, but I feel it is not good practice to expose a security problem to everyone and make it much bigger than it needs to be). So probably after the security issue is plugged will I be more than happy to share the web site with everyone.
Actually, I was hoping the observation made by VentraIP and myself can be somehow reproduced on other people's web sites (assuming there are people here who have one). In fact, I would invite the Mozilla team to try it on its own web site (maybe even this one). I'm certainly curious to know if it is just VentraIP and myself who are seeing this problem with FireFox 39 or whether it really is a common problem for other Firefox users having their own web sites to manage.
Or is it really an Apache server issue?
I've tried your recommended Ctrl+Shift+R (for PC) or Command+Shift+R (for Mac) for clearing FireFox cache (a good suggestion, I haven't tried that specific key combination). I've even used OnyX on OSX Yosemite to clear all forms of caches (not jiust the internet-related ones). My machine has been restarted a number of times to clear what's in RAM now that Apple likes to retain certain data in RAM and have some of this information sent to its servers while people are online.
And FireFox 39 has not one add-on enabled. I can't get it much more basic than that I reckon.
Oh, I almost forgot, I did also start up in OS X Mountain Lion and used the previous FireFox 38 version, and it too is showing the contents of a hotlink protected file from the server (namely style.css).
I can vouch for the fact that none of the above steps have helped.
And the code to hotlink protect files on the .htaccess file is suppose to be this:
RewriteCond %{HTTP_REFERER} !^http://[yourwebname].com/.*$ [NC] RewriteCond %{HTTP_REFERER} !^http://[yourwebname].com$ [NC] RewriteCond %{HTTP_REFERER} !^http://www.[yourwebname].com/.*$ [NC] RewriteCond %{HTTP_REFERER} !^http://www.[yourwebname].com$ [NC] RewriteRule.*\.(jpg|jpeg|gif|png|bmp|css|ini|mp3|swf)$ http://www.[yourwebname].com/404.shtml [R,NC]
Without revealing [yourwebname] to everyone, does anyone disagree with the code for hotlink protecting files as I have it? Certainly Safari and Google Chrome sees no problem with it.
Arrrgh!
An gyara
This can never be a problem with the browser as this is a server side measure. I do not that much about how such a rewrite rule works, so I don't know why the server doesn't obey these rules.
Did you verify that Firefox is at least sending a referrer because a missing referrer might be accepted as well like I wrote above?
You can use the Live Http Headers extension.
Installed the live http headers addon.
I have let the addon gather information during the test and saved the results. So in summary, I visited the home page.
Referer: http://www.[websitename].com/
I visited http://www.[websitename].com/style.css. The browser shows the text in the stylesheet :(
Referer: None
I haven't tried this before, but I decided to refresh the style.css page. Lo and behold the browser (or server) decided to show the 404.shtml page :)
Referer: http://www.[websitename].com/404.shtml
Perhaps you might be right. No referer may cause problems. But is it really a server issue? Or is FireFox 39 getting lazy in not sending the right information for the server to know what the referer is and therefore show the right web page?
Here is the full headers text. See if you can see anything odd in it (please note that web site name has been changed to [wesitename] for security reasons):
http://www.[websitename].com/images/slideshow/free.jpg
GET /images/slideshow/free.jpg HTTP/1.1 Host: www.[websitename].com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0 Accept: image/png,image/*;q=0.8,*/*;q=0.5 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate DNT: 1 Referer: http://www.[websitename].com/ Connection: keep-alive
HTTP/1.1 200 OK Date: Wed, 08 Jul 2015 08:10:25 GMT Server: Apache/2.4.12 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Last-Modified: Fri, 14 Mar 2014 08:18:54 GMT Etag: "26a0a-4f48cb8733780" Accept-Ranges: bytes Content-Length: 158218 Cache-Control: public Expires: Fri, 07 Aug 2015 08:10:25 GMT Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: image/jpeg
http://www.[websitename].com/style.css
GET /style.css HTTP/1.1 Host: www.[websitename].com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate DNT: 1 Connection: keep-alive If-Modified-Since: Mon, 06 Jul 2015 12:09:26 GMT If-None-Match: "46c8-51a33c6e5b36e-gzip" Cache-Control: max-age=0
HTTP/1.1 302 Found Date: Wed, 08 Jul 2015 08:14:15 GMT Server: Apache/2.4.12 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Location: http://www.[websitename].com/404.shtml Cache-Control: max-age=60 Expires: Wed, 08 Jul 2015 08:15:15 GMT Content-Length: 341 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: text/html; charset=iso-8859-1
http://www.[websitename].com/404.shtml
GET /404.shtml HTTP/1.1 Host: www.[websitename].com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate DNT: 1 Connection: keep-alive
HTTP/1.1 200 OK Date: Wed, 08 Jul 2015 08:14:15 GMT Server: Apache/2.4.12 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Accept-Ranges: bytes Vary: Accept-Encoding Content-Encoding: gzip Cache-Control: max-age=60 Expires: Wed, 08 Jul 2015 08:15:15 GMT Content-Length: 2233 Keep-Alive: timeout=5, max=99 Connection: Keep-Alive Content-Type: text/html
http://www.[websitename].com/images/logo.png
GET /images/logo.png HTTP/1.1 Host: www.[websitename].com User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:39.0) Gecko/20100101 Firefox/39.0 Accept: image/png,image/*;q=0.8,*/*;q=0.5 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate DNT: 1 Referer: http://www.[websitename].com/404.shtml Connection: keep-alive If-Modified-Since: Wed, 15 Aug 2012 07:39:52 GMT If-None-Match: "653a-4c74908432200" Cache-Control: max-age=0
HTTP/1.1 304 Not Modified Date: Wed, 08 Jul 2015 08:14:15 GMT Server: Apache/2.4.12 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Connection: Keep-Alive Keep-Alive: timeout=5, max=98 Etag: "653a-4c74908432200" Expires: Fri, 07 Aug 2015 08:14:15 GMT Cache-Control: public
The same thing occurs with images (which are hotlink protected). Refresh the page and the proper 404.shtml page is shown.
What do you reckon might be causing this?
An gyara