Google safe browsing using poor quality engine
Google safe browsing is using a poor quality engine or fault algorithm and is false flagging sites as containing malware and false flagging download files as containing malware/viruses.
Here is one example:
Google flagged AutoHotkey_104805.zip as containing malware or a virus.
Analysis of AutoHotkey_104805.zip in June of 2018 by virustotal.com (https://www.virustotal.com/ro/file/c8bf1c3dc4622559963b6626316ba1d083bb8a8af605f78382e371e5294d435a/analysis/) shows that out of 59 engines used to test the file only Cylance detected a problem with the file. Cylance is not a top rated engine and that detection is almost certain to be a false positive.
The follow engines passed AutoHotkey_1.1.28.00.zip as CLEAN:
Ad-Aware, AegisLab, AhnLab-V3, Alibaba, ALYac, Arcabit, Avast, Avast Mobile, AVG, Avira, AVware, Babable, Baidu, BitDefender, Bkav, CAT-QuickHeal, ClamAV, CMC, Comodo, DrWeb, Emsisoft, ESET-NOD32, F-Prot, Fortinet, GData, Ikarus, Sophos ML, Jiangmin, K7AntiVirus, K7GW, Kaspersky, Kingsoft, Malwarebytes, MAX, McAfee, McAfee-GW-Edition, Microsoft, eScan, NANO-Antivirus, Panda, Qihoo-360, Rising, SUPERAntiSpyware, Symantec, TACHYON, Tencent, TheHacker, TotalDefense, TrendMicro, TrendMicro-HouseCall, VBA32, VIPRE, ViRobot, Webroot, Yandex, Zoner
NOTE: 9 of the these engines are in the top 10 anitvirus/malware engines (UNDERLINED). All reported that AutHotkey_1.1.28.00.zip is CLEAN. F-Secure was not run on this file.
Google safe browsing claims that this file is INFECTED.
Similar analysis of multiple other Google Safe Browsing "infected" files from this site show similar CLEAN results when tested by virustotal.com.
Mozilla should report this issue to Google and demand that Google employ only quality engines to screen files. Google should employ better quality algorithms to prevent false flagging of files and web sties.
If Firefox is going to block web access to sites and to downloads based upon bad data from Google, then I am going to be forced to switch to another browser.
Mozilla. Stand up, take action, and provide the quality service you have provided IN THE PAST.
THERE IS A SECOND MAJOR ISSUE. Once a download file is flagged as containing malware there is a bug in Firefox. The only options offered by Firefox are "REMOVE FILE" or "OPEN". "OPEN" should not be an option at this point. The options should be "DOWNLOAD ANYWAY" or "REMOVE FILE". With the current options and an .exe file (if downloaded) will be immediately executed resulting in infection if the file is tainted. With my suggested options, If a user chooses to "DOWNLOAD ANYWAY" then Firefox should remind the user the file may be infected and should recommend that the file be scanned by the user's antivirus/malware software before opening.
All Replies (13)
Safe Browsing does not guarantee 100% accuracy, nor does any anti-virus. It is a way for us to help users stay safe, and while sometimes things will slip through, or things will be blocked that shouldn't be, it's a best effort method.
Feel free to contact Google and let them know you feel that file is safe, and see if they will remove it.
You are right. Google Safe Browsing is not 100% accurate. The problem is that Google Safe Browsing is unacceptably inaccurate. Mozilla needs to demand that Google work to fix this issue.
I can list numerous files that Google Safe Browsing has falsely labeled as containing either a virus or malware that do not contain either. What would be the point?
The issue is not with the files this issue is the extremely poor performance of Google Safe Browsing when compared to the industry standards.
In the example I provided 58 of 59 virus/malware search engines (including 9 of the best engines) found AutoHotkey_1.1.28.00.zip to be CLEAN. Google Safe Browsing says the file is infected.
Every file labeled by Google Safe Browsing as infected when checked through virustotal.com results in similar findings. In each case the top virus/malware engines (AhnLab, Avira, Bitdefenter, Comodo, Kapersky, McAffe, Symantec, TrendMicro, and Vipre) said that the files are CLEAN. Google should be ashamed of its performance and Mozilla should take issue with that poor performance. CRYING WOLF when there is no wolf is not a service. HOW CAN I TRUST YOU?
MOZILLA needs to ask GOOGLE to clean up their act and to start using better engines in Google-Safe-Browsing.
I have not been able to find a way to report to Google that a file has been misflagged by Google Safe Browsing. Another moderator responding to an unrelated question did provide me with a link to https://developers.google.com/safe-browsing/v4/
In that article, I found a reference to where a user (and not a site owner) can report URLs that are currently on Google's malware list but shouldn't be. That link is not to Google. Instead, Google sends you to https://www.stopbadware.org/request-review.
When I tried to request a review of AutoHotkey104805.zip stopbadware.org returned:
"Your search for (URLs below) returned no results."
https://autohotkey.com/download/1.0/AutoHotkey104805.zip https://autohotkey.com/download/1.0/
Followed by:
"Please try a different search or check back later. There can be a delay between the time our data providers blacklist a URL and when that URL is searchable in our Clearinghouse. Usually, this delay is no more than a few hours."
This page and this file have been black listed by Google Safe Browsing for several days and should have been in the Clearinghouse.
YET ANOTHER REASON WHY MOZILLA SHOULD GET INVOLVED.
drposts said
You are right. Google Safe Browsing is not 100% accurate. The problem is that Google Safe Browsing is unacceptably inaccurate. Mozilla needs to demand that Google work to fix this issue.
drposts said
MOZILLA needs to ask GOOGLE to clean up their act and to start using better engines in Google-Safe-Browsing.
In both of these responses FF isn't Google and you have to ask Google what your asking.
Are you the developer of autohotkey? It's usually best if they request a review of their own software, and fix any suspicious things in their code
No I am not the developer of autohotkey. Yes they have requested a review.
Tyler you clearly do not want to address the issue I have posed. Google Safe Browsing is using a poor quality antivirus/malware engine. It falsely labels files as infected they are not. It is repeatedly out performed by most other other similar engines. I like Mozilla's concept here but we have enough people providing fake news. Mozilla should not play that game. PLEASE gIve us an antivirus/malware engine that is on par with the rest of the world.
What are you using as a metric for "high quality engine"? Every anti-virus has false positives, and while I understand your frustrations, this is just part of the game.
drposts said
https://autohotkey.com/download/1.0/AutoHotkey104805.zip
https://autohotkey.com/download/1.0/
Firefox displays:
The site ahead may contain harmful programs
Firefox blocked this page because it might try to trick you into installing programs that harm your browsing experience (for example, by changing your homepage or showing extra ads on sites you visit).
I think the translation of this to "contains a virus" on the Downloads list is where we might be overinterpreting what SafeBrowsing is saying. In Chrome, they display "[file] is dangerous, so Chrome has blocked it" but the nature of the danger is not specified, and their "Learn more" link lists multiple possibilities: https://support.google.com/chrome/answer/6261569
WHAT AM IS USING FOR A METRIC. THE BEST INDUSTRY STANDARD TOP ANTIVIRUS/MALWARE ENGINES: AhnLab, Avira, Bitdefenter, Comodo, Kapersky, McAffe, Symantec, TrendMicro, and Vipre.
WHAT IS YOUR STANDARD (GOOGLE SAFE BROWSING) WORTH?
TAKE A LOOK AT THE ISSUE. HERE ARE SOME FACTS IF YOU NEED THEM. I SUSPECT YOU ALREADY KNOW THE ANSWER. IF YOU DON'T WHY ARE YOUR REPLYING TO MY QUESTIONS WITH ANSWERS THAT REPEATEDLY FAIL TO ADDRESS THE ISSUE:
1. Using firefox I screened https://autohotkey.com/download/1.1/ and every file I checked was passed by GSB. 1a. I checked all of the Ahk2Exe...zip files, plus all of the AutoHotkey112207..., ...112300..., ...112400..., ...1.1.25.00..., ...1.1.26.00..., ...1.1.27.00..., ...1.1.28.00..., ...1.1.29.00..., and 1.1.29.01... files
2. I next checked all of the https://autohotkey.com/download/2.0/ files. All are blocked except: AutoHotkey 2.0-a078-31+g72dc326.zip AutoHotkey v2.0-a074.zip I have no idea why those 2 *.zip files were passed by GSB and all of the others are flagged as containing "a virus or malware".
3. Finally I screened the https://autohotkey.com/download/1.0/ files. 3a. All of the downloads from AutoHotkey1000.exe throught AutoHotkey104311 Install.exe are blocked under firefox except: AutoHotkey1000.exe AutoHotkey1016.exe AutoHotkey1023.exe AutoHotkey102514.exe AND AutHotkey102701.exe
3b. All of the *.zip files on the page. All are blocked except: AutohotKey104500.zip
3c. I checked all of the AutoHotkey104404..., ...104500..., ...104600..., ...104700..., ...104800..., and ...104805... files. All are blocked except: AutoHotkey104404 sc bin min size.zip AND AutohotKey104500.zip (as noted above)
3d. In addition AHK-binaries.zip is not blocked.
4. I next checked the following URLs with virustotal.com
4a. URL https://autohotkey.com/download/1.1/ Host autohotkey.com No engines detected this URL Last analysis 2018-06-15 02:35:37 UTC
4b. URL https://autohotkey.com/download/1.0/ One engine of 67 engines detected this URL. Google Safebrowsing: "Malicious". DNS8: "Suspicious". 65 other engines (including the industry standards): CLEAN!!! Last analysis 2018-06-15 02:35:09 UTC
4c. URL https://autohotkey.com/download/2.0/ Host autohotkey.com On April 17, 2019 no engines detected this URL including Google Safe Browsing. Last analysis 2018-04-17 09:43:06 UTC That is not the case today (as noted above).
4e. URL https://autohotkey.com/download/2.0/AutoHotkey_2.0-a097-60f26de.zip (CURRENT VERSION OF AHK 2.0) Host autohotkey.com 2 engines out of 68 detected this URL. ADMINUSLabs: Malicious Google Safebrowsing: Malicious DNS8: Suspicious. 65 other engines (including the industry standards): CLEAN!!! Last analysis 2018-06-27 09:13:39 UTC
WHAT DOES THIS SAY ABOUT GOOGLE SAFE BROWSING?
Tyler this is not a game.
Do you have any financial associations with Google?
I do. I own stock in the company. And, I find this gross failure by Google to meet industry standards distressing.
Then take it up with Google as you have been asked to do several times as this is not the place to messaging long stores of how unfair it is.
Even when it maybe so. Mozilla Firefox does not have a say as to what they block so please file with them or have the owners/Developers of the program do so.
This is a common tactic of rivals of software and websites is to lay false claims and the block goes up for several days then comes down on appeal, only to be repeated.
So do know that we are understanding of the issue but we can not do anything for you.. Regards.
Several of the anti-virus you listed are generally not accepted as high quality.
Regardless, this discussion as devolved into insults and rudeness (from both PK and drtools) so I'll be locking this. You were given a remediation method, safebrowsing is used to keep users safe from potentially malicious downloads, you can disable it (however, that is highly discouraged), and your feedback has been received. Thank you.