Web Privacy Security Crawl Using Selenium Blocking Enhanced Tracking Protection
Hi,
I am working on a web crawler that scrapes set of websites analyzing certain privacy strings that are present before and after Global Privacy Signals are sent. We make use of Selenium Web Driver to perform the crawl. Part of the data we collect is the urlClassification of third party sites that are tracking users on the site, via Firefox's Enhanced Tracking Protection. However when performing the crawl, Enhanced Tracking Protection data is no longer available while Selenium has automatic control over Firefox, even though there is indeed third party sites (fingerprinters, cross-site tracking cookies) active on these sites when they are manually visited not using our crawler. Does anyone have an idea why our software may be interfering with Firefox's Enhanced Tracking Protection? Thanks!
Alla svar (1)
Additional note for above: It is important to note that the crawler worked as expected in early June 2024, before failing to log the urlClassification data provided by ETP in July 2024.