How to Whitelist Similarweb on Cloudflare

If you are using Cloudflare to improve your website’s security and performance, our crawler may be blocked by Cloudflare’s firewall, hence stopping you from performing a crawl on your website. You can lift this restriction by following the simple steps below.

1. Log into your Cloudflare account
2. On your left sidebar, click on Security, then click on WAF
   
3. On that page, click on the “Create firewall rule” button
   
4. Name your firewall rule and select either of the two options below:
   1. For IP address whitelisting, you’ll need to enter our Site Audit Crawler IP addresses for crawling. To do this, choose IP Source Address as the field, equals as the Operator and 52.5.118.182 as the Value. You will then need to click Or to add a second line for IP address 52.86.188.211.
      
   2. If you want to crawl with a regional IP or custom proxy, add ‘similarweb’ as an allowed user agent. You need to choose User Agent as the field, Contains as the Operator and enter 'similarweb' as the Value.
      
5. Once that information is entered, select Allow from the Action dropdown and click on the “Deploy firewall rule” button