Bots are blocked from crawling website when x-theme is active

Support
#22

It turns out that the default Mod Security rules in Cpanel, created by Jolt servers, are causing the issue. This conflicts with the Pro theme causing a 403 error and prevents bots from crawling the site. I believe this is the case with all Pro themes being used on jolt shared servers as it happened on another server account that was created on the same server.

If anyone else is having this issue, go to Mod Security in Cpanel and turn off the mod security rules for your domain. Hopefully this will help people with the same problem in the future.

#23

Hey @callandcreative,

Thanks for sharing this. We really appreciate for letting us know that you have figure out the cause of the issue. This is very useful information for other users too.

Best Regards.

#24

Just a note from me on this. I recently had a 403 issue on Cloudflare and I ended up looking into this thread. It turned out to be something else, but just in case it helps anyone - we had a lot of global bot signups so used Cloudflare to Recaptcha challenge all countries other than the UK. It worked for the bots, but what I hadn’t figured on was that Cloudflare’s ‘Firewall Rules’ does not respect their global whitelists - in other words it was blocking SEO crawlers and showing a 403 Forbidden error in Sucuri.

Instead, Cloudflare support suggested this page https://developers.cloudflare.com/firewall/cf-firewall-rules/actions/ and then use cf.client.bot for the rule writing.

I realise this is unrelated other than Cloudflare and 403 but thought I’d drop this in in case it helps anyone having similar issues. Btw this was not a Cloudflare plugin, this was within the Cloudflare Dash.

#25

Thank you for sharing this Paul.

#26