Hello @rohansukne,
Sorry to hear you’re facing the issues with Google crawling the site.
“Hostload Exceeded” error means that the Google crawling limit has been hit. Google does that to prevent reaching the estimated server capacity, which could lead to slowing down the site.
This error may be a result of increased activity on the server, like a temporary traffic spike, or lack of the server resources. SmartCrawl itself does not directly affect the metrics used by Google to determine the server capacity.
First of all you can try recrawling the URLs after waiting for a while. If you get the same issue over and over, check the pages with Google Page Speed Insights tool, and see if you can perform some optimization, to make them load faster, and improve the page speed (e.g. activate cache with a caching plugin like Hummingbird).
Depending on the available server resources, you can also try increasing the memory_limit value in PHP options to the maximum.
Hope this helps. Let us know if you have any questions.
Best Regards,
Dmytro
Hi @rohansukne,
We haven’t heard from you in a while, I’ll go and mark this thread as resolved. If you have any additional questions or require further help, please let us know!
Kind Regards,
Kris