mpoepping30
Forum Replies Created
-
Forum: Plugins
In reply to: [XML Sitemap Generator for Google] Sitemap URLs blocked by robots.txt file@ecokleensolar, I did not find an actual solution anywhere, and seem to have the same troubles you do with finding any help in getting an answer.
After really combing through my client’s Search Console and site audit (from AHRefs) I did end up fixing any and all errors I could however. From there I saw the sitemap issues cleared after resubmitting it to Search Console.
I don’t necessarily think that’s a direct correlation, as I really think the virtual sitemap just finally updated based on my robots.txt changes (see below for what I landed on trying for this), or I think removing a Schema Markup plugin I had installed on the site (which I also did to try and fix this issue) may have also helped, but I’ll never know for sure.
I think that plugin (I can’t remember the name of the one we were using) was creating the virtual sitemap, and Yoast may have been as well, which could’ve also caused the bot crawl confusion.
Sorry I can’t be more help, but hopefully this gives you some ideas on what to try/where to look!
Sample Robots.txt file:
User-Agent: *
Allow: /
Allow: /wp-content/uploads/
Disallow: /wp-content/plugins/
Disallow: /readme.html