Google Search Console noindex

  • Resolved zkedrov

    (@zkedrov)


    4 months, 4 weeks ago

    Hello,

    I am experiencing an issue with the XML sitemaps generated by Yoast SEO on my website blisstver.ru.

    Google Search Console refuses to process my sitemap and returns the error:

    “Couldn’t fetch / Sitemap could not be processed.”

    After checking the HTTP response, I discovered that the sitemap files generated by Yoast include the following header:

    X-Robots-Tag: noindex, follow

    Because of this header, Google does not accept the sitemap.

    Here are the verified facts:

    1. When I deactivate Yoast SEO, the sitemap returns a 404 error — confirming that Yoast is generating the sitemap.
    2. All regular website pages return normal headers without noindex.
    3. Only the sitemap files created by Yoast SEO return:

    X-Robots-Tag: noindex, follow

    1. My hosting provider (REG.ru) confirmed that the header is not added by the server (nginx), but specifically by the Yoast SEO plugin.They wrote:“The X-Robots-Tag: noindex header is generated by the WordPress SEO plugin, not by the Nginx server.”

    The problem is that I cannot find any setting inside Yoast SEO that would enable or disable the noindex header for XML or sitemap files.Environment:

    HTTP/2 200
    Content-Type: text/xml
    X-Robots-Tag: noindex, follow

    I kindly request your assistance with the following:

    1. Where exactly in Yoast SEO (in my version) is the setting located that controls the noindex header for XML/sitemap files?
    2. How can I disable Yoast SEO from adding X-Robots-Tag: noindex to sitemap files?
    3. Is it possible that this noindex header is added automatically if some content types or sections in Yoast are set to “Do not show in search results”?If so, please specify where I should check this.

    I need the sitemap to be indexable (or at least not include a noindex header), otherwise Google Search Console cannot process it.

    Thank you in advance for your help!

    The page I need help with: [log in to see the link]

Viewing 4 replies - 1 through 4 (of 4 total)
  • biglyash1

    (@biglyash1)

    4 months, 4 weeks ago

    I am having the same issue – my homepage is returning <meta name=’robots’ content=’noindex, follow’ /> which has prevented google from indexing certain pages. Have checked robots.txt and updated rules, but that is unlikely to be the issue anyway since some pages are indexing and do not have the no index meta tags. OP if you check individual URLs you might find similar thing.

    Would appreciate further assistance.

    Plugin Support Maybellyne

    (@maybellyne)

    4 months, 3 weeks ago

    Hello @zkedrov,

    Thanks for reaching out. I understand Google Search Console hasn’t fetched or processed your sitemap. This has nothing to do with X-Robots-Tag: noindex, follow. That directive is added to ensure the sitemap itself is not listed in search results. It does not prevent Google will indexing the URLs in the sitemap.

    Where exactly in Yoast SEO (in my version) is the setting located that controls the noindex header for XML/sitemap files?

    Yoast SEO generates the XML sitemap dynamically, so the header is added automatically. You cannot remove it.

    How can I disable Yoast SEO from adding X-Robots-Tag: noindex to sitemap files

    No, it cannot be disabled

    Is it possible that this noindex header is added automatically if some content types or sections in Yoast are set to “Do not show in search results”?If so, please specify where I should check this.

    It is unconnected to content types or the sections in the plugin you have referred to.

    To the main issue, ensure you submitted only the sitemap index. Do not remove and resubmit as this sort of restarts the process. Google needs some time to process the sitemap and will do so eventually.

    @biglyash1: Your issue is unrelated. Please open a new topic.

    desmondrafferty

    (@desmondrafferty)

    4 months, 3 weeks ago

    Hi I have a similar issue.

    a lot of pages not indexed. Some of these pages were created in 2020.

    https://the-mkgroup.com/

    How do I add images to this?

    Plugin Support Maybellyne

    (@maybellyne)

    4 months, 3 weeks ago

    @desmondrafferty: Your issue is unrelated. Please open your own thread.

Viewing 4 replies - 1 through 4 (of 4 total)

You must be logged in to reply to this topic.