Speculation Rules causing Google bot crawl errors
-
Have been having some trouble with this code, which gets injected in the footer via wp_footer(). My SEO team has indicated that it is responsible for causing google bots to refuse re-crawling pages.
{"prefetch":[{"source":"document","where":{"and":[{"href_matches":"\/*"},{"not":{"href_matches":["\/wp-*.php","\/wp-admin\/*","\/wp-content\/uploads\/*","\/wp-content\/*","\/wp-content\/plugins\/*","\/wp-content\/themes\/my_theme\/*","\/*\\?(.+)"]}},{"not":{"selector_matches":"a[rel~=\"nofollow\"]"}},{"not":{"selector_matches":".no-prefetch, .no-prefetch a"}}]},"eagerness":"conservative"}]}Here are the articles my team provided:
- https://support.google.com/webmasters/answer/9012289#will_i_be_indexed
- https://ww.wp.xz.cn/support/topic/some-of-my-pages-cannot-be-indexed/
- https://support.google.com/webmasters/thread/10533591/url-will-be-indexed-only-if-certain-conditions-are-met?hl=en
Essentially, it appeared that this “
prefetch” installed on the site (that hadn’t been present before) was causing google to avoid re-crawling pages. Pages could be submitted but once read they would not be indexed again.Thanks,
Cameron
The topic ‘Speculation Rules causing Google bot crawl errors’ is closed to new replies.