Title: Speculation Rules Script & Googlebot
Last modified: December 22, 2025

---

# Speculation Rules Script & Googlebot

 *  [farnely](https://wordpress.org/support/users/farnely/)
 * (@farnely)
 * [5 months, 1 week ago](https://wordpress.org/support/topic/speculation-rules-script-googlebot/)
 * I recently received a notification from Google that they had found reasons preventing
   pages from being indexed. (Bear with me; this IS about WordPress). When I checked
   the Search Console, the following URLs are listed as returning 404s:-
 * /wp-*.php, /wp-admin/*, /wp-content/uploads/*, /wp-content/*, /wp-content/plugins/*,/
   wp-content/themes/my-theme/*, /wp-content/themes/my-theme/*
 * They’re not “valid” URLs but they match EXACTLY the list used in the speculation
   rules script loaded by WP. That’s not a coincidence IMHO.
 * Normally, when pages are not being indexed, I check my settings for the affected
   URLs, make any required changes then mark as fixed in Search Console. I don’t
   see how I can do that in this case so can someone please explain what I can do
   to stop Google crawling these URLs.

Viewing 2 replies - 1 through 2 (of 2 total)

 *  Moderator [threadi](https://wordpress.org/support/users/threadi/)
 * (@threadi)
 * [5 months, 1 week ago](https://wordpress.org/support/topic/speculation-rules-script-googlebot/#post-18764425)
 * The paths mentioned belong to the WordPress core and are not and do not need 
   to be indexed by search engines.
 * What does your robots.txt look like? You can use it to exclude these paths.
 * Do you use an SEO plugin? That would take care of the appropriate robots.txt 
   entries for you.
 *  Thread Starter [farnely](https://wordpress.org/support/users/farnely/)
 * (@farnely)
 * [5 months ago](https://wordpress.org/support/topic/speculation-rules-script-googlebot/#post-18768270)
 * The entries in my robots file relating to Google have remained (rightly or wrongly)
   unchanged for years but now, since WP 6.8 was released in April, Google has got
   the idea that (taking the first item in the list) the URL _[https://my-site/wp-*.php](https://my-site/wp-*.php)_
   exists when it doesn’t, hence the 404s. The only place wp-*.php appears is in
   the speculation rules script that loads since WP 6.8.
 * Excluding those paths in robots would be a request to not crawl any of those 
   locations. In other words, “don’t crawl my site”.

Viewing 2 replies - 1 through 2 (of 2 total)

You must be [logged in](https://login.wordpress.org/?redirect_to=https%3A%2F%2Fwordpress.org%2Fsupport%2Ftopic%2Fspeculation-rules-script-googlebot%2F%3Foutput_format%3Dmd&locale=en_US)
to reply to this topic.

## Tags

 * [googlebot](https://wordpress.org/support/topic-tag/googlebot/)

 * In: [Fixing WordPress](https://wordpress.org/support/forum/how-to-and-troubleshooting/)
 * 2 replies
 * 2 participants
 * Last reply from: [farnely](https://wordpress.org/support/users/farnely/)
 * Last activity: [5 months ago](https://wordpress.org/support/topic/speculation-rules-script-googlebot/#post-18768270)
 * Status: not resolved

## Topics

### Topics with no replies

### Non-support topics

### Resolved topics

### Unresolved topics

### All topics