Title: Robots problem?
Last modified: May 25, 2020

---

# Robots problem?

 *  Resolved [ai4k Web Design](https://wordpress.org/support/users/soylaostia/)
 * (@soylaostia)
 * [6 years ago](https://wordpress.org/support/topic/robots-problem/)
 * I´ve noticed a lot of 404’s from Google spider trying to index plugins, and I’ve
   disallowed it in the robots.
 * Disallow: /wp-content/plugins/
    Disallow: /plugin/ Disallow: /plugins/
 * Can you check it, please?
 * Thanks in advance.

Viewing 2 replies - 1 through 2 (of 2 total)

 *  [bloup](https://wordpress.org/support/users/bloup/)
 * (@bloup)
 * [6 years ago](https://wordpress.org/support/topic/robots-problem/#post-12888124)
 * It happens to everyone, it is not related to all in one.
    In fact, googlebot 
   crawls a HTML page, and then later on, you may uninstall plugins. However, the
   HTML page referred to those plugins in the HTML code (Path to CSS file, path 
   to JS files, or WOFF files). So, Googlebot will still think there are there as
   long as it didn’t recrawl the HTML page. That’s why you get so many 404 errors.
   But at some point, you will less and less of them because googlebot will re-crawl
   the HTML pages. Keep in mind, that googlebot separately crawls HTML pages and
   CSS files, images, JS files.
    -  This reply was modified 6 years ago by [bloup](https://wordpress.org/support/users/bloup/).
 *  Plugin Support [Steve M](https://wordpress.org/support/users/wpsmort/)
 * (@wpsmort)
 * [6 years ago](https://wordpress.org/support/topic/robots-problem/#post-12900135)
 * You shouldn’t block Google from the /wp-content/plugins/ directory. They need
   to be able to access any plugins that have any JS or CSS required in order to
   render your website in Chrome. If they can’t render your site in Chrome because
   you’ve blocked access to these files then this will affect whether they index
   your content or not.

Viewing 2 replies - 1 through 2 (of 2 total)

The topic ‘Robots problem?’ is closed to new replies.

 * ![](https://ps.w.org/all-in-one-seo-pack/assets/icon.svg?rev=2443290)
 * [All in One SEO – Powerful SEO Plugin to Boost SEO Rankings & Increase Traffic](https://wordpress.org/plugins/all-in-one-seo-pack/)
 * [Frequently Asked Questions](https://wordpress.org/plugins/all-in-one-seo-pack/#faq)
 * [Support Threads](https://wordpress.org/support/plugin/all-in-one-seo-pack/)
 * [Active Topics](https://wordpress.org/support/plugin/all-in-one-seo-pack/active/)
 * [Unresolved Topics](https://wordpress.org/support/plugin/all-in-one-seo-pack/unresolved/)
 * [Reviews](https://wordpress.org/support/plugin/all-in-one-seo-pack/reviews/)

 * 2 replies
 * 3 participants
 * Last reply from: [Steve M](https://wordpress.org/support/users/wpsmort/)
 * Last activity: [6 years ago](https://wordpress.org/support/topic/robots-problem/#post-12900135)
 * Status: resolved