Title: Robots.txt is blocking googlebots
Last modified: August 24, 2016

---

# Robots.txt is blocking googlebots

 *  [jmasis09](https://wordpress.org/support/users/jmasis09/)
 * (@jmasis09)
 * [11 years ago](https://wordpress.org/support/topic/robotstxt-is-blocking-googlebots/)
 * hi!
    I have problems because google webmaster was notified me that googlebots
   can’t access to my site because of the robots.txt file.
 * I have check my robots.txt and it’s actually “virtual” because i can find it.
 * I don’t know how do the robots is blocking the googlebots.
 * Can someone help me!
 * The site I have problems is [http://www.tamcostarica.com/](http://www.tamcostarica.com/)

Viewing 5 replies - 1 through 5 (of 5 total)

 *  [barnez](https://wordpress.org/support/users/pidengmor/)
 * (@pidengmor)
 * [11 years ago](https://wordpress.org/support/topic/robotstxt-is-blocking-googlebots/#post-6146557)
 * I doesn’t look like there is anything in your robots.txt file that would block
   google:
 *     ```
       User-agent: *
       Disallow: /wp-admin/
       Disallow: /wp-includes/
   
       User-agent: Googlebot
       Allow:
       Sitemap: http://tamcostarica.com/sitemap.xml
       ```
   
 * You can test it in Google Webmaster Tools >> Crawl >> robots.txt.Tester
 * Is that a historical warning from a day in the past, or this a current issue 
   now?
 * EDITED:
    Try running this past your robits.txt Tester with the trailing slash
   after **Allow: /**
 *     ```
       User-agent: *
       Disallow: /wp-admin/
       Disallow: /wp-includes/
   
       User-agent: Googlebot
       Allow: /
       Sitemap: http://tamcostarica.com/sitemap.xml
       ```
   
 *  Thread Starter [jmasis09](https://wordpress.org/support/users/jmasis09/)
 * (@jmasis09)
 * [11 years ago](https://wordpress.org/support/topic/robotstxt-is-blocking-googlebots/#post-6146569)
 * Is a current issue…
 * google webmaster notified this:
    Over the last 24 hours, Googlebot encountered
   29 errors while attempting to access your robots.txt. To ensure that we didn’t
   crawl any pages listed in that file, we postponed our crawl. Your site’s overall
   robots.txt error rate is 100.0%.
 * If I Crawl as google the site is says “It’s out of access”
 *  [barnez](https://wordpress.org/support/users/pidengmor/)
 * (@pidengmor)
 * [11 years ago](https://wordpress.org/support/topic/robotstxt-is-blocking-googlebots/#post-6146639)
 * When I run your robots.txt through the Google Webmasters tester it comes up fine
   without any problems.
 * Have you tried:
 *     ```
       User-agent: Googlebot
       Allow: /
       ```
   
 * or
 *     ```
       User-agent: Googlebot
       Allow: *
       ```
   
 * Are you running any security plugins? Try disabling all plugins and then crawl
   as Google. If you can, then reactivate one by one until you can identify the 
   conflicting plugin.
 *  Thread Starter [jmasis09](https://wordpress.org/support/users/jmasis09/)
 * (@jmasis09)
 * [11 years ago](https://wordpress.org/support/topic/robotstxt-is-blocking-googlebots/#post-6146684)
 * I try with
 *     ```
       User-agent: *
       Disallow: /wp-admin/
       Disallow: /wp-includes/
   
       User-agent: Googlebot
       Allow: /
       Sitemap: http://tamcostarica.com/sitemap.xml
       ```
   
 * And now is working. Also I use a anti-malware scan, and fix a file that it have
   a Backdoor script so made, that was the problem generator.
 *  [barnez](https://wordpress.org/support/users/pidengmor/)
 * (@pidengmor)
 * [11 years ago](https://wordpress.org/support/topic/robotstxt-is-blocking-googlebots/#post-6146696)
 * Glad to hear that Google can see you again!

Viewing 5 replies - 1 through 5 (of 5 total)

The topic ‘Robots.txt is blocking googlebots’ is closed to new replies.

## Tags

 * [robots](https://wordpress.org/support/topic-tag/robots/)

 * In: [Fixing WordPress](https://wordpress.org/support/forum/how-to-and-troubleshooting/)
 * 5 replies
 * 2 participants
 * Last reply from: [barnez](https://wordpress.org/support/users/pidengmor/)
 * Last activity: [11 years ago](https://wordpress.org/support/topic/robotstxt-is-blocking-googlebots/#post-6146696)
 * Status: not resolved

## Topics

### Topics with no replies

### Non-support topics

### Resolved topics

### Unresolved topics

### All topics