Title: Robots.txt internal server error
Last modified: August 21, 2016

---

# Robots.txt internal server error

 *  [sephcot](https://wordpress.org/support/users/sephcot/)
 * (@sephcot)
 * [12 years, 4 months ago](https://wordpress.org/support/topic/robotstxt-internal-server-error/)
 * Hi all,
 * Any attempt to access the robots.txt file for one of the websites I run — [idolchatteryd.com](https://wordpress.org/support/topic/robotstxt-internal-server-error/idolchatteryd.com?output_format=md)—
   comes back with a 500 Internal Server Error. That happens when I try to do so
   by typing [idolchatteryd.com/robots.txt](https://wordpress.org/support/topic/robotstxt-internal-server-error/idolchatteryd.com/robots.txt?output_format=md)
   into the url, and (more disturbingly) it happens when I try to “fetch as Google”
   using Google Webmaster, which tells me the file is unreachable. For this reason,
   Google has — *gasp* — deindexed the site.
 * I really would like the site to be indexed, obviously, so what the heck do I 
   do?
 * My Googling has told me that the .htaccess might be the problem, but I don’t 
   really know what to do with that information, or what I’d be looking for that
   could be problematic. So…help, please?
 * Thanks!

Viewing 3 replies - 1 through 3 (of 3 total)

 *  [kmessinger](https://wordpress.org/support/users/kmessinger/)
 * (@kmessinger)
 * [12 years, 4 months ago](https://wordpress.org/support/topic/robotstxt-internal-server-error/#post-4557514)
 * Have you checked to see if the file is really there?
 * And your links above are incorrect.
 *  Thread Starter [sephcot](https://wordpress.org/support/users/sephcot/)
 * (@sephcot)
 * [12 years, 4 months ago](https://wordpress.org/support/topic/robotstxt-internal-server-error/#post-4557520)
 * Now that you mention it, it isn’t in the FTP. Sorry, I’m pretty new to this. 
   But the issue remains: Google gets that internal server error when trying to 
   figure out what bots are allowed to crawl the site when it looks for a robots.
   txt. Any suggestions?
 * And my bad: [idolchatteryd.com](http://www.idolchatteryd.com/) and [idolchatteryd.com/robots.txt](http://www.idolchatteryd.com/robots.txt/)
 *  [kmessinger](https://wordpress.org/support/users/kmessinger/)
 * (@kmessinger)
 * [12 years, 4 months ago](https://wordpress.org/support/topic/robotstxt-internal-server-error/#post-4557532)
 * An internal service error means something is set up wrong in the server. If it
   was just a file missing you would have a 404 error.
 * You don’t have to have a robots.txt file to be indexed by Google or anyone. Bots
   would look for it and follow the instructions or if they don’t see it they should
   go about their work. The trouble is that when they look they get the error.
 * Time to ask GoDaddy for some assistance. Good luck.

Viewing 3 replies - 1 through 3 (of 3 total)

The topic ‘Robots.txt internal server error’ is closed to new replies.

## Tags

 * ["internal server error"](https://wordpress.org/support/topic-tag/internal-server-error/)
 * [robots.txt](https://wordpress.org/support/topic-tag/robots-txt/)
 * [unreachable](https://wordpress.org/support/topic-tag/unreachable/)

 * In: [Fixing WordPress](https://wordpress.org/support/forum/how-to-and-troubleshooting/)
 * 3 replies
 * 2 participants
 * Last reply from: [kmessinger](https://wordpress.org/support/users/kmessinger/)
 * Last activity: [12 years, 4 months ago](https://wordpress.org/support/topic/robotstxt-internal-server-error/#post-4557532)
 * Status: not resolved

## Topics

### Topics with no replies

### Non-support topics

### Resolved topics

### Unresolved topics

### All topics