Google crawler can't access robots.txt
-
I’m getting errors from my Google Webmaster-Tools account. It says, it can’t access my robots.txt
I’m experiencing this problem on 2 websites I’m running.
First website:
Wordpress 4.0.1 (german)
WP Slimstat 3.8.3
Server: Apache HTTPD 1.3.x
PHP: 5.5.11Second website:
Wordpress 4.0.1 (german)
WP Slimstat 3.8.3
Server: nginx/1.2.1
PHP: 5.4.35If I manually trigger fetching robots.txt from here
https://www.google.com/webmasters/tools/googlebot-fetch
I also get the error “Unreachable” IF fetching as desktop client.
When selecting “Mobile: Smartphone” instead, everything’s fine.This is what I get in my Apache’s error_log:
[Mon Nov 24 05:02:26 2014] [error] [client 66.249.78.253] PHP Fatal error: Unsupported operand types in /path/to/webroot/wordpress/wp-content/plugins/wp-slimstat/wp-slimstat.php on line 725This is what I get in my Apache’s access_log, if using “Mobile: Smartphone” for fetching:
66.249.78.247 – – [24/Nov/2014:05:02:24 +0100] “GET /robots.txt HTTP/1.1” 500 243 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”If I comment out line 725, error_log says (when Google-fetching in desktop mode):
[Mon Nov 24 05:36:45 2014] [error] [client 66.249.78.247] PHP Warning: array_key_exists() expects parameter 2 to be array, boolean given in /path/to/webroot/wordpress/wp-content/plugins/wp-slimstat/wp-slimstat.php on line 723Workaround:
Replace line 725 with this:
$search = ((string)$search) + ((string)$value);When accessing robots.txt via webbrowser, everything looks fine (I’m using Mozilla Firefox 33.0, operating-system is openSUSE Linux 13.2).
By the way:
There isn’t any real robots.txt on my webspace. WordPress just pretends there is. This is the content of the virtual robots.txt (as I get it when accessing with Firefox):
User-agent: *
Disallow: /wp-admin/
The topic ‘Google crawler can't access robots.txt’ is closed to new replies.