HI:
You can tweak your robots.txt file in the file editor module.
Here’s a reference guide on robots.txt
Best,
Jeffrey
Hi Jeffrey,
Thanks for that. In the file editor module (Appearance > Editor) cannot find any .php or template that refers to the Robots.txt. Is there a specific place for it?
We tried to do it directly in the server and everytime it we changed, the website shut down. Any similar experience in the past?
Best,
Hugo
Sorry, it was not meant the above comment.
I went to file editor and change the robots.txt to be:
User-agent: *
Disallow:
Still, no changes are applied: http://blog.ev-box.com/robots.txt.
Suggestions?
Hi Hugo:
Just use ftp and upload your new robots.txt as many hosting permissions block direct modification of robots.
FTP is the best way.
Jeffrey
Hello,
I have edited my robots.txt file and did that through ftp but it didn’t change anything – when I go to
http://www.playnwisconsin.com/robots.txt
it still says…
User-agent: *
Disallow: /
This is what is in the htaccess box in my SEO file editor-
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ – [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress
Is this messing something up???
Thanks for your help!
@emphoto4:
Just looked now, seems your pages are crawled and indexed just fine and can’t see robots.txt now, so, I assume you deleted?
Best,
Jeffrey
