WordPress generates strange links

  • WPuser3

    (@wpuser3)


    14 years, 1 month ago

    In my Webmaster tools I often see strange 404 errors. For example – “
    tag/nvidia/contact-me/” or “tag/official/feed/” or “tag/guide/feed/www.twitter.com” – they don’t even make sense.

Viewing 3 replies - 1 through 3 (of 3 total)
  • Paweł Knapek

    (@adpawl)

    14 years, 1 month ago

    Use robots.txt file to disallow this links.

    eg.

    User-agent: *
Allow: /
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /tag/

    Thread Starter WPuser3

    (@wpuser3)

    14 years, 1 month ago

    What about this instead?

    Sitemap: http://www.example.com/sitemap.xml

# Google Image
User-agent: Googlebot-Image
Disallow:
Allow: /*

# Google AdSense
User-agent: Mediapartners-Google*
Disallow:

# digg mirror
User-agent: duggmirror
Disallow: /

# global
User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /trackback/
Disallow: /feed/
Disallow: /comments/
Disallow: /category/*/*
Disallow: */trackback/
Disallow: */feed/
Disallow: */comments/
Disallow: /*?
Allow: /wp-content/uploads/
    Paweł Knapek

    (@adpawl)

    14 years, 1 month ago

    Use

    Sitemap: http://www.example.com/sitemap.xml

# Google Image
User-agent: Googlebot-Image
Disallow:
Allow: /*

# Google AdSense
User-agent: Mediapartners-Google*
Disallow:

# digg mirror
User-agent: duggmirror
Disallow: /

# global
User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /trackback/
Disallow: /feed/
Disallow: /tag/
Disallow: /comments/
Disallow: /category/*/*
Disallow: */trackback/
Disallow: */feed/
Disallow: */comments/
Disallow: /*?
Allow: /wp-content/uploads/

Viewing 3 replies - 1 through 3 (of 3 total)

The topic ‘WordPress generates strange links’ is closed to new replies.