googlebot hitting admin-ajax.php

  • deafeed

    (@deafeed)


    9 years, 3 months ago

    Recent logs show googlebot is hitting admin-ajax.php even though called off limits in robots.txt

    That normal behavior? To disrespect the robots.txt

Viewing 2 replies - 1 through 2 (of 2 total)
  • Pablo Pacheco

    (@karzin)

    9 years, 3 months ago

    Hello,

    I don’t know if it’s a good idea to block googlebot from reading admin-ajax.
    I say this because google wants to have access to all css and js. Blocking this file can block access to these files sometimes either.

    And considering this file also appears on front-end i think it’s good to allow google to read it. Maybe it’s the reason your robots is not being respected.

    Therefore i think robots could be something like this

    Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

    WordPress itself is allowing it by default

    • This reply was modified 9 years, 3 months ago by Pablo Pacheco.
    Thread Starter deafeed

    (@deafeed)

    9 years, 3 months ago

    Muchas gracias

Viewing 2 replies - 1 through 2 (of 2 total)

The topic ‘googlebot hitting admin-ajax.php’ is closed to new replies.