Title: additional useragent added
Last modified: July 29, 2023

---

# additional useragent added

 *  Resolved [janardhan](https://wordpress.org/support/users/janardhanjp/)
 * (@janardhanjp)
 * [2 years, 10 months ago](https://wordpress.org/support/topic/additional-useragent-added/)
 * User-agent: * Disallow: /wp-admin/
 * Allow: /wp-admin/admin-ajax.php
 * User-agent: * Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
 * Please remove User agent * in the last line.
 * The page I need help with: _[[log in](https://login.wordpress.org/?redirect_to=https%3A%2F%2Fwordpress.org%2Fsupport%2Ftopic%2Fadditional-useragent-added%2F%3Foutput_format%3Dmd&locale=en_US)
   to see the link]_

Viewing 6 replies - 1 through 6 (of 6 total)

 *  Thread Starter [janardhan](https://wordpress.org/support/users/janardhanjp/)
 * (@janardhanjp)
 * [2 years, 10 months ago](https://wordpress.org/support/topic/additional-useragent-added/#post-16932926)
 * the following is the robots.txt of my website:
 * User-agent: * Disallow: /wp-admin/
 * Allow: /wp-admin/admin-ajax.php
 * User-agent: * Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
 * Please help me remove ” User agent * ” in the last line.
    -  This reply was modified 2 years, 10 months ago by [janardhan](https://wordpress.org/support/users/janardhanjp/).
 *  Plugin Contributor [Venkat Raj](https://wordpress.org/support/users/webulous/)
 * (@webulous)
 * [2 years, 10 months ago](https://wordpress.org/support/topic/additional-useragent-added/#post-16935441)
 * [@janardhanjp](https://wordpress.org/support/users/janardhanjp/) It is not possible
   to remove single line. You can remove all the lines added by WP-Optimize by adding
   following code in your theme’s `functions.php` file
 *     ```wp-block-code
       add_action('after_setup_theme', 'ab123_remove_robots_txt_entry');
       function ab123_remove_robots_txt_entry() {
       	if (function_exists('WP_Optimize')) {	
       		remove_filter('robots_txt', array(WP_Optimize(), 'robots_txt'), 99, 1);
       	}
       }
       ```
   
    -  This reply was modified 2 years, 10 months ago by [Venkat Raj](https://wordpress.org/support/users/webulous/).
    -  This reply was modified 2 years, 10 months ago by [Venkat Raj](https://wordpress.org/support/users/webulous/).
 *  Thread Starter [janardhan](https://wordpress.org/support/users/janardhanjp/)
 * (@janardhanjp)
 * [2 years, 10 months ago](https://wordpress.org/support/topic/additional-useragent-added/#post-16935909)
 * the following is th Robots.txt as on today:
 * User-agent: *
 * Disallow: /wp-admin/
 * Allow: /wp-admin/admin-ajax.php
 * User-agent: *
 * Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
 * the robots text analysis is telling like this,
 * Results of robots.txt analysis
 * 1 error
 * lines used
 * User-agent: *     Multiple ‘User-agent: *’ rules found
 * So there are two User Agent:*
 * So pl try to remove the last but one line in robots text
 * that is remove User Agent:* before
 * Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
 * which is repetitive
 *  Plugin Contributor [Venkat Raj](https://wordpress.org/support/users/webulous/)
 * (@webulous)
 * [2 years, 9 months ago](https://wordpress.org/support/topic/additional-useragent-added/#post-16946544)
 * [@janardhanjp](https://wordpress.org/support/users/janardhanjp/) At this moment
   there is no option to remove that single line. To implement this, could you please
   let us know which `robots.txt` analyzer throws error? Thanks
 *  Thread Starter [janardhan](https://wordpress.org/support/users/janardhanjp/)
 * (@janardhanjp)
 * [2 years, 9 months ago](https://wordpress.org/support/topic/additional-useragent-added/#post-16959488)
 * Yandex Site diagnostics says,
 * Robots.txt contains errors. This can lead to incorrect crawling and indexing 
   of your site.
 * [Check](https://webmaster.yandex.com/site/https:www.samskrutijanardhan.com:443/tools/robotstxt/)
   the robots.txt file.
 * User-agent: *
 * Disallow: /wp-content/uploads/wpo-plugins-tables-list.json
 * WP Optimize was added by me recently. After that I found this error message in
   Yandex.
 *  Thread Starter [janardhan](https://wordpress.org/support/users/janardhanjp/)
 * (@janardhanjp)
 * [2 years, 9 months ago](https://wordpress.org/support/topic/additional-useragent-added/#post-16986240)
 * Thanks Venkat Raj. I have added the code given by you in functions.php. And the
   lines added by WPoptimize are gone. Thank you very much.

Viewing 6 replies - 1 through 6 (of 6 total)

The topic ‘additional useragent added’ is closed to new replies.

 * ![](https://ps.w.org/wp-optimize/assets/icon-256x256.png?rev=1552899)
 * [WP-Optimize – Cache, Compress images, Minify & Clean database to boost page speed & performance](https://wordpress.org/plugins/wp-optimize/)
 * [Frequently Asked Questions](https://wordpress.org/plugins/wp-optimize/#faq)
 * [Support Threads](https://wordpress.org/support/plugin/wp-optimize/)
 * [Active Topics](https://wordpress.org/support/plugin/wp-optimize/active/)
 * [Unresolved Topics](https://wordpress.org/support/plugin/wp-optimize/unresolved/)
 * [Reviews](https://wordpress.org/support/plugin/wp-optimize/reviews/)

 * 6 replies
 * 2 participants
 * Last reply from: [janardhan](https://wordpress.org/support/users/janardhanjp/)
 * Last activity: [2 years, 9 months ago](https://wordpress.org/support/topic/additional-useragent-added/#post-16986240)
 * Status: resolved