Title: double user agents
Last modified: February 14, 2023

---

# double user agents

 *  Resolved [wazifalab](https://wordpress.org/support/users/wazifalab/)
 * (@wazifalab)
 * [3 years, 3 months ago](https://wordpress.org/support/topic/double-user-agents/)
 * I have double user agents in my website robots.txt file,
   The second one was generated
   by wp optimize plugin automatically, how may i keep the rule and delete the second
   agent?
 * The page I need help with: _[[log in](https://login.wordpress.org/?redirect_to=https%3A%2F%2Fwordpress.org%2Fsupport%2Ftopic%2Fdouble-user-agents%2F%3Foutput_format%3Dmd&locale=en_US)
   to see the link]_

Viewing 1 replies (of 1 total)

 *  Plugin Contributor [Venkat Raj](https://wordpress.org/support/users/webulous/)
 * (@webulous)
 * [3 years, 3 months ago](https://wordpress.org/support/topic/double-user-agents/#post-16476371)
 * [@wazifalab](https://wordpress.org/support/users/wazifalab/) The no. of user 
   agents doesn’t affect your site. Each user agent directive only applies to the
   rules that follows them. See documentation page where two user agents are present
   
   [https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt](https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt)
   By default, there is no `robots.txt` file in WP setup. Mostly SEO plugins create
   them. What if the plugin that created it, also removes it in later stage? The`
   Disallow` rule will be there without any user agent.If you want to remove the
   rules added by WP Optimize, add the following code in your theme’s `functions.
   php`
 *     ```wp-block-code
       add_action('after_setup_theme', 'ab123_remove_robots_txt_entry');
       function ab123_remove_robots_txt_entry() {
         if (function_exists('WP_Optimize')) {
           remove_filter('robots_txt', array(WP_Optimize(), 'robots_txt'), 99, 1);
         }
       }
       ```
   

Viewing 1 replies (of 1 total)

The topic ‘double user agents’ is closed to new replies.

 * ![](https://ps.w.org/wp-optimize/assets/icon-256x256.png?rev=1552899)
 * [WP-Optimize – Cache, Compress images, Minify & Clean database to boost page speed & performance](https://wordpress.org/plugins/wp-optimize/)
 * [Frequently Asked Questions](https://wordpress.org/plugins/wp-optimize/#faq)
 * [Support Threads](https://wordpress.org/support/plugin/wp-optimize/)
 * [Active Topics](https://wordpress.org/support/plugin/wp-optimize/active/)
 * [Unresolved Topics](https://wordpress.org/support/plugin/wp-optimize/unresolved/)
 * [Reviews](https://wordpress.org/support/plugin/wp-optimize/reviews/)

 * 1 reply
 * 2 participants
 * Last reply from: [Venkat Raj](https://wordpress.org/support/users/webulous/)
 * Last activity: [3 years, 3 months ago](https://wordpress.org/support/topic/double-user-agents/#post-16476371)
 * Status: resolved