Invalid robots.txt

Hey guys,

The plugin is fantastic.

It omits one single thing – a rule set by Google regarding the robots.txt and that says “At a group-member level, in particular for allow and disallow directives, the most specific rule based on the length of the [path] entry will trump the less specific (shorter) rule. The order of precedence for rules with wildcards is undefined.”
That means that if you
Disallow: /wp-content/
Allow: /.js
Allow: /.css

the allow rules are invalid, since they are shorter. I have tested this and the CSS and JS does not get crawled if you use the rules above.

Thanks.

• This topic was modified 7 years, 9 months ago by miketica.

The page I need help with: [log in to see the link]