High CPU usage with redirects enabled

  • Resolved Kyle Duncan

    (@kyleduncan)


    7 years, 8 months ago

    Hi,

    I have a site with about 17,000 redirects (unfortunately they have to be unique, as the old site we migrated from has no patternable structure to the old URLs).

    I have imported the redirects in groups of 2,000, and when I go above about 7,000-8,000, the MySQL process on my server starts to run high when traffic to my site increases.

    An example redirect regex I have (there are about 15,500 of these) is:

    /(oldsite|newsite)/article/(optional-article-slug-here/)?5556/?

    I know it’s a lot of redirects, but just wondered if there’s anything I can do differently in the plugin to make it faster? For example:
    1. Should I use the Apache module instead of WordPress?
    2. Would a stricter regex on each redirect help? E.g. defining the start and end of each URL precisely using ^ and $ respectively?
    3. Are there any other optimisations I could make?

    Our server is pretty powerful (an Amazon Lightsail instance, with 8 vCPUs, 32GB RAM) but it’s just the MySQL process that struggles when all the redirects are enabled.

    Thanks very much for any advice!

Viewing 8 replies - 1 through 8 (of 8 total)
  • Plugin Author John Godley

    (@johnny5)

    7 years, 8 months ago

    17000 regular expressions? With that amount you would be better making them plain redirects, then the database can use an index

    Plugin Author John Godley

    (@johnny5)

    7 years, 8 months ago

    Also, it may be worth considering putting the logic in the application layer (i.e. PHP). There may be someway of parsing the URLs in PHP that you can’t achieve with regular expressions

    Thread Starter Kyle Duncan

    (@kyleduncan)

    7 years, 7 months ago

    Hi John!

    Thanks very much for your quick reply. Could you explain what you mean by “plain redirects”? Do you mean like Redirect 301 oldpath newpath rather than using conditions and rules? I might be able to turn off the “regex” option, but:

    1. some articles will have been bookmarked by users with an annoying hashtag at the end because of our old comments system, and I wanted the redirects to work with or without the hashtag. Would I have to create 2 redirects for each article because of this?
    2. Some articles may have been bookmarked before or after friendly URLs were added to the old site, so I needed URLs like /article/12345/ AND /article/name-of-post-here/12345/ to both work, hence the regex. I also can’t use a universal regex for all articles, because the old site’s friendly URLs are terrible and don’t nicely match the new wordpress URLs in most cases. So again, would I need 2 redirects for each article?

    So re: the above issues, would I actually need 4 x 17,000 redirects in order to do it all without regexes? And, if I were to go that route, what would I need to do to setup a database index? Sorry if this is beyond the scope of your plugin, I’m not clear if you’re referring to a feature within your plugin or not.

    And lastly, with putting the logic in PHP, could you point me in the direction of the kind of thing you’re thinking of?

    Cheers

    Plugin Author John Godley

    (@johnny5)

    7 years, 7 months ago

    By plain redirect I meant a non-regular expression redirect.

    You can’t redirect anything after the # character anyway, so there’s no point making regular expressions to handle those.

    For (2) you could convert it to two plain redirects. It does mean more redirects, but the URLs are indexed by the database and so this won’t have any effect on performance (this is handled by the plugin).

    My comment about PHP is if it’s not possible to simplify the old URLs, or somehow convert them into plain redirects, then doing the redirect in PHP may be an option. You would need to code your own plugin that looks at the URL and parses it in a way that may be easier using the full power of PHP.

    Thread Starter Kyle Duncan

    (@kyleduncan)

    7 years, 7 months ago

    Great thank you John. So plain redirects will perform much faster than regexes? if that’s the case, I’ll definitely try 34,000 plain redirects instead of 17,000 regex redirects, sounds promising!

    On the hashtag, will a URL with a hashtag match a plain redirect just fine?

    so if the redirect is for /oldsite to http://www.test.com/newsite and somebody goes to http://www.test.com/oldsite#12345 will the plain redirect work?

    Plugin Author John Godley

    (@johnny5)

    7 years, 7 months ago

    No, hashtags are not sent to the server and it’s not possible to redirect based on them. This applies to any server redirect, not just Redirection (see https://redirection.me/support/faq/)

    You’ll need to redirect in the client using JavaScript if the hash value is important.

    Thread Starter Kyle Duncan

    (@kyleduncan)

    7 years, 7 months ago

    Sorry I wasn’t clear. I don’t care about the hashtag value or saving it, I just want the redirect to work. So I meant: will both of these URLs:

    http://www.test.com/oldsite
    http://www.test.com/oldsite#12345

    redirect to this URL:

    http://www.test.com/newsite

    if the plain redirect is:

    Redirect 301 /oldsite http://www.test.com/newsite

    Thread Starter Kyle Duncan

    (@kyleduncan)

    7 years, 7 months ago

    I just tested your method, and the results are fantastic! CPU usage halved, even though there are now 34,000 redirects in place for all the permutations, and the hashtags are just passed through as you said (i see they’re actually maintained in the URL, which is fine, they don’t do any harm on the new site).

    Thanks again John, for the great plugin and the tip!

Viewing 8 replies - 1 through 8 (of 8 total)

The topic ‘High CPU usage with redirects enabled’ is closed to new replies.