hilrap2

Hi edo888,

thank you for the clarification and for taking the time to explain how gtranslate.php operates as a reverse proxy.

We understand that gtranslate.php primarily performs network I/O and that, when translations are served from your side, the translation step itself is usually very fast. This is also reflected in our analytics data.

For reference, our average translation times are:

• Last year: 22 ms
• Last month: 18 ms
• Last week: 14 ms

So from our perspective, translation performance itself appears consistently fast and stable.

We have also contacted your live chat as suggested and were advised that our request would be forwarded to the plugin author, which we understand to be you. We therefore wanted to follow up in writing with the relevant technical context.

What remains unclear to us is how this behavior is handled under concurrent access patterns.

From our observations:

• During crawl bursts, we see multiple concurrent executions of gtranslate.php, all blocking in curl_exec()
• These executions occur even though the origin is generally responsive under normal load
• The cumulative effect of multiple simultaneous blocking connections leads to PHP-FPM queue buildup and temporary service degradation

In other words, while a single stalled or slow origin fetch may not be problematic, many parallel origin fetches triggered at the same time appear to be the main contributing factor to the issue we are seeing.

To better understand and mitigate this, we would appreciate your guidance on the following:

1. Is there any form of request coordination, locking, or reuse to prevent multiple concurrent origin fetches for the same URL?
2. Are there recommended or configurable timeout values for the curl requests initiated by gtranslate.php?
3. Are there plugin settings or best practices aimed at reducing the impact of crawl bursts when using URL-based translations?
4. Is this concurrency behavior expected, or has there been any recent change in how origin fetches are handled?

Our goal is to ensure stable operation under crawl load while continuing to use URL-based translations, which have been working well for us for several years.

Thank you again for your support. We appreciate your time and look forward to your guidance.

Best regards

Hi,

thank you for your reply.

We have reviewed the GTranslate analytics as suggested. They show a sharp decrease in requests over the last weeks, which correlates with geo-blocking measures we introduced to combat excessive bot traffic. Despite this significant reduction in request volume, the PHP-FPM exhaustion issue persists during crawl bursts.

To address your other points:

• Origin response time: Our origin response time for non-translated pages is stable and has not materially changed. The slowdowns we observe occur specifically when multiple PHP-FPM workers are blocked inside curl_exec() during GTranslate processing, not due to general site slowness.
• Firewall blocking: We have verified that requests are not being blocked mid-flight. Direct access to gtranslate.php is explicitly blocked, and Cloudflare logs confirm there are no direct hits to that endpoint. The executions are triggered internally during normal translated page requests.
• PHP-FPM limits: Our hosting provider has confirmed that PHP-FPM limits have not been reduced recently.

Given this, the issue does not appear to be explained by increased request volume, slower origin responses, firewall interference, or reduced PHP-FPM capacity.

What remains consistent across multiple affected sites is the following pattern:

• multiple concurrent PHP-FPM workers
• all blocked in curl_exec()
• originating from wp-content/plugins/gtranslate/url_addon/gtranslate.php (around line 192)
• no request deduplication or locking
• blocking outbound calls with a default timeout of up to 60 seconds

As mentioned previously, this setup worked reliably for several years under comparable traffic conditions, and the overload behavior is a recent phenomenon.

To reiterate and clarify the open questions we would appreciate guidance on:

1. Are there any recent or planned changes in URL mode that affect how original content is fetched or refreshed under crawl load?
2. Is it possible to make outbound HTTP behavior configurable, specifically:
   • shorter or configurable curl timeouts
   • reduced or disabled retries
   • fail-fast behavior under load
3. Is there any internal roadmap for making URL mode more resilient to modern crawl patterns (high concurrency, bursty crawlers), without requiring sub-domain migration?

In addition, we would like to suggest the following potential improvements for consideration:

1. Introduce per-URL locking or cache-stampede protection so only one PHP-FPM worker performs an origin fetch for a given URL at a time.
2. Allow configuration of curl timeouts and retry behavior via plugin settings or constants.
3. Provide a non-blocking or deferred execution option for translation fetches to avoid tying up PHP-FPM workers.
4. Publish clear guidance on the scalability limits of URL mode, especially for long-standing, SEO-established sites.

We understand the recommendation to increase PHP-FPM processes or migrate to sub-domains. However, for established sites with years of indexed content, these are significant architectural changes, and it is concerning that there are currently no plugin-level mitigations for concurrency and blocking behavior.

We would appreciate clarification on whether improvements in this area are planned, or whether the current behavior is considered the final and expected behavior of URL mode.

Thanks for your time and clarification.

Hi @mrclayton,

After editing a typo, my post went in for manual moderation… Will try to repost.

Thank you for all your efforts, much appreciated!