I’m also curious about the status of this (be it chunked up beyond the total number of pages being primed at a time for its own processing of things & triggered via more traditional/existing means, potentially implemented with https://www.php.net/manual/en/intro.parallel.php [per https://www.php.net/manual/en/intro.pthreads.php being considered dead] if/when available [both of which effectively requiring PHP 7.2+ for safe execution either way], relying on WP-CLI to some degree, or some other implementation.) I would imagine any of these would help scale the preloading cache for some of the larger sites out there (sites with an active and/or long-running blog, event calendar, directory listings, etc. and/or a combination of things can really have things add up.)
If there isn’t a good method now (with any number of those options above [and maybe some I didn’t think of] potentially/hopefully being considered for a future release), would the best approach in the meantime be to have it simply trigger a cache preload more frequently where they could potentially have separate operations run alongside each other if the previous request didn’t finish yet, or does that actually check to see if a process is already running to prime the cache & waits until that’s done to start another?
-
This reply was modified 3 years, 11 months ago by
KZeni.
Great, hopefully we can find a solution.
I haven’t dug in to the plugin code myself yet but maybe will tomorrow if we don’t get a response.
Hello @stevewolfe555
Thank you for your question and I am happy to answer.
At the moment there is no possibility to do this. The problem with this as @kzeni mentioned is the PHP version.
So for now, the option si to rely on wp-cron or, a custom cron job to visit the website at a specific time period (ideally while wp-cron is disabled), and of course, wp-cli, so, instead of calling wp-cron via crontab – you can call it directly via
wp w3-total-cache pgcache_prime
and track the output of which pages/sitemap positions are primed by script.
I can add this as a ticket for potential future enhancement in our GitHub repository.
Thanks!
Ideally, we would be able to dispatch jobs to the cron queue that can be picked up if not already running.
Then each time the cron is hit we can start a new php process.
@vmarko Thanks for the details. It sure would be nice if W3TC was to be updated to make this a more accessible feature (possibly version-checking PHP 7.2+ and the parallel extension [per pThreads not being advisable anymore, it seems] to be able to offer it as a progressive enhancement rather than limiting it to the lowest common denominator [possibly similar to how it checks to see what caching/minify/compression/etc. modules/extensions are available to then offer it & maybe even be the default when available since I don’t really envision a downside to this action being multithread-capable once that part of things has been built out & shown as working well with it then on a hosting setup that supports it.]) I definitely understand this is likely an involved thing to plan & implement. Happy to just hear it’s under consideration.
Hello @kzeni @stevewolfe555
Thank you all for your suggestions.
I’ve had a chat with the team and this will be added to a roadmap for future releases, once we make sure that this can be used properly.
Thank you again!
