mjvito

Happy to help. Standing by for your update.

I tried two rounds of testing, but I’m getting unexpected behavior.

In each of the iterations below, when I manually trigger a preload, a “Cache Preload Started” email is sent, but this is the only notification I receive.

Round 1

Started with a loaded cache from 1.9.4. I triggered a manual preload, and at first it seemed to work. The dialog at the top of the Preload tab showed tags being cached. With previous versions of the alpha, it took about 80 minutes for a full cache preload, so I checked progress after a half hour. At that point, the Preload tab appeared as it would after a preload cycle had completed, with no dialog box. No preload stalled or completed emails were sent. The Cached Pages count in the Contents tab was unchanged from before I started the preload.

After that, I triggered another manual preload. This time, the dialog box showed it resuming from somewhere in the middle of the tags. The dialog box message did not update and Cached Pages count did not change.

After that, I canceled the preload, deleted the cache, and triggered another manual preload. This time, the dialog box showed it resuming from the same tag where it had attempted to start the previous time. The dialog box message did not update and Cached Pages count did not change.

Round 2

I tried again at a later time, starting with a mostly empty cache. The only message in the dialog box is the initial note that preloading will begin after 10 seconds. The dialog disappears after the tab is refreshed. No preloading happens.

Thanks for incorporating my feedback. I know this plugin is not just for me 🙂

The only additional information sent from DreamHost was, “could not locate any network-related issue or anything suggesting your VPS services are running out of resources…could not find anything meaningful in your MySQL’s slow query log”

I went back to 1.9.4 to observe how it behaves with the new max children and PHP memory. The most it asked for was 6 processes.

It is the case that some of my posts are very heavy, with large amounts of text, images and hyperlinks. In many cases there are also a few embeds, which pull extra js for loading facades and social media trackers. In a few cases, there are a large number of embeds. But the server manages to get through a preload on 1.9.4 with no noticeable impairment.

The slow down with 1.10.0-alpha (without the 1 second sleep) doesn’t persist for the entire time posts are preloading, only at certain points. Some posts may just be too heavy to fly through at warp speed.

• This reply was modified 3 years ago by mjvito.

You were correct. Max children was set at 3, the default for a DreamHost managed VPS. They increased this to 8.

With the new cap, during preloading of tags, there are occasionally 4 PHP processes running at a time, but generally only 2 or 3, and TTFB is normal as before. However, during preloading of posts and pages, there are never more than 3 processes, and the TTFB spike still occurs.

What other bottlenecks could cause this?

I know the release version will be different from this alpha, but sending additional notes in case they are useful.

With 1.10.0-alpha, on some preload cycles it can complete everything quickly, caching around 1350 items in approximately 75 minutes. Other times, it appears to process very slowly, especially tags, or gets stuck in a loop where it reprocesses the first 50 tags many times. Maybe the looping is related to the email issue. Sometimes, it can exit the loop by itself and finish. Other times, I abandon and restart after a few hours.

When preload does get to caching posts and pages, sometimes but not always, there is a significant TTFB increase for the site, about 5x the average. This lasts through the period when posts and pages are being cached, around 25-30 minutes. It does not appear to be a resource issue. There is ample memory headroom and CPU is near idle. Perhaps the new preload is too aggressive and is impacting the normal page serving to visitors.

Hi @donncha, no worries. The situation over here is the same.

One unexpected good thing has happened in the meantime. My host increased the FastCGI read data timeout to 300 seconds. Now preload can complete its fetch cycles before the timeout. There are still things to fix, but the pressure (for me, anyway) has been alleviated.

Now all posts and pages are cached during preload.

The child categories are still deleted. This still needs a fix.

A new problem I observed, which may have been present earlier but I am only noticing now, is that preload does not cache all of my tags. It caches the first 440-450 out of 628 tags, then stops. I have not done debug log diving, but there are no read data timeouts in my server logs. This is not a high priority for me, though.

When manually triggering preloads at 24 hour intervals, there are usually only a few file deletions at the end of a preload cycle, and sometimes there are none.

I ran the “take a day off” test again, this time with 48 hours between preload runs. About 10% of cache files were deleted in this case, much less than before. The large deletions observed earlier were potentially an outcome of the cache state resulting from frequent read data timeouts.

Hi @thingalon,

Thanks for checking into this. I look forward to solving it together with you.

For the child category issue, I have a log already, so I’ll address that now. For observing the mass cache file deletion at the end of preload, I’ll need to carve out time to set that up. Hopefully I can get to that in the next few days. I also inadvertently made a new observation that might shed more light on that, and note that at the end of this post.

Confirming that Cache Timeout is set to 0 (garbage collection disabled). In case it is relevant, noting that Scheduler is set to 86400 seconds. I have Notification Emails checked, but don’t receive any.

Child Categories

My site has a parent category Urbanism with five children. All child categories contain posts. When I run debug, I see the children are refreshed first. Let me know if you want to see the debug messages for that part. After they are refreshed, the following sequence occurs:

• The last child is fetched at the end of its refresh sequence. wp_cron_preload_cache: fetched <path>/category/urbanism/train-culture/

• The parent is deleted at the beginning of its refresh sequence. wp_cron_preload_cache: delete <path>category/urbanism/
• The compressed and normal versions of the cached page for a child category are deleted. prune_super_cache: deleted <path>/category/urbanism/train-culture/index-https.html.gz prune_super_cache: deleted <path>/category/urbanism/train-culture/index-https.html
• The folder for the child category is deleted. gc: deleted <path>/category/urbanism/train-culture, older than 0 seconds
• This repeats for each child.
• The parent category completes its refresh sequence at the end.

I cannot view cache contents through the UI because there are too many files, but I can see what is happening over SSH. The parent category folders and cached pages are present, but the child category folders and cached pages are not. When I browse to these child category pages on the site, they load correctly and subsequently appear in the cache folder.

Mass Cache File Deletion

When I observed that the deletions were happening only after scheduled preloads, but not after manual preloads, I didn’t consider the condition that I was starting these manual preloads very frequently. Usually I would start one in the morning with just posts and pages, then start the second with tags and categories as soon as the first finished in the afternoon. There was never more than 12 hours between the end of the second preload of the day and the beginning of the preload the next morning.

The other day, I took a one day break. After the break, I started a morning preload with just posts and pages, and at the end of that cycle, the significant deletion also occurred, even though it had been started manually. There was approximately 36 hours between the end of the previous cycle and beginning of the new one. I can try reproducing this with tags and categories checked, to see if the same thing happens. Potentially, the problem is not scheduled preloads, but something related to the age of the cached files.