Mark (a11n)

Hi @malcolm12boxes

I’m one of the developers on the Boost team. I’ve taken a look at your site and I think I’ve figured out what is going on. It looks like your site has an HTTP header set which blocks your site from loading inside an iframe:

Content-Security-Policy: frame-ancestors 'none';

Our manual Critical CSS generator runs from your wp-admin page by creating a hidden iframe, and using it to load various pages on your site at various sizes to determine which CSS rules are needed to render the “above-the-fold” portion of your site quickly.

I’m afraid it can’t work with frame-ancestors set to none, as it prevents your site from loading inside any iframe. If you are willing to change your site settings, you could change this rule to:

Content-Security-Policy: frame-ancestors 'self'

This would still disallow your site from being loaded inside an iframe from other sites but would allow it to load in iframes created on the same domain. That would let Boost load your site in an iframe to generate Critical CSS for you.

You can read more about the frame-ancestors settings here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors

Alternately, our automated Critical CSS generator (available when you upgrade Boost to a paid plan) does not use iframes to generate Critical CSS. Instead it loads your site inside a virtual browser on our servers to generate Critical CSS for you. That should work regardless of your frame-ancestors policy.

It sounds like you’ve taken the right steps, but if those pages are still being cached then something isn’t quite right.

First, may I ask how you are checking if the pages are still cached? The easiest way is to load the pages in an incognito/private browser tab, right click on the page, and select “View Source”. At the bottom of the source code, you will find an HTML comment like the following if the page is cached:

<!-- Cached page generated by WP-Super-Cache on ... -->

Next, let’s double-check the paths. When you load your contact page, does it typically have a / at the end of the URL, or does that get removed?

If the / is not present at the end of the URL, then you should remove it from your Reject URLs settings. e.g.: just /contact instead of /contact/.

Finally, have you added each of the paths you want excluded on a line of their own in the rejected URL list?

Hopefully one of these checks will shake out what’s going wrong here.

Hi Dimal!

I remember you were looking at your DEBUG logs a few months ago, and reported that you saw “ABSPATH” in your paths. Did you figure out where that came from, and if fixing your ABSPATH in wp-config would fix that issue?

Unfortunately, without a better understanding of how/where your preload is failing, we aren’t sure what problem to fix. We test Super Cache on sites with many posts, and aren’t seeing it stop after 20.

We’ve been iterating on the preload stuff a bit; making it work more reliably on hosts with tight limits. We’ll be releasing it in 1.10 soon once we work out a bug to do with gzip compression. But I see you’re already running 1.10-alpha now.

Where did you get it from? We don’t typically release an alpha build; that’s the version we use for the absolute latest in our repository. If you’re keeping up-to-date with our absolute latest from our repository, then installing 1.10 once it’s released won’t fix your preloading.

If you’d like us to check out what is going on with your preload, would you mind sending us your DEBUG log?

You’re welcome to post it here, but if you would like to send it to us privately, you can use the following contact form:

https://jetpack.com/contact-support/?rel=support&hpi=1

If you do, please include a link to this forum thread for context.

Great discussion, all – and thanks in particular to @dimalifragis for suggesting simple mode as a way to let WordFence run first.

I just wanted to chime in with a few points to consider:

When WPSC is running in Expert mode, it configures your webserver to serve static HTML files directly from the cache without executing any WordPress PHP code. (That’s why Wordfence doesn’t get a chance to run first).

I don’t think that’s necessarily a conflict. While WordFence may block some traffic to prevent attacks, serving static HTML at this stage means your site isn’t vulnerable to attack, as no dynamic content or PHP code is being executed.

If WordFence is blocking traffic from specific IP addresses to mitigate DDoS attacks, serving static HTML content for a page is likely even faster and lighter on server resources than running enough PHP code to filter the request.

That doesn’t solve the traffic reports issue, so if you rely on the traffic report from Wordfence, I’d recommend leaving WP Super Cache in simple mode as @dimalifragis suggests.

Hi @mjvito,

Thanks for investigating and reporting the strange behaviour you’ve experienced. I’ve just tried to recreate both issues in my test sites, and have not been able to so far.

I’d be happy to check over any debug logs you have. Please be sure to redact your site url from the debug logs if you would like to keep it private.

Preload mode creates the child categories first. Then when it creates the parent category, it deletes the children

On my test site, I created categories Brie and Cheddar, both of which are children of Cheese. I ensured that Brie contained posts, but Cheddar did not.

I ran the preload process, and was able to observe:

• A log message stating wp_cron_preload_cache: delete <path>/category/cheese/brie appeared in my logs, which was then followed by further log messages indicating files in the brie directory were fresh and so were not deleted. This occurs when Super Cache is considering deleting directories, and may not indicate that the indicated path is actually getting deleted.
• The same logs appeared for Cheddar.
• When I browsed to Contents > Show file list in the Super Cache settings page, I could see all three cheese related categories under the list of “Fresh WP-Cached files”.

You mention that you saw this deletion in your Debug logs.

If you view the contents of your cache after pre-loading, do all of your category pages appear there? Finally, if you browse to any category page which is missing from your cache does it load correctly, and does it appear in your cache contents after you load it manually?

The significant bug is a behavior that occurs at the end of a preload refresh cycle, but ONLY when preload is on a schedule. When preload is run manually (“Refresh preloaded cache files” set to 0 minutes), everything appears to function as intended. The total cached pages indicated in the UI is close to what I expect. It stays approximately at that level through subsequent manually started preload cycles. However, when preload refresh is set to run on a schedule, it deletes a large number of cached pages at the moment the cycle finishes. This can be up to 50% of the pages. This happens both with “Preload tags, categories and other taxonomies.” checked and unchecked. I observed this only through the UI.

That does sound like a severe bug.

I have attempted to recreate this on my test site, but so far haven’t been able to. I would love to see Debug logs around this occurring.

When you click the “Preload now” button, it internally schedules a preload for 10 seconds in the future – using the same code as the scheduled preloads.

At the moment, the best working theory I have is that another cron-job is running after your scheduled job, which is deleting the cache files.

In your original post, you mentioned that you set Garbage Collection to 0, but that it did not fix your original issue. Can I please verify that it’s still set to 0?

Hi @occupygh,

WP Super Cache does not add any custom JavaScript files to your site’s front-end. I can see two possible causes behind the issue you experienced:

1. Another plugin added a script to your amp pages, which Super Cache included in its cache. When you deactivated Super Cache, it fixed the problem because your site was no longer serving a cached “incorrect” AMP page, or
2. Your site was mixing up AMP pages with non-AMP pages, and serving the same content to both versions of the page.

Testing possibility 1 is very easy: Simply re-enable WP Super Cache, and click the “Delete Cache” button on the “Easy” tab in WP Super Cache to clear out your cache. If the problem persists after doing this, then we are probably dealing with the second possibility.

In order to fix the second possibility, can you please try checking the following settings:

1. Make sure that “Tracking Parameters” under “Advanced” does not include amp,
2. Add amp to the “Rejected URL Strings” setting under “Advanced” in a new line of its own.

Finally, can you please visit the amp version of your page in your browser when WP Super Cache is enabled, to check whether it looks the same as your main site, or if it looks like a simplified version.