Missing minified files

Resolved

(@kubiq)

Hello,

as you already marked recent topic as resolved, I’m opening new one – my own topic as I was only commenting on someone else topic (https://ww.wp.xz.cn/support/topic/asset-cleanup-breaks-my-website-every-few-days/)

As I wrote you in last comment on that topic, I’ve created small plugin to monitor and fix missing minified files from Asset CleanUp (https://files.kubiq.sk/asset-clean-up-fix.zip) and even after last update and after setting the timing to 7 days, there are still many incidents.

Here is log, as you can see, file names are always the same – these files are static – from theme or plugins so I don’t understand why they are deleted almost every day:

[17-Jan-2021 09:08:01 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/weiss_cookies_style-ve0886a4718043283a9693fc5a61ad999f136ef3b.css
[19-Jan-2021 10:57:20 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[19-Jan-2021 10:57:30 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/smush-lazy-load-vda585b65c7726a84036fa1f91ac77565d67c14e1.js
[19-Jan-2021 10:57:35 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[19-Jan-2021 10:57:50 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/weiss_cookies_style-ve0886a4718043283a9693fc5a61ad999f136ef3b.css
[19-Jan-2021 11:58:20 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[19-Jan-2021 11:58:20 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[21-Jan-2021 05:12:16 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/smush-lazy-load-vda585b65c7726a84036fa1f91ac77565d67c14e1.js
[21-Jan-2021 06:13:26 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[21-Jan-2021 11:39:29 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[21-Jan-2021 12:56:26 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/weiss_cookies_style-ve0886a4718043283a9693fc5a61ad999f136ef3b.css
[21-Jan-2021 12:56:31 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[21-Jan-2021 13:57:09 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[24-Jan-2021 02:57:22 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/rank-math-review-snippet-v85076d38dc20b0983e1debc138002e43db644071.css
[24-Jan-2021 02:57:29 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/smush-lazy-load-vda585b65c7726a84036fa1f91ac77565d67c14e1.js
[24-Jan-2021 02:57:34 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[24-Jan-2021 02:57:45 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/weiss_cookies_style-ve0886a4718043283a9693fc5a61ad999f136ef3b.css
[24-Jan-2021 02:57:51 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[25-Jan-2021 04:13:24 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/smush-lazy-load-vda585b65c7726a84036fa1f91ac77565d67c14e1.js
[25-Jan-2021 04:13:27 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[25-Jan-2021 04:13:37 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/weiss_cookies_style-ve0886a4718043283a9693fc5a61ad999f136ef3b.css
[25-Jan-2021 04:13:42 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[25-Jan-2021 05:14:27 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[26-Jan-2021 03:39:43 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[26-Jan-2021 03:39:47 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/smush-lazy-load-vda585b65c7726a84036fa1f91ac77565d67c14e1.js
[26-Jan-2021 03:39:54 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[26-Jan-2021 03:39:58 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/weiss_cookies_style-ve0886a4718043283a9693fc5a61ad999f136ef3b.css
[26-Jan-2021 17:00:21 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[26-Jan-2021 18:01:13 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[27-Jan-2021 02:40:14 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[27-Jan-2021 22:00:19 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js
[27-Jan-2021 22:00:23 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/smush-lazy-load-vda585b65c7726a84036fa1f91ac77565d67c14e1.js
[27-Jan-2021 22:00:31 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[28-Jan-2021 07:48:10 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[28-Jan-2021 07:48:29 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/weiss_cookies_script-v2441fb27d561d7a0e5bf3bd93e1807d42ac97c2d.js
[28-Jan-2021 07:48:31 UTC] Asset Fix: /wp-content/cache/asset-cleanup/js/item/main_script-v985aa68a26cfdcb73e2feb80907cc0728f500c78.js

Here are my settings: https://files.kubiq.sk/asset-cleanup-lite-exported-everything-from-www.mobatime.com-28-Jan-2021-07.58.json

Basically, once you are using Asset CleanUp with cache plugin, then your web is broken almost every day. I use Cache Enabler mostly, but I use it on hundreds of websites and some of them are using SuperCache or W3Cache and there were same problems.

Let me know, how can I help you with solving this issue.

Viewing 9 replies - 1 through 9 (of 9 total)

Plugin Author Gabe Livan
(@gabelivan)

5 years, 4 months ago

@kubiq Asset CleanUp works well with Cache Enabler and on my tests, there were no such problems. If Cache Enabler clears its cache on a daily basis as it should (depending on the settings that are set), there shouldn’t be any problems in terms of loading assets that are missing. I also use it on http://www.gabelivan.com/ and so far, there weren’t any issues like this in the past months. Very few incidents like the ones you mentioned were reported and it was sometimes due to the fact that whenever Asset CleanUp clears its caching and has to delete something, there were old references in Cache Enabler to Asset CleanUp Pro’s files that were no longer relevant or even missing. This has been fixed now and whenever Asset CleanUp caching is cleared, Cache Enabler’s one is cleared as well (check the “Changelog” for the latest version of the plugin).

Now, as far as the plugin you developer, I’m trying to understand how it actually solves the problem. As far as I noticed, it doesn’t recreate the file. Perhaps I missed something there or you didn’t share the latest version of the fixed plugin you’ve developed?

I even made tests on my website using 0 as a value from “Settings” -> “Plugin Usage Preferences” -> “Clear previously cached CSS/JS files older than (x) days” and as I expected it kept the current valid cached files and deleted all the unused ones that are no longer relevant. More precisely if the value is ‘0’, it keeps all the assets that were created in the past 8 hours. So, if your Cache Enabler pages are not cleared every 8 hours, but every 12 hours (for instance), then yes, you could get references to non-existent files. That’s only if you have ‘0’ as a value set. Otherwise, they will be deleted after 8 hours + the number of days set.

Note that the default value is 8 hours which was set as an average for the time it takes until a cached page is refreshed. WP Rocket recommended the same amount of time. Usually, it’s best to keep the number of days to at least 4 until the older files will be cleared, in case very old cached pages and even Google cache previews are made on a page and we want the visitors to load a functional page. ‘0’ is usually set for those that know their static HTML pages are refreshed after a shorter time than 8 hours and want to have fewer files kept in the caching directory (e.g. because there are too many of them).

A solution would be to enabler a logger that will write to a file whenever the files are deleted by Asset CleanUp. Perhaps those CSS/JS files are cleared by an external source. This way, one will know for sure if the files were cleated by Asset CleanUp too soon (which would be a bug indeed).
Thread Starter kubiq
(@kubiq)

5 years, 4 months ago
Why would you clear cache every day? It makes no sense…
You want to have cache valid until page update, or something changed.
Yes I’m testing newest version.

About my plugin fix – if there is missing file in asset cleanup folder, then it has 404 header – my plugin catch this and will call URL with ‘?wpacu_debug’ at the end – it will regenerate all needed files.
You can test it easily – if you have cache enabler active and you go to FTP and manually delete some CSS file from /cache/asset-cleanup/ folder, then on refresh (logged-out user) you will see broken layout, because HTML version from Cache enabler will load link of non-existing CSS from /cache/asset-cleanup/ folder…
With my plugin activated it will regenerate that CSS, so next time you refresh page everything will work.
You will see notice in debug.log about this incident – that’s how I know how often and when and where it happen ( of course don’t forget to enable WP_DEBUG and WP_DEBUG_LOG, but you can disable WP_DEBUG_DISPLAY )

In Cache enabler I have disabled ‘Cached pages expire X hours after being created.’, but I have enabled ‘Clear the site cache if a plugin has been activated, updated, or deactivated.’ and ‘Clear the site cache if any post type has been published, updated, or trashed (instead of only the page and/or associated cache).’ because I don’t want to regenerate cache each day, as most websites have same content for weeks or months.
- This reply was modified 5 years, 4 months ago by kubiq.
Thread Starter kubiq
(@kubiq)

5 years, 4 months ago

I just wrote new version – this will load url with ‘?wpacu_debug’ through CURL and then it will redirect that file header to the file itself, so it will load correct file and web is never broken again…
Yes load-time for such a file is about 2-3 seconds on that specific reload, so it’s just hotfix solution, but still better than broken web.
Here: https://files.kubiq.sk/asset-clean-up-fix-v2.zip

Thread Starter kubiq
(@kubiq)

5 years, 4 months ago

New version for fewer request and some additional conditional checks
https://files.kubiq.sk/asset-clean-up-fix-v3.zip

We really need to make sure that all needed cache files exists,
otherwise you need to load original files,
or do some request to create missing files firstly before page load like I do in my fix plugin

Plugin Author Gabe Livan
(@gabelivan)

5 years, 4 months ago

@kubiq thanks for the plugin fix! I will consider implementing it somehow in the current plugin. I double-checked the code behind the cache clearing from the plugin and there are instances when indeed, some CSS/JS files get deleted while Cache Enabler can still reference them.

For instance, Asset CleanUp could delete the CSS/JS and re-create new ones with different versions, thus different file names as the “ver” value is encoded in the file name as it should be. This could happen after (just an example) 4 days. Depending on the settings from Cache Enabler, the static version of the page (HTML one) could stay the same way for over a week (even much more), depending on the settings set in “Cache Enabler Settings”.

So, it would be a good idea to write a guide about Cache Enabler and Asset CleanUp, but more importantly, I’ve started rewriting the code for the cache clearing where older files aren’t as easily deleted. When I’m saying older files, I’m referring to the “ver” value which could change whenever a plugin is updated or when you updated, for instance, your theme’s CSS and you append a new “ver” value.

If the “ver” never changes, then the files would be still be checked if there’s a chance there and re-create them (having the same file names) in case the developer updated the CSS/JS but forgot to change the “ver”. It happens a lot really and some guys even emailed me about it saying that Asset CleanUp still loads the old version of the file from the cache. That was because “ver” was the same. Now, the plugin also checks for the filemtime() value and depending on the value, it re-creates the cache accordingly. This is something that needs to be improved. Anyway, a golden rule would be that whenever you change files directly in your theme/plugins, just clear the caching manually instead of waiting for the plugin to do it, as there could be delays.

You’re using cURL to access the page via /?wpacu_debug which is good enough, apart from staging websites that are password protected and will require in the CURL, the username and password in order to access the page, which is not ideal. I’ll think of something that would emulate the HTTP request. It’s important that the same CSS/JS is recreated. After the changes I mentioned would be implemented and your fix will come into play (hopefully, in very rare cases), then we shouldn’t have issues like broken layouts anymore. In the worst case, the TTFB will be delayed when the page loads if the assets are re-created, in case they are render-blocking (hopefully not).

PS: I want to reward you with a Pro license for finding out these issues and taking the time to write that plugin fix. You can write to me in private via https://www.gabelivan.com/contact/ if you’re interested.

Thread Starter kubiq
(@kubiq)

5 years, 4 months ago

I didn’t investigate your source code, so I’m not aware of all aspects how it works now, but I think that old files should be deleted ONLY when new ones are created.

Also use-case of caching on most classic websites is that you don’t want to clear cache each X days, because if there are very rare changes, then it’s waste of resources and possible broken layouts like now.

Yes filemtime() is exactly what should be used 😉 maybe setup some CRON to scan filemtime for cached files to be sure that everything is up to date, or any other way how it is done right now.

Yes I know that my current solution isn’t bulletproof, actually it’s just “hotfix”… But I’m updating it as you can see… according what I see in logs

Maybe you have some public function in your plugin that can be called instead of CURL… if you have database record with filenames then you can easily pair cached non-existing file by its URL with non-cached version?
I didn’t have time to go through your plugins code, so that was just first idea I had…

But sure, I like your plugin, so I will be happy to help you with testing or some improvements 😉
I wrote you with licence 00000000000000000000000000000000 😀
Thanks
Thread Starter kubiq
(@kubiq)

5 years, 4 months ago
Hm, today I’ve noticed something really weird
```
[01-Feb-2021 19:16:03 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/wpbs-style-vfe9fff58e0abaf4b60e9b327d0dd6a49910c0043.css
[01-Feb-2021 21:31:19 UTC] Asset Fix: /wp-content/cache/asset-cleanup/css/item/wpbs-style-vfe9fff58e0abaf4b60e9b327d0dd6a49910c0043.css
```
original file is in theme /wp-content/themes/mobatime/style.css and last change of this file was on 28.01.2021, but as you can see there were 2 incidents with missing file yesterday… I’m really curious how this can happen o.O
I’ll probably check whole code of your plugin during the weekend and maybe i’ll set there some error_log calls to log where, when and why it is deleted so often
Plugin Author Gabe Livan
(@gabelivan)

5 years, 4 months ago

@kubiq the caching mechanism has been updated for the individual cached files as follows:

– The most recently created versions of a file is stored in a special whitelist directory
– That file is never deleted when the cache clears
– When a new “ver” of that file is cached, the now old one is removed from the whitelist directory and it will get deleted in one day + the number of days set in “Settings” -> “Plugin Usage Preferences” -> “Clear previously cached CSS/JS files older than (x) days” – so, if it’s the default 4 days when it will get deleted whenever the cache is cleared IF it’s older than 5 days. Obviously, whenever the caching is cleared in Asset CleanUp, it’s also cleared in Cache Enabler.

The certainty that the most recent files (which can be referenced for weeks, even months) will never be deleted will ensure errors like 404 not found are less likely to occur. The other thing that needs some improvement is the combined CSS/JS files as the most recent ones should also be stored in a whitelist. However, this is a bit more difficult with these ones as there could be a lot of such files, depending on how many pages are on the website.

Thread Starter kubiq
(@kubiq)

5 years, 3 months ago

I’ll mark this topic as resolved, because we continue testing and debugging in mail communication 😉

Viewing 9 replies - 1 through 9 (of 9 total)

The topic ‘Missing minified files’ is closed to new replies.