Nope, it will take the same amount of time either way though, and usually faster when scanning is done first.
If you havae time, I do have a few questions though, as I’ve been trying to find ways to speedup the scanner for some sites. On most sites, it takes no more than a minute or two, but certain configurations slow it down for some reason:
1. While the scanner is running, watch to see how many images it does in each batch – each time the counter decreases, which should be every 15 seconds.
2. Who is your webhost?
3.Try adding this line to your functions.php to speed up the scanner and let me know how many images it processes in each batch:
set_transient( 'ewww_image_optimizer_low_memory_mode', 1, 600 );
To answer your questions:
1. The gap is around 45 seconds for each batch, and it processes 4-5 images per batch. Of course, these numbers are somewhat approximate.
2. It’s a powerful dedicated server, dual CPU, lots of memory, etc. Regular CentOS with cPanel (no CloudLinux), so I don’t think we’re hitting any resource limits.
3. The scan is still running now after some sort of ‘access token expired’ error. I refreshed the page and it continued. Do I add the code while the scan is running, or should I somehow terminate it, add the code and start again?
Thanks.
On #3, you can add the code while it is scanning and watch to see if it changes.
With #2, is the database on a separate server, or is it on the same server?
Ok, I added the code and scanning speed increased considerably. Looks like it’s now going in 15-20 seconds gaps and nearly 1k images per batch.
Also, the following message appeared: Increasing PHP’s memory_limit setting will allow for faster scanning with fewer database queries. Please allow up to 10 minutes for changes to memory limit to be detected.
In this case PHP memory limit is set to 2GB.
On a separate note, is there a similar way to increase optimization speed? The server has a LOT of power and barely any load.
The message about the memory_limit is because that line of code I gave you tricks the plugin into changing it’s scanning method to what is required in “low memory” environments. Basically, here’s the two scanning methods:
1. Load the entire ewwwio_images table into memory, and check that for each image. On larger sites, the table can be 80MB or more, so it requires more memory, which takes us to #2.
2. Do direct db queries for each image found.
Additionally, when you’re seeing the counter decrease by 1,000 in each batch, it’s actually scanning at least 5,000 images: 1 original image for each upload, and a minimum of 4 resizes for each upload.
Now, the weird part here is that #1 should always be faster, as it has the entire table in memory. With a dedicated server like you have, it should be able to churn through 5,000+ attachments in 15 seconds. But it’s only making it as far as each time/memory check (every 4-5 images), which is the slowest I’ve seen out of any of the “problem” sites. I would be interested in seeing a debug log when “low memory mode” is NOT enabled. If you are willing to do a bit more debugging on this, shoot me a note here: https://ewww.io/contact-us/
As to optimization speed, you can try enabling parallel optimization on the advanced options, and see if that helps.
I had forgotten that parallel mode no longer affects bulk operations, it only affects processing time when a new image has been added. The bulk optimize itself is single-threaded, so it’s already running as fast as it can in that mode.
Yeah, I had parallel mode enabled from the very beginning.
Will check other things you mentioned shortly.
