Forum Replies Created

Viewing 15 replies - 1 through 15 (of 69 total)
  • Thread Starter LucasRolff

    (@lucasrolff)

    Hello,

    Considering it’s a constant, I should be able to modify this in wp-config.php? Or is there any particular reason you’re suggesting we go change hundreds of website’s files only for those changes to be overwritten upon next upgrade?

    Thread Starter LucasRolff

    (@lucasrolff)

    With regards to your initial question, this is simply a limitation of php, it’s not limited to UpdraftPlus.

    The way you add batches of files to a zip, is not a limitation of PHP, but rather how UpdraftPlus decides to implement adding files into batches in the zip.

    As I’ve already pointed out, this doesn’t seem to be an issue with a bunch of other WordPress backup plugins, if it was a PHP limitation, all backup plugins would face the same issue, but they do not.

    While “this is a limitation of PHP”-excuse may work for your average customer who doesn’t know any better, it ain’t gonna work in case. Once again, you should really look into reducing your write amplifications in the plugin. While it’s currently likely done for preventing “limits” from being hit, you’re causing backups to become slower and slower, because files are being rewritten for every single “batch” of 1000 files (for example) that’s done. The result is backups that takes significantly longer than what they should because you’re likely to hit disk IO throttling as the bottleneck (especially for certain providers out there that limit people to 5-10MB/s).

    One way could for example be to try to batch more files at a time, this reduces the amplification quite significantly, the number of files could be determined based on e.g. available memory_limit, and max_execution_time settings, e.g. try to “push” it to find a nice balance. The constant of 1000 files for example makes no sense in many cases.

    Batching things into smaller files like 400MB zip files are fine, I’m not against it, and it does save in terms of required storage. however, that does not justify the write amplification caused by UpdraftPlus. And creating smaller zip files than 1 big file, does not equal to high write amplification. What equals high write amplification is the way you batch add files to those zip files.

    So no, it’s not a limitation of PHP, and you know that.

    Thread Starter LucasRolff

    (@lucasrolff)

    Hello,

    Running backups during low-traffic periods (such as overnight) does not fix the write amplification. And if all customers run backups during low-traffic periods, you’ll end up having servers writing several gigabytes per second (combined). We’re seeing several terabytes of data being written to disk per day due to UpdraftPlus’s write amplification issue.

    While we use expensive enterprise drives, disk writes are (still) finite, that’s simply the reality of NAND chips. Many other plugins seemingly can do backups just fine without causing write amplification issues, but Updraft is different. As a hosting provider we can’t simply go blacklist a given plugin, if we could, Updraft would be on the list in all honesty due to the fact this is even an issue.

    I ask that you bring this to your developers so they fix the issue instead of simply acknowledging that your plugin indeed has a write amplification issue. If the “fix” is to reduce the default chunk size to something smaller, then go for it. But a plugin should not cause 8x write amplification out of the box.

    Hi tores54,

    Testing the images in the Google Drive zip file, the majority of the images does indeed not end up with a generated webp image:
    – 7NDNMLIqak0.jpg
    – Fk5xo6hxgIY.jpg
    – FRSbHtCKINQ.jpg
    – o-PpIxUWSMA.jpg
    – SkRpLtCsJRk.jpg
    – STci4WCoyfA.jpg
    – ZkrxrclxCQ0.jpg

    krWpoj_yMH0.jpg does get a webp image.

    The way the decision is made whether there’s going to be a webp image, is based on a filesize comparison.

    Whenever you send an image for optimization 3 things happens:

    – We check the filesize of the original image fetched from your site, and record this (original_filesize).
    – We optimize the image using a set of algorithms to generate the lowest possible size, with a slight quality loss (unless you have lossless optimization enabled). We then record the size (optimized_filesize).
    – We generate a webp image based on the original image. The webp image will still have a slight quality loss. We then record this size as well (optimized_webp_filesize).

    We then do a check whether the generated webp image’s filesize is smaller than the optimized jpeg/png filesize. If it’s smaller, you’ll get the webp image returned. If it’s bigger you’ll not get the webp image returned. Since you’re then increasing the size of the page, which is what you’re trying to prevent in the first place by using the image optimize function.

    Since WordPress has different sizes (thumbnail + scaled/resized versions) for each image, the outcome may differ for each image as well. In some cases there can be optimizations, in other cases, there won’t be.

    If we take 7NDNMLIqak0.jpg as an example:

    – The size when uploaded to WP is 1299518 bytes.
    – The optimized jpeg size is 447307 bytes.
    – The webp imag size is 457024 bytes.

    The webp image is thus 9717 bytes larger than the jpeg.

    Since the webp image is larger, it won’t be returned.

    It’s also worth noting that webp generally turns out larger for large images, it’s just how the webp compression works.

    The IP being used September 27/28/29 ends on .163, at this time the `
    notify_img` endpoint continued to return “404 not found”.

    The IP ending on .163 is also the one configured in my.quic.cloud.

    The IP change was registered by the image optimization service at 2020-09-26 19:44:19 UTC.

    Thus the 404 is not related to the incorrect IP, assuming the .163 IP that got configured the 26th of September is indeed correct.

    Best Regards,
    Lucas Rolff

    Forum: Plugins
    In reply to: [LiteSpeed Cache] Question

    The swcfpc query string is a part of the “WP Cloudflare Super Page Cache” plugin, you might want to check their support page whether it’s possible to remove it.

    You could use https://ww.wp.xz.cn/plugins/disable-json-api/ instead which allows whitelisting specific endpoints.

    Your website does not accept requests to /wp-json/litespeed/v1/notify_img, it simply gives a timeout.

    You have to ensure the site (or server) will accept traffic from IPs listed on https://quic.cloud/ips

    Currently if doing a request it results in the following:

    
    * Hostname <DOMAIN> was found in DNS cache
    *   Trying 82.79.xxx.xxx...
    * TCP_NODELAY set
    * connect to 82.79.xxx.xxx port 443 failed: Connection timed out
    * Failed to connect to <DOMAIN> port 443: Connection timed out
    * Closing connection 0
    curl: (7) Failed to connect to <DOMAIN> port 443: Connection timed out
    

    Neither of the sites linked in GAJTARPE and SMKERJHK reports have done any requests the past 13 days to the image optimization service.

    You’d have to do more requests for debugging to take place.

    > Are new images automatically optimized on upload?

    Yes, as long as you go to LiteSpeed Cache -> Image Optimization -> [2] Image Optimization Settings, and make sure “Auto Request Cron” is set to “On” (and ideally “Auto Pull Cron” is set to “On” as well).

    Then the plugin will use the WP cron to request optimization for any image that has not yet been optimized (new or existing).

    You can do 200 images per 5 minutes, so 5000 images could be done as fast as roughly 2 hours. However, it depends on your WordPress site and hosting as well, how fast it can actually push and pull the images to the optimization system.

    The 1000 images will go into a “fast” queue, the remaining images will go into a standard queue, which may be slower compared to the fast queue.

    Standard queue is free, and will always be free – and you’re indeed limited to maximum 200 images per 5 minutes regardless of being in the fast or the standard queue. Except if you buy the “jumbo group” option, then you’re limited to 1000 images per 5 minutes.

    Hi @duylbk

    I can see it was due to your origin site returning timeout errors when trying to establish the connection. However, I then tried to push the images to optimization again and it seems to have worked this time.

    I’ve added some additional information about the connection happening, which may reveal more information if the problem happens again in the future.

    So if you experience it again, then please let me know! Then I can check the logs.

    The notify endpoint (/wp-json/litespeed/v1/notify_img) is giving a 404 not found page.

    This usually happens if the backend IP changes or is configured incorrectly on my.quic.cloud.

    Another reason can be that it’s being blocked/intercepted by some kind of application firewall either on the hosting providers systems or the site itself.

    If you do not have a my.quic.cloud user, you can try doing a domain key refresh in WordPress -> LiteSpeed Cache -> General, and click on “Refresh Domain Key”.

    Can you try once again? I had to restart a service that didn’t auto-restart during the code update! Sorry!

Viewing 15 replies - 1 through 15 (of 69 total)