Bulk upload stuck

Resolved cbrherms
(@cbrherms)

1 year, 2 months ago

Testing out AdvMO on our staging site. Cloudflare R2 bucket is set up and configured, and connection check seems good.

I’ve been able to successfully upload a file using the media admin panel, and this appears in our bucket and is accessible via our custom domain.

Settings wise, i’ve set it to smart local cleanup, and set a custom path (wp-content/uploads).

Issue is, after attempting a batch offload, this stayed stuck at 0% for a good while (around an hour), and none of the 1799 found images were offloaded. I then attempted to cancel, but that also now seems to be stuck. I removed the advmo_bulk_offload_* related database rows as mentioned in another support thread but the media overview page still appears to be stuck in the same cancelling state even after a page refresh.

Any thoughts?

Viewing 5 replies - 1 through 5 (of 5 total)

Plugin Author Masoud Golchin
(@masoudin)

1 year, 2 months ago

Hi @cbrherms

Pressing cancel will stop the bulk offloader after completing the current item and clear the queue. It appears this file is causing the process to stall. Do you have error.log enabled?
That log would help us diagnose the issue.

Alternatively, we’re happy to examine your setup at no cost to debug the problem.
Thread Starter cbrherms
(@cbrherms)

1 year, 2 months ago
Hi @masoudin – Thanks for the quick response.

Due to the nature of our setup, we wouldn’t be able to enable external user access, but i’ve tried again on a fresh clone of our prod, with the debug.log enabled in config.php to capture any potential problems.

The logs do seem quite sparse though. Below you can see from install to running some uploads.
```
[27-Mar-2025 16:01:46 UTC] auditor:event=upgrader_process_complete_plugin {"type":"plugin","action":"install","plugin":"advanced-media-offloader","blog_id":1,"event":"upgrader_process_complete_plugin","current_user_id":8,"remote_addr":"*sanitised*"}
[27-Mar-2025 16:01:54 UTC] auditor:event=activated_plugin {"plugin":"advanced-media-offloader/advanced-media-offloader.php","network_activation":false,"blog_id":1,"event":"activated_plugin","current_user_id":8,"remote_addr":"*sanitised*"}
[27-Mar-2025 16:01:54 UTC] ADVMO - Error: Unsupported cloud provider: 
[27-Mar-2025 16:01:55 UTC] ADVMO - Error: Unsupported cloud provider: 
[27-Mar-2025 16:02:03 UTC] ADVMO - Error: Unsupported cloud provider: 
[27-Mar-2025 16:02:04 UTC] ADVMO - Error: Unsupported cloud provider: 
[27-Mar-2025 16:02:25 UTC] ADVMO - Error: Unsupported cloud provider: 
[27-Mar-2025 16:02:25 UTC] ADVMO - Error: Unsupported cloud provider: 
[27-Mar-2025 16:03:41 UTC] PHP Warning:  Undefined array key "filesize" in /nas/content/live/*sanitised*/wp-content/plugins/advanced-media-offloader/includes/Traits/OffloaderTrait.php on line 106
[27-Mar-2025 16:23:14 UTC] PHP Warning:  Undefined array key "filesize" in /nas/content/live/*sanitised*/wp-content/plugins/advanced-media-offloader/includes/Traits/OffloaderTrait.php on line 106
```
The unsupported cloud provider only appeared before I selected Cloudflare R2 in the options so pre-configuration of the plugin.

The first PHP warning about an undefined array was logged when the first file was manually added via the media library admin panel.

After that was successful, I enabled the bulk media offload via media overview of the offloader plugin. This didn’t generate any debug logs, nor errors in our normal error capturing logs.

The second undefined array error was captured when I uploaded a second file via the media library admin panel.

Additionally, now almost an hour after starting, checking the database options entries for the offload data shows last_update as roughly 16:04 UTC, so no changes since the process was started.
```
advmo_bulk_offload_data:
a:6:{s:5:"total";i:50;s:6:"status";s:10:"processing";s:9:"processed";i:0;s:6:"errors";i:0;s:17:"oversized_skipped";i:0;s:11:"last_update";i:1743091477;}

advmo_bulk_offload_media_process_batch_ab98c8acef408334645db50bf:
a:50:{i:0;s:3:"288";i:1;s:3:"289";i:2;s:3:"385";i:3;s:3:"504";i:4;s:3:"505";i:5;s:3:"506";i:6;s:3:"714";i:7;s:3:"731";i:8;s:3:"769";i:9;s:3:"779";i:10;s:3:"890";i:11;s:3:"892";i:12;s:3:"895";i:13;s:3:"896";i:14;s:3:"906";i:15;s:3:"923";i:16;s:3:"925";i:17;s:3:"928";i:18;s:3:"938";i:19;s:3:"940";i:20;s:3:"941";i:21;s:3:"942";i:22;s:3:"943";i:23;s:3:"944";i:24;s:3:"950";i:25;s:4:"1004";i:26;s:4:"1006";i:27;s:4:"1007";i:28;s:4:"1018";i:29;s:4:"1053";i:30;s:4:"1054";i:31;s:4:"1058";i:32;s:4:"1067";i:33;s:4:"1069";i:34;s:4:"1072";i:35;s:4:"1099";i:36;s:4:"1102";i:37;s:4:"1109";i:38;s:4:"1110";i:39;s:4:"1111";i:40;s:4:"1112";i:41;s:4:"1113";i:42;s:4:"1114";i:43;s:4:"1118";i:44;s:4:"1126";i:45;s:4:"1128";i:46;s:4:"1130";i:47;s:4:"1132";i:48;s:4:"1139";i:49;s:4:"1140";}
```
Thread Starter cbrherms
(@cbrherms)

1 year, 2 months ago
Actually, saying that, i’ve just checked the nginx logs as well on this install and can see that there are 401 unauthorized errors every (roughly) 5 minutes relating to the plugin. Does the background offloader retry every 5 minutes?
```
{
  "installName": "*sanitised*",
  "environment": "STAGE",
  "domain": "*sanitised*",
  "ipAddress": "*sanitised*",
  "date": "2025-03-27T17:10:08.000Z",
  "type": "nginx",
  "verb": "POST",
  "resource": "/wp-admin/admin-ajax.php?action=advmo_bulk_offload_media_process&nonce=bf026d2065",
  "protocol": "HTTP/1.0",
  "status": 401,
  "referer": null,
  "userAgent": "WordPress/6.7.1; *sanitised*",
  "uuid": "*sanitised*"
}
```
Thread Starter cbrherms
(@cbrherms)

1 year, 2 months ago
Just to follow up (apologies, should have updated last night):

It turns out the root cause was a non-standard config on our end. Specifically, we have HTTP basic auth enabled on this environment, which was causing the offload call to fail with a 401 error (as shown in the log in my previous comment). Temporarily disabling basic auth allowed the plugin to offload the full batch of 50 files successfully.

That said, I was wondering if it might be possible to enhance the plugin to handle such cases more gracefully? Perhaps:
- Detecting and surfacing a clearer error when a 401 error occurs, so users can identify the issue quickly / avoid the background task getting stuck as stalled
- Adding an advanced option to allow users to set HTTP basic auth credentials for the offload call, ensuring the process works seamlessly in environments like ours
Thank you for your time and efforts in looking into this and for this plugin in general – much appreciated!
Plugin Author Masoud Golchin
(@masoudin)

1 year, 2 months ago

Hello @cbrherms

Thanks for sharing these helpful updates with us.

Detecting and surfacing a clearer error when a 401 error occurs, so users can identify the issue quickly / avoid the background task getting stuck as stalled

Thanks for the suggestion. We will definitely work on logging REST API events to catch such errors.

Adding an advanced option to allow users to set HTTP basic auth credentials for the offload call, ensuring the process works seamlessly in environments like ours

I would refine this to:

The feature has been added to our backlog, but implementation depends on user demand. If more requests come in, we’ll prioritize it. Meanwhile, a general solution would be to bypass HTTP basic auth for internal requests, since whitelisting backend requests should be straightforward.

Viewing 5 replies - 1 through 5 (of 5 total)

The topic ‘Bulk upload stuck’ is closed to new replies.