Crawler not working properly

Resolved dbs1990
(@dbs1990)

1 month, 1 week ago

I am having issues with the LiteSpeed crawler on multiple WordPress sites.

The cron job does seem to trigger correctly, but the crawler itself often does not appear to actually start.
When it does start, it frequently ends in stopped_reset or stays stuck on end.

If I start the crawler manually from the WordPress admin, it usually works.
My rapport number is: UKZRNFLV
what am i doing wrong or is going wrong?

Thank you in advance!
DBS

Viewing 5 replies - 1 through 5 (of 5 total)

niburu
(@niburu)

1 month, 1 week ago

I just noticed, it cant read my sitemap anymore. Ima getting “No valid sitemap parsed for crawler”. i did reset my optino, used thw word press sitemap instead of rank maths. Few days ago, it worked like it should.

Plugin Support qtwrk
(@qtwrk)

1 month, 1 week ago

if you manually run it , and it works, then it should work

do you have properly set up the wp cron ?

ref: https://docs.litespeedtech.com/lscache/lscwp/admin/#disable-wp-cron
Thread Starter dbs1990
(@dbs1990)

1 month, 1 week ago
Hi,

I believe my cron job is set up correctly.

I have disabled the default WordPress cron (DISABLE_WP_CRON) and I am using a server-side cron with the following command:

8,38 * * * * php -q “/home/hostmap/domains/mydomain.be/public_html/wp-cron.php” >> /home/hostmap/domains/wp-cron.log 2>&1

However, I’m experiencing inconsistent behavior across different sites:
- On some sites, every 30 minutes one crawler thread completes normally.
- On other sites, the crawler sometimes progresses, but often nothing happens at all.
- In some cases, the crawler remains stuck on “end” for an entire day.
- Occasionally, I also see a “stopped_reset” status.
So even though the cron seems to be running, the crawler behavior is very inconsistent.

Has anyone experienced similar issues or has an idea what could be causing this?

Thanks in advance!
- This reply was modified 1 month, 1 week ago by dbs1990.
- This reply was modified 1 month, 1 week ago by dbs1990.
Plugin Support litetim
(@litetim)

1 month, 1 week ago

@niburu please open a different topic and I can help you with your issue.
Thank you

Plugin Support qtwrk
(@qtwrk)

1 month ago

please try this

set crawler interval to 600 seconds

go to toolbox -> debug setting -> set debug log to on , debug level to advance, add wp-cron and async_litespeed into debug include URI

then go to log view , clean up existing log if any

then wait for 30 minutes , check log view again , copy paste and share log via https://pastebin.ubuntu.com/

Viewing 5 replies - 1 through 5 (of 5 total)

You must be logged in to reply to this topic.