Impossible to run Crawler manually…

  • Resolved Armsportstore.com

    (@armbreakersweden)


    3 years, 2 months ago

    I have run Litespeed Cache for a long time and never had any big issues with it. Just the other day something weird happened though. I usually clear the cache when I make changes on my site just to make sure everything is working properly with checkout etc. So cleared the cache and clicked on “Manually run” as usual on the Crawler page. Have 9 cookie varies for currency, so 9 separate threads for cache. The crawler suddenly got stuck on crawler number 8. And for some really odd reason it showed that it was on 3792 or something similar in the sitemap, which is impossible, as the sitemap only has 1354 objects.

    So I tried everything. I clicked reset position and tried to run the crawler manually again, which didn’t work. I cleared the folder wp-content/litespeed/crawler and also cleared the sitemap on map-tab. And then clicked on the Manually run again. The sitemap is created as usual and map-tab is filled with 1354 objects as it should, but the crawler doesn’t start. Normally I have to press the Manually run first and wait for the sitemap to be created and then press it again and refresh the page, and the green outline is shown with the position the crawler is at. But this time nothing nothing happens. And also strangely is that the meta.data.reset is created everytime I click the Manually run button, even though the crawler is at position 1 on crawler 1.

    Even contacted my host to see if something could be done from the server side, but they checked that everything was fine from their side.

    Next I tried de-activating the plugin and enable it again, nothing changed.

    Here is the really strange thing, the crawler can still be run through the Crawler cron. So as I tried everything, I tried setting the Crawl interval to something small. And the crawler started working as it should and everything got cached, which made me really happy, as I thought everything was working again.

    Today I tried running it manually again, and realized that the issue still exists… It is impossible to run it manually. If I click the Manually run and wait for the sitemap creation and then hit it again and refresh the page, nothing happens. And if I open the “Watch Crawler status” I only see “Start watching” and dots that move but nothing ever happens.

    So what could have caused the manual feature to just stop working and any ideas on how I can fix it? (Can’t find anything to fix in the database either as I have gone through all tables related to the crawler).

    The page I need help with: [log in to see the link]

Viewing 9 replies - 1 through 9 (of 9 total)
  • serpentdriver

    (@serpentdriver)

    3 years, 2 months ago

    Press the button for “Manually run” twice.

    Thread Starter Armsportstore.com

    (@armbreakersweden)

    3 years, 2 months ago

    I know I wrote a long text, but I clearly wrote that I always do just that, so that doesn’t solve my problem. I have been running it manually probably thousand of times during the years I used Litespeed, so I know how to run it. It doesn’t WORK.

    I am happy to get help, but I am not stupid and if you think I spent two full days without trying the basic things, you think wrong. I am usually very patient but right now I am really annoyed by this as the time I spent trying to get it working just sucks. So please suggest other things then common knowledge.

    serpentdriver

    (@serpentdriver)

    3 years, 2 months ago

    Nobody said you were stupid. However, it was obvious from your description that your problem would be solved by pressing the button twice.

    Thread Starter Armsportstore.com

    (@armbreakersweden)

    3 years, 2 months ago

    Don’t know were you want to go with this, I clearly wrote it doesn’t WORK. I will wait for someone else to answer as this doesn’t help me at all. The manual crawler feature is totally dead and it doesn’t matter if I click the button 10-15 times, it doesn’t work. Have tried everything possible.

    And by the way, if it is obvious from my description, then you haven’t read what I wrote. So if you really want to help, and not just try to be patronizing, then read the whole text again. It is easier to respond correctly if you do that and not just guessing. Try it 🙂

    Thread Starter Armsportstore.com

    (@armbreakersweden)

    3 years, 2 months ago

    Wanted to update with Report number: TPCFGMED

    Plugin Support qtwrk

    (@qtwrk)

    3 years, 2 months ago

    please try set crawler interval to 61 , then wait for 2 minutes and manually start it

    also make sure you don’t exceed the server load limit.

    Thread Starter Armsportstore.com

    (@armbreakersweden)

    3 years, 2 months ago

    Sadly I already tested that, as I saw that you wrote that in another thread. I went through the whole forum here for anything related to Crawler. It doesn’t work. When the issue occurred there was a period when the crawler’s end reason was due to high_load. This has never been an issue before and as stated I have run it thousands of times before this.

    After the high_load was displayed I changed the threads to 3 and lowered the crawler interval and waited a couple of hours just in case there was a server limitation applied. But since then the Manually run haven’t worked a single time.

    And just to clarify, the routine when I do the manual crawling is like this:

    1. Click the button Manually run
    2. After a while the message “Sitemap created successfully: 1354 items” is displayed.
    3. I hit the Manually run again and wait a few seconds and refresh the page.
    4. The first crawler has a green outline around the position and the crawler is running.

    What happens now is that when I click the button the second time is that the message “Sitemap created successfully: 1354 items” is displayed again, which has never happened before. So regardless of how many times I click it, this message is displayed but there is no green outline and the “Show crawler status” has no activity at all, only displays “Start watching…”

    Thread Starter Armsportstore.com

    (@armbreakersweden)

    3 years, 2 months ago

    Just a question, I am not very technical on how things work in the background. But can there be something wrong with the WP-Cron that should trigger the manual run? Is it possible to delete the WP-Cron job and re-create it?

    Plugin Support qtwrk

    (@qtwrk)

    3 years, 2 months ago

    in crawler setting , disable and re-enable the crawler will delete/recreate the cron job

    but no really , manually run does not involve the cron I think , since you triggering it manually

    please create a ticket by mail to support at litespeedtech.com with reference link to this topic , we will investigate further.

Viewing 9 replies - 1 through 9 (of 9 total)

The topic ‘Impossible to run Crawler manually…’ is closed to new replies.