Hi @galbaras,
Trust you are doing well, and thank you for reaching out to us.
I see that you are using the Local version of the plugin. Could you please give a try using the new and improved Cloud version, which has a better user experience? Please find more about how to exclude URLs in the Cloud version here: https://wpmudev.com/docs/wpmu-dev-plugins/broken-link-checker/#ignoring-links
Please feel free to get back to us if you need any further assistance. We are here to help.
Best Regards,
Nebu John
Hi Nebu,
I tried the cloud checker for a while, and it was way more hassle to use it than the local checker. Also, the cloud checker excludes the specific links, which means a particular page, and I’m asking to exclude the entire domain.
The most frequenly-reported reason for me is 403 – access denied, which is generally not specific to a page, but rather a domain policy that blocks the user agent used by the plugin. So I need to exclude entire domains and would love such a feature on the local checker.
Finally, some sites that reject checker access are served via Cloudflare, so maybe you could look into reaching some agreement with Cloudflare that prevents this from happening.
Thank you,
Gal
-
This reply was modified 1 month ago by
Gal Baras.
Hello @galbaras ,
I hope you are doing well, and thank you for the details.
With the Cloud version, you can exclude specific domains and URLs, as well as filter by HTTP status codes. In your case, you can configure exclusions for 403 errors to prevent them from being flagged for specific domains.
Please check this screenshot: https://prnt.sc/TPA9K775E97F
The tool also supports both regular URLs and regex patterns, giving you flexibility in defining these rules.
For more information, please refer to the documentation:
https://wpmudev.com/docs/wpmu-dev-plugins/broken-link-checker/#ignore-url-rules
Kind Regards,
Fida Al Hasan
Thanks for this. Looks like the cloud option has evolved since I used it last, so I’ve activated the cloud connection on one of my sites to test. I’ll update once I see the report.
Hi @galbaras ,
I hope you’re doing well.
Sure! Let us know how it goes, please.
Best Regards,
Williams Valerio
I’ve run a few scans now, and I’m not getting any broken links. I’m not sure if this is because the cloud scanner gets through where the local scanner fails, or because the cloud scanner is under-reporting.
Is there no chance of adding the requested feature to the local scanner?
Hello @galbaras
The Cloud Scanner should be able to scan links that the local one can’t, as some sites and security services may be aware of the BLC Cloud User Agent, and bypass their requests.
If you notice a link that is broken, but BLC doesn’t detect it, please share us that link so we can test it and investigate why BLC is not detecting it.
Kind regards,
Jair.
Thank you for your reply.
I would feel better if I could see the list of URLs that were checked and confirm that previously-broken links now pass. Is this possible? I can’t find any debugging options.
Side note: my site has link elements with rel="icon" in the head, but the hub doesn’t seem to be able to use them as the site thumbnail, and when I upload a custom site thumbnail, it isn’t displayed in the site list (at least within 5 minutes).
Hello @galbaras
Unfortunately, there is no option to see the list of links that checked by the plugin, you need to run the scan again to see if it can catch the mentioned link.
Side note: my site has link elements with rel=”icon” in the head, but the hub doesn’t seem to be able to use them as the site thumbnail, and when I upload a custom site thumbnail, it isn’t displayed in the site list (at least within 5 minutes).
HUB doesn’t use your site icon in site list. Where do you check this to see the site icon?
Best Regards
Amin
Hello Amin,
I’ve saved the local checker’s list of broken links in a PDF file for you to compare. The cloud report listed 8 pages within the local domain, showing them as “internal request timeout”, likely because Cloudflare blocked excessive access attempts, but NONE of the locally broken links were listed.
See Local Broken Links – Family Matters.pdf – broken links are from YouTube, LinkedIn and Canva, but over time, I’ve had many broken links from Pixabay, Freepix and various news sites, which I marked as “not broken”, all of which got 403 HTTP statuses.
So please help me find a way to trust the cloud report.
BTW, in the HUB, I can only ignore links, which, to me, is like the “dismiss” local action, but I can’t mark links as “not broken”, edit them, fix redirects or even recheck them. And the HUB interface doesn’t include filters like the local checker does (broken, redirects, warnings, images, dismissed).
I’ve just looked at the site thumbnail again, and it seems to be a screen capture, not the site icon.
Thanks again,
Gal
Hello @galbaras,
Thank you for your reply.
I was able to check the PDF and compare the current BLC Cloud scan results. The main difference here, when it comes to the BLC Local Version and the BLC Cloud version, is that the BLC Cloud version is made to skip sites that generally block bot traffic. You can check the sites that it will skip from scanning https://wpmudev.com/docs/wpmu-dev-plugins/broken-link-checker/#scanned-skipped
Moreover, the BLC Local Version relies on scanning links from the database whereas the BLC Cloud version will scan links from the frontend page via the BLC UA. Ref: https://wpmudev.com/docs/wpmu-dev-plugins/broken-link-checker/#broken-link-checker-user-agent
but I can’t mark links as “not broken”, edit them, fix redirects or even recheck them.
Regarding the Not Broken functionality, it is essentially the same as ignoring a link on BLC Cloud. But yes, the exact “Mark as not broken” is not available on the Cloud engine as the link highlighted as broken can be either ignored, fixed by editing or can be unlinked altogether via BLC Cloud. Regarding editing, have you already tried the options available at the Broken link list? Ref: https://wpmudev.com/docs/wpmu-dev-plugins/broken-link-checker/#broken-link-list
I am referring to the options here: https://monosnap.ai/file/qAvTE3NtFQjaBZuOWFgWrhowGFvSGR
I’ve just looked at the site thumbnail again, and it seems to be a screen capture, not the site icon.
Regarding the screencapture/image, can you please confirm if the link is within the image element? Could you please share the screenshot of the element with us so that we can take a closer look?
That said, I would also like to mention that there is an already reported bug in the BLC Cloud version in which the Cloud engine may fail to detect external broken links that are added to the site. At the moment, there is no confirmed ETA as to when this bug will be fixed, but it should be soon as the BLC team is already working on a fix. You can follow the updates by following to our roadmap page here: https://wpmudev.com/roadmap/
Kind Regards,
Saurabh
Hi Saurabh,
Looks like I haven’t reacquainted myself with the HUB. Thank you for the tour.
Skipping sites wholesale means that when, say, a YouTube video goes missing, which has happened before, I won’t get a chance to check this for myself. In that respect, the local scanner listing false positives is actually better.
Regarding the screencapture/image, can you please confirm if the link is within the image element? Could you please share the screenshot of the element with us so that we can take a closer look?
In the HUB, in site Settings > Site Thumbnail, I thought this was meant to be the site icon, but it’s a screen capture of the site’s front-end. Looking at this one, for example, it looks nothing like the Pacific Springs home page, so something isn’t right in the capture, but it’s not an important issue (I’m not even sure where it’s used).
Thanks again,
Gal
Hi @galbaras,
Thanks for pointing out regarding the Thumbnails, can confirm it’s a bug and our team is already working to get that sorted on the Hub side. At the moment, there isn’t an exact ETA but can confirm it’ll be fixed in future updates.
Regards,
Nithin
Hi Nithin,
Thanks for the update.
Can you confirm that YouTube, Cloudflare, etc, are only skipped when they return a 403 status, but NOT when they return, say, a 404? A 404 status indicates a true broken link, unlike a 403.
Thanks again,
Gal
HI @galbaras,
The sites mentioned here are skipped because they block the BLC crawler by default, as it only returns 403 and no other status:
https://wpmudev.com/docs/wpmu-dev-plugins/broken-link-checker/#scanned-skipped
So even if a URL is broken, the BLC crawl will only get a 403 instead of a 404, so there won’t be a way for the BLC cloud version to determine whether the link is actually working, as it’s blocked.
Regards,
Nithin
