• Can there please be a fix, a setting or filter to not report “403 Forbidden errors” as broken links (for local)?

    There are more and more websites that block your link checks and your plugin reports them all as broken while they actually work. I understand that the plugin can’t tell if a link is broken or not when the link checker is being blocked, but in that case it shouldn’t be reported as broken.

    I know I can mark them as ‘not broken’, but for some reason, when the same link is checked again, it is also reported as broken again.

    Instead I suggest:

    • Best to fix the issue by better mimicking a real browser
    • Have a setting to filter ‘broken’ link reports by server status codes and to exclude them from the email reports.

    I have searched the support posts and this is being reported multiple times over the past few years. Can you please do something about it? Thanks!

    Log: === HTTP code: 403 === Response headers

    HTTP/1.1 403 Forbidden
    Cache-Control: no-store, must-revalidate, no-cache
    Pragma: no-cache
    Connection: close
    Content-Type: text/html
    Content-Length: 244
    X-Frame-Options: SAMEORIGIN
    X-XSS-Protection: 1; mode=block
    X-Content-Type-Options: nosniff Request headers

    GET /this-is-the-slug HTTP/1.1
    Host: externaldomain.nl
    User-Agent: WPMU DEV Broken Link Checker Local Engine
    Accept: /
    Accept-Encoding: deflate, gzip, br
    Referer: https://www.mydomain.nl
    Connection: close
    Range: bytes=0-2048 Response HTML

    Request RejectedThe requested URL was rejected. Please consult with your administrator.

    Your support ID is: <5782400494749519032>
Viewing 7 replies - 1 through 7 (of 7 total)
  • Plugin Support Jair – WPMU DEV Support

    (@wpmudevsupport15)

    Hello @jpnl

    Thanks for the suggestion. I can see something like that can improve the plugin, as sometimes one would be fine by not receiving notifications from broken links that happen because a 403: forbidden error.

    I’m sharing this suggestion to our development team. Take note that these things can take time to investigate and implement, so I don’t have an ETA of when this feature may be deployed in the future.

    Kind regards,
    Jair.

    Thread Starter John-Pierre Cornelissen

    (@jpnl)

    Thanks. I can tell it takes a long time. I found a ticket about is from 3 years ago.

    peteskitoo

    (@peteskitoo)

    I have a similar issue. The URL being checked works ok but obviously didn’t like the robotic activity. The other error I seem to get is 415, Unsupported Media Type, as well as 403. I was hoping clicking Dismiss or Not Broken would take a URL off of this list of ones to check in the future. Maybe into a list that I can manually check later.

    Plugin Support Nithin – WPMU DEV Support

    (@wpmudevsupport11)

    Hi @jpnl,

    I would like to update that our team will be improving the current down the roadmap. At the moment, we don’t have an exact ETA, but we hope it will be soon.

    @peteskitoo You can check our doc for what each status code error would mean:
    https://wpmudev.com/docs/wpmu-dev-plugins/broken-link-checker/#broken-link-status-codes

    If you need any further assistance, we would highly recommend that you open a new thread. Please use the following URL to open a new thread:

    https://ww.wp.xz.cn/support/plugin/broken-link-checker/#new-topic-0

    Regards,

    NIthin

    Thanks Nithin,

    I just looked at the ‘how to fix’ for the 403 error. It says

    If you can access the page please make sure to allow BLC User Agent and BLC IP addresses to scan the page. See the Troubleshooting & Notes chapter below.

    Unfortunately that only works if you have control over the target site. You can’t do that for 403 links that point to social media sites like Facebook, or stock sites like Pexels.

    Ideally, it does a re-check with another user agent and/or from a different IP address. If that still doesn’t work, then it can be reported a broken. If a user then marks the link as not broken, it shouldn’t report it again until the status code has changed to something else.

    Thanks
    JP

    I’d be willing to manually check my many 403 errors if I was assured that if I mark the link to be dismissed or as not broken, then it would stay dismissed for an extended period of time, not reappear in the next scan.

    • This reply was modified 4 months, 3 weeks ago by sallijane.

    Hi everyone

    As said above, this was already forwarded to our developers for future improvement, and I am marking this thread as resolved, but if you have any additional questions, feel free to let us know.

    Kind Regards,
    Kris

Viewing 7 replies - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.