• AI Visibility Score. When we analyze how well our site is optimized to be discovered.

    We have this message (5 wordpress websites) :

    Error analyzing page

    Could not connect to the site: cURL error 28: Connection timed out after 10000 milliseconds

    And we have : Crawlers that visited your site in the last 30 days despite being in your Disallow list. What can we do ? We have block crawlers at PHP level by User-Agent .

Viewing 3 replies - 1 through 3 (of 3 total)
  • Plugin Author Fernando Tellado

    (@fernandot)

    Bonjour @zepolo,

    Thanks for the details. These are actually two separate things, and the good news is that neither is a bug in the plugin nor a problem with your content.

    1. “Could not connect / cURL error 28: timed out”

    To score a page, VigIA requests your own site over HTTP (a “loopback” request). This error means your server can’t connect back to its own public URL within 10 seconds. It’s a server/network issue, not a slow page and not the plugin itself. It’s very common when a site sits behind Cloudflare, a CDN/proxy, or a security firewall that silently drops these self-requests.

    Could you check two things so we can pinpoint it?

    • In WordPress, go to Tools → Site Health. If you see a failed notice about “loopback requests”, that confirms the cause.
    • Let me know whether these sites are behind Cloudflare / a CDN / a proxy, and whether you run a security firewall or rules that block by User-Agent or IP. That’s likely what’s blocking the analyzer, and I’ll tell you exactly what to whitelist.

    2. “Crawlers that visited despite being in your Disallow list”

    This one is expected behavior, not an error:

    • Disallow in robots.txt is a request, not a lock. Well-behaved bots obey it; misbehaving ones ignore it. That list is simply showing you which crawlers ignore your robots.txt.
    • Your PHP User-Agent block is still working. When it blocks a bot, the bot gets a 403 Forbidden and receives no content, but the request still reached your site, so it gets logged. So “visit” here means “knocked on the door”, not “got in and copied your content.”

    You can confirm your block is doing its job: in VigIA’s visit log, those crawlers should show HTTP status 403. If they do, you’re fully protected and can safely ignore them. (If any show 200, send me the User-Agent and I’ll help you tighten the rule.)

    One tip: User-Agent blocking happens after the bot already reached your site (so it’s always logged), and bad bots often fake their User-Agent. If you want to stop them reaching the server at all, the most effective place is at the edge: Cloudflare’s “Block AI bots” / WAF, or blocking by IP/ASN.

    Just send me the Site Health result and your hosting/firewall setup, and I’ll close out the connection issue.

    Thread Starter zepolo

    (@zepolo)

    hola Fernando,

    This is site health :

    Lors du test de l’API REST, une erreur s’est produite :

    Point de terminaison de l’API REST : https://www.xxx.com/xxx/wp-json/wp/v2/types/post?context=edit
    Réponse de l’API REST : (http_request_failed) cURL error 28: Connection.

    I don’t use cloudflare. I use wordfence and plugin BBQ pro, Really Simple Security Pro.

    Plugin Author Fernando Tellado

    (@fernandot)

    Perfect, that’s confirms the cause.

    The Site Health test shows WordPress itself can’t reach its own REST API (same cURL error 28), so this is a server “loopback” issue, not VigIA and not your content. Any plugin that checks your site this way will hit the same wall.

    The most likely cause is your security plugins overlapping. To fix it, temporarily deactivate them one at a time and re-run the Site Health test after each (Wordfence → BBQ Pro → Really Simple Security Pro). The one that makes the test pass is the one blocking the self-request: re-enable it and add your server’s own IP to its allowlist.

    As a side note, running three security plugins together, each with its own firewall, is what tends to cause exactly this kind of conflict. Consolidating to a single one avoids it. Wordfence already covers most of what the other two do. Another solid, completely free option is Vigilant, my own plugin.

    Since this sits on the server/security side rather than VigIA, your host or those plugins’ own support are the best people to help you apply the fix. Once the Site Health loopback test passes, VigIA’s score will work on its own.

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.